A cache, computing device, and computing system

Abstract

The present disclosure relates to a cache, a computing device and a computing system, the cache comprising: request receiving circuitry configured to receive a target memory access request sent by a computing unit, the target memory access request comprising a target memory access request index; request executing circuitry configured to execute the target memory access request to obtain target response data of the target memory access request; request returning circuitry configured to send the target response data to the computing unit; performance counting circuitry configured to obtain the target memory access request index, determine a number of target clock cycles between sending the target response data and last response data by the request returning circuitry, and generate target performance data representing an execution time of the target memory access request according to the memory access request index of the target memory access request and the number of target clock cycles. In this way, subsequent reading and analysis by a developer can be facilitated, and a specific request affecting memory access performance can be accurately located, thereby providing a fine-grained and traceable data basis for program performance optimization.

Description

Technical Field

[0001] This disclosure relates to the technical field of caching, and more specifically, to a cache, a computing device, and a computing system. Background Technology

[0002] Computing devices, represented by Graphics Processing Units (GPUs), are widely used in general computing and have become core hardware in applications such as Artificial Intelligence (AI) and scientific computing. Within GPUs, the Last Level Cache (LLC), acting as the final buffer between the CPU and main memory, is a key hardware resource affecting program performance. Its performance directly determines the memory access latency of computing cores, system throughput, and hardware resource utilization, thus having a decisive impact on program efficiency. As computer applications extend to high-performance computing, artificial intelligence, and cloud services, program complexity continues to increase, placing higher demands on the accuracy of LLC performance monitoring. Precise LLC performance data has become a core basis for program performance optimization and hardware resource scheduling.

[0003] Most existing solutions treat LLC as an isolated hardware module, failing to establish its relationship with the computing core. They ignore the inherent connection between LLC's memory access behavior and the execution of instructions by the computing core, making it impossible to accurately map each specific memory access behavior of LLC to the specific assembly instructions being processed by the computing core, and failing to establish a one-to-one correspondence between memory access behavior and instruction execution.

[0004] The aforementioned shortcomings mean that existing solutions can only provide developers with coarse-grained LLC performance conclusions, such as overall average cache hit rate, average memory access latency, and overall hardware resource utilization. They cannot precisely pinpoint which specific assembly instructions are causing insufficient LLC memory hits or excessive memory access latency, nor can they identify which instructions are consuming excessive LLC hardware resources. This coarse-grained performance data makes it impossible for developers to clearly determine the specific source of program performance bottlenecks. Summary of the Invention

[0005] One objective of this disclosure is to provide a new technical solution for a cache that records the performance of memory access requests, thereby enabling the recording of the performance of each memory access request.

[0006] According to a first aspect of this disclosure, a cache is provided, comprising: The request receiving circuit is configured to receive the target memory access request and the target memory access request index of the target memory access request sent by the computing unit. The request execution circuit is configured to execute the target memory access request and obtain the target response data for the target memory access request. The request return circuit is configured to send the target response data to the computing unit; A performance counting circuit is configured to acquire the target memory access request index, determine the target clock cycle number between the target response data sent by the request return circuit and the previous response data, and generate target performance data characterizing the execution time of the target memory access request based on the memory access request index and the target clock cycle number.

[0007] Optionally, the performance counting circuit includes: A clock cycle counter is configured to count clock cycles; The encoding module is configured to: detect whether the request return circuit sends response data in each clock cycle; if the response data sent by the request return circuit is the target response data, obtain the count of the clock cycle counter as the target clock cycle number, reset the count of the clock cycle counter to zero, and encode the target clock cycle number to obtain the target performance data.

[0008] Optionally, the encoding module is configured to: determine the target encoding format corresponding to the target clock cycle number and the target format identifier corresponding to the target encoding format based on the numerical range of the target clock cycle number; encode the target clock cycle number according to the target encoding format to obtain target encoded data; and obtain the target performance data based on the target format identifier and the target encoded data.

[0009] Optionally, the bit width of the data encoded by the encoding format corresponding to a smaller numerical range is smaller than the bit width of the data encoded by the encoding format corresponding to a larger numerical range.

[0010] Optionally, the encoding module is further configured to: detect whether the target memory access request is valid and obtain a detection result; and determine the target encoding format and the target performance data based on the detection result.

[0011] Optionally, the performance counting circuit further includes a storage module; The encoding module is also configured to: write the target performance data into the data packet buffer area in the storage module for caching; The storage module is configured to: when the amount of data cached in the data packet buffer reaches a set value, package the data in the data packet buffer to obtain a target data packet with a set bit width; write the target data packet into memory and clear the target data packet buffer.

[0012] Optionally, the target memory access request is the first memory access request that matches the data packet buffer received by the request receiving circuit after the target data packet buffer is cleared; The encoding module is further configured to: generate a data packet tag, write the data packet tag into the data packet buffer, and reset the clock cycle counter to zero; wherein the data packet tag includes at least a timestamp of receiving the target memory access request.

[0013] Optionally, the data packet label further includes at least one of the following: a data packet sequence number, used to identify the cache index of the cache, used to identify the computing unit index of the computing unit, and a memory access type identifier used to identify the memory access type corresponding to the associated performance data.

[0014] Optionally, the cache further includes a judgment circuit and a performance counting circuit that corresponds one-to-one with various memory access types; The determination circuit is configured to detect the memory access type of the target memory access request and send the target memory access request index to the performance counting circuit corresponding to the memory access type of the target memory access request.

[0015] According to a second aspect of this disclosure, a computing device is also provided, the computing device comprising: Multiple computing units, each computing unit including a level-one cache; Cache, said cache being the cache according to the first aspect of this disclosure; and, An on-chip bus is provided, wherein the level 1 cache is coupled to the on-chip bus, and the on-chip bus is also coupled to the cache.

[0016] Optionally, the first-level cache is configured as follows: In response to the target memory access request derived from the target memory access instruction by the corresponding computing unit, it detects whether the target memory access request hits the first-level cache, allocates a corresponding target memory access request index for the target memory access request; if the target memory access request does not hit the first-level cache, it carries the target memory access request index into the target memory access request, and sends the target memory access request carrying the target memory access request index to the cache.

[0017] According to a third aspect of this disclosure, a computing system is also provided, the computing system including a control device and the computing device according to a second aspect of this disclosure, the control device being used to control the computing device to perform computing tasks.

[0018] Through the embodiments of this disclosure, a performance counting circuit is set in the cache, enabling accurate collection and storage of memory access request performance data. By associating with the target memory access request index, the response interval clock cycle count of each memory access request is accurately counted, generating target performance data that corresponds one-to-one with specific memory access requests. This facilitates subsequent reading and analysis by developers, allowing them to accurately locate specific requests affecting memory access performance and providing fine-grained, traceable data for program performance optimization. Simultaneously, the performance counting circuit works collaboratively with the request receiving circuit, request execution circuit, and request return circuit, without affecting the normal processing flow of memory access requests, ensuring the efficiency and reliability of memory access processing, and adapting to scenarios with stringent memory access performance requirements, such as high-performance computing and artificial intelligence.

[0019] The features and advantages of the embodiments of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of these embodiments.

[0021] Figure 1 This is a schematic diagram of the structure of a computing system provided in this disclosure; Figure 2 This is a structure diagram of the cache based on some embodiments; Figure 3 This is a schematic diagram of the memory access request processing architecture according to some embodiments; Figure 4 This is a schematic diagram of a cache according to some other embodiments; Figure 5 This is a schematic diagram of a performance counting circuit according to some embodiments; Figure 6 These are schematic diagrams of the structure of a computing device according to some embodiments; Figure 7 This is a schematic diagram of the structure of a computing system according to some embodiments. Detailed Implementation

[0022] Various exemplary embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0023] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the embodiments of this specification or their application or use.

[0024] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0025] It should be noted that all data acquisition actions in this disclosure were carried out in compliance with the relevant data protection laws and policies of the country where the data is located, and with the authorization of the relevant equipment owner.

[0026] This disclosure relates to a cache access request control scheme that dynamically adjusts the receiving speed of cache access requests to balance the consumption of cache processing resources by read and write requests. Figure 1 This is a schematic diagram of the structure of a computing system to which the memory access request control scheme provided in the embodiments of this disclosure can be applied. Figure 1 As shown, the computing system 100 may include a computing device 110, a control device 120, and a storage device 130, and the computing device, control device, and storage device may be connected to each other via a bus network or other means.

[0027] This storage device can be used to store instructions and / or data, which can be retrieved and used by a control device or a computing device. For example, the storage device can store program instructions executed by the control device or computing device, as well as data such as text, images, audio, and configuration parameters. The control device can control the computing device to execute related processes or tasks to achieve related system functions, such as artificial intelligence, scientific computing, or image processing functions based on user needs. For example, the control device can be a central processing unit (CPU); the computing device can be a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a neural network processing unit (NPU), or a tensor processing unit (TPU); and the storage device can be static random access memory (SRAM), read-only memory (ROM), or erasable programmable read-only memory (EPROM). This storage device can also be called memory, and this computing device can also be called a chip, processor, or artificial intelligence chip.

[0028] Continue as Figure 1As shown, the computing device 110 may include a computing unit 111 and a cache 112. The computing unit may be the core component in the computing device that performs computing tasks, and may also be referred to as a computing core, arithmetic core, or processing core. The cache may be used to cache instructions and / or data, and may also be referred to as a cache circuit, on-chip memory circuit, or on-chip memory.

[0029] This computing device contains multiple computing units, which can execute a process or task together or individually. Taking a GPU as an example, the computing unit can be a GPU Core, and a GPU consists of multiple GPU Cores.

[0030] The processing efficiency of the cache in this computing device is greater than that of external storage devices, but the storage capacity is smaller. This cache can be used to store recently used instructions and / or data to improve the processing efficiency of the computing device. For example, the computing device may include multi-level caches (at least two levels), such as a Level 1 cache (L1 Cache) and a Level 2 cache (L2 Cache), or a Level 1 cache (L1 Cache), a Level 2 cache (L2 Cache), and a Level 3 cache (L3 Cache). The Level 2 or Level 3 cache, acting as the Last Level Cache (LLC), is the Level 1 cache with the largest capacity and closest to the external storage device 130. It can directly access the storage device 130, for example, by reading instructions from or writing data to the storage device 130 via a bus. For example, each computing unit can have its own independent level-one cache, allowing each level-one cache to be used independently by a single computing unit. Alternatively, multiple computing units within the computing device can be grouped into multiple computing unit groups (e.g., SPCs), with each computing unit group having its own independent level-one cache, allowing each level-one cache to be shared by the computing units within that group. The last-level cache can be configured to be shared by all or some of the computing units in the computing device; this last-level cache can also be called the final level cache, shared cache, etc.

[0031] For example, a cache can employ a set-associative organization structure. For instance, sibling caches can be divided into multiple sets, each containing multiple ways, and each way containing one cache line (also called a cache block). In this way, a cache line can be uniquely identified by an index within the (set, way) group. A specific memory address in memory can be mapped to any cache line, thus improving processing efficiency when a computing unit needs to access a specific memory address by accessing the mapped cache line.

[0032] The compute unit performs read and write operations on the cache through memory access requests. For caches with a large number of compute units, such as the last-level cache, one possible approach to monitor cache performance is to deploy counters in the LLC to collect key performance parameters, including but not limited to core metrics such as the number of read requests, the number of write requests, the number of read hits, and the number of write hits. Based on the data collected by these counters, the LLC-related performance parameters of the program within a given runtime range are further calculated, such as cache hit rate, memory access latency, and hardware resource utilization. However, this approach can only provide developers with coarse-grained LLC performance conclusions, such as the overall average cache hit rate, average memory access latency, and overall hardware resource utilization. It cannot accurately pinpoint which specific assembly instructions caused insufficient LLC memory hits or excessive memory access latency, nor can it identify which instructions consumed excessive LLC hardware resources.

[0033] To address the issues of the above solutions, this embodiment of the disclosure achieves accurate collection and storage of memory access request performance data by setting a performance counting circuit in the cache. By associating with the target memory access request index, the response interval clock cycles for each memory access request are accurately counted, generating target performance data that corresponds one-to-one with specific memory access requests. This facilitates subsequent reading and analysis by developers, allowing them to accurately pinpoint the specific requests affecting memory access performance and providing fine-grained, traceable data for program performance optimization. Simultaneously, the performance counting circuit works collaboratively with the request receiving circuit, request execution circuit, and request return circuit, without affecting the normal processing flow of memory access requests, ensuring the efficiency and reliability of memory access processing, and adapting to scenarios with stringent memory access performance requirements, such as high-performance computing and artificial intelligence.

[0034] The following combination Figure 1 The computing system described herein illustrates various embodiments.

[0035] <First Embodiment> Figure 2 A schematic diagram of the cache structure according to some embodiments is shown.

[0036] like Figure 2 As shown, the cache 112 may include a request receiving circuit 210, a request execution circuit 220, a request return circuit 230, and a performance counting circuit 240.

[0037] The request receiving circuit 210 is configured to receive a target memory access request sent by the computing unit, the target memory access request including a target memory access request index.

[0038] The request execution circuit 220 is configured to execute the target memory access request and obtain the target response data for the target memory access request.

[0039] The request return circuit 230 is configured to send the target response data to the computing unit.

[0040] The performance counting circuit 240 is configured to acquire the target memory access request index, determine the target clock cycle number between the target response data sent by the request return circuit 230 and the previous response data, and generate target performance data to characterize the execution time of the target memory access request based on the memory access request index and the target clock cycle number.

[0041] In this embodiment, the target memory access request may carry the memory access type (read type or write type) and the memory access address.

[0042] The request execution circuit 220 executes the corresponding memory access operation based on the memory access type and memory access address carried in the target memory access request. This operation may include: when the memory access type of the target memory access request is read, the request execution circuit 220 accesses the corresponding cache or memory according to the memory access address, reads the storage data corresponding to that address, and uses the read storage data as the target response data. At the same time, it associates the corresponding target memory access request index to ensure that the response data and the memory access request are accurately matched. When the memory access type of the target memory access request is write, the request execution circuit 220 writes the data to be written carried in the request to the corresponding cache or memory according to the memory access address. After the write is completed, it generates a write success confirmation signal as the target response data and associates it with the corresponding target memory access request index so that the computing unit can confirm the execution result of the write operation.

[0043] Cache 112 may also include a request scheduling circuit. When there are multiple concurrent memory access requests, the request scheduling circuit will schedule them according to the priority of the memory access requests (such as real-time requirements, data importance, etc.), so that the request execution circuit 220 will prioritize the execution of high-priority memory access requests, avoid memory access blocking, and improve memory access processing efficiency.

[0044] The request return circuit 230 establishes a data connection with the request execution circuit 220, receives the target response data and the corresponding target memory access request index transmitted by the request execution circuit 220, and then sends the target response data to the corresponding computing unit through the data return interface. The target response data is transmitted in conjunction with the corresponding target memory access request index. The computing unit can use this index to match the received target response data with its own sent target memory access requests, determining the response result for each memory access request and ensuring the accuracy of data interaction.

[0045] Furthermore, the request return circuit 230 supports both batch and single transmission of response data, and the transmission mode can be flexibly adjusted according to the amount of response data output by the request execution circuit 220.

[0046] In this embodiment, the target memory access request index is a unique identifier assigned by the computing unit to each target memory access request. It is used to distinguish different memory access requests and avoid correspondence errors between memory access requests and response data. The index can be a string of continuous numerical codes or a custom identifier, and its length can be flexibly set according to the actual concurrency of memory access requests.

[0047] Each computing unit can be configured with a first-level cache, in which a corresponding target memory access request index can be allocated for the target memory access request, and the mapping relationship between the instructions to be recorded and the memory access requests is maintained based on the format shown in Table 1 below.

[0048] Table 1

[0049] As shown in Table 1, each mapping relationship can consist of 4 domain segments. The kernel function index refers to the dynamic kernel number of the kernel function within the multi-core processor module to which the current instruction belongs. In one example, a computing unit can run 4 kernel functions in parallel; therefore, the bit width of a kernel function index is 2 bits. Here, a kernel function refers to a task submitted by the software. A single kernel function can run 8 thread bundles in parallel, and each thread bundle can contain 32 threads. The computing unit can issue 256 read requests and 256 write requests in parallel. Therefore, the mapping relationship between instructions and memory access requests is constructed using a 2-bit kernel function index, a 3-bit thread bundle index, an 8-bit memory access request index, and a 6-bit request index.

[0050] In the L1 cache, each thread bundle of each kernel function needs to maintain an independent table, which contains a mapping relationship arranged in sequence as shown in the diagram above. At any given time, it can generally be configured by software to maintain only one or more thread bundles of one or more kernel functions that need to be monitored.

[0051] Instructions are executed by a thread bundle. Since instructions are issued sequentially by a thread bundle, the order of the mapping relationships in the table is consistent with the order in which the L1 cache receives memory access instructions. Therefore, the instruction corresponding to the mapping relationship can be determined based on the order, and then the instruction corresponding to the target memory access request can be determined based on the mapping relationship.

[0052] If a thread bundle of 32 threads generates 32 different memory access requests, then they may be assigned a maximum of 32 consecutive memory access request indices. In this case, the number of memory access indices can be recorded as 32. If the memory access request is hit in the L1 cache and does not need to be sent to the cache, then this number of memory access indices can be recorded as 0.

[0053] In one embodiment, such as Figure 3As shown, a computing unit can include four or more computing cores, each of which runs one or more thread bundles in parallel. All thread bundles within multiple computing cores share the same L1 cache. If a memory access request received by the L1 cache does not hit the L1 cache, the memory access request is sent to the on-chip bus, and ultimately a read / write operation is performed on the cache. If the cache miss is also not found, a read / write operation needs to be performed on the main memory.

[0054] In one embodiment, such as Figure 4 As shown, cache 112 may also include a hit detection circuit 250 and a request scheduling circuit 260. After a memory access request enters the cache, it is first temporarily stored in the request receiving circuit 210, then the hit detection circuit 250 determines whether it hits cache 112, and then the request scheduling circuit 260 performs unified scheduling, sorting, and path allocation. The request scheduling circuit 260 internally maintains at least one scheduling queue for temporarily storing and sorting received memory access requests, and transmits the received memory access requests to the request execution circuit 220 for execution in sequence. If the memory access request hits cache 112, the request execution circuit 220 will perform read and write operations on the cache lines of cache 112; if the memory access request misses cache 112, the request execution circuit 220 will perform read and write operations on the memory lines. After the memory access request is executed, the response data of the memory access request executed by the request execution circuit 220 is sent to the request return circuit 230, which queues the response data and returns it to the corresponding computing unit in sequence.

[0055] In this embodiment, the request receiving circuit 210 can send the target memory access request index to the performance counting circuit 240 through the hit detection circuit 250.

[0056] In this embodiment, the request return circuit 230 can send one memory access request response data per clock cycle. When the request return circuit 230 sends the target response data to the computing unit, it indicates that the target memory access request has been completed.

[0057] The target clock cycle number is the interval between the target response data sent by the request return circuit 230 and the previous response data. Specifically, it is the difference between the clock cycle number when the request return circuit 230 sent the target response data and the clock cycle number when the request return circuit 230 sent the previous response data. This target clock cycle number represents the clock cycles required to execute the target memory access request.

[0058] The data format for the target performance data can be as shown in Table 2 below.

[0059] Table 2

[0060] As shown in Table 2, a target performance data point consists of two domain segments. Since a single computing unit can issue 256 read or write requests simultaneously, the memory access request index can be 8 bits.

[0061] In this embodiment, the target clock cycle count directly reflects the response interval time cost corresponding to a single target memory access request. Combined with the target memory access request index, a one-to-one correspondence between memory access requests and performance data can be achieved, enabling developers to accurately understand the response efficiency of each memory access request. Moreover, by binding the target performance data to the target memory access request index and the target clock cycle count, the performance of each memory access request can be traced, avoiding the disconnect between performance data and specific memory access behavior. This helps developers identify the specific source of performance bottlenecks, pinpoint the specific memory access requests that cause excessive memory access latency and low processing efficiency, and then optimize the program's instruction sequence and data access patterns accordingly, thereby improving the overall program performance.

[0062] Furthermore, based on the mapping relationship between the memory access request index in the target performance data and the instructions and memory access requests recorded in the L1 cache, the target memory access request can be mapped to the instructions processed by the computing unit, so that developers can perform analysis.

[0063] Through the embodiments of this disclosure, a performance counting circuit is set in the cache, enabling accurate collection and storage of memory access request performance data. By associating with the target memory access request index, the response interval clock cycle count of each memory access request is accurately counted, generating target performance data that corresponds one-to-one with specific memory access requests. This facilitates subsequent reading and analysis by developers, allowing them to accurately locate specific requests affecting memory access performance and providing fine-grained, traceable data for program performance optimization. Simultaneously, the performance counting circuit works collaboratively with the request receiving circuit, request execution circuit, and request return circuit, without affecting the normal processing flow of memory access requests, ensuring the efficiency and reliability of memory access processing, and adapting to scenarios with stringent memory access performance requirements, such as high-performance computing and artificial intelligence.

[0064] In some embodiments, such as Figure 5 As shown, the performance counting circuit 240 includes a clock cycle counter 241 and an encoding module 242. The clock cycle counter 241 is configured to count clock cycles. The encoding module 242 is configured to: detect whether the request return circuit sends response data in each clock cycle; if the response data sent by the request return circuit 230 is detected to be the target response data, acquire the count of the clock cycle counter 241 as the target clock cycle number, reset the count of the clock cycle counter to zero, and encode the target clock cycle number to obtain the target performance data.

[0065] The clock cycle counter 241 uses the system clock of the cache as its counting reference, and its clock frequency is consistent with the entire cache to ensure the uniformity and accuracy of the counting. The clock cycle counter 241 adopts a synchronous counting method. After each clock cycle, the counter value is automatically incremented by 1. The counting range can be preset according to the actual hardware design requirements to meet the memory access response interval counting requirements in different scenarios.

[0066] The clock cycle counter 241 has a reset function. It can receive a reset signal sent by the encoding module 242. After receiving the reset signal, it clears the current count value to zero and restarts counting from 0, ensuring that the clock cycle count corresponding to each target response data is independent and accurate. Within each clock cycle, if the request return circuit 230 sends a response data, it sets the clock cycle counter 241 to 0.

[0067] Encoding module 242 can detect the response data transmission status of request return circuit 230 in each clock cycle, that is, detect whether request return circuit 230 has sent response data in the current clock cycle. This detection process is synchronized with the clock cycle to ensure that no response data transmission is missed, providing accurate triggering for obtaining the target clock cycle number in subsequent cycles.

[0068] When the encoding module 242 detects that the response data sent by the request return circuit 230 is the target response data, the clock cycle counter 241 counts, which directly reflects the total number of clock cycles elapsed from the last clock cycle count reset to the current target response data transmission. In other words, it represents the target clock cycle number between the time the request return circuit 230 sends the target response data and the previous response data.

[0069] The purpose of encoding module 242 in encoding the target clock cycle count is to convert the original clock cycle count value into an encoding format that conforms to the storage specifications and is easy to read and analyze later. At the same time, it can combine the target memory access request index to associate the target memory access request index with the encoded target clock cycle count, so as to ensure that the target performance data can correspond one-to-one with the specific target memory access request.

[0070] Specifically, the encoding method can adopt binary encoding, BCD encoding, or other hardware encoding methods commonly used in this field.

[0071] In this embodiment, the performance counting circuit is subdivided into a clock cycle counter and an encoding module. The continuous counting and on-demand resetting of the clock cycle counter can ensure that the target clock cycle number corresponding to each target memory access request is independent and accurate, avoiding counting interference, so that the obtained performance data can accurately reflect the performance of the corresponding target memory access request. The transmission detection and encoding processing of the encoding module can not only realize the real-time generation of performance data, but also ensure that the performance data is easy to store, read and analyze through encoding format optimization.

[0072] In some embodiments, the encoding module 242 encodes the target clock cycle number based on a uniform encoding format.

[0073] In some embodiments, the encoding module 242 is configured to: determine the target encoding format corresponding to the target clock cycle number and the target format identifier corresponding to the target encoding format according to the numerical range of the target clock cycle number; encode the target clock cycle number according to the target encoding format to obtain target encoded data; and obtain target performance data according to the target format identifier and the target encoded data.

[0074] In this embodiment, to save storage space and bandwidth, a variable-length encoding method can be used to encode the target clock cycle number.

[0075] In one embodiment, multiple encoding formats can be preset, and a format identifier and numerical range corresponding to each encoding format can be set. For example, a first encoding format, a second encoding format, and a third encoding format can be set. The first encoding format corresponds to a 2-bit width, the second encoding format corresponds to an 8-bit width, and the third encoding format corresponds to a 16-bit width. The format identifier of the first encoding format is "0" (1 bit), the format identifier of the second encoding format is "10" (2 bits), and the format identifier of the third encoding format is "110" (3 bits). The numerical range corresponding to the first encoding format is [0,1], the numerical range corresponding to the second encoding format is [2,63], and the numerical range corresponding to the third encoding format is [64,∞).

[0076] When the target clock cycle count `delay_cycle` is 0, it indicates that the clock cycle in which the target memory access request completes is exactly adjacent to the clock cycle in which the previous memory access request completes, with no extra interval. When `delay_cycle` is 1, it indicates that there is a 1-clock-cycle interval between the two. When using this encoding format, the format identifier "0" is followed by one significant bit. This significant bit is used to represent the specific value of the target clock cycle count `delay_cycle` (0 corresponds to significant bit 0, and 1 corresponds to significant bit 1), forming a 2-bit target encoded data.

[0077] When the target clock cycle number delay_cycle∈[2,63], 8-bit encoding can completely represent this value range, while taking into account both storage efficiency and data integrity. The format identifier "10" is followed by 6 valid bits, which can cover the value range of 0~63, which exactly matches the applicable range of this encoding format. After encoding, the target encoded data is obtained as 8-bit.

[0078] It's important to note that if the actual value of `delay_cycle` is greater than 4095, it will be uniformly recorded as 4095 (16-bit encoding can cover the value range of 0 to 4095; this setting avoids encoding overflow and meets the performance statistics requirements in practical applications). When using the third encoding format, the format identifier "110" is followed by 12 significant bits, which can cover the value range of 0 to 4095, resulting in 16-bit target encoded data.

[0079] This embodiment flexibly selects the encoding method and format identifier based on the numerical range of the target clock cycle count, ensuring that target clock cycle counts with different numerical ranges can obtain appropriate encoding processing. This avoids data loss due to encoding overflow and reduces resource waste caused by bit width redundancy, ensuring that each piece of target performance data can completely and accurately represent the response interval characteristics of memory access requests. When generating target performance data, the target format identifier is associated and integrated with the target encoded data. The format identifier can directly represent the encoding format type, so when parsing the target performance data, it is not necessary to traverse and determine the encoding bit width. Only by parsing the format identifier can the encoding format and the parsing length of subsequent effective bits be quickly determined, reducing the computational load and complexity of parsing the target performance data and improving parsing efficiency and accuracy.

[0080] In some embodiments, the bit width of the data encoded by the encoding format corresponding to the smaller counting range is smaller than the bit width of the data encoded by the encoding format corresponding to the larger counting range.

[0081] In this embodiment, the bit width of the data encoded by the first encoding format is the first bit width, the bit width of the data encoded by the second encoding format is the second bit width, and the bit width of the data encoded by the third encoding format is the third bit width. The first bit width is smaller than the second bit width, and the second bit width is smaller than the third bit width. For example, the first bit width can be 2 bits, the second bit width can be 8 bits, and the third bit width can be 16 bits.

[0082] In this embodiment, the bit width increases with the counting range. Combined with typical application scenarios (when the hardware utilization rate is high, delay_cycle is mostly in the [0,1] interval), most memory access requests can use the narrowest bit width encoding, without having to use 16-bit wide bit width as in fixed bit width schemes. This can reduce the storage occupation of a single performance data. At the same time, multiple narrow bit width performance data can be closely arranged in the data packet, increasing the data packet capacity, reducing the occupancy of the internal cache unit of the encoding module, and further reducing storage overhead, avoiding storage waste caused by bit width redundancy.

[0083] In some embodiments, the encoding module 242 is further configured to: detect whether the target memory access request is valid and obtain a detection result; and determine the target encoding format and target performance data based on the detection result.

[0084] If the target memory access request is detected as invalid and cannot be executed, the range of the target clock cycle number delay_cycle will no longer be checked, and the third encoding format will be used directly.

[0085] Correspondingly, when the target clock cycle number is encoded using the third encoding format, the format identifier "110" is followed by a 1-bit error flag and 12-bit valid bits.

[0086] If an invalid target memory access request is detected, the error flag (err_flag) can be set to 1 to indicate that the target memory access request is invalid. If a valid target memory access request is detected, the corresponding target encoding format and the target format flag are determined based on the range of the target clock cycle number delay_cycle. Then, the target clock cycle number delay_cycle is encoded according to the target encoding format to obtain the target encoded data. Finally, the target performance data is generated by combining the target format flag and the target encoded data. If the target memory access request is valid and the target encoding format is the third encoding format, the error flag is set to 0.

[0087] To further clarify the specific implementation process of variable-length encoding, a concrete example is provided: Assuming the target memory access request is a valid request and delay_cycle=0, the encoding module 242 determines to use the first encoding format, with the target format identifier being "0". Encoding delay_cycle=0 yields one valid bit "0". Combined with the 8-bit target memory access request index (assumed to be 0x01), the resulting target performance data is "00000001". 00 (the first 8 bits are the request index, and the last 2 bits are the format identifier + valid bits); if delay_cycle=30, it belongs to the range [2,63], the second encoding format is used, the target format identifier is "10", and 30 is encoded to get 6 valid bits "01111", combined with the 8-bit request index (0x01), the target performance data is "000000011001111"; if delay_cycle=100, it belongs to the range ≥64, the third encoding format is used, the target format identifier is "110", and 100 is encoded to get 13 valid bits "000000110100", combined with the 8-bit request index (0x01), the target performance data is "00000001". 110000110100”; If the current memory access request is detected as invalid, regardless of the delay_cycle value, the third encoding format is used, err_flag is set to 1, and combined with the 8-bit request index (0x01), the target performance data is “00000001 11010000000001” (err_flag is set to 1, and the highest bit of the corresponding valid bits is 1).

[0088] When parsing the target performance data, it can be parsed sequentially from the most significant bit to the least significant bit. First, parse the first 8 bits of the target performance data to obtain the target memory access request index and determine the target memory access request. Then, sequentially check the value of the next bit and determine the target encoding format based on the bit combination: if the next bit is "0", it is determined to be the first encoding format, and the next 1 significant bit is parsed to obtain the specific value of the target clock cycle number delay_cycle; if the next two bits are "10", it is determined to be the second encoding format, and the next 6 significant bits are parsed to obtain the specific value of the target clock cycle number delay_cycle; if the next three bits are "110", it is determined to be the third encoding format. Check if err_flag is 1. If it is 1, the target memory access request is considered invalid; if it is 0, the target memory access request is considered valid. Continue parsing the next 12 significant bits to obtain the specific value of the target clock cycle number delay_cycle (if the parsing result is greater than 4095, it is uniformly treated as 4095).

[0089] In this embodiment, the encoding module first checks the validity of the target memory access request. For invalid target memory access requests, the encoding format is no longer selected based on the numerical range of the target clock cycle number. Instead, the third encoding format is fixed and the error flag is set to 1. This clearly distinguishes the performance data of invalid requests from those of valid requests, avoids the mixing of erroneous performance data of invalid memory access requests with valid data, provides an accurate and reliable data foundation for subsequent performance analysis, and prevents developers from making misjudgments due to invalid data.

[0090] In some embodiments, the performance counting circuit 240 may write the target performance data into memory for storage once the target performance data is obtained.

[0091] This embodiment enables persistent storage and centralized management of performance data, preventing data loss due to power outages, resets, or circuit resetting, thus ensuring the integrity and availability of performance data. Simultaneously, storing performance data in memory facilitates subsequent reading, parsing, and analysis, eliminating the need for on-chip temporary caches, reducing cache resource consumption, and improving the flexibility of data access.

[0092] In some embodiments, such as Figure 5 As shown, the performance counting circuit 240 also includes a storage module 243. The encoding module 242 is further configured to write the target performance data into the data packet buffer in the storage module 243 for caching. The storage module 243 is configured to: when the amount of data cached in the data packet buffer reaches a set value, package the data in the data packet buffer to obtain a target data packet of a set bit width; write the target data packet into memory and clear the target data packet buffer.

[0093] The setting value can be pre-defined based on the application scenario or specific requirements, and the setting bit width matches the setting value. For example, the setting value can be 512 bytes, and the setting bit width can also be 512 bytes.

[0094] In this embodiment, the storage module 243 has a pre-set data packet buffer area, which is used to specifically cache the target performance data sent by the encoding module 242. The core function of the storage module 243 is to continuously cache the target performance data and continuously package data packets. Whenever a complete 512-byte target data packet is packaged, the sending process is immediately triggered, and the data packet buffer area is cleared.

[0095] Storage module 243 receives the target performance data written by encoding module 242 and writes the received target performance data sequentially into the data packet buffer according to a tightly packed, non-redundant arrangement rule, thus completing temporary caching. During the caching process, storage module 243 will count the total amount of data already cached in the data packet buffer in real time to ensure that the performance data is not lost or disordered.

[0096] When the storage module 243 detects that the amount of data cached in the data packet buffer has reached the set value, it indicates that the data in the data packet buffer has met the packaging conditions. At this time, the storage module 243 immediately triggers the data packet packaging mechanism to package all data (including all target performance data and tags) in the data packet buffer. According to the preset data packet setting bit width, a complete target data packet is generated to ensure that the bit width of the target data packet meets the setting requirements and that the data in the data packet is arranged in a standardized and complete manner, consistent with the writing rules of the encoding module.

[0097] After the target data packet is packaged, the storage module 243 writes the target data packet into the preset memory area in one go through the communication interface with the memory, thus completing the batch transmission of performance data. After the data packet is written, the storage module 243 immediately clears all data in the data packet buffer area, releases the buffer space, and prepares for receiving new target performance data written by the encoding module 242, ensuring the continuity and stability of the buffer operation.

[0098] It should be noted that if the storage module detects that the amount of data already cached in the data packet buffer has not reached the set value, it can continue to maintain the cached state and not perform the packing, writing, and clearing operations until the amount of cached data reaches the set value before triggering the corresponding process.

[0099] After writing the target data packet into memory, the storage module 243 immediately clears the data packet cache area, which can release the cache space in time, avoid cache overflow, ensure that subsequent performance data can be continuously and stably written to the cache, ensure the continuity of the caching, packaging and writing process, and avoid data loss or processing interruption due to insufficient cache space.

[0100] In this embodiment, the encoding module writes the target performance data into the data packet buffer of the storage module for dedicated caching. This enables centralized management of performance data, avoiding data loss and corruption caused by scattered storage. The tightly packed writing method maximizes cache space utilization, reduces storage redundancy, and lays an efficient data foundation for subsequent packaging operations. The storage module employs a mechanism that packages data only after the data volume reaches a set value, enabling batch packaging and transmission of performance data. This replaces the mode of transmitting individual data entries, significantly reducing data transmission frequency, bandwidth usage, and transmission latency. Especially in scenarios with high concurrency of memory access requests, it effectively avoids bandwidth bottlenecks affecting data transmission efficiency. The data packet design with a set bit width precisely adapts to memory storage specifications and transmission requirements, avoiding transmission failures or resource waste caused by data packet bit width mismatches, ensuring smooth data transmission.

[0101] In some embodiments, the target memory access request is the first memory access request that matches the data packet buffer received by the request receiving circuit 210 after the data packet buffer has been cleared. The encoding module 242 is also configured to: generate a data packet tag, write the data packet tag into the data packet buffer, and reset the clock cycle counter to zero; wherein the data packet tag includes at least a timestamp of the time when the target memory access request was received.

[0102] In an embodiment where the target memory access request is the first memory access request received by the request receiving circuit after the data packet buffer has been cleared and matches the data packet buffer, the encoding module 242 generates a data packet tag when the request receiving circuit 210 receives the target memory access request. The data packet tag includes a timestamp of the received target memory access request, which ensures that the data packet tag accurately corresponds to the performance data of subsequent caches.

[0103] Based on this, if the target performance data is obtained, the target performance data can be cached and arranged after the packet label in the packet buffer area.

[0104] Furthermore, based on the timestamp recorded in the data packet tag and the target clock cycle number represented by the target performance data, the actual execution completion time of the target memory access request can be determined.

[0105] In one example, the timestamp could be 32 bits.

[0106] This embodiment limits the triggering time of packet tag generation and clock cycle counter reset to the first target memory access request after the packet buffer is cleared. This allows for precise binding of packet tags to each batch of cached data, ensuring that each batch of packaged target packets has a unique starting identifier. This completely eliminates the confusion between performance data from different batches, guarantees the independence of performance data from each batch, and provides a clear basis for subsequent batch-by-batch parsing and comparison of memory access performance data at different time periods, avoiding misjudgments in performance analysis due to batch confusion. Moreover, the timestamp contained in the packet tag not only accurately marks the start reception time of each batch of memory access requests but also links with the target clock cycle number represented by the target performance data, accurately calculating the true execution completion time of the target memory access request. This provides more comprehensive and accurate data support for refined analysis of memory access performance and timing optimization. In addition, resetting the clock cycle counter to zero at the beginning of each batch completely eliminates the interference of the previous batch's count residue on the current batch's target clock cycle number statistics, ensuring the accuracy of the cycle number statistics corresponding to each batch of memory access requests. This, in turn, guarantees the accuracy of the encoded target performance data, providing a data foundation for memory access performance bottleneck location and program optimization.

[0107] In some embodiments, the data packet label further includes at least one of the following: a data packet sequence number, a cache index for identifying the cache, a computing unit index for identifying the computing unit, and a memory access type identifier for identifying the memory access type corresponding to the associated performance data.

[0108] The cache index is used to identify cache 112, facilitating the subsequent identification of access hotspots and performance bottlenecks. This cache index can correspond to the actual hardware structure of cache 112 and can be dynamically configured according to the capacity and partitioning of cache 112. The cache index can be 5 bits.

[0109] The compute unit index identifies the compute unit monitored by the performance counting circuit of cache 112. It distinguishes different sources of memory access requests, facilitating independent performance statistics and analysis for memory access requests from different sources, and adapting to the needs of multi-client memory access scenarios. This compute unit index can be 8 bits.

[0110] The memory access type identifier is used to identify the memory access type corresponding to the memory access request of the subsequent associated performance data. In this embodiment, read and write types can be counted using independent performance counting circuits. Therefore, the memory access type identifier in the data packet generated by the performance counting circuit corresponding to the read type is different from the memory access type identifier in the data packet generated by the performance counting circuit corresponding to the write type, achieving separate statistics for the performance data of read and write requests, thus facilitating the differentiation of performance differences between different types of memory access requests. For example, when the memory access type identifier is 1, it indicates that all memory access requests corresponding to performance data in this data packet belong to the write type; when the memory access type identifier is 0, it indicates that all memory access requests corresponding to performance data in this data packet belong to the read type.

[0111] The data packet sequence number is used to identify the order of data packets output by the performance counting circuit. In this embodiment, read and write types can use independent performance counting circuits for performance counting, and their respective data packet sequence numbers increment independently. That is, the data packet sequence number corresponding to the performance counting circuit for the read type starts from 0, and increments by 1 for each data packet generated. The data packet sequence number corresponding to the performance counting circuit for the write type also starts from 0, and increments by 1 for each data packet generated. The data packet sequence numbers corresponding to different types of performance counting circuits increment independently without interference, ensuring clear management of the sequence numbers of data packets for read and write types and avoiding performance data tracing errors caused by sequence number confusion.

[0112] Specifically, the cache index can clearly identify the cache level and location corresponding to performance data, making it easier to locate cache access conflicts and latency sources; the computing unit index can distinguish the memory access characteristics of different processing units, supporting performance difference analysis in multi-core scenarios; the data packet sequence number can achieve orderly management and timing alignment of data packets, avoiding out-of-order transmission and loss of multiple packets; and the memory access type identifier can distinguish the performance of different operations such as read and write, enabling targeted optimization.

[0113] In this embodiment, the data packet tag further includes at least one of the following: cache index, computing unit index, data packet sequence number, and memory access type identifier. This enriches the dimensional information of the performance data and enables refined tracing and accurate analysis of memory access behavior.

[0114] In some embodiments, the encoding module writes the target data packet into memory by deriving a write request and sending it to the request receiving circuit 210. The request receiving circuit 210 then sends the write request to the request execution circuit 220, which executes the write request and writes the target data packet into memory.

[0115] In some embodiments, cache 112 further includes a judgment circuit 270 and a performance counting circuit that corresponds one-to-one with various memory access types.

[0116] The determination circuit is configured to detect the memory access type of the target memory access request and send the target memory access request index to the performance counting circuit corresponding to the memory access type of the target memory access request.

[0117] Furthermore, the performance counting circuit corresponding to the memory access type of the target memory access request is configured to: obtain the target memory access request index, determine the target clock cycle number between the target response data sent by the request return circuit 230 and the previous response data corresponding to the memory access type of the target memory access request, generate target performance data of the target memory access request based on the memory access request index and the target clock cycle number, and write the target performance data into memory.

[0118] Specifically, when the memory access type of the target memory access request is read, the target memory access request index can be obtained by the performance counting circuit corresponding to the read type, the target clock cycle number between the target response data sent by the request return circuit 230 and the previous response data corresponding to the read type can be determined, the target performance data of the target memory access request can be generated according to the memory access request index of the target memory access request and the target clock cycle number, and the target performance data can be written into memory.

[0119] When the memory access type of the target memory access request is write type, the target memory access request index can be obtained by the performance counting circuit corresponding to the write type, the target clock cycle number between the target response data sent by the request return circuit 230 and the previous response data corresponding to the write type can be determined, the target performance data of the target memory access request can be generated according to the memory access request index of the target memory access request and the target clock cycle number, and the target performance data can be written into memory.

[0120] In this embodiment, the memory access type of the memory access request is the memory access type corresponding to the response data of the memory access request.

[0121] In this embodiment, the judgment circuit can detect the memory access type of the target memory access request in real time and accurately distribute the target memory access request to the corresponding performance counting circuit for processing. This achieves automatic memory access type identification and performance data classification and collection, enabling independent analysis and comparison of the execution time of memory access requests of different types. This facilitates accurate identification of whether the bottleneck is read or write, providing a more granular and targeted basis for cache architecture optimization, instruction scheduling optimization, and bandwidth allocation strategies. Moreover, the independent counting of multiple channels does not interfere with each other, maintaining statistical stability in high-concurrency scenarios, improving the robustness and scalability of system performance monitoring, and allowing for direct expansion of the corresponding counting circuit when adding new memory access types without affecting the original logic.

[0122] In some embodiments, the cache 112 is a last-level cache of the computing device with a large memory access volume. For example, the cache 112 may be the last-level cache of the computing device.

[0123] <Second Embodiment> This disclosure also provides a computing device. Figure 6 This illustrates one hardware architecture of the computing device. The computing device can be any device capable of parallel computing, such as a GPU, GPGPU, NPU, or TPU.

[0124] like Figure 6 As shown, the computing device 110 includes multiple computing units 111, a cache 112, and an on-chip bus 113. Each computing unit includes a Level 1 cache, which is coupled to the on-chip bus 113, and the on-chip bus 113 is also coupled to a cache memory access request control unit. The computing units 111 can access the cache 112 via the on-chip bus.

[0125] In some examples, the first-level cache is configured to: in response to the target memory access request derived by the corresponding computing unit based on the target memory access instruction, detect whether the target memory access request hits the first-level cache, allocate a corresponding target memory access request index for the target memory access request; if the target memory access request does not hit the first-level cache, carry the target memory access request index into the target memory access request, and send the target memory access request carrying the target memory access request index to the cache.

[0126] In some examples, cache 112 can be a cache according to the first embodiment described above.

[0127] In some examples, this cache is the last-level cache of the computing device with a large number of memory accesses.

[0128] <Third Embodiment> This disclosure also provides a computing system, such as Figure 7 As shown, the computing system 100 includes a control device 120 and a computing device 110. The computing device 110 may include a cache according to the third embodiment, or it may be a computing device according to the fourth embodiment, and it may also be configured to execute the memory access request control method according to the first embodiment. The control device 120 is used to control the computing device 110 to perform computing tasks, and the control device 120 may be, for example, a CPU.

[0129] This computing device can be deployed in any type of electronic device, such as any type of terminal device, workstation, or server, without any limitation.

[0130] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0131] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and apparatuses according to various embodiments of this specification. In this regard, each block in a flowchart or block diagram may represent a module, unit, or part of a circuit. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented in hardware that performs the specified function or action, or in a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are equivalent.

[0132] Various embodiments of this specification have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A cache, characterized in that, include: A request receiving circuit is configured to receive a target memory access request sent by a computing unit, the target memory access request including a target memory access request index; The request execution circuit is configured to execute the target memory access request and obtain the target response data for the target memory access request. The request return circuit is configured to send the target response data to the computing unit; A performance counting circuit includes a clock cycle counter and an encoding module, wherein the clock cycle counter is configured to count clock cycles; The encoding module is configured to obtain the target memory access request index, detect whether the request return circuit sends response data in each clock cycle, and if the response data sent by the request return circuit is the target response data, determine the count of the clock cycle counter as the target clock cycle number between the time interval ... and the time interval between the time interval between the time interval between the time interval and the time interval between the time interval between the time interval, generate target performance data to characterize the execution time of the target memory access request based on the memory access request index of the target memory access request and the encoded target clock cycle number.

2. The cache according to claim 1, characterized in that, The encoding module is configured to: determine the target encoding format corresponding to the target clock cycle number and the target format identifier corresponding to the target encoding format according to the numerical range of the target clock cycle number; and encode the target clock cycle number according to the target encoding format to obtain target encoded data. The target performance data is obtained based on the target format identifier and the target encoding data.

3. The cache according to claim 2, characterized in that, The bit width of data encoded by the encoding format corresponding to a smaller numerical range is smaller than the bit width of data encoded by the encoding format corresponding to a larger numerical range.

4. The cache according to claim 2, characterized in that, The encoding module is further configured to: detect whether the target memory access request is valid and obtain a detection result; and determine the target encoding format and the target performance data based on the detection result.

5. The cache according to claim 1, characterized in that, The performance counting circuit also includes a storage module; The encoding module is also configured to: write the target performance data into the data packet buffer area in the storage module for caching; The storage module is configured to: when the amount of data cached in the data packet buffer reaches a set value, package the data in the data packet buffer to obtain a target data packet with a set bit width; write the target data packet into memory and clear the target data packet buffer.

6. The cache according to claim 5, characterized in that, The target memory access request is the first memory access request that matches the data packet buffer received by the request receiving circuit after the target data packet buffer is cleared; The encoding module is further configured to: generate a data packet tag, write the data packet tag into the data packet buffer, and reset the clock cycle counter to zero; wherein the data packet tag includes at least a timestamp of receiving the target memory access request.

7. The cache according to claim 6, characterized in that, The data packet label further includes at least one of the following: a data packet sequence number, used to identify the cache index of the cache, used to identify the computing unit index of the computing unit, and a memory access type identifier used to identify the memory access type corresponding to the associated performance data.

8. The cache according to claim 1, characterized in that, The cache also includes a judgment circuit and a performance counting circuit that corresponds one-to-one with various memory access types; The determination circuit is configured to detect the memory access type of the target memory access request and send the target memory access request index to the performance counting circuit corresponding to the memory access type of the target memory access request.

9. A computing device, characterized in that, include: Multiple computing units, each including a first-level cache; The cache is the cache according to any one of claims 1 to 8; as well as, An on-chip bus is provided, wherein the level 1 cache is coupled to the on-chip bus, and the on-chip bus is also coupled to the cache.

10. The computing device according to claim 9, characterized in that, The first-level cache is configured as follows: In response to the target memory access request derived by the corresponding computing unit based on the target memory access instruction, it is detected whether the target memory access request hits the first-level cache, and a corresponding target memory access request index is allocated for the target memory access request; If the target memory access request does not hit the first-level cache, the target memory access request index is carried into the target memory access request, and the target memory access request carrying the target memory access request index is sent to the cache.

11. A computing system, characterized in that, include: The control device and the computing device according to claim 9 or 10, wherein the control device is used to control the computing device to perform computing tasks.

Images