A method and system for bypassing access to optimize cache latency

Abstract

The application discloses a bypass access method and system for optimizing cache delay, comprising a core queue, a shared cache and an order-keeping queue, the core queue sends a request to the order-keeping queue through an order-keeping channel, and the order-keeping queue sends the received request to the shared cache; the order-keeping queue comprises creating logic and selection and sending logic, and the core queue comprises an advance channel and the order-keeping channel; the application further comprises the advance channel; the core queue sends a request to the shared cache through the advance channel, thereby realizing advance access; the advance channel is used for sending to the shared cache, so that parallelism is realized in the mode of advance access of the shared cache by a request, then correlation matching is realized by separating control and associating with the ID of a read-write request, and the request without correlation after matching can directly realize advance access of the shared cache, thereby reducing the access delay of the shared cache.

Description

Technical Field

[0001] This invention relates to a bypass access method and system for optimizing cache latency. Background Technology

[0002] A shared cache is a cache structure shared by multiple cores to reduce access latency. Read and write requests from each core must first reach a pre-ordered queue before being added to the shared cache. Before being added to the pre-ordered queue, requests in the core queue are sent to the creation logic. The creation logic checks if there are resources available to add the request to the pre-ordered queue. If the request can be added, the creation logic checks if the address of the new read / write request matches that of an existing request in the pre-ordered queue. If they match, the request is marked with the relevant entry in the pre-ordered queue when it is added. After all entries are processed, the request is sent to the shared cache via a selection and sending logic. If there is no match, the request is sent directly to the shared cache after being added to the pre-ordered queue. Existing access logic is as follows: Figure 1 As shown.

[0003] In actual operation, there are more cases where there is no match. In this case, the new read and write requests are not related and do not need any relatedness marking. However, since the existing access logic is that the ordered queue and the shared cache are accessed serially, even if there is no match, the read and write requests still need to be created in the ordered queue first, and then go through the selection and sending logic before they can reach the shared cache. This will cause additional latency. If the creation logic checks whether the request can be created in the ordered queue and finds that the resources are full and does not create the request in the ordered queue in time, the time when the request accesses the cache will be delayed indefinitely. Summary of the Invention

[0004] The purpose of this invention is to provide a bypass access system and method for optimizing cache latency in order to address the shortcomings of existing technologies. By parallelizing the serial access logic that requires requests from the core queue to pass through the order-keeping queue before reaching the shared cache, the latency of core queue requests accessing the shared cache can be reduced.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A bypass access method for optimizing cache latency includes the following steps:

[0007] Step 1. Create a lead-ahead channel and a keep-order channel. The core queue sends requests to the shared cache through the lead-ahead channel, and the core queue sends requests to the keep-order queue through the keep-order channel.

[0008] Step 2. Requests sent by the core queue through the order-keeping channel arrive at the creation logic before reaching the order-keeping queue. If the creation logic detects that there are resources available, it can then create the request into the order-keeping queue.

[0009] Step 3. The creation logic checks the address correlation between the request sent by the current core queue and the existing request in the ordering queue, and creates the request into the ordering queue, recording the correlation in the newly created ordering queue entry; when the creation logic successfully creates the request into the ordering queue, it broadcasts a set of creation information to each pipeline stage of the shared cache;

[0010] Step 4. After receiving the creation information broadcast by the creation logic, each pipeline stage of the shared cache performs matching. If a pipeline stage matches the ID of the request in its own pipeline, and if the valid signal in the order-keeping queue is cleared, the created item number is forwarded to the pipeline stage, and this information is passed down to subsequent pipeline stages. At the same time, the pipeline stage sends a signal to the selection sending logic to cancel the access request for that item. When the shared cache is in the last pipeline stage, it sends the cache information of the ahead-accessed item to the entry of the corresponding item number in the order-keeping queue.

[0011] By adopting the above technical solution, requests issued by the core queue can directly reach the shared cache through the advance channel. After receiving the creation information broadcast by the creation logic, it is determined whether there is any relevance. If there is no relevance, the request can directly achieve advance access in the shared cache, avoiding the additional access delay caused by the original serial access logic.

[0012] Furthermore, if the creation logic in Step 2 fails to detect any idle resources, the request will be added to the ordering queue. The request from the core queue will not receive a broadcast message when it reaches the last pipeline stage of the shared cache. In this case, the request from the core queue that reaches the shared cache through the advance channel will be invalidated.

[0013] Furthermore, if the valid signal for matching the order-keeping queue is set in Step 4, the request in the current pipeline level will be cleared, and the advance access will be invalidated. After the correlation in the order-keeping queue is removed, the order-keeping queue will send a request to the shared cache through the selection and sending logic.

[0014] A bypass access system that optimizes cache latency, including

[0015] Core queue;

[0016] Shared cache;

[0017] The core queue sends requests to the order-preserving queue through the order-preserving channel, and the order-preserving queue sends the received requests to the shared cache.

[0018] Its features are:

[0019] It also includes a lead channel; the core queue sends requests to the shared cache through the lead channel to achieve lead access.

[0020] Furthermore, there is creation logic between the core queue and the order-keeping queue; the creation logic is used to detect whether there are resources to create a request into the order-keeping queue and whether the address of the new request is related to the address of the existing request in the order-keeping queue; the creation logic is connected to the shared cache data.

[0021] Furthermore, if the creation logic does not detect any available resources, it will request creation into the ordering queue. This will cause the request from the core queue to reach the last pipeline stage of the shared cache without receiving the broadcast information. In this case, the request from the core queue that reaches the shared cache through the advance channel will be invalidated.

[0022] Furthermore, if the creation logic detects that there are idle resources, it will request to create them in the ordering queue. When creating the request in the ordering queue, the creation logic will broadcast a set of creation information to each pipeline stage of the shared cache.

[0023] Furthermore, the creation information includes:

[0024] A valid signal has been created.

[0025] The project number created;

[0026] Create the ID for the request;

[0027] Match the valid signal of the order-preserving queue.

[0028] Furthermore, after receiving the creation information broadcast by the creation logic, each pipeline stage of the shared cache performs a match. If a pipeline stage matches the ID of the request in its own pipeline, and if the valid signal in the order-keeping queue is cleared, the created item number is forwarded to the pipeline stage, and this information is passed down to subsequent pipeline stages. At the same time, the pipeline stage sends a signal to the selection sending logic to cancel the access request for that item. When the shared cache reaches the last pipeline stage, it sends the cache information of the ahead-accessed item to the entry with the corresponding item number in the order-keeping queue.

[0029] Furthermore, a selection and sending logic is provided between the ordered queue and the shared cache. This logic is used to select requests that are confirmed to be irrelevant in the ordered queue and send them to the shared cache, thereby enabling data transmission between the shared cache and the ordered queue.

[0030] Furthermore, after each pipeline level in the shared cache receives the creation information broadcast by the creation logic, it performs matching. If a pipeline level matches the ID of the request in its own pipeline, and if the valid signal of the ordering queue is set, the request in the current pipeline level is cleared, and the advance access is invalidated. After the correlation in the ordering queue is removed, the ordering queue will send a request to the shared cache through the selection and sending logic.

[0031] Furthermore, arbitration is required between the advanced channel and the order-maintaining channel, with the order-maintaining channel having higher priority than the advanced channel.

[0032] The present invention, by adopting the above-described technical solution, has the following beneficial effects:

[0033] 1. This invention can parallelize the serial logic that requests from the core queue must first pass through the order-keeping queue before reaching the shared cache, allowing unrelated requests to directly achieve advance access, thereby reducing the latency of shared cache access.

[0034] 2. In this invention, requests sent by the core queue through the advance channel will be sent to the shared cache regardless of whether the order-keeping queue has resources available for creation. As long as the request from the core queue is still in the pipeline of the shared cache when the order-keeping queue is successfully created, the request can be directly determined in the shared cache whether to access it in advance. This overlaps some of the waiting time for resources, further increasing parallelism.

[0035] 3. In this invention, if the core queue request matching through the advance channel finds no correlation, advance access to the shared cache can be achieved. At the same time, the pipeline sends a signal to the selection and sending logic to cancel the access request for that item. When the shared cache is in the last stage of the pipeline, it sends the cache information of the advance access to the order-keeping queue of the corresponding item number, thereby preventing the sending of duplicate requests and the return of incorrect results. Attached image description:

[0036] The present invention will be further described below with reference to the accompanying drawings:

[0037] Figure 1 A flowchart illustrating the existing access logic;

[0038] Figure 2 This is a flowchart illustrating all scenarios of the kernel queue request in this invention;

[0039] Figure 3 This is a schematic diagram illustrating the process by which the kernel queue request can access the shared cache ahead of time, as per the present invention.

[0040] Figure 4 This is a schematic diagram of the process of requesting related kernel queues to follow the order-preserving channel in this invention;

[0041] Figure 5 This is a schematic diagram illustrating the process by which the order-preserving queue of this invention was never successfully created. Detailed Implementation

[0042] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0043] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0044] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0045] Example 1

[0046] A bypass access system for optimizing cache latency includes a core queue, a shared cache, and a pre-order queue;

[0047] There is creation logic between the core queue and the order-preserving queue;

[0048] There is a selection and sending logic between the ordered queue and the shared buffer;

[0049] The core queue sends requests to the order-keeping queue through the order-keeping channel, and the order-keeping queue sends the received requests to the shared cache;

[0050] The core queue can send requests to the pre-order queue through the pre-order channel and to the shared cache through the lead-out channel;

[0051] In most cases, the core queue will send requests to both the advance channel and the order-keeping channel simultaneously.

[0052] In some cases, a request is accepted by the shared cache first. If the resources in the ordering queue are busy, it will be recorded that the request has been sent to the shared cache through the advance channel to avoid sending the request repeatedly.

[0053] Therefore, the transmission times of the advance channel and the order-keeping channel can be different and decoupled in essence;

[0054] Requests sent from the core queue do not go through the pre-order channel first and then the lead-up channel; because once a request in the pre-order channel can be set in the pre-order queue, the request in the shared cache will be popped, and the head of the lead-up channel will become the next request.

[0055] By joining the advanced channel, the original... Figure 1 The sequential access logic of the queue and shared cache shown is changed to parallel logic that includes look-ahead access, which can reduce the latency of accessing the shared cache.

[0056] The creation logic is used to check if there are resources in the ordered queue, and requests can be created into the ordered queue.

[0057] The creation logic is also used to detect whether the address of a new request is related to that of an existing request in the ordering queue;

[0058] The existing requests in the order-keeping queue refer to requests sent by multiple cores to the order-keeping queue. The first request in the core queue has been created and entered into the order-keeping queue, which is the existing request in the order-keeping queue. The creation logic needs to check whether the addresses of subsequent requests are related to the addresses of requests that have already been created and entered into the order-keeping queue.

[0059] Requests sent to the pre-order queue via the pre-order channel will reach the creation logic first;

[0060] If the creation logic detects that there is an idle resource, it will request to create it into the ordering queue. When creating the request into the ordering queue, the creation logic will broadcast a set of creation information to each pipeline stage of the shared cache.

[0061] Creation information includes:

[0062] A valid signal has been created.

[0063] The project number created;

[0064] Create the ID for the request;

[0065] Match the valid signal of the order-preserving queue.

[0066] The select-to-send logic is used to select one request from all entries in the order-keeping queue that are ready to access the shared cache and send it to the shared cache.

[0067] After receiving the creation information from the creation logic broadcast, the shared cache performs a matching process. If a pipeline level matches the ID of the request in that pipeline level, and the valid signal in the ordering queue is cleared, it means that the request is irrelevant. The created item number is forwarded to the pipeline level, and this information is passed down to subsequent pipeline levels. At the same time, the pipeline level sends a signal to the selection sending logic to cancel the access request for that item. When the shared cache reaches the last pipeline level, it sends the cache information of the ahead-accessed item to the entry with the corresponding item number in the ordering queue.

[0068] If the valid signal for matching the ordered queue is set, it indicates that the request is related. The shared cache needs to clear the current pipeline request, and this advance access is invalidated. This request needs to follow the traditional access logic. After the relevance of the request in the ordered queue is removed, the selection and sending logic will be used to send a request to the shared cache.

[0069] If a request in the core queue has completed the last pipeline in the shared cache and the creation logic has not detected any resources in the ordering queue that the request can be created in the ordering queue, then the request will not receive broadcast information in the last pipeline in the shared cache and therefore cannot obtain the item number. In this case, the request that entered the shared cache through the advance channel in the core queue will be invalidated.

[0070] Arbitration is required between the advance channel and the order-keeping channel. The order-keeping channel has higher priority than the advance channel because requests sent from the core queue do not go through the order-keeping channel first and then the advance channel. After the order-keeping channel is successfully sent, the head of the advance channel queue becomes the next request. Therefore, for performance reasons, requests in the order-keeping channel that come first need to be served first. This ensures that earlier requests are responded to first, which is in line with the expected return order of the program.

[0071] A bypass access method for optimizing cache latency includes the following steps:

[0072] Step 1. Create a lead-ahead channel and a keep-order channel. The core queue sends requests to the shared cache through the lead-ahead channel, and the core queue sends requests to the keep-order queue through the keep-order channel.

[0073] Step 2. Requests sent by the core queue through the order-keeping channel arrive at the creation logic before reaching the order-keeping queue. The creation logic detects that there are resources available and can create the request into the order-keeping queue.

[0074] Step 3. The creation logic checks the address correlation between the request sent by the current core queue and the existing request in the ordering queue, and creates the request into the ordering queue, recording the correlation in the newly created ordering queue entry; when the creation logic successfully creates the request into the ordering queue, it broadcasts a set of creation information to each pipeline stage of the shared cache;

[0075] Step 4. After receiving the creation information broadcast by the creation logic, each pipeline stage of the shared cache performs matching. If a pipeline stage matches the ID of the request in its own pipeline, and if the valid signal in the order-keeping queue is cleared, the created item number is forwarded to the pipeline stage, and this information is passed down to subsequent pipeline stages. At the same time, the pipeline stage sends a signal to the selection sending logic to cancel the access request for that item. When the shared cache is in the last pipeline stage, it sends the cache information of the ahead-accessed item to the entry of the corresponding item number in the order-keeping queue.

[0076] By employing the above steps, unrelated core queue requests can achieve prioritized access, thereby optimizing access latency. The above steps are as follows: Figure 3 As shown.

[0077] After a core queue request arrives at the shared cache through the advance channel, the following scenarios also apply:

[0078] (1) If the creation logic consistently fails to detect any free resources in the pre-ordered queue, and requests from the core queue do not receive broadcast information when they reach the last pipeline stage of the shared cache, then requests from the core queue that arrive at the shared cache via the lead-ahead channel are invalidated. Figure 5 As shown.

[0079] (2) In Step 4, each pipeline stage of the shared cache receives the creation information broadcast by the creation logic and performs matching. If a pipeline stage matches the ID of the request in its own pipeline, and if the valid signal of the ordering queue is set, indicating that the request is related, the request in the current pipeline stage is cleared, and the current advance access is invalidated. After the relatedness in the ordering queue is resolved, the ordering queue initiates a request to the shared cache through the selection and sending logic, such as... Figure 4 As shown.

[0080] This bypass access method establishes a lookahead channel between the core queue and the shared cache, changing the original serial access logic into a parallel access logic. This allows unrelated requests to directly access the lookahead channel, reducing the waiting time for creating the order-keeping queue and sending the selection logic, thus optimizing the shared cache access latency. At the same time, requests in the core queue are accessed in the shared cache. Since lookahead access has replaced the traditional access logic, the shared cache sends a cancellation signal to the selection logic and sends the lookahead access cache information to the corresponding item number in the order-keeping queue at the last stage of the pipeline, avoiding sending duplicate requests.

[0081] When there are no resources available to create a pre-order queue for an extended period of time, the request flow in the shared cache will not receive the creation information broadcast by the creation logic until the last level, and thus will be unable to obtain the item number. The advance access will be invalidated because if the request occupies the pipeline level of the shared cache without being released, subsequent requests will also be blocked. If this request depends on the completion of subsequent requests that jump out of the pre-order queue, a deadlock will occur. By invalidating the request, the deadlock situation can be avoided.

[0082] Once the creation logic successfully creates the request and puts it into the ordering queue, it broadcasts the creation information to every pipeline level of the shared cache. The shared cache synchronizes by matching the request ID. If there is a correlation, the advance access in the shared cache will be invalidated and the request will wait for access in the ordering queue. By matching correlations, the occurrence of incorrect results can be avoided.

[0083] The above are merely specific embodiments of the present invention, but the technical features of the present invention are not limited thereto. Any simple changes, equivalent substitutions, or modifications made based on the present invention to achieve substantially the same technical effect are all covered within the protection scope of the present invention.

Claims

1. A bypass access method for optimizing cache latency, comprising the following steps: Step 1. Create a lead-ahead channel and a keep-order channel. The core queue sends requests to the shared cache through the lead-ahead channel, and the core queue sends requests to the keep-order queue through the keep-order channel. Step 2. Requests sent by the core queue through the pre-order channel arrive at the creation logic before reaching the pre-order queue. If the creation logic detects that there is a resource, it can create the request into the pre-order queue. If the creation logic does not detect any free resources, it can create the request into the pre-order queue. Requests from the core queue will not receive broadcast information when they are transmitted to the last pipeline stage of the shared cache. In this case, requests from the core queue that arrive at the shared cache through the advance channel are invalidated. Step 3. The creation logic checks the address correlation between the request sent by the current core queue and the existing request in the ordering queue, and creates the request into the ordering queue, recording the correlation in the newly created ordering queue entry; when the creation logic successfully creates the request into the ordering queue, it broadcasts a set of creation information to each pipeline stage of the shared cache; Step 4. After receiving the creation information broadcast by the creation logic, each pipeline stage of the shared cache performs a match. If a pipeline stage matches the ID of the request in its own pipeline, and the valid signal in the order-keeping queue is cleared, the created item number is forwarded to the pipeline stage, and this information is passed down to subsequent pipeline stages. At the same time, the pipeline stage sends a signal to the selection sending logic to cancel the access request for that item. When the shared cache is in the last pipeline stage, it sends the cache information of the ahead-accessed item to the entry of the corresponding item number in the order-keeping queue.

2. The bypass access method for optimizing cache latency according to claim 1, characterized in that: If the valid signal for matching the ordering queue is set in Step 4, the request in the current pipeline level will be cleared, and the advance access will be invalidated. After the correlation in the ordering queue is removed, the ordering queue will send a request to the shared cache through the selection and sending logic.

3. A bypass access system for optimizing cache latency, comprising a core queue and a shared cache; The system includes a pre-order queue, whereby the core queue sends requests to the pre-order queue via a pre-order channel, and the pre-order queue sends the received requests to a shared cache. The system is characterized by: further including a look-ahead channel; the core queue sends requests to the shared cache via the look-ahead channel to achieve look-ahead access; a creation logic is provided between the core queue and the pre-order queue; the creation logic is used to detect whether there are resources available to create a request into the pre-order queue and whether the address of the new request is related to the address of an existing request in the pre-order queue; the creation logic is data-connected to the shared cache; if the creation logic does not detect any available resources to create a request into the pre-order queue, resulting in the core queue's request still not receiving broadcast information even after reaching the last pipeline stage of the shared cache, then the request arriving at the shared cache via the look-ahead channel in the core queue is invalidated; if the creation logic detects available resources to create the request into the pre-order queue, the creation logic... When the logic creates the request into the ordering queue, it broadcasts a set of creation information to each pipeline stage of the shared cache. The creation information includes: a creation success signal, the created item number, the creation request ID, and a matching ordering queue validity signal. After receiving the creation information broadcast by the creation logic, each pipeline stage of the shared cache performs a matching. If a pipeline stage matches the ID of the request in its own pipeline, and finds that the matching ordering queue validity signal is zero, it forwards the created item number to the pipeline stage and passes this information down to subsequent pipeline stages. At the same time, the pipeline stage sends a signal to the selection sending logic to cancel the access request for that item. When the shared cache reaches the last pipeline stage, it sends the cache information of the ahead-accessed item to the entry with the corresponding item number in the ordering queue.

4. The bypass access system for optimizing cache latency according to claim 3, characterized in that: The ordered queue and the shared cache are provided with a selection and sending logic. The selection and sending logic is used to select requests that are confirmed to be irrelevant in the ordered queue and send them to the shared cache; thereby realizing the data transmission between the shared cache and the ordered queue.

5. The bypass access system for optimizing cache latency according to claim 4, characterized in that: After receiving the creation information broadcast by the creation logic, each pipeline stage of the shared cache performs matching. If a pipeline stage matches the ID of the request in its own pipeline stage, and if the valid signal of the ordering queue is set, the request of the current pipeline stage is cleared, and the advance access is invalidated. After the correlation in the ordering queue is removed, the ordering queue will send a request to the shared cache through the selection and sending logic.

6. The bypass access system for optimizing cache latency according to claim 3, characterized in that: The two paths, the advance channel and the order-preserving channel, need to be arbitrated, with the order-preserving channel having a higher priority than the advance channel.

Images