Using Predictions of Outcomes of Cache Memory Access Requests for Contolling Whether A Request Generator Sends Memory Access Requests To A Memory In Parallel With Cache Memory Access Requests

Abstract

An electronic device handles memory access requests for data in a memory. The electronic device includes a memory controller for the memory, a last-level cache memory, a request generator, and a predictor. The predictor determines a likelihood that a cache memory access request for data at a given address will hit in the last-level cache memory. Based on the likelihood, the predictor determines: whether a memory access request is to be sent by the request generator to the memory controller for the data in parallel with the cache memory access request being resolved in the last-level cache memory, and, when the memory access request is to be sent, a type of memory access request that is to be sent. When the memory access request is to be sent, the predictor causes the request generator to send a memory request of the type to the memory controller.

Description

GOVERNMENT RIGHTS[0001]This invention was made with government support under the PathForward Project with Lawrence Livermore National Security (prime contract no. DE-AC52-07NA27344, subcontract no. B620717) awarded by the DOE. The government has certain rights in this invention.BACKGROUNDRelated Art[0002]Some electronic devices include processors that execute program code for performing various operations. For example, the electronic devices may include one or more central processing unit (CPU) cores or graphics processing unit (GPU) cores that execute program code for software applications, operating systems, etc. Many of these electronic devices, along with a memory (e.g., a “main” memory) and a mass-storage device, include one or more cache memories for storing data (“data,” as used herein, is a generic term that is intended to cover program code, data, and other information and values). For example, some electronic devices include a hierarchy of cache memories in which each of t...

AI Technical Summary

Benefits of technology

This approach reduces unnecessary traffic and memory access operations, improves data access speed, and maintains low last-level cache miss recovery times, enhancing overall device performance and user satisfaction.

Problems solved by technology

Although accessing copies of data that are present in cache memories is relatively fast, processors are still exposed to the delay of acquiring data from memory when such data is not already present in cache memories.

Although the speculative data fetching technique can be helpful for speeding up data acquisition when cache memory access requests miss in the cache memory, the technique has detriments.

For example, when the cache memory access request hits—and therefore can be satisfied using data that was already present in the cache memory—the speculative memory access request causes unnecessary traffic on a fabric in the electronic device (i.e., a memory bus or other communication route), causes the memory controller to perform unneeded memory access operations, etc.

Images