Master Processor Prefetching for Coprocessor Data Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer processors face delays in accessing data from memory hierarchies, especially when data is shared among multiple processors, leading to increased computation time and the need for large caches, which are costly and inefficient.
Innovation Solution
A system comprising a master processing unit and consumer processing units, where the master processing unit decodes machine instructions, determines data needs, and sends read requests to memory, with data stored in local read storage accessible to consumer processing units, eliminating the need for multiple cache copies and minimizing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in L1 cache to provide faster access, then access speed is improved, but cache size and cost increase
Solution Approach 1:
The master processing logic sends read requests to memory in advance, before the consumer processing logic actually needs the data. This preliminary action allows data to be fetched from memory while the instruction is being decoded and prepared, so that data arrives at the consumer logic ready for immediate use, eliminating the need for large L1 cache to hold data.
2Loss of time
If data is stored in L1 cache for faster access, then computation time is reduced, but hardware cost increases
Solution Approach 1:
Data is fetched from memory in advance during the instruction decode phase, so that when the consumer processing logic needs the data for execution, it is already available in local storage. This eliminates computation time delays without requiring expensive large-capacity L1 cache hardware.
3Loss of time
If speculative pre-fetching is used to bring data closer to the processor, then data access time is reduced, but additional hardware is required
Solution Approach 1:
The master processing logic proactively sends read requests to memory as soon as it decodes an instruction that will need data, rather than waiting for the consumer logic to request it. This timing allows data to arrive just in time for execution without requiring complex speculative pre-fetching hardware or prediction logic.
Solution Approach 2:
The master processing logic acts as an intermediary that coordinates between memory and consumer processing logic. It sends read requests to memory with destination addresses, and data is routed through this coordination mechanism to the appropriate consumer logic, reducing the need for complex distributed cache management hardware.
4Adaptability or versatility
If data is shared by multiple processors, then data reusability is improved, but storing copies in L1 cache becomes inefficient
Solution Approach 1:
Data is extracted from the traditional cache hierarchy and delivered directly to consumer processing logic via targeted read requests from master logic. This removes the need to store multiple copies of shared data in L1 cache across different processors, as each processor receives data directly when needed through the master logic coordination.
Data Source
AI summary
Technology for providing data to a processing unit is disclosed. A computer processor may be divided into a master processing unit and consumer processing units. The master processing unit at least partially decodes a machine instruction and determines whether data is needed to execute the machine instruction. The master processing unit sends a request to memory for the data. The request may indicate that the data is to be sent from the memory to a consumer processing unit. The data sent by the memory in response to the request may be stored in local read storage that is close to the consumer processing unit for fast access. The master processing unit may also provide the machine instruction to the consumer processing unit. The consumer processing unit may access the data from the local read storage and execute the machine instruction based on the accessed data.