DATA AS COMPUTE

The DaC cache addresses the memory wall issue by representing data as a function in a computing system, reducing memory accesses and improving performance for large datasets in HPC and ML applications.

JP2024543766APending Publication Date: 2024-11-26ADVANCED MICRO DEVICES INC
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2024515311
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-11-30
Filing Date
2022-08-22
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The increasing demand for memory in computing systems to accommodate large datasets for high-performance computing (HPC) and machine learning (ML) applications has led to higher latency and lower bandwidth in hierarchical memory systems, with cache compression techniques failing to effectively handle large reuse distances, exacerbating the memory wall issue.

Method used

A data-on-compute (DaC) cache system represents data as a function using curve-fitting techniques, allowing data elements to be reconstructed from a local cache memory instead of accessing remote or high-latency memory, reducing memory accesses and improving performance by caching large datasets with varying reuse distances.

Benefits of technology

The DaC cache reduces memory bandwidth consumption and access latency by reconstructing data elements from local cache memory, enhancing performance and accommodating large datasets with varying reuse distances, suitable for ML training and remote memory access scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024543766000001_ABST
    Figure 2024543766000001_ABST
Patent Text Reader

Abstract

The method includes storing a function representative of a set of data elements stored in a backing memory, and in response to a first memory read request for a first data element of the set of data elements, calculating a function result representative of the first data element based on the function.
Need to check novelty before this filing date? Find Prior Art

Description

[Background technology]

[0001] Modern computing systems have one or more central processing units (CPUs) that typically include and rely on multiple caches in a cache hierarchy to improve processor and memory performance. Caches are smaller, faster memory devices that store data that is frequently accessed or expected to be used in the near future so that the data can be accessed with low latency compared to main memory. Such cache devices are often implemented in static random access memory (SRAM) and typically sit between the processing units that issue memory requests and the main memory devices.

[0002] Memory bandwidth is a critical resource in modern systems (e.g., CPUs, Graphics Processing Units (GPUs), or heterogeneous systems) and significantly impacts the performance of many High Performance Computing (HPC) and Machine Learning (ML) applications.

[0003] The present disclosure is illustrated by way of example, and not by way of limitation, in the accompanying figures. [Brief description of the drawings]

[0004] [Figure 1] FIG. 1 illustrates an embodiment of a computing device including a Data as Compute (DaC) cache. [Diagram 2] FIG. 2 illustrates a processor core and a cache device in a computing device, according to one embodiment. [Diagram 3] FIG. 2 illustrates one embodiment of a DaC cache. [Figure 4]FIG. 2 illustrates a data element and a miss vector according to one embodiment. [Diagram 5] FIG. 2 illustrates a curve fitting process via Hough transform according to one embodiment. [Figure 6] FIG. 2 illustrates an accumulator matrix for determining the correlation degree of multiple candidate functions according to one embodiment. [Figure 7] FIG. 2 illustrates a DaC cache representing data with different error tolerances according to one embodiment. [Figure 8] FIG. 2 illustrates a process for caching data in a DaC cache according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] In the following description, numerous specific details are set forth, such as examples of specific systems, components, methods, etc., to enhance understanding of the embodiments. However, it will be apparent to one of ordinary skill in the art that at least some of the embodiments may be practiced without these specific details. In other instances, well-known components or methods are not described in detail or are presented in simplified block diagram form to avoid unnecessarily obscuring the embodiments. Thus, the specific details described are merely exemplary. Particular embodiments may differ from these example details and still be considered to be within the scope of the embodiments.

[0006] The increasing demand for memory in computing systems to accommodate large data sets for HPC and ML applications has led to the emergence of hierarchical memory systems such as Fabric-Attached Memory (FAM). Hierarchical memory structures can increase capacity in computing systems, but hierarchical memory accesses incur high latency, low bandwidth, and increased energy consumption. These issues can usually be addressed by exploiting locality through some form of caching. However, the benefits provided by data caching depend on the reuse distance and size of the working set. While many applications can benefit from data caching, data caching does not work as well for large data sets with large reuse distances (e.g., loading an ML training dataset over an epoch). Cache compression techniques can be used to increase the effective capacity of the cache and thus reduce the number of memory accesses. However, the benefits of cache compression may not be fully realized if the expanded capacity cannot accommodate the large reuse distances. As the disparity between processor speed and memory speed continues to grow, the resulting memory wall remains a major obstacle to improving performance.

[0007] One embodiment of the computing system filters the number of memory accesses sent to a remote or other high latency memory location by representing the data as a function. A function f(x) that can represent the data is derived using curve fitting techniques, so that the particular data element being requested can be obtained by calculating f(x) with appropriate inputs, such as the array index or address offset of the data element. Thus, memory accesses to obtain the data are replaced with f(x) calculations, for which parameters representing f(x) are stored and read from a faster local cache memory. This reduces the number of memory accesses to the remote or high latency memory, resulting in lower bandwidth consumption, access latency (e.g., by eliminating remote memory accesses), and higher performance.

[0008] It is often unlikely that a function can be derived that can fit all data points within a given memory range. Thus, in one embodiment, f(x) is used to provide data in a manner similar to a cache, and an access is a hit if f(x) can reconstruct the requested data element exactly (i.e., within a given error tolerance) and a miss otherwise. With this approach, the number of hits is not sensitive to the size or reuse distance of the working set, but instead depends on the stored data elements. This makes it suitable for caching large datasets (e.g., datasets for ML training over epochs, graph datasets for traversal, datasets stored in remote memories such as FAM, etc.). A Data-at-Compute (DaC) cache implementing this approach can be placed in the memory hierarchy of a computing system to filter and reduce memory accesses sent to remote or high-latency memories. In various embodiments, the DaC cache may be located at the last level cache in the memory hierarchy to filter memory accesses directed to main memory, or at the local Dynamic Random Access Memory (DRAM) level to filter accesses directed to remote memory or storage.

[0009] FIG. 1 illustrates one embodiment of a computing device 100 including a DaC cache. Generally, computing device 100 may represent any of a number of different types of devices, including, but not limited to, a laptop or desktop computer, a mobile phone, a server, and the like. Computing device 100 includes a number of components 102-108 that can communicate with each other via an interconnect 101. In computing device 100, each of components 102-108 can communicate with any of the other components 102-108 directly via interconnect 101 or via one or more of the other components 102-108. Components 101-108 in computing device 100 are contained within a single physical casing, such as the chassis of a laptop or desktop computer or the casing of a mobile phone. In an alternative embodiment, some of the components of computing device 100 are embodied as peripheral devices such that the entire computing device 100 is not present within a single physical casing.

[0010] Computing device 100 also includes user interface devices for receiving information from a user or providing information to a user. Specifically, computing device 100 includes input devices 102, such as a keyboard, mouse, touch screen, or other device for receiving information from a user. Computing device 100 displays information to the user via display 105, such as a monitor, a Light-Emitting Diode (LED) display, a Liquid Crystal Display, or other output device.

[0011] Computing device 100 further includes a network adapter 107 for transmitting and receiving data over a wired or wireless network. Computing device 100 also includes one or more peripheral devices 108. Peripheral devices 108 may include mass storage devices, location sensing devices, sensors, input devices, or other types of devices that may be used by computing device 100.

[0012] Computing device 100 includes one or more processing units 104 that receive and execute instructions 106a stored in main memory 106. As referred to herein, each of processing units 104 represents a processor "pipeline" and may include a central processing unit (CPU) pipeline, a graphics processing unit (GPU) pipeline, or other computing engines that support memory operations using virtual addresses. Main memory 106 is part of a memory subsystem of computing device 100 that includes memory devices used by computing device 100, such as random access memory (RAM) modules, read-only memory (ROM) modules, hard disks, and other non-transitory computer-readable storage media.

[0013] In addition to main memory 106, the memory subsystem may also include cache memory, such as a level 2 (L2) or level 3 (L3) cache, and / or registers. Such cache memory and registers may reside within processing unit 104 or on other components of computing device 100.

[0014] FIG. 2 illustrates components within a processing unit 104, according to one embodiment. The processing unit 104 includes four processor cores 210A-210D and a cache hierarchy subsystem including L1 caches 215A-215D, L2 caches 220A-220D, and an L3 cache 230 shared between the processor cores 210A-210D. The cache subsystem includes high-speed cache memories (e.g., L1 caches 215A-215D, L2 caches 220A-220D, and L3 cache 230) that store blocks of data (not shown). As used herein, a "block" is a set of bytes stored in contiguous memory locations in a cache memory and is treated as a unit for coherency purposes. Additionally, each of the terms "cache block," "block," "cache line," and "line" are interchangeable herein. In some embodiments, a block may also be a unit of cache allocation and deallocation. The block may be retrieved from other caches within computing system 100, such as from other processing units or from main memory.

[0015] L3 cache 230 is shared by each of processor cores 210A-210D such that data used in the execution of a thread in any of cores 210A-210D may be cached in L3 cache 230. L3 cache 230 is coupled to interconnect 101 such that L3 cache may read from and write to main memory subsystem 106 and other devices (e.g., 102, 105, 107, 108) in the computing system. In other embodiments, processing unit 104 may include fewer or more processor cores (e.g., processor core 210A) and corresponding L2 caches (e.g., L2 cache 220A). In still other embodiments, processing unit 104 may include other types of addressable cache subsystems that may include different numbers, types, and levels of cache memory.

[0016] Each of the cache devices 215A-215D, 220A-220D, and 230 includes a cache controller that intercepts read and write requests from the processor cores 210A-210D that are directed to the main memory 106. Each cache controller then copies the data being accessed to its respective cache. When the cache controller adds new data to an already full cache, the cache controller selects victim lines to remove from the cache according to a replacement policy, which is a set of rules for identifying the least valuable data to keep in the cache (e.g., data that is least likely to be accessed again in the near future). In one embodiment, the cache controller of the shared cache monitors several metrics for each user of the shared cache, identifies similar users based on the monitored metrics, and groups similar users together for the purpose of applying different policies to each group. In the processing unit 104, this functionality is implemented in the shared L3 cache controller 231 of the shared L3 cache 230.

[0017] 3 illustrates one embodiment of a memory hierarchy including a data-at-computation (DaC) cache 320 in a computing system 100. The processor core 210A is coupled to a memory hierarchy including a cache hierarchy 300 and a backing memory 330. As illustrated, the DaC cache 320 is disposed as a last level cache in the cache hierarchy 300 between a higher level cache 310 (e.g., L1 cache 215, L2 cache 220) and a backing memory 330, which represents one or more regions in the main memory 106, a remote memory in another computing node, and / or other memory accessible to the computing system 100 with higher latency and / or lower bandwidth than any of the cache memories in the cache hierarchy 300. Thus, the higher level cache 310 is disposed between the processor core 210A and the DaC cache 320 in the memory hierarchy. In alternative embodiments, the processor core 210A may be another type of processing unit (e.g., a CPU, a GPU, or other processing unit or processing element).

[0018] When executing instructions for an application, the processor core 210A accesses data elements stored in the backing memory 330 by issuing memory read and write requests to the memory hierarchy. A data element is information that can be stored in memory, such as an integer value, a floating point value, a data structure, etc. The data element requested by the memory read request is returned to the processor core 210A via a response issued by a device in the memory hierarchy. As shown in FIG. 3, the processor core 210A requests any of a set of data elements stored in the backing memory 330 by issuing a request 341 to the memory hierarchy. If the requested data element is in the upper level cache 310, a response 342 is provided from the cache 310. However, if the data element misses in the upper level cache 310, the upper level cache 310 requests 351 the value from the DaC cache 320.

[0019] The DaC cache 320 represents a set of data elements in the backing memory 330 as a function. The function is recorded as a type of function (e.g., the degree of an nth degree polynomial function) and one or more function parameters (e.g., coefficients of an nth degree polynomial function), which are stored in a cache memory 323 in the DaC cache 320. For example, the function may be a quadratic curve with three parameters (i.e., three coefficients of a quadratic curve) recorded in the memory 323, or the function may be a 14-parameter Neural Network (NN)-based curve with 14 parameters recorded in the memory 323.

[0020] In addition, the metadata stored in the cache memory 323 associates a function with a memory address range in the backing memory 330 (e.g., indicating a start address and an end address) that contains a set of data elements to be reconstructed by the function, and a set of miss indexes associated with the function that indicate which of the data elements in the address range can be accurately (within a predetermined tolerance) reconstructed using the function. The set of miss indexes includes an index that identifies each data element in the associated memory address range that is not reconstructed by the function because the function result is not within the predetermined tolerance of the data element. Alternatively, a set of hit indexes can be used that stores the indexes of data elements that can be reconstructed (within a predetermined tolerance) using the function.

[0021] If one miss index value corresponds to one data element, storing the miss index directly can scale with the data size. In an alternative embodiment, the miss index is maintained as a bit vector (e.g., a 128-byte miss vector for a window size of 1024 values) or a Bloom filter with conservative hit prediction to limit storage overhead. In an alternative embodiment, some of the metadata is embedded in other memories in the system, such as the Translation Lookaside Buffer (TLB), etc. For example, since all memory instructions reference the TLB, the miss vector embedded in the TLB easily enables curve fitting as soon as the address translation is done (e.g., after a miss in the L1 cache).

[0022] In one embodiment, the DaC cache 320 tracks multiple memory regions by their address ranges and associates a function, a miss index (or miss vector) and metadata with each tracked memory region. In one embodiment, polynomial curve fitting is used to accurately fit a quadratic curve to the data set. Since fitting a single quadratic curve across the entire large target memory region may result in a reduced percentage of hits across the region, the large region is divided into smaller sub-regions with one or more curves fitted to each. Thus, the entire large region is represented using multiple functions (e.g., up to five quadratic curves across different sub-regions of the large region). The sub-region size (window size) may affect the accuracy of the fit. A smaller window size is likely to result in a more accurate fit, but will result in a higher parameter storage overhead across the entire target region as more functions and their parameters are recorded. In one embodiment, each single window or sub-region may have multiple functions fitted across it, with each fitted function associated with its own corresponding miss vector or miss index.

[0023] Depending on the location of the DaC cache 320 in the hierarchy, the request can be made using a virtual or physical address. In the case of virtual addresses, each captured memory region is further annotated with a process identifier to avoid address aliasing problems during context switching.

[0024] The data reconfiguration logic 321 includes logic circuitry for reconfiguring the requested data element based on a function. When the DaC cache 320 receives a request 351 for a data element, the data reconfiguration logic 321 determines whether the requested data element is within an address range associated with any of the tracked memory regions. If so, the data reconfiguration logic 321 also determines whether the miss index of the memory region includes the index associated with the requested data element. If the data element is not within the address range of the tracked memory region or its index is included in the miss index, the request 351 is treated as causing a miss in the DaC cache 320. Thus, a memory request 361 is sent to a lower level in the memory hierarchy (e.g., to the backing memory 330) to fetch the data element.

[0025] If the index corresponding to the data element is not in the miss index, this indicates that the data element can be reconstructed using a function, and the request 351 is treated as a hit. The function associated with the region containing the data element is then used to calculate a value to return to the processor core 210A. The inputs to the function depend on the type of function. For example, the function inputs may be an array index indicating the location of the data element in an array, a memory address offset, a previous or adjacent data element in an array, etc.

[0026] In one embodiment, the function inputs ideally have a strong correlation with their corresponding data elements and are readily available (e.g., available upon request). The offset or array index of a data element is a candidate that is easily derivable from the address of the requested data, but does not always have a strong correlation with the data element. Alternatively, a previous data element can be used as an input (e.g., time series data), which may be highly correlated with the data element, but may not be readily available.

[0027] Data reconfiguration logic 321 includes one or more function units 324 that contain dedicated logic circuitry for computing a function result based on appropriate function inputs. Once the function result is computed, it is returned to upper level cache 310 via response 352 and to processor core 210A via response 342. In an alternative embodiment, the function and its parameters are returned to host processor core 210A, which computes the function result.

[0028] Given an example set of data elements data1={0, 1, 4, 9, 16, . . .}, we have the function f(x)=x 2 is used to represent a value in data1. For a particular data element in data1, the input x is the index of the data element in the data1 array. When processor core 210A requests any of the values ​​in data1, the index is used as an input to calculate the requested value using a function f(x). Given an example set of data elements data2={0, 108, 4, 21, 16, . . .}, the values ​​in data2 can be calculated using the same function f(x)=x 2 where the miss index is {1, 3, ...}. Because index "3" is included in the miss index, a request to access the value data2[3] results in a miss in the DaC cache 320. Therefore, the requested value "21" is retrieved from the backing memory 330 instead of being calculated by the function f(x).

[0029] Reconstructing data in the DaC cache 320 reduces the number of memory requests sent to the remote memory 330. As an example, the processor core 210A requests 4 kilobytes of data to be read from the backing memory 330. The data to be read is checked in the DaC cache 320. If the function representing the requested data in the DaC cache 320 can accurately reconstruct 25% of the requested data according to the miss index or miss vector, the amount of data read from the backing memory 330 is reduced to 3 kilobytes. The remaining 1 kilobyte is calculated in the function unit 324 of the DaC cache 320 using the function. If the input to the function is available at the start of the transaction (e.g., the input is based on an array index), the function result can be calculated in parallel with the remote memory access. Thus, the calculation latency is completely hidden because the function result can be calculated before the remote memory access is completed. The hit latency of the DaC cache 320 is determined by the latency to reconstruct data by calculating the function result, which is shorter than the memory access latency to retrieve data from the backing memory 330. The latency of function computation depends on the complexity of the function derived to represent the data - the simpler the function, the lower the latency - so a function that is simple enough that it can be computed before a remote memory access is completed is selected.

[0030] In one embodiment, the DaC cache 320 includes function deduction logic circuitry 322 that performs curve fitting calculations to determine a function for representing a set of data elements. The function deduction logic 322 determines the function type, parameters that define the function, a miss index, and other metadata stored in memory 323. In one embodiment, the function deduction logic 322 determines the function for representing a set of data elements using a single pass curve fitting technique during initialization of the data elements or when the data elements are first read. The function deduction logic 322 also uses a multi-pass curve fitting technique to deduce the function after repeated reading and loading of the data elements (e.g., multiple epochs during machine learning training). In an alternative embodiment, the function, miss index and metadata are calculated outside the DaC cache 320 (e.g., in software executed by the processor core 210A, in network interface hardware, in a Processing In Memory (PIM) element 331 of the remote memory 330, etc.) and sent to the data reconstruction logic 321 before the data is reconstructed.

[0031] In one embodiment, the miss index is used as a measure of the applicability of a given curve fitting technique to a particular data set. In one such embodiment, multiple machine learning fitting techniques are tried in parallel and the function deduction logic 322 selects the technique that results in the fewest misses to represent the data set in the DaC cache 320.

[0032] Additionally, if the ratio of misses to hits indicated in the miss index exceeds a configurable threshold (e.g., the ratio of misses to hits is greater than 2:1), the function deduction logic 322 does not record a function to represent the data. This can occur if the curve fitting technique used is unable to match the data within a given tolerance (e.g., if the data is random).

[0033] In one embodiment, a curve fitting technique is used to derive a function for representing the sign and exponent values ​​of floating-point data. Since the randomness of the mantissa bits is usually much greater than that of the sign and exponent bits, finding a function for representing the mantissa or reconstructing the data due to the complexity of the function may be computationally expensive. In one embodiment, the mantissa bits are explicitly stored in the DaC cache 320, while the sign and exponent bits are represented by the function f(x). For data types such as 16-bit floating-point numbers (where the sign and exponent bits represent 37.5% of the data value), this embodiment can save memory capacity in the DaC cache 320.

[0034] If the processor core 210A stores or modifies a data element in any of the tracked memory regions, an index for the stored or modified value is added to a list of miss indexes, so that the function is not used to reconstruct the data element in response to future requests. Because the data element has been modified after the curve fitting process was performed, it is unknown whether the function will be able to reconstruct the data element within a given tolerance. Subsequent requests for the data element are serviced from the backing memory 330 or other devices in the memory hierarchy (e.g., other caches). If a data element in a tracked memory region has been significantly modified (e.g., more than a threshold percentage of the data elements in the region have been modified since the curve fitting was performed), the region is removed from the cache. The region is removed from the cache by marking its entry (including the function, miss index, and metadata) as invalid. Alternatively, significant modification of data elements in a region may trigger the curve fitting process to be performed again, or another recalibration process in which the function type and / or parameters are updated to fit the modified data.

[0035] In one embodiment, the functionality of the DaC cache 320 is implemented in software (e.g., as a library) instead of hardware. In a software embodiment, a function that represents a set of data elements is deduced by the processor core 210A during data preparation before the data is sent to the remote backing memory 330 for storage. The function, miss index, and metadata information are stored in the memory hierarchy like other data used by the application.

[0036] For hardware or software implementations, even if the curve fitting process of the DaC cache 320 fits a relatively simple function (e.g., a quadratic polynomial, an n-parameter neural network (NN), etc.) to the data, deriving such a function incurs computational overhead. If the function deduction is performed in hardware (e.g., in a network interface), the curve fitting calculations can be performed as network packets are being generated and processed, without preventing the primary processor core 210A from making progress. Thus, offloading the function deduction and curve fitting calculations to another processing element in the system avoids placing the overhead from the function deduction in the critical path of the application.

[0037] The curve fitting process in different embodiments can be performed either online (while the data is actively used or available for use in computation) or offline (while the data is not being used) depending on the constraints and use case. Workloads that are read-only (e.g., image / video classification or segmentation, graph input for traversal) or mostly read (analysis of periodically updated security data) are fitted to the function offline or during data initialization. Curve fitting for frequently updated data is performed online periodically. The accuracy and computational overhead of the curve fitting depend on several factors, such as the degree of correlation between the function inputs and the data elements, the number of data elements over which the curve is fitted (window size), and the curve fitting technique (polynomial, NN, other ML technique) used to derive the function f(x).

[0038] There are a wide variety of curve fitting techniques (e.g., polynomial, neural network, or other machine learning techniques), each offering a different tradeoff between accuracy and computational overhead. Each technique also has parameters (e.g., number of layers for NN techniques, learning rate, etc.) that can be adjusted to achieve different results. The choice of curve fitting technique also determines the number of passes over the data used to obtain the representative curve. For example, polynomial curve fitting techniques are performed in a single pass, while NN techniques are performed in multiple passes.

[0039] In one embodiment, the curve fitting is performed using a polynomial curve fitting technique based on the Hough transform, a feature extraction technique used in image analysis, computer vision and digital image processing to fit functions over large data sets using array index based inputs. This technique allows for accurate and approximate curve fitting to data sets (treated as large arrays) with low overhead.

[0040] As mentioned above, the load addresses or array indices of the data elements are readily available when required, thus allowing the computation of f(x) to begin as soon as the load instruction is issued to the core. However, the load addresses or array indices do not always have a sufficiently high correlation with their corresponding data values. In one embodiment, the rank() function is used to achieve a better match with the data elements using inputs derived from the array indices. For a given index idx, the function input is equal to rank(miss_vector, idx)-1. In the miss vector, a bit set to 1 indicates that the corresponding data element can be reconstructed exactly within a given error tolerance using the function, while a bit set to 0 indicates that the data element cannot be reconstructed exactly within the error tolerance. The rank function returns the number of elements in the miss vector that are equal to "1" from position 0 to position idx.

[0041] 4 illustrates an array of data elements 401 and a miss vector 402 associated with the data elements 401, according to one embodiment. The data element array 401 includes elements {1, 21, 19, 2, 3, 41, 4, 11, 5, 6} and is associated with a miss vector={1, 0, 0, 1, 1, 0, 1, 0, 1, 1}. For values ​​of index idx for which the miss vector 402 indicates a hit, the value of the data element data[idx]=f(k idx )=k idx +1, where k idx =rank(miss_vector,idx)-1.

[0042] FIG. 5 illustrates the use of the Hough transform to determine a line defined by the function y=mx+b that passes through two points (x1,y1) and (x2,y2). 2,y2) are shown in Cartesian space 501 and transformed into parameter space 502 with axes m and b to define a linear curve y=mx+b. Each point (x,y) in Cartesian space 501 is transformed into a line in parameter space 502. Thus, points (x1,y1) and (x 2, The points (x1, y1) and (x2, y2) are transformed into lines b=-x1m+y1 and b=-x2m+y2, respectively, in parameter space 502. The points where these lines intersect provide the parameters of the line that passes through the points (x1, y1) and (x2, y2) in Cartesian space.

[0043] The Hough transform fitting process is performed by scanning over a finite range of values ​​of m and b to determine whether a given point (x,y) lies on the line y=mx+b, and then using a voting technique to find the values ​​of m and b that define the line that passes through the maximum number of points. Pseudocode for this process is shown below in Table 1. [Table 1]

[0044] In Table 1, Points is the set of all points (x,y) to be fitted, M is the set of all m values ​​to be tested, and B is the set of all b values ​​to be tested. The accumulator matrix Accum[m][b] is used to determine the degree of correlation between each candidate function (represented by a unique combination of parameters m and b) and the data elements. The element in the accumulator matrix Accum[m][b] with the highest count represents the line that passes through the most points in Points and is therefore most correlated with the set of data elements. For example, assuming M={0, 1, 2, 3}, B={0, 1, 2, 3}, and Points=(0, 3), (1, 3), (2, 5), (3, 7), (4, 7), (5, 11), FIG. 6 shows the accumulator matrix Accum[m][b] after the voting portion of the Hough transform process has been performed, according to one embodiment. Different combinations of m and b, represented by Accum[m][b], represent a set of candidate functions from which the best-fit candidate function is selected to represent the data. As shown in FIG. 6, element Accum[2][1] has the highest value of "4", indicating that the line y=2x+1 intersects with most points in Points. This means that given an input value idx, the line y=2x+1 produces a function result equal to the data element corresponding to idx in more instances than the other candidate functions. Therefore, the line y=2x+1 is selected as the best-fit function to represent the data element in the DaC cache 320.

[0045] Table 2 shows pseudocode for a modified Hough transform fitting process that fits a curve across N data points, according to one embodiment. [Table 2] In Table 2, the pseudocode iterates (per line 1) through each of the N data points, and for each data point (with index i and value y), it iterates through each value of m and b in M ​​and B, respectively, where M and B are the set of possible m and b values ​​being evaluated. Accum[m][b] is the accumulator matrix. Each element of the Accum[m][b] matrix is ​​a vector that holds the array index of the value being fitted. LastIndex is an array that contains the current size of each vector in Accum[m][b]. Err is a user-determined error tolerance (as a fraction) for the curve fit. A candidate function is considered to be able to reconstruct the data if the function result differs from the data element by less than this predetermined tolerance, as shown in the sixth row of Table 2.

[0046] After the curve fitting process is complete, the entry in Accum[m][b] that holds the longest array indicates the curve that covers the most points. Thus, the modified Hough transform fitting process identifies a linear sequence of not necessarily adjacent data points while preserving their relative order. The curve that covers the most points is selected as the function that is most highly correlated with the data set and represents the data elements in the DaC cache 320.

[0047] As an example, given points (0,1), (1,21), (2,19), (3,2), (4,3), (5,41), (6,4), (7,11), (8,5), and (9,6), the entry in Accum[1][1] (corresponding to f(x)=x+1) fits to point (0,1) with input of x=0 (corresponding to j in the pseudocode of Table 2). In addition, this line fx=x+1 also fits to point (3,2) with x=1, point (4,3) with x=2, point (6,4) with x=3, point (8,5) with x=4, and point (9,6) with x=5. The miss vector is generated by asserting, for each data element, a corresponding miss bit value if the corresponding function result differs from the data element by less than a predetermined tolerance. Thus, the miss vector, miss_vector, is {1, 0, 0, 1, 1, 0, 1, 0, 1, 1} for the above dataset, with "1" values ​​corresponding to points that are on the line fx=x+1 (within the tolerance defined by Err) and "0" values ​​corresponding to points that are not on the line. Each of the entries for x listed above is equal to rank(miss_vector, idx)-1. The top K accumulator entries in Accum[m][b] are selected to obtain the top K curves that fit the largest number of points within the specified tolerance.

[0048] A key step in the Hough transform curve fitting process is determining the sets M and B to evaluate. Larger sets allow for more accurate curve fitting, but with increased computational overhead. In one embodiment, the adaptive Hough transform starts with a coarser range of values ​​of m and recursively refines the possible values ​​of m.

[0049] The previous example shows the use of the Hough transform to fit a curve to integer data, which allows m and b to be integers, thus reducing the range of possible values ​​to be evaluated. For floating-point data, the floating-point value is divided into a mantissa and an exponent value. Two functions, one each of the mantissa and exponent components, are fitted to each floating-point value. The actual coverage is the intersection of both functions. That is, a floating-point data element is generated by the DaC technique if both the mantissa and the exponent can be calculated from the respective functions. Thus, a miss index or miss vector indicates a hit for a floating-point data element if both the mantissa and the exponent values ​​of the floating-point data element can be calculated by their respective functions within their respective error tolerances.

[0050] FIG. 7 illustrates an embodiment of the DaC cache 320 that uses multiple functions 710-712 with different tolerances to represent a set of data elements. Some applications tolerate errors in their working data sets. For example, machine learning tasks can tolerate some errors in their training data, with different models tolerating different amounts of error. Thus, an embodiment of the DaC cache 320 fits multiple curves for the same memory region, with different error tolerances for each curve. Thus, the hit rate varies depending on which curve is used to reconstruct the requested data. An approximately fitted curve (i.e., with a higher error tolerance) achieves a higher hit rate than a more accurately fitted curve. Machine learning models and accelerators that tolerate errors can achieve higher memory performance by requesting data elements from the DaC cache 320 using a more approximately fitted curve that has a higher hit rate. A model with a lower tolerance for error can still request the same data with higher precision using a more accurately fitted curve.

[0051] 7, a memory region containing a set of data elements stored in a backing memory 330 is tracked by a DaC cache 320. A memory region is tracked by three functions: f0(x) 710, f5(x) 711, and f 10 The DaC cache 320 also stores a respective miss vector 720-722 for each of the functions 710-712. The three different accelerators 701-703 (or three different machine learning or artificial intelligence models) may tolerate 0%, 5% and 10% error in the data. The DaC 320 caches the data requested by the different accelerators 701-703 using functions f0(x), f5(x) or f(x) that adapt to the error tolerance of the requesting accelerators 701-703. 10 (x). Compared to accelerator 701, accelerator 703 benefits from higher memory performance because DaC 320 trades data precision for a higher hit rate. The original copy of the data in backing memory 330 remains unmodified (i.e., free of lossy compression or other inaccuracies) so that the data set may be used by other devices.

[0052] 8 illustrates a process 800 for caching data in the DaC cache 320, according to one embodiment. Process 800 may be performed by hardware and / or firmware components of computing system 100, by software executing within system 100, or by a combination of hardware and software.

[0053] Process 800 begins at block 801, where a curve fitting process is used to determine a function type and function parameters for representing a set of data elements stored in remote backing memory 330. Blocks 801-809 represent operations performed by function deduction logic 322, or in software executed in processing unit 104, to determine a function (i.e., function type and parameters), a miss vector or miss index, and other metadata for representing the data. In sub-block 803 of block 801, function deduction logic 322 scans through multiple combinations of parameters (e.g., different combinations of m and b parameters of a line function), each of which represents a different candidate function, and for each candidate function, determines a degree of correlation between the function and the data set. For each parameter combination, function deduction logic 322 determines a degree of correlation for the candidate function by determining, for each data element in the data set, whether the candidate function produces a function result that differs from the data element by less than a predetermined error tolerance. In one embodiment, the index of the data element, or a value derived from the index, is used as an input to the candidate function. The more function results that are within the error tolerance from the data element values, the more highly correlated the candidate function is with the data set.

[0054] In sub-block 805, the function deduction logic 322 selects the function with the highest degree of correlation with the dataset to represent the data. In one embodiment, the function is selected in response to determining that the number of function results that are within a predetermined error tolerance of the corresponding data elements in the dataset exceeds a predetermined threshold. Thus, if the function with the highest degree of correlation does not provide a sufficiently high hit rate, the region is not represented by the function. In some embodiments, the function deduction logic 322 can fit the data to different types of functions or use different curve fitting techniques (e.g., polynomial curve fitting, machine learning techniques, etc.) and perform curve fitting in parallel using different techniques and function types to determine the function type and parameters that result in the highest degree of correlation with the dataset values.

[0055] In sub-block 807, a miss vector for the function is generated by asserting a miss bit value in the miss vector for each data element that can be reconstructed by the function within an error tolerance (i.e., the function result corresponding to the data element does not differ from the data element by more than the predetermined error tolerance). Alternatively, an index of a data element is added to a list of miss indexes if its corresponding function result differs from the data element by more than the predetermined error tolerance.

[0056] At block 809, the function deduction logic 322 stores in memory 323 of the DaC cache 320 the selected function, the parameters, the miss vector or miss index, and metadata indicating the memory range of the backing memory 330 that contains the data element. In one embodiment, blocks 801-809 are performed multiple times for each of multiple data sets in different regions of the backing memory 330 and / or for the same data set (e.g., to represent the same data using multiple functions with different error tolerance levels).

[0057] At this point, the DaC cache 320 is ready to process a memory request issued by the processor core 210A directed to the set of data elements. The processor core 210A issues a memory request 341 to the memory hierarchy, which is received as a request 351 in the DaC cache 320. The request 351 is processed according to blocks 811-821, and blocks 811-819 are executed by the data reorganization logic 321 in the DaC cache 320. In block 811, if the memory request 351 is a read request, the process 800 continues to block 813. In block 813, the address of the request is checked against metadata in the DaC cache memory 323 to determine whether the address is within a tracked memory region (i.e., within an address range associated with one or more of the functions). If the requested address is within one of the tracked memory regions, the process 800 continues to block 815.

[0058] In block 815, the miss vector is checked to determine whether a function (or functions) associated with the tracked memory region containing the requested address is used to reconstruct the data element. In the miss vector, a miss bit that has the same array index or memory address offset as the data element is asserted if the data element can be reconstructed by the associated function. If the miss bit corresponding to the data element is asserted, process 800 continues from block 815 to block 817. In embodiments that use a list of miss indexes instead of a miss vector, the index of the requested data element is looked up in the list of miss indexes. If the index is not in the list of miss indexes, the associated function can reconstruct the data element within a predetermined error tolerance, and process 800 continues to block 817.

[0059] In block 817, if multiple functions are used to represent the data with different tolerance levels (e.g., functions 710-712 shown in FIG. 7), one of these functions is selected based on the acceptable tolerance level of the requesting application or device. For example, if a data element is requested by accelerator 702, which can tolerate 5% error, function 711 is selected to reconstruct the data element. If there is only one function, then that function is selected by default.

[0060] In block 819, the data reconfiguration logic 321 uses the selected function to calculate a function result that represents the requested data element. In one embodiment, an index of the data element that identifies the location of the data element in the array is used as an input to the function. Alternatively, another value, such as an address offset of the data element, a previous data element in the same data set or array, or other value, may be used as an input, as appropriate. The function result is then calculated by applying the input value to the input of one or more functional units 324 for performing the function. The latency for calculating the function result is less than the latency for fetching a data element (e.g., any of the data elements in the data set) from the backing memory 330. The calculated function result is returned (as a response 352) to the requesting core 210A via the upper cache hierarchy 310 to fulfill the request 341. From block 819, the process 800 continues to block 823.

[0061] The processor core 210A issues a second memory read request 351, and in block 823, the DaC cache 320 determines whether the request 351 is a write request. Since the request 351 is not a write request, the process 800 returns to block 811. From block 811, the process 800 continues to block 813 because this second request 351 is a read request. If, in block 813, the address of the request 351 is not within the tracked memory region, or, in block 815, the miss index or miss vector indicates that the requested data element is a miss in the DaC cache 320, the process 800 continues to block 821 from block 813 or 815. In block 821, the data element is fetched from the backing memory 330 instead of being reconstructed by the function, and the data value is returned to the requesting processor core 210A. From block 821, the process 800 continues to block 823.

[0062] The process 800 reaches block 823 from block 821 or from block 811 if the received memory request is not a read request. In block 823, if the memory request is a write request directed to a memory location that stores the data element, the process 800 continues to block 825. In block 825, if the memory location of the request is within the tracked memory region, the function deduction logic 322 adds the index of the data element being modified to a list of miss indexes. Alternatively, if a miss vector is being used, the function deduction logic 322 deasserts the miss bit corresponding to the data element in the miss vector. If the memory location that stores the data element being modified is not within the tracked memory region, the function deduction logic 322 does not update the miss index or the miss vector. From block 825, the process 800 continues to block 827.

[0063] If the number of miss indexes in the list of miss indexes (or the number of deasserted miss bits in the miss vector) does not exceed the threshold in block 827, the process 800 returns to block 811 to continue processing memory read requests and memory write requests. If the number of miss indexes or deasserted miss bits exceeds the threshold in block 827, this indicates a high number of misses relative to hits for the function (i.e., a low hit / miss ratio). In this case, the function is deemed not to be effective enough to reconstruct the data elements in its associated memory region. Thus, as provided in block 829, the function's entry in the DaC memory 323 is invalidated. Thus, the association between the function and the set of data elements is removed such that subsequent requests for the data elements cannot be satisfied by using the function to reconstruct the requested data. In one embodiment, the process 800 returns to block 801 to redo the curve fitting process to find a new function to represent the data set in the DaC cache 320.

[0064] The method includes storing a function representative of a set of data elements stored in a backing memory, and in response to a first memory read request for a first data element of the set of data elements, calculating a function result representative of the first data element based on the function.

[0065] The method further includes calculating a function type of the function and one or more parameters of the function by performing a curve fitting process based on the set of data elements.

[0066] The method further includes selecting a function to represent the set of data elements in response to determining that a number of function results of the function that are within a predetermined tolerance of corresponding data elements in the set of data elements exceeds a predetermined threshold.

[0067] The method further includes, for each parameter combination of the plurality of parameter combinations, determining a degree of correlation between a candidate function defined according to the parameter combination and the set of data elements by determining, for each data element within a memory address range that includes the set of data elements, whether the candidate function produces a function result that differs from the data element by less than a predetermined tolerance based on an index associated with the data element, selecting the candidate function having the highest degree of correlation as the function representing the set of data elements, and for each data element in the set of data elements, asserting a miss bit if the function result corresponding to the data element differs from the data element by less than the predetermined tolerance.

[0068] The method further includes storing metadata associating the function with a memory range in the backing memory, the set of data elements being stored in the memory range.

[0069] In the method, computing the function result includes applying an index of the first data element to an input of a functional unit for performing the function, the index corresponding to a position of the first data element in the array.

[0070] In this method, the function is any of a plurality of functions for representing the set of data elements, each of the plurality of functions representing the set of data elements within a different tolerance.

[0071] The method further includes storing a miss vector including a miss bit for each data element in the set of data elements. The computing of the function result is performed in response to determining that the first miss bit corresponding to the first data element is asserted. The method also includes fetching a second data element from the backing memory in response to a second memory read request for a second data element in response to determining that the second miss bit corresponding to the second data element is deasserted. The second data element is stored in a memory location between two data elements of the set of data elements.

[0072] The method further includes deasserting a first miss bit in response to a memory write request directed to a memory location storing the first data element, and removing an association between the function and the set of data elements if a number of deasserted miss bits in the miss vector exceeds a threshold.

[0073] In this method, the latency to compute the function result is less than the latency to fetch any of the set of data elements from the backing memory.

[0074] The computing device includes a first memory for storing a function representative of a set of data elements stored in a second memory, and data reconstruction logic for computing a function result representative of the first data element based on the function in response to a first memory read request for a first data element of the set of data elements.

[0075] The computing device further includes function deduction logic coupled to the first memory for calculating a function type of the function and one or more parameters of the function by performing a curve fitting process based on the set of data elements. The first memory is further configured to store metadata associating the function with a memory range of the second memory. The set of data elements is stored in the memory range.

[0076] The computing device further includes function deduction logic coupled to the first memory to determine, for each parameter combination of the plurality of parameter combinations, a degree of correlation between a candidate function defined according to the parameter combination and the set of data elements by determining, for each data element within a memory address range that includes the set of data elements, whether the candidate function produces a function result that differs from the data element by less than a predetermined tolerance based on an index associated with the data element, selects the candidate function having the highest degree of correlation as a function representing the set of data elements, and asserts a miss bit for each data element in the set of data elements when a function result corresponding to the data element differs from the data element by less than the predetermined tolerance.

[0077] In the computing device, the data restructuring logic further includes one or more functional units for computing a function result based on an index of the first data element, the index corresponding to a position of the first data element in the array.

[0078] In the computing device, the first memory further stores a miss vector including a miss bit for each data element in the set of data elements. The data reconfiguration logic calculates a function result in response to determining that the first miss bit corresponding to the first data element is asserted, and in response to a second memory read request for a second data element, causes a second data element to be fetched from the backing memory in response to determining that the second miss bit corresponding to the second data element is deasserted. The second data element is stored in a memory location between two data elements of the set of data elements.

[0079] In the computing device, the first memory is a cache memory and the second memory is a backing memory that has a higher access latency than the first memory.

[0080] The computing system includes a backing memory for storing a set of data elements, a processing unit coupled to the backing memory for issuing a memory request for a first data element of the set of data elements, and a cache device coupled to the processing unit, the cache device including a cache memory for storing a function representing the set of data elements, and data reconfiguration logic for computing a function result representing the first data element based on the function in response to the memory request.

[0081] In the computing system, the cache device further includes function deduction logic coupled to the cache memory for calculating a function type of the function and one or more parameters of the function by performing a curve fitting process based on the set of data elements. The cache memory further stores metadata that associates the function with a memory range of the backing memory. The set of data elements is stored in the memory range.

[0082] In the computing system, the cache memory further stores a miss vector including a miss bit for each data element in the set of data elements. The data reconfiguration logic calculates a function result in response to determining that the first miss bit corresponding to the first data element is asserted. In response to a second memory read request for the second data element, the cache device causes the second data element to be fetched from the backing memory in response to determining that the second miss bit corresponding to the second data element is deasserted. The second data element is stored in a memory location between two data elements of the set of data elements.

[0083] In a computing system, the cache device is a last-level cache in a memory hierarchy of the computing system that includes one or more higher-level cache devices between the processing unit, the cache device, and a backing memory.

[0084] As used herein, the term "coupled" may mean directly or indirectly coupled through one or more intervening components. Any of the signals provided via the various buses described herein may be time-shared with other signals and provided via one or more common buses. In addition, the interconnections between circuit components or blocks may be depicted as buses or single signal lines. Each of the buses may alternatively be one or more single signal lines, and each of the single signal lines may alternatively be a bus.

[0085] Certain embodiments may be implemented as a computer program product that may include instructions stored on a non-transitory computer-readable storage medium. These instructions may be used to program a general-purpose or special-purpose processor to perform the described operations. A computer-readable storage medium includes any mechanism for storing or transmitting information in a form (e.g., software, processing application) readable by a machine (e.g., a computer). Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage media (e.g., floppy diskettes), optical storage media (e.g., CD-ROMs), magneto-optical storage media, read-only memory (ROM), random access memory (RAM), erasable programmable memory (e.g., EPROM and EEPROM), flash memory, and other types of media suitable for storing electronic instructions.

[0086] Additionally, some embodiments may be practiced in a distributed computing environment where a computer-readable medium is stored on and / or executed by two or more computer systems. Additionally, information transferred between computer systems may be either pulled or pushed through a transmission medium connecting the computer systems.

[0087] In general, the data structures representing computing device 100 and / or portions thereof embodied in a computer-readable storage medium may be databases or other data structures that are readable by a program and may be used directly or indirectly to manufacture hardware including computing device 100. For example, the data structures may be behavioral or register-transfer level (RTL) descriptions of hardware functionality in a high-level design language (HDL) such as Verilog or VHDL. The descriptions may be read by a synthesis tool that may synthesize the descriptions to generate a netlist that includes a list of gates from a synthesis library. The netlist includes a set of gates that also represent the functionality of the hardware including computing device 100. The netlist may then be placed and routed to generate a data set that describes the geometric shapes to be applied to a mask. The mask may then be used in various semiconductor manufacturing processes to manufacture the semiconductor circuit(s) corresponding to computing device 100. Alternatively, the database on the computer readable storage medium may be a netlist (with or without a synthesis library) or a data set, or Graphic Data System (GDS) II data, as desired.

[0088] Although the operations of the methods herein are shown and described in a particular order, the order of operations of each method may be changed such that certain operations may be performed in the reverse order, or such that certain operations may be performed at least in part concurrently with other operations. In alternative embodiments, instructions or sub-operations of individual operations may be performed intermittently and / or alternately.

[0089] In the foregoing specification, the embodiments have been described with reference to certain exemplary embodiments thereof. However, it will be apparent that various modifications and changes can be made thereto without departing from the broader scope of the embodiments as set forth in the appended claims. The specification and drawings are therefore to be regarded in an illustrative rather than a restrictive sense.

Claims

1. 1. A method comprising: storing a function representative of the set of data elements stored in a backing memory; in response to a first memory read request for a first data element of the set of data elements, calculating a function result representative of the first data element based on the function. method.

2. calculating a function type of the function and one or more parameters of the function by performing a curve fitting process based on the set of data elements.

2. The method of claim 1.

3. selecting a function that represents the set of data elements in response to determining that a number of function results of the function that are within a predetermined tolerance of corresponding data elements in the set of data elements exceeds a predetermined threshold.

2. The method of claim 1.

4. determining, for each combination of parameters, a degree of correlation between a candidate function defined according to the combination of parameters and a set of data elements by determining, for each data element within a memory address range that includes the set of data elements, whether the candidate function produces a function result that differs from the data element by less than a predetermined tolerance based on an index associated with the data element; selecting the candidate function having the highest degree of correlation as the function representing the set of data elements; and for each data element in the set of data elements, asserting a miss bit if a function result corresponding to the data element differs from the data element by less than a predetermined tolerance.

2. The method of claim 1.

5. storing metadata associating the function with a memory range of the backing memory; the set of data elements is stored in the memory range; 2. The method of claim 1.

6. Computing the function result includes applying an index of the first data element to an input of a functional unit for performing the function; the index corresponds to the position of the first data element within an array.

2. The method of claim 1.

7. the function is any of a plurality of functions for representing the set of data elements; each of the plurality of functions represents the set of data elements within a different tolerance; 2. The method of claim 1.

8. storing a miss vector including a miss bit for each data element in the set of data elements, wherein the calculating of the function result is performed in response to determining that a first miss bit corresponding to the first data element is asserted; and in response to a second memory read request for a second data element, fetching the second data element from the backing memory in response to determining that a second miss bit corresponding to the second data element is deasserted; the second data element is stored in a memory location between two data elements of the set of data elements.

2. The method of claim 1.

9. deasserting the first miss bit in response to a memory write request directed to a memory location storing the first data element; removing an association between the function and the set of data elements if a number of deasserted miss bits in the miss vector exceeds a threshold.

9. The method of claim 8.

10. a latency for computing the function result is less than a latency for fetching any of the set of data elements from the backing memory; 2. The method of claim 1.

11. 1. A computing device comprising: a first memory configured to store a function representative of a set of data elements stored in a second memory; and data reconfiguration logic configured to, in response to a first memory read request for a first data element of the set of data elements, calculate a function result representing the first data element based on the function. Computing device.

12. and a function deduction logic coupled to the first memory and configured to calculate a function type of the function and one or more parameters of the function by performing a curve fitting process based on the set of data elements; the first memory is configured to store metadata associating the function with a memory range of the second memory; the set of data elements is stored in the memory range; The computing device of claim 11.

13. function deduction logic coupled to the first memory; The functional deductive logic comprises: determining, for each combination of parameters, a degree of correlation between a candidate function defined according to the combination of parameters and a set of data elements by determining, for each data element within a memory address range that includes the set of data elements, whether the candidate function produces a function result that differs from the data element by less than a predetermined tolerance based on an index associated with the data element; selecting the candidate function having the highest degree of correlation as the function representing the set of data elements; for each data element in the set of data elements, asserting a miss bit if a function result corresponding to the data element differs from the data element by less than a predetermined tolerance; 4. The method of claim 3, The computing device of claim 11.

14. the data restructuring logic includes one or more functional units configured to calculate the function result based on an index of the first data element; the index corresponds to the position of the first data element within an array. The computing device of claim 11.

15. the first memory is configured to store a miss vector including a miss bit for each data element in the set of data elements; The data reconstruction logic includes: calculating the function result in response to determining that a first miss bit corresponding to the first data element is asserted; in response to a second memory read request for a second data element, fetching the second data element from the backing memory in response to determining that a second miss bit corresponding to the second data element is deasserted; The device is configured to: the second data element is stored in a memory location between two data elements of the set of data elements. The computing device of claim 11.

16. the first memory is a cache memory; the second memory being a backing memory having a higher access latency than the first memory; The computing device of claim 11.

17. 1. A computing system comprising: a backing memory configured to store a set of data elements; a processing unit coupled to the backing memory and configured to issue a memory request for a first data element of the set of data elements; a cache device coupled to the processing unit; The cache device is a cache memory configured to store a function representative of the set of data elements; and data restructuring logic configured to calculate a function result representing the first data element based on the function in response to the memory request. Computing system.

18. The cache device is function deduction logic coupled to the cache memory and configured to calculate a function type of the function and one or more parameters of the function by performing a curve fitting process based on the set of data elements; the cache memory is configured to store metadata associating the functions with memory ranges of the backing memory; the set of data elements is stored in the memory range; 20. The computing system of claim 17.

19. the cache memory is configured to store a miss vector including a miss bit for each data element in the set of data elements; the data reconstructing logic is configured to calculate the function result in response to determining that a first miss bit corresponding to the first data element is asserted; the cache device is configured to fetch the second data element from the backing memory in response to a second memory read request for the second data element and in response to determining that a second miss bit corresponding to the second data element is deasserted; the second data element is stored in a memory location between two data elements of the set of data elements.

20. The computing system of claim 17.

20. the cache device is a last level cache in a memory hierarchy of the computing system; The memory hierarchy comprises: one or more upper level cache devices between the processing unit and the cache device; and the backing memory.

20. The computing system of claim 17.

Citation Information

Patent Citations

  • Memory integrated circuit

    JP1986296599A

  • Dynamic image transmission device and dynamic image transmission method

    JP1992261585A

  • Chiplet-integrated machine learning accelerator

    JP2022541461A

  • Verifiable Cacheable Calclulations

    US20220365921A1

  • Methods and apparatus for reducing false hits in a non-tagged, n-way cache

    US6470438B1