87 results about "Memory hierarchy" patented technology

The invention relates to the technical field of data processing, and discloses a multilevel cloud storage cluster construction and data storage method. The method comprises the steps that S1, multiple levels of storage layers are constructed and comprise a cache layer, a performance layer and a capacity layer, and the storage layers are provided with different storage media which are used for storing hot data, warm data and cold data respectively; s2, determining data popularity by monitoring data access frequency, access time interval and data importance; and S3, dynamically adjusting the storage hierarchy of the data according to the data popularity, and triggering hierarchical migration of the data. By constructing a multi-level storage layer and adopting a hierarchical architecture that the cache layer stores high-frequency data, the performance layer stores intermediate-frequency data and the capacity layer stores low-frequency data, high-heat data is preferentially stored in a high-performance medium, access delay is reduced, and data reading and writing efficiency is improved. Meanwhile, in combination with a dynamic data migration strategy, the storage position can be adjusted in real time according to the data access popularity, and the overall performance of the system is further optimized.

The invention discloses a GPU (Graphics Processing Unit) and parallel IO (Input / Output) collaborative optimization method based on mode heterogeneous computing, and particularly relates to the field of heterogeneous computing optimizing.The method comprises the following steps: monitoring mode characteristics during execution of a target computing task in real time through an instruction analyzer, and synchronously collecting dynamic parameters of a heterogeneous computing environment to generate a multi-dimensional resource situation matrix; based on the mode and the resource situation matrix, collaborative optimization operation of the binding equipment dimension and the scheduling time dimension is executed; and constructing a three-level adaptive transmission mechanism and periodically implementing feedback optimization according to an execution state of the three-level adaptive transmission mechanism. According to the GPU and parallel IO collaborative optimization method based on mode heterogeneous computing, the data prefetching hit rate is increased by dynamically sensing the matching relation between a heterogeneous computing mode and an I / O access mode; end-to-end delay is reduced and task contention conflicts are reduced through a dynamic matching strategy of an equipment capability portrait and a task type; and the heterogeneous storage bandwidth peak value utilization rate and the I / O throughput are improved by differentially utilizing the hierarchical characteristics of the multi-stage storage.

The invention discloses a distributed time sequence library data management method supporting cold and hot data level-to-level management, and relates to the technical field of computer databases, comprising: receiving a write-in request of time sequence data, and executing preliminary write-in data aggregation and sorting; dynamically identifying data cold and hot attributes based on a multi-dimensional data cold and hot degree calculation model in combination with a write-in behavior and an access behavior of time series data; according to the cold and hot attribute recognition result, in combination with the hierarchy boundary of self-adaptive division, automatic hierarchical storage of the time series data is executed; based on a cold and hot data dynamic migration and scheduling mechanism, according to the access frequency and time decay characteristics of the time series data, dynamically adjusting the storage hierarchy of the time series data, and executing a data migration task; and constructing a hierarchical index system adaptive to different cold and hot attribute time series data, and combining query frequency dynamic identification and index structure automatic upgrading. By adopting a cold and hot data automatic identification and hierarchical storage mechanism, the overall performance and the resource utilization rate of the system are improved.

The technical scheme can be applied to the field of financial science and technology / medical health. The invention discloses a mercuric chloride chip-based large model reasoning operator optimization method, device and equipment, and the method comprises the steps: carrying out the blocking processing of original input data according to the parallel calculation capability of a mercuric chloride chip, and generating the blocking data of an adaptive chip calculation unit; optimizing a memory access path of a matrix multiplication operator by combining a memory hierarchical structure and a calculation core type of a mercuric chloride chip based on the block data, and adjusting a sliding step length and a filling mode of convolution operation; a plurality of operators continuously executed in the large model are fused into a composite operator, the data storage process of the composite operator is optimized, and collaborative execution is achieved by dynamically allocating computing resources; and integrating and decoding block calculation results after collaborative optimization, and adjusting an input data block strategy and operator execution parameters through a verification feedback mechanism to form closed-loop optimization. According to the technical scheme, the reasoning efficiency of a large model can be improved.

A cache writeback circuit is disclosed for writing back, without invalidating, dirty cache lines. The cache writeback circuit is configured to enter an active state based on detecting a trigger condition indicative of cache misses to a memory cache circuit within a memory hierarchy of a computer memory subsystem causing cache line eviction activity. During the active state, the cache writeback circuit is configured to identify a set of dirty cache lines in the memory cache circuit, and write back, without invalidating, cache lines of the identified set of dirty cache lines from the memory cache circuit to a memory circuit within a lower level of the memory hierarchy, such as DRAM. The cache writeback circuit may further be configured to identify the dirty cache lines via a cache walk operation, which can be suspended, for example, when a higher-priority cache operation occurs.

An apparatus has cache circuitry providing a cache storage to store data for access by processing circuitry, and request handling circuitry arranged to process requests, each request providing an address indication for associated data. The request handling circuitry determines with reference to the address indication whether the associated data is available in the cache circuitry. The cache circuitry forms a given level of a multi-level memory hierarchy, and the request handling circuitry is responsive to determining that the associated data is unavailable in the cache circuitry to issue an onward request to cause the associated data to be retrieved into the cache circuitry from a lower level of the multi-level memory hierarchy than the given level. Prefetch circuitry issues, as one type of request to be handled by the request handling circuitry, prefetch requests, and the request handling circuitry is arranged in response to a given prefetch request to retrieve into the cache circuitry the associated data in anticipation of that associated data being requested by the processing circuitry. In addition, trigger circuitry, responsive to a specified condition being detected in respect of the given prefetch request, issues a prefetch trigger signal for receipt by control circuitry associated with further cache circuitry at a higher level of the multi-level memory hierarchy, to cause a higher level prefetch procedure to be triggered by the control circuitry to retrieve the associated data from the cache circuitry into the further cache circuitry.

The invention discloses a two-dimensional code data security detection method and system based on a block chain, relates to the technical field of data security detection, and is used for solving the problems that in the prior art, no fine dynamic management of storage hierarchy and access frequency exists, data compliance and integrity verification generally depends on a single-node or centralized mode, and data security is influenced. The dynamic adaptive capacity to data access frequency and risk is lacked, and anomaly detection and response mechanisms of most systems are passive; through a block chain consensus mechanism and Hash verification, the non-tampering and consistency of data are ensured, the layered storage architecture and the intelligent contract are dynamically adjusted, and the real-time detection and automatic response ability to potential threats is improved by using the combination of an isolated forest algorithm and the intelligent contract. And the system can dynamically adjust the strategy according to the access frequency and risk of the data to adapt to different demand changes.

The concepts and technologies disclosed herein are directed to software runtime assisted co-processing acceleration with a memory hierarchy augmented with compute elements. An example system disclosed herein includes one or more switches and a plurality of hardware compute nodes connected via the one or more switches. Each hardware compute node of the plurality of hardware compute nodes includes an in-memory compute (IMC) element configured to perform in-memory processing operations on data, such as graph data. The system also includes a near-memory compute (NMC) element configured to perform near-memory processing operations on the data. The system also includes a far-memory compute (FMC) element configured to perform far-memory processing operations on the data.

A processor core is coupled to a memory hierarchy. The processor core is configured to execute vector instructions, scalar instructions, and micro-operations. A dispatch unit within the processor core receives a vector memory operation. The dispatch unit sends the vector memory operation to a first vector input queue of multiple vector input queues. The sending is based on the memory addressing mode. A micro-operation sequencer splits the vector memory operation into one or more memory micro-operations, which includes forwarding each micro-operation within the one or more micro-operations to a first memory queue within multiple memory queues. A memory operation is then issued to a load-store unit within the processor core. The issuing includes selecting, from the multiple memory queues, the memory operation. The vector memory operation comprises either a vector load operation or a vector store operation.

In one embodiment, a system for managing a virtualization environment comprises a plurality of host machines, one or more virtual disks comprising a plurality of storage devices, a virtualized file server (VFS) comprising a plurality of file server virtual machines (FSVMs), wherein each of the FSVMs is running on one of the host machines and conducts I / O transactions with the one or more virtual disks, and a virtualized file server self-healing system configured to identify one or more corrupt units of stored data at one or more levels of a storage hierarchy associated with the storage devices, wherein the levels comprise one or more of file level, filesystem level, and storage level, and when data corruption is detected, cause each FSVM on which at least a portion of the unit of stored data is located to recover the unit of stored data.

An apparatus is provided in which processing circuitry performs processing in one of a fixed number of at least two domains. One of the domains is subdivided into a variable number of execution environments one of which is a management execution environment configured to manage the execution environments. Memory protection circuitry defines a point of encryption after at least one unencrypted storage circuit of a memory hierarchy and before at least one encrypted storage circuit of the memory hierarchy. The at least one encrypted storage circuitry uses a key input to perform encryption or decryption on the data of a memory access request issued from within a current one of the domains. The key input is different for each of the domains and for each of the execution environments and the management execution environment is configured to inhibit issuing a maintenance operation to the at least one encrypted storage circuit of the memory hierarchy.

The invention discloses a software and hardware collaborative optimization method for a hybrid in-memory architecture, which comprises the following steps of: performing joint feature representation on a target AI algorithm and an in-memory computing architecture, extracting context features, and parameterizing in-memory computing unit configuration, a neural network processor assembly line, a multi-core interconnection topology and a storage level interface of the in-memory computing architecture; an off-line reference data set is constructed, a predictive agent model is trained, discrete architecture parameters are processed by the model by adopting an embedding method, feature association is learned by applying an encoder with a self-attention mechanism, joint prediction of multi-dimensional PPA indexes is realized through a parallel prediction network, and a feasible region constraint learning mechanism is introduced in training; the trained agent model is embedded into a multi-objective evolutionary algorithm, the energy efficiency ratio, the average computing power utilization rate, the model execution delay and the like serve as optimization objectives, chip area efficiency and power consumption constraints are met at the same time, and a Pareto optimal in-memory computing architecture configuration set is searched.

The present invention discloses a method for collaborative optimization of GPU and parallel IO based on pattern heterogeneous computing, specifically relating to the field of heterogeneous computing optimization, including real-time monitoring of pattern characteristics during the execution of target computing tasks through an instruction analyzer, synchronously collecting dynamic parameters of the heterogeneous computing environment to generate a multi-dimensional resource situation matrix; based on the pattern and resource situation matrix, performing collaborative optimization operations of the binding device dimension and the scheduling time dimension; constructing a three-level adaptive transmission mechanism and periodically implementing feedback optimization according to its execution status. The method for collaborative optimization of GPU and parallel IO based on pattern heterogeneous computing improves the data prefetch hit rate by dynamically perceiving the matching relationship between heterogeneous computing patterns and I / O access patterns; reduces end-to-end latency and reduces task contention conflicts through a dynamic matching strategy between device capability profiles and task types; and improves heterogeneous storage bandwidth peak utilization and I / O throughput by differentially utilizing multi-level storage hierarchy characteristics.

This invention proposes a GQA-LUT method, utilizing a genetic algorithm and LUT-based circuit to efficiently approximate non-linear operators in Transformers. It adaptively finds optimal solutions for various non-linear functions, outperforming conventional neural network methods. A novel rounding mutation (RM) algorithm enhances approximation accuracy during quantization, improving low-bit integer precision. The invention also introduces a LayerNorm folding strategy as a near-memory computing principle, reducing IO and energy overheads with a two-stage memory hierarchy. Additionally, an additive partial sum quantization method is proposed to reduce energy consumption by quantizing accumulated PSUMs in matrix multiplication, alongside a PSQ-APSQ grouping strategy and floating-point regularization.

Aspects of the disclosure are directed to dynamic memory prefetching. In accordance with one aspect, the disclosure includes a main memory configured to store data; a memory hierarchy coupled to the main memory, the memory hierarchy configured to augment the main memory; and a trusted zone element coupled to the memory hierarchy, the trusted zone element configured to read a shared memory to determine a specific control register and one or more prefetch values based on a secure monitor call (SMC). In another aspect, the disclosure includes reading a shared memory to determine a specific control register and one or more prefetch values based on a secure monitor call (SMC); and writing the one or more prefetch values to an exception level register.

A system includes a machine-learning model implemented on a data processing apparatus, which features a parallel processor with a memory hierarchy. The machine-learning model is a recurrent neural network (RNN) with a multi-head architecture, comprising multiple sub- vectors that process parallel data streams. The RNN's weight matrix is structured as a block-diagonal matrix, allowing for parallel processing of the sub-vectors. A fused computational kernel executes an entire time-series processing loop for the multi-head RNN, maintaining the weight matrix blocks in on-chip memory and performing matrix multiplications and element-wise operations for each sub-vector in a single kernel execution.

This application provides a method, system, electronic device, and medium for caching codebooks based on vector quantization. The method includes: acquiring a codebook to be processed; sorting the codebook to be processed to obtain a target codebook index; obtaining an index boundary based on the target codebook index and a codebook cache; the codebook cache being a cache for storing the codebook to be processed; comparing the index boundary and the target codebook index to obtain a judgment result; and storing the codebook to be processed in the corresponding cache within the codebook cache based on the judgment result. This application improves memory performance and execution efficiency by placing codebook entries at different locations in the GPU's memory hierarchy based on the codebook's usage frequency, thus solving the problems of low efficiency in shared memory and global memory.

A system and method are provided for artificial intelligence (AI) model selection and loading. The method includes receiving an input data stream for an AI application and analyzing the stream to determine temperature scores based on topic frequency, importance weightage, and access frequency. The method also includes mapping these temperature scores to relevant expert AI models and predicting the temperature and context of the next prompt using historical data patterns through a sliding window that smooths temporary anomalies. The system dynamically selects AI models based on temperature scores and predicted contexts, then pre-loads these models into a memory hierarchy where higher-temperature models are placed in faster memory. The method processes incoming prompts using these pre-loaded models and outputs responses, improving processing efficiency through predictive model management. The system addresses challenges of managing large-scale AI models by implementing intelligent pre-loading mechanisms that anticipate and prepare for upcoming processing needs.

In disclosed embodiments, compression circuitry may compress data blocks and compressed write data blocks and corresponding compression metadata to a certain level in a cache / memory hierarchy. In some embodiments, the compression circuitry is configured to detect a pre-determined set of data values in a block of data to be compressed. In response, the compression circuitry may write a special metadata value for the data block to data storage circuitry to indicate compression of the data block, without writing a compressed version of the data block to the data storage circuitry. This may advantageously improve performance and reduce power consumption for data blocks with certain data values (e.g., having uniform pixel / texel values).

Disclosed embodiments provide techniques for improved performance in processing vector instructions. A processor core is accessed. The processor core is coupled to a memory hierarchy, and the processor core includes one or more vector execution units (VUs), and one or more load store units (LSUs). The processor core includes a vector register file (VRF). The VRF includes multiple vector registers, and each vector register includes multiple vector elements. Vector elements that have a source or destination in contiguous memory are identified. Load store units (LSUs) take advantage of the contiguous memory condition by executing a vector load or vector store operation as a single memory access, requiring a reduced number of clock cycles. The single memory access satisfies each memory operation for each vector element within the vector register file.

An electronic device includes at least one compression - decompression functional block and a cache memory hierarchy having a first cache memory and a second cache memory. The at least one compression - decompression functional block receives data in an uncompressed state, compresses the data using one of a first compression or a second compression, and after compressing the data, provides the data to the first cache memory for storage therein. When retrieving data from the first cache memory for storage in the second cache memory, when the data was compressed using the first compression, the compression - decompression functional block decompresses the data to reverse the effect of the first compression on the data, thereby restoring the data to the uncompressed state, and provides the data compressed using the second compression or in the uncompressed state to the second cache memory for storage therein.

Systems, apparatus, articles of manufacture, and methods are disclosed. An example apparatus includes main memory, a memory hierarchy in circuit with the memory, buffer circuitry in circuit with the memory, and control circuitry to selectively couple programmable circuitry to the main memory via the cache circuitry or to the main memory via the buffer circuitry.

A cache controller of a cache assigns a dirty tracking bit for each dirty byte of a cache line. Once a predetermined interval has elapsed without any accesses to the cache line or to a cache set that includes the cache line, the cache controller compresses contiguous dirty tracking bits for each portion of the cache line. Compressing the dirty tracking bits for contiguous dirty portions of the cache line allows the cache to store more dirty data using fewer dirty tracking bits, reducing area cost and bandwidth among levels of a memory hierarchy.

An apparatus includes a processor core and a memory hierarchy. The memory hierarchy includes a main memory and one or more caches between the main memory and the processor core. A plurality of hardware prefetchers are coupled to the memory hierarchy, and a prefetch control circuit is coupled to the plurality of hardware prefetchers. The prefetch control circuit is configured to compare changes in the one or more cache performance metrics within the two or more sampling intervals and control operation of the plurality of hardware prefetchers in response to changes in the one or more performance metrics between at least a first sampling interval and a second sampling interval.

Disclosed techniques enable vector instruction processing. A processor core is accessed. The processor core is coupled to a memory hierarchy, and is configured to execute vector operations, scalar operations, and micro-operations. A decode unit decodes a vector memory operation. The vector memory operation is associated with a unit stride addressing mode. The decoding includes dividing the vector memory operation into one or more vector memory micro-operations. A dispatch unit sends at least one vector micro-operation within the one or more vector micro-operations to a scalar request queue within a plurality of request queues. The at least one vector micro-operation is issued to a load-store unit within the processor core. The issuing includes selecting, from the plurality of request queues, the at least one vector memory micro-operation.

Methods and systems for extracting a one-dimensional (1D) signal from a two-dimensional (2D) digital image along a projection line are provided herein. The method and system store the digital image in a memory hierarchy, where a non-blocking prefetch operation may fetch pixels from a main memory to a data cache. In response to the direction of the projection line, a prefetch plan, a pixel processing plan, and a prefetch distance are selected. The prefetch plan uses a first address order designed to facilitate efficient extraction of pixels from the main memory to the data cache for a given direction. The pixel processing plan uses a second address order designed to facilitate calculation of a one-dimensional signal along the projection line. A pixel processing plan is used in coordination with a prefetch plan to compute the one-dimensional signal such that the pixel is fetched from the main memory to the data cache in advance in response to an amount of time of the prefetch distance before the pixel operation uses the pixel.

A system and method are provided for artificial intelligence (Al) model selection and loading. The method includes receiving an input data stream for an Al application and analyzing the stream to determine temperature scores based on topic frequency, importance weightage, and access frequency. The method also includes mapping these temperature scores to relevant expert Al models and predicting the temperature and context of the next prompt using historical data patterns through a sliding window that smooths temporary anomalies. The system dynamically selects Al models based on temperature scores and predicted contexts, then pre-loads these models into a memory hierarchy where higher-temperature models are placed in faster memory. The method processes incoming prompts using these pre-loaded models and outputs responses, improving processing efficiency through predictive model management. The system addresses challenges of managing large-scale Al models by implementing intelligent pre-loading mechanisms that anticipate and prepare for upcoming processing needs.

The invention provides a CUDA (Compute Unified Device Architecture) code generation method based on data flow analysis, which comprises the following steps: converting a hardware sensing algorithm into a multi-memory hierarchical data flow diagram based on architecture information of a multi-layer memory in target hardware, including a plurality of task nodes representing different operation tasks in the algorithm, a plurality of affine edges and a plurality of cache nodes representing memories, the cache nodes comprise memory hierarchies and memory data, and the affine edge comprises memory access information between two cache nodes which are connected with the affine edge and represent memories of adjacent hierarchies; mapping each task node into target hardware to obtain a target hardware unit allocated to the operation task of each task node; by taking maximization of the performance of all the operation tasks as an optimization target, adjusting a blocking strategy of the memory data of each cache node to obtain a memory data blocking result; and based on the data flow diagram, the memory data partitioning result and the memory access information of each affine edge, generating a CUDA code for indicating each target hardware unit to execute the operation task.

The invention discloses an agent interaction memory cooperation method and device based on memory grading and a medium, and the method comprises the steps: obtaining the original interaction data of a user, and carrying out the context interaction analysis of the original interaction data, so as to obtain an interaction state variable; based on the interaction state variable, determining user interaction multiplexing state data through cross-session multiplexing of the user unique identifier; obtaining an iteratable shared knowledge base through high-frequency multiplexing knowledge precipitation according to the user interaction multiplexing state data; performing data interaction control on the interaction state variable, the user interaction multiplexing state data and the iteratable shared knowledge base to determine a directional calling state of the memory data; and based on the directional calling state of the memory data, through multi-level cache matching of agent demand analysis, obtaining agent interactive memory collaborative storage data. Through the method, the technical problem that an artificial intelligence interaction memory function cannot meet real-time interaction dynamic and personalized requirements in the prior art is solved.

Techniques related to eviction control for cache lines that store register data are disclosed. In some embodiments, a memory hierarchy circuit is configured to provide memory support for register operand data in one or more cache circuits. The lock circuitry may control a first set of lock indicators for a set of registers of a first thread, including one or more lock indicators to assert registers indicated by the decoding circuitry to be utilized by decoded instructions of the first thread. The lock circuit may hold register operand data in the one or more cache circuits, including to prevent eviction of a given cache line from the cache circuit with an assertion-based lock indicator. The lock circuit may clear the first set of lock indicators in response to a reset event. The disclosed techniques may advantageously retain relevant register information in a cache having a limited control circuit area.