271 results about "Cache miss" patented technology

The invention discloses a hybrid expert model reasoning method based on cooperation of a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), and belongs to the field of deep learning. According to the method, a CPU-GPU computing framework of a hybrid expert model is constructed, heterogeneous computing resource loads are effectively balanced, and the hardware utilization rate is remarkably increased; an intelligent cache management mechanism based on dynamic priority scores is provided, high-demand experts are reserved preferentially, and the transmission overhead caused by cache missing is reduced; through pipeline parallel design for separating calculation and transmission tasks, CPU calculation and PCIe transmission are overlapped in the GPU execution period, and delay is effectively hidden. In addition, in combination with a multi-layer expert activation prediction prospective prefetching mechanism, the expert cache hit rate is improved. The method is compatible with hybrid expert models of different scales and structures, and stable and efficient reasoning acceleration is realized on a resource-limited heterogeneous platform.

The invention relates to the technical field of computers, and provides a request management method and device, electronic equipment and a storage medium, and the method comprises the steps: when it is detected that a data access request is subjected to a cache miss, based on a first address carried by the data access request, executing the cache miss on the data access request; checking whether an uncompleted request entry corresponding to the first address exists in a preset request management structure or not, wherein the preset request management structure is a pre-constructed multi-path group associative cache structure used for performing grouping management on a cache miss request; and adding the data access request into a waiting queue of the uncompleted request entry under the condition that the uncompleted request entry exists in the preset request management structure. According to the method and the device, the cache miss requests are subjected to grouping management by adopting a multi-path group associative cache structure, so that the parallel search of the cache miss requests can be realized, and the processing efficiency of the cache miss requests is greatly improved.

The invention relates to a federal knowledge retrieval and large language model enhancement system and method. The system comprises a server and a hardware accelerator. The server preprocesses an input query and extracts a head word of the query; the server retrieves the head word in the caching process through a cache module of the server, and if the cache of the cache module is not hit, the server sends a retrieval instruction to the hardware accelerator so as to retrieve in a local cache of the hardware accelerator; if the local cache is still not hit, the hardware accelerator generates parallel subtasks associated with the head word in a mode of segmenting a local knowledge graph, so that deep search is carried out, and noise is added into knowledge items retrieved by the parallel subtasks to protect sensitive information; and the hardware accelerator integrates the knowledge items added with the noise, performs reasoning through a large language model and generates an enhanced answer. According to the method, a two-stage dynamic cache system is constructed, the cross-device communication frequency can be effectively reduced, the query request is responded preferentially through the local high-frequency cache, and the overall network load of the system is reduced.

The invention relates to the technical field of computers, in particular to an adaptive method and device based on Warp scheduling and cache replacement and a medium. The method comprises the following steps: dynamically monitoring an access behavior of a Warp in an L1 data cache through a locality sensing unit, and identifying locality preference of a workload; according to the locality preference, adaptively selecting a Warp scheduling algorithm; periodically collecting cache access count and cache missing count through a cache state monitor, calculating the memory access missing rate of the current sampling period, and comparing the memory access missing rate with the missing rate of the previous sampling period; if the memory access missing rate of the current sampling period is higher than that of the previous sampling period, switching a cache replacement strategy; if not, the current strategy is kept. According to the method, the Warp scheduling algorithm and the cache replacement strategy are adaptively adjusted, so that the cache utilization rate and the performance of the GPGPU are optimized.

Techniques for cache management for LLM processing are described. Example embodiments include a signal hashing model that generates a key for particular context data. An LLM output corresponding to the context data is stored in a cache along with the key. For a user input received by the system, a cache lookup is performed using a key for context data corresponding to the received user input. For a cache hit, the stored output is used to respond to the user input. For a cache miss, a LLM processes the context data and the user input to generate an output within a first timeout. If the LLM is unable to generate an output within the first timeout, then in some cases, the LLM is allowed to continue processing until a second timeout, and a final or partial output from the LLM is stored in the cache.

Methods and systems are presented for providing multi-tiered cache system that works with an artificial intelligence (AI)-based conversation system for facilitating a conversation with users and processing transactions for the users. The multi-tiered cache system includes multiple tiers of cache modules that use different structures for caching and / or querying data. As a new utterance is received, the cache system uses each of the cache modules in sequence to determine whether a cache hit occurs. If a cache miss occurs at a first cache module, the cache system determines if a cache hit occurs at a second cache module. When a response is obtained from one of the cache modules and / or the AI model, the cache system updates the cache modules using the response.

The invention discloses an instruction scheduling method, device and system, a product and a medium, and relates to the technical field of processor design. The invention provides an instruction scheduling method aiming at the problem that the dynamic change in the instruction transmitting time cannot be accurately scheduled by the existing out-of-order scheduling strategy. The historical delay information of each instruction is stored through the delay cache, and when the instruction execution delay corresponding to the instruction is dynamic delay, the dynamic delay is predicted through the historical delay information stored in the delay cache, so that instruction scheduling is performed. According to the method, the dynamic delay of which the delay time cannot be predetermined is included in the consideration range of instruction scheduling based on the dynamic delay prediction mechanism. On the basis of the method, for dynamic change scenes in instruction transmission time such as memory access delay and cache miss, the optimal instruction transmission opportunity can be accurately predicted, so that the effect of instruction scheduling on performance improvement of a processor is further improved.

The invention relates to the technical field of message pushing, in particular to an SaaS (Software as a Service) intelligent concurrent pushing system which comprises a tenant flow monitoring module, a resource binding module, a lock-free object writing module, a handle layout construction module and a cache prefetching delivery module. According to the method, the high-frequency business object is identified by monitoring the flow characteristics of the tenants, and the complete Slab page is dynamically applied and bound to the thread exclusive local allocation buffer area, so that physical isolation of bottom layer resources is realized to eliminate global lock competition and context switching overhead in a multi-thread environment; lock-free memory writing is executed based on a privatized pointer, the object construction efficiency is greatly improved, a compact storage structure in which a bitmap is adjacent to a connection handle is constructed, a hardware cache prefetching mechanism is triggered by utilizing bitmap reading, connection data is loaded to a CPU cache line in advance, and the connection data is stored in the cache line. And the cache miss rate is reduced and the concurrent delivery performance of massive messages is remarkably improved by utilizing a memory locality principle.

Caching techniques can include: receiving, from a host, a read I / O operation requesting to read current content of a logical address; determining whether a data cache includes a data cache entry corresponding to the logical address; responsive to determining the data cache includes the data cache entry corresponding to the logical address, performing data cache hit processing to service the read I / O operation using the data cache entry; responsive to determining the data cache does not include the data cache entry corresponding to the logical address, performing data cache miss processing including: determining whether a mapping cache includes a descriptor corresponding to the logical address; and responsive to determining the mapping cache includes the descriptor corresponding to the logical address, performing mapping cache hit processing to service the read I / O operation using the descriptor of the mapping cache.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for dynamic distance adjustment for stride prefetcher. One of the systems includes a processor; a stride prefetcher; and a cache subsystem, wherein the stride prefetcher is configured to dynamically compute a stride prefetch distance based on a measure of cache miss latency and a duration between requests in a sequence of strided memory accesses issued by the processor.

A cache writeback circuit is disclosed for writing back, without invalidating, dirty cache lines. The cache writeback circuit is configured to enter an active state based on detecting a trigger condition indicative of cache misses to a memory cache circuit within a memory hierarchy of a computer memory subsystem causing cache line eviction activity. During the active state, the cache writeback circuit is configured to identify a set of dirty cache lines in the memory cache circuit, and write back, without invalidating, cache lines of the identified set of dirty cache lines from the memory cache circuit to a memory circuit within a lower level of the memory hierarchy, such as DRAM. The cache writeback circuit may further be configured to identify the dirty cache lines via a cache walk operation, which can be suspended, for example, when a higher-priority cache operation occurs.

Caching techniques can include: receiving, from a host, a read I / O operation requesting to read current content of a logical address; determining whether a data cache includes a data cache entry corresponding to the logical address; responsive to determining the data cache includes the data cache entry corresponding to the logical address, performing data cache hit processing to service the read I / O operation using the data cache entry; responsive to determining the data cache does not include the data cache entry corresponding to the logical address, performing data cache miss processing including: determining whether a mapping cache includes a descriptor corresponding to the logical address; and responsive to determining the mapping cache includes the descriptor corresponding to the logical address, performing mapping cache hit processing to service the read I / O operation using the descriptor of the mapping cache.

A system includes a multilevel memory such as a two level memory (2LM), where a first level memory acts as a cache for the second level memory. A memory controller or cache controller can detect a cache miss in the first level memory for a request for data. Instead of automatically performing a swap, the controller can determine whether to perform a swap based on a swap policy assigned to a memory region associated with the address of the requested data.

The invention discloses a method for accelerating secure metadata access in a secure memory system, a memory controller and a system, and belongs to the field of secure memory systems, and the method comprises the following steps: when a page table item corresponding to a logic page where data accessed by a processor is located does not hit a TLB, obtaining the page table item from a memory page table, extracting a physical page address from the page table item, and storing the physical page address in a memory; a counter corresponding to a physical page where the data to be accessed is located and a father node of the counter in the integrity tree are prefetched through the physical page address; adding a replacement dirty block address in a miss request sent by the last level of cache, after receiving the miss request containing a field of the replacement dirty block address, executing conventional memory reading and decryption, positioning a counter corresponding to the replacement dirty block address, and performing prefetching by using an idle memory bandwidth; in addition to the secure metadata cache, the prefetching queue is maintained to temporarily store the prefetched metadata. The cache hit rate of the security metadata in the security memory system can be improved, and the performance overhead caused by the cache miss can be reduced.

An apparatus and a method for performing an authenticated encryption with associated data (AEAD) operation of an encrypted instruction and a golden tag stored in a memory device in an event of a cache miss are provided. The apparatus includes a bus control circuit, a block buffer, a tag buffer and an AEAD circuit. The bus control circuit receives a read address from a cache for reading the encrypted instruction and the golden tag from the memory device. The block buffer receives and stores the encrypted instruction from the bus control circuit, wherein a size of the block buffer is preset to be N times a size of one cache line. The tag buffer receives and stores the golden tag from the bus control circuit. The AEAD circuit performs the AEAD operation upon the encrypted instruction and the golden tag to check whether the encrypted instruction is tampered or not.

Embodiments of the present application relate to the technical field of computers, and provide a prefetching method and apparatus, an electronic device, and a readable storage medium. The method comprises: when a first cache block corresponding to a memory access address exists in a cache, determining whether a target pointer exists in the first cache block, wherein said target pointer points to an address space in which a cache miss occurs in a historical period; if the target pointer pointing to the address space in which a cache miss occurs in the historical period exists in the first cache block, increasing the value of a first parameter of the first cache block by n; and performing continuous prefetching by taking the target pointer in the first cache block as a trigger starting point for prefetching, and stopping the prefetching operation until a prefetching termination condition is met.

The invention discloses a distributed data preloading and querying method and device and a storage medium. The method comprises the steps of searching data corresponding to a query key in a cache region integrated by an application server; if the query result of the query key is hit in the cache region, returning the query result; if the query result of the query key is not hit in the cache region, determining whether a target query key which is the same as the query key exists in the executed database query; and if the target query key does not exist, initiating a query request based on the query key to the database, and storing a query result returned by the database into a cache region integrated by the application server to respond to the query of the same query key. According to the method and the device, the cache is integrated in the application server, the resource overhead of an independent cache server is eliminated, the concurrent query when the cache is missed is merged and processed, and the concurrency is changed into single query, so that the repeated request pressure of the database is reduced fundamentally, and the risk of cache breakdown is effectively prevented.

The invention discloses an optimization method and device for an embedded linked list container, electronic equipment and a storage medium, and relates to the technical field of computers, the method comprises the steps that a linked list node is embedded into a host data structure to serve as a member variable, extra memory occupation of a pointer domain in a traditional linked list is omitted, and the optimization efficiency is improved. Physical storage of the nodes and the host data structure is continuous, so that cache missing can be reduced; meanwhile, the stability of the linked list under high concurrency is guaranteed through atomized insertion and deletion operations, and a traditional independent node structure and a non-atomized operation are not adopted; the technical problems that in the prior art, a traditional linked list node pointer domain occupies an extra memory, so that expenditure is increased, discontinuous node physical distribution causes cache missing, access delay is increased, and system abnormal safety and operation reliability are affected by iterator failure under high concurrency can be solved. And the technical effects of reducing the memory overhead, reducing the access delay and improving the abnormal safety and the operation reliability of the system are achieved.

The invention relates to the technical field of computers, and discloses a cache replacement method, system and device based on binary tree status bit management and a medium. The method comprises the following steps: firstly mapping N cache lines of a cache unit into N leaf nodes of a binary tree, then setting status bits of internal nodes and leaf nodes of the binary tree, preferentially replacing the empty cache lines when accessing the cache, then judging whether the cache is hit or not, and if the cache is hit, judging whether the cache is hit or not. Determining state transition of the internal node according to whether the path node corresponding to the hit cache line is the same as the internal node, and updating the state of the leaf node; when the cache is not hit, the historical queue is checked firstly, then the sub-tree is selected according to the node state value to search and replace the cache line, and whether replacement is carried out or not is determined according to the leaf node and the state bit of the new cache. According to the method, relatively low hardware overhead can be kept, frequency locality and time locality can be better balanced, a higher cache hit rate is provided, and relatively good performance is achieved when data of different access modes are processed.

The invention discloses a processor performance optimization method and device, electronic equipment and a storage medium, and can solve the problems that the performance of a processor cannot be analyzed from multiple angles, and the performance of the processor cannot be accurately and effectively optimized. Acquiring real-time operation data of the target processor, wherein the real-time operation data comprises hardware sensing information, software sensing information and load information; multi-dimensional feature extraction is carried out on the operation data to obtain target load features, and the target load features comprise a cache miss rate and / or a branch prediction error rate; determining an optimization strategy corresponding to the target load feature according to the target load feature and a target performance optimization strategy model; the target performance optimization strategy model is obtained through model training in advance; and performing performance optimization on the target processor through the optimization strategy so as to reduce the cache miss rate and / or the branch prediction error rate.

The invention discloses a filter type cache replacement method based on reinforcement learning and a related device, and the method employs a dynamic learning rate technology, pays attention to the cache miss condition during the operation of the method at any time, quantifies the miss condition into a loss value, and adjusts the learning rate in the direction of reducing the loss value through a gradient descent method. And the cache replacement method is always in better performance. A cache replacement problem is abstracted into a dobby machine problem, and cache replacement is guided by using an LRU algorithm and an LFU algorithm with weight values. The weight of the expert algorithm is adjusted according to the cache miss condition so as to adaptively change the workload. For a cache structure, the cache structure is logically divided into three layers, the first two layers of the cache quickly filter non-frequent access data, and the third layer of the cache retains data which are frequently accessed in the future as much as possible, so that the hit rate of a cache replacement algorithm is increased. In addition, the method can dynamically adjust the cache structure, and the adaptive capacity of the method is further enhanced.

Methods and systems for cache miss monitoring and fulfillment are disclosed. A disclosed method comprises receiving, from a processing unit, a cache access request, determining, based on a cache failing to fulfill the cache access request, that a cache miss has occurred, populating a request buffer based on the cache miss and the cache access request, populating a transaction buffer with a transaction entry for the request buffer based on the cache miss request and the cache access request, determining by the request buffer, that information requested in the cache access request should be retrieved from main memory, determining, by the transaction buffer if information was not retrieved, that the cache access request satisfies criteria for creating a cache miss tag, creating the cache miss tag based on the cache access request, and storing the cache miss tag in a portion of the cache.

The invention provides a shared cache management method and electronic equipment, and relates to the technical field of electronic equipment control. According to the method, the running frequency points of the processor and the shared cache can be matched, so that the processor can obtain data from the shared cache in time, and the running speed of the application is ensured. The method comprises the following steps: in response to a first event, setting a running frequency point of a shared cache as a first frequency point based on a focus application, user operation information, performance information of each processing core and thread running information; wherein the focus application is the application operated by the user for the last time, the user operation information comprises the type and the number of times of the operation acting on the focus application by the user within the preset time, and the performance information of each processing core is used for reflecting the shared cache missing rate of the processing core; the thread running information is used for reflecting running time proportions of the plurality of thread groups on the N processing cores, and each thread group comprises one or more threads running on the electronic equipment.

A method of reducing cache thrashing in a processing system is provided. M threads are issued to process a workload, and a memory access request associated with the M threads is transmitted to a first-level cache of the processing system. The memory access request is then transmitted to a second-level cache of the processing system in response to the first cache miss at the first-level cache. The memory access request is transmitted to a main memory of the processing system in response to the second cache miss at the second-level cache. The value of M is decreased when the relationship between the hit rates of the second-level cache and the first-level cache satisfies a predetermined criterion. A storage capacity and an access latency of the second-level cache are higher than those of the first-level cache.

The invention relates to a malicious file detection method and device, electronic equipment and a storage medium, and the method comprises the steps: reading fixed N-byte data at the head of a to-be-detected file, calculating a Hash value for the N-byte data through employing a Hash algorithm, and generating a head fingerprint, N being an integer greater than 1; a local cache is inquired according to the head fingerprints, the local cache is used for storing local head sample fingerprints and corresponding judgment results, and the judgment results are used for indicating malicious fingerprints or benign fingerprints; if the local cache does not hit the head fingerprint, a cloud cache is inquired according to the head fingerprint, and the cloud cache is used for storing a global head sample fingerprint and a corresponding judgment result; if the cloud cache does not hit the head fingerprint, a malicious rule set of the cloud is called to perform feature scanning on the to-be-detected file, and the malicious rule set is used for identifying whether the to-be-detected file has malicious properties or not. The malicious file detection efficiency is improved.

The invention provides a key value cache multiplexing method for a retrieval enhancement generation system. The method comprises three stages of knowledge retrieval, prompt construction and reasoning generation. In the knowledge retrieval stage, the system carries out vectorization on user input and retrieves related documents, and retrieval results serve as enhanced information; in the prompt construction stage, user input and enhancement information is coded into a token sequence, and the hash value and length of each part of the token sequence are calculated; in the inference generation stage, whether the key value cache is hit or not is judged through hash comparison, the key value cache is directly reused if the key value cache is hit, differential video memory allocation is executed if the key value cache is not hit, a new key value cache is generated in combination with a partition position coding strategy, and finally a result is generated through large language model inference and returned to a user. According to the method, the key value cache reuse rate of the large language model in multi-round reasoning of the retrieval enhancement generation system can be improved, so that the calculation amount and the video memory overhead are reduced.

The invention discloses an Ethernet-based distributed cache coherence protocol, an implementation method, an implementation device and an implementation system. A plurality of computing nodes are connected through a packet switching network. Each computing node comprises a CPU / GPU (Central Processing Unit / Graphics Processing Unit) and a local cache thereof, and is provided with a cache agent. The far-end memory is organized in home nodes, and each home node manages a part of physical address space and is equipped with a directory controller. When the CPU of the computing node accesses a far-end memory address and does not hit in the local cache, the CA of the computing node replaces the far-end memory address and communicates with the DC managing the address through the network so as to maintain the cache consistency of the data among all the nodes. Based on a cache consistency protocol of a directory, the CXL.cache consistency of a plurality of independent computing nodes can be maintained in a low-overhead and high-reliability mode on a high-delay and lossy packet switching network, and broadcast storm caused by a monitoring protocol is avoided.

The invention relates to the technical field of computers, in particular to a data processing method, a readable storage medium, a program product and electronic equipment. The data processing method is applied to the electronic equipment, the electronic equipment comprises a processor, if continuous cache miss occurs in the process that the processor executes a first instruction and a second instruction which are continuous, the processor sends a first message for obtaining first data of the first instruction to a memory, and in the process that the memory returns the first memory data corresponding to the first message, the processor can also send a second message for obtaining the second data of the second instruction to the memory. And after the processor acquires the first memory data, acquiring second memory data corresponding to the second message. Therefore, the processor only needs to set one buffer area to store the memory data acquired from the memory, so that the time loss of the cache miss of the processor is reduced, and the speed of acquiring the data from the memory by the processor under the continuous cache miss is improved.

The invention discloses an optimization method and system for direct I / O read performance under a Linux system. The method comprises the steps that a cache control mark used for controlling cache access is introduced; in the file opening stage, whether a cache mechanism is started or not is judged, if yes, whether a cache is hit or not is detected when direct I / O reading operation is executed, data are directly read from the cache and returned when the cache is hit, standard direct I / O reading and data returning are called when the cache is not hit, meanwhile, asynchronous caching is conducted on the data, and a cache mark is set; when the direct I / O write operation is executed, whether a cache mechanism is started or not is judged, if yes, whether the cache is hit or not is detected firstly, when the cache is hit, original cache data is set to be invalid, asynchronous caching is conducted on the data, meanwhile, standard direct I / O write-in data is called, and after asynchronous caching and data write-in operation are completed, the cache data is set to be valid. According to the invention, the reading performance of direct I / O can be improved.

The invention discloses a design space exploration method for a cache hierarchical structure in a multi-core particle system. The method aims at optimizing the cache subsystem in the multi-core particle system, and the system performance is improved by reasonably configuring the cache hierarchical structure and the interconnection network topology between the core particles. The method comprises the following specific steps: 1) modeling a cache miss rate and network delay: modeling the cache miss rate and the network delay as a function of a cache hierarchical structure and interchip interconnection network parameters; 2) optimization problem definition: defining an optimization objective and minimizing concurrency average storage access time (C-AMAT) under the constraint of cost and power consumption; and 3) solving by using a double-layer optimization algorithm: respectively optimizing the cache subsystem and the interconnection network between the chip grains through the double-layer optimization algorithm. The method provides an effective solution for cache optimization of the multi-core particle system, and has a wide application prospect.