95 results about "Hit ratio" patented technology

The invention relates to the technical field of data storage and processing, in particular to an RAID card static cache management method and device based on data popularity, and the method comprises the steps that the physical position of a data block needing to be accessed in an SSD array is acquired according to a read-write request; historical data access information of the RAID card is collected, a historical data set is generated, and a neural network model is trained by using the historical data set to obtain a cache management model; predicting and outputting data blocks which are possibly accessed in the future and the access probability of the data blocks through the cache management model; the RAID controller dynamically adjusts a cache strategy in combination with an LRU strategy according to a prediction result output by the cache management model and a read-write request of a file system, and optimizes a storage position and an updating mechanism of a data block in a cache space; and when the hit rate or the data access efficiency does not reach the preset value, adjusting the parameters of the cache strategy and updating the cache strategy. According to the method, cache resources can be more effectively distributed, the cache hit rate is improved, the cache replacement overhead is reduced, and therefore the performance of a storage system is optimized.

The invention discloses a cache space control method of a storage system, electronic equipment and a storage medium, and relates to the technical field of data caching, and the method comprises the steps of identifying dirty data, comparing the size relation between the hit rate of cached data in a cache space and a first hit rate threshold value and a second hit rate threshold value, and when the hit rate is greater than the first hit rate threshold value, controlling the cache space of the storage system. When the hit rate is smaller than a first hit rate threshold value, executing an expansion instruction to control the cache module to expand a cache space, and when the hit rate is smaller than a second hit rate threshold value, controlling the cache module to brush dirty data down to a rear-end memory, executing a reduction instruction to control the cache module to reduce the cache space, and performing a cache hit rate judgment mechanism and a linkage telescopic cache mechanism in a full random access scene. The problems of performance reduction, high delay of data access, high power consumption and the like of the cache system due to the fact that the cache space is fixed and cannot adapt to different application scenes and changes of data loads in the prior art are solved, performance self-adaption of the IO model is achieved, the response speed is increased, and the performance of the cache system in multiple storage scenes is improved.

The invention belongs to the field of integrated circuits and computer system structures, and provides a caching method and system for a memory access unit of a superscalar processor, and the method comprises the steps: receiving a plurality of memory access instructions in the same period, and determining a corresponding Bank in a to-be-accessed cache through the memory access instructions; after the memory access instruction obtains a cache access permission, if cache line missing occurs, generating a missing request, merging all the missing requests, and performing parallel prefetching training on the merged missing requests by utilizing a mode of fusing a constant step length prefetching mode and a complex step length prefetching mode to obtain a prefetching request and a prefetching cache address corresponding to the prefetching request; requesting a missing cache line from the first-level cache to the second-level cache based on the missing queue, and writing the missing cache line back to the cache line of the corresponding data cache in the first-level cache; and storing the bus consistency request by using the sniffing queue, judging whether the data in the multi-core cache are consistent or not by using the consistency request, and performing consistency modification according to a judgment result. The cache hit rate and the bandwidth utilization rate are improved.

The invention provides a large model reasoning method and system based on a multi-level cache mechanism, electronic equipment and a storage medium, and relates to the technical field of computers. The large model reasoning method based on the multi-level cache mechanism comprises the following steps: receiving an original request; based on the key value cache distribution and the load condition of each instance, determining a pre-filling instance and a decoding instance corresponding to the original request; determining a plurality of key value cache blocks of the original request based on a three-level cache mechanism through a pre-filling instance; based on the key value cache block, executing calculation in a pre-filling stage to obtain a first output mark in the pre-filling stage; performing calculation in a decoding stage based on the first output mark in the pre-filling stage and the key value cache block through the decoding instance to obtain a complete output mark corresponding to the original request; and determining a reasoning result of the original request based on the complete output mark. Therefore, the hit rate of the prefix cache can be greatly improved, and the problem of load imbalance is effectively relieved.

The embodiment of the invention discloses a self-adaptive frequency modulation method and device which are used for improving the frequency modulation accuracy of a processor. The method comprises the steps that the computing device collects real-time monitoring data of the processor, the real-time monitoring data comprises one or more events monitored by a performance monitoring unit (PMU) of the processor, and the events comprise one or more of the following items: the number of instruction execution times, the cache hit rate and the branch prediction error rate. A predicted performance change rate of the processor is calculated based on the real-time monitoring data, the predicted performance change rate is determined based on a current performance change rate of the processor, the current performance change rate is determined based on the real-time monitoring data, and the predicted performance change rate is used for indicating the proportion of a reference performance value to an operation performance value of the processor. And determining a target frequency of the processor based on the target performance loss rate and the predicted performance change rate, wherein the target performance loss rate is used for indicating the predicted performance change rate tolerable by the processor.

The invention provides a cache elimination method and device, equipment, a medium and a program product, relates to the technical field of data processing, and aims to solve the problem of low cache hit rate. The method comprises the following steps: monitoring a key operation event of a Redis node based on an EBPF program to obtain a first key name of a first key value pair operated by the key operation event and access information corresponding to the key operation event; inputting a first service label corresponding to the first key name and the access information into a preset machine learning model for judgment to obtain a target cache elimination strategy of the first key value pair; updating a preset strategy mapping table based on the first service label and the target cache elimination strategy to obtain a first updated strategy mapping table; and performing cache elimination on the key value pair of the Redis node based on the first updating strategy mapping table. The cache hit rate can be improved.

The invention relates to the technical field of computers, and discloses a cache replacement method, system and device based on binary tree status bit management and a medium. The method comprises the following steps: firstly mapping N cache lines of a cache unit into N leaf nodes of a binary tree, then setting status bits of internal nodes and leaf nodes of the binary tree, preferentially replacing the empty cache lines when accessing the cache, then judging whether the cache is hit or not, and if the cache is hit, judging whether the cache is hit or not. Determining state transition of the internal node according to whether the path node corresponding to the hit cache line is the same as the internal node, and updating the state of the leaf node; when the cache is not hit, the historical queue is checked firstly, then the sub-tree is selected according to the node state value to search and replace the cache line, and whether replacement is carried out or not is determined according to the leaf node and the state bit of the new cache. According to the method, relatively low hardware overhead can be kept, frequency locality and time locality can be better balanced, a higher cache hit rate is provided, and relatively good performance is achieved when data of different access modes are processed.

The invention discloses a filter type cache replacement method based on reinforcement learning and a related device, and the method employs a dynamic learning rate technology, pays attention to the cache miss condition during the operation of the method at any time, quantifies the miss condition into a loss value, and adjusts the learning rate in the direction of reducing the loss value through a gradient descent method. And the cache replacement method is always in better performance. A cache replacement problem is abstracted into a dobby machine problem, and cache replacement is guided by using an LRU algorithm and an LFU algorithm with weight values. The weight of the expert algorithm is adjusted according to the cache miss condition so as to adaptively change the workload. For a cache structure, the cache structure is logically divided into three layers, the first two layers of the cache quickly filter non-frequent access data, and the third layer of the cache retains data which are frequently accessed in the future as much as possible, so that the hit rate of a cache replacement algorithm is increased. In addition, the method can dynamically adjust the cache structure, and the adaptive capacity of the method is further enhanced.

The invention provides a distributed large model reasoning method and system based on modular cache, and the method comprises the steps: disassembling a prompt into modules which can be independently reused through modular division driven by Schema, and calculating the attention Key-Value state of each module in advance before reasoning for caching. When a user request arrives, the system firstly performs module analysis and cache assembly on a prompt, loads a cache as required and supplements a missed part, and then injects a complete KV state into a reasoning process. Meanwhile, a layered and distributed scheduling mechanism is designed, a global scheduler selects a target GPU based on a cache hit rate and a node load, and a local scheduler manages cache copying and elimination in a node, so that load balancing and cache multiplexing are both considered. According to the scheme, repeated calculation can be remarkably reduced, the first Token delay is reduced, the system throughput rate is increased, and the method is suitable for high-repetition prompt scenes such as code generation and long text question and answer.

Embodiments of the present application disclose an adaptive frequency modulation method and apparatus, which are used for improving the frequency modulation accuracy of a processor. The method in the embodiments of the present application comprises: a computing device acquiring real-time monitoring data of a processor, the real-time monitoring data comprising one or more events monitored by a performance monitoring unit (PMU) of the processor, and the events comprising one or more of the following: the number of times of instruction execution, a cache hit rate, and a branch prediction error rate; on the basis of the real-time monitoring data, computing a predicted performance change rate of the processor, the predicted change rate being determined on the basis of a current performance change rate of the processor, the current performance change rate being determined on the basis of the real-time monitoring data, and the predicted performance change rate being used for indicating the ratio of a reference performance value of the processor relative to an operating performance value; and, on the basis of a target performance loss rate and the predicted performance change rate, determining a target frequency of the processor, the target performance loss rate being used for indicating a predicted performance change rate tolerable by the processor.

The invention relates to a distributed storage cache hotspot prediction method and system based on an access mode. The method comprises the following steps: acquiring an access log to obtain original data; the gradient change of the access frequency between the adjacent data blocks is analyzed in real time to divide the hot spot range and identify continuous hot spot data blocks; constructing a time sequence based on the historical access record, and pre-judging an access hotspot in a future time period by applying a time sequence trend prediction algorithm; constructing a multi-dimensional feature vector and inputting the multi-dimensional feature vector into a machine learning prediction model to output a hotspot probability of the corresponding data object in a future time period; according to the hotspot probability and a popularity threshold value, scheduling the data predicted as the hotspot into a cache, and allocating corresponding cache levels or storage paths for the data objects with different popularity at the same time; and periodically updating the prediction model and the popularity statistics to obtain dynamic prediction and cache optimization of the hotspot data. According to the invention, the prediction of the hotspot data is realized, and the hit rate and the system performance of the distributed cache are improved.

The present application belongs to the technical field of cloud computing and provides a computing system, a multi-round session inference method, an apparatus and a computing device cluster. In the system, an external storage device stores a historical key value cache of a completed session. As the capacity of the external storage device is far greater than the capacity of an HBM in an accelerator, the hit rate of the historical key value cache can be increased, thereby avoiding recomputing the key value cache. In addition, when the accelerator processes a session, a host can preload a historical key value cache of a session to be processed in a task queue from the external storage device to an internal memory of the host; when performing i-th layer computing on the session, the accelerator can preload from the internal memory of the host the historical key value cache required for the (i+1)-th layer computing. As the computing process and the data loading process are synchronously performed, the computing process does not need to wait for the completion of data loading, such that the time overheads of the accelerator accessing the external storage device can be hidden, improving the efficiency of multi-round session inference.

The embodiment of the invention provides a multi-level cache access method, system and device and a storage medium, and relates to the technical field of data cache access, and the method comprises the following steps: obtaining a key field in a user auditing request, and converting the key field into a feature vector; searching whether a reasoning result corresponding to the feature vector exists in a first preset cache or not; if yes, a reasoning result is returned to the user; if not, a corresponding semantic partition is found in a second preset cache, and a historical cache vector with the highest similarity and a corresponding reasoning result are determined from the semantic partition; and determining a reasoning result corresponding to the feature vector according to the feature vector, a historical cache vector and a reasoning result corresponding to the historical cache vector. In this way, a two-stage cache mechanism based on the feature vector is constructed, a corresponding reasoning result is found in combination with semantic partition, the cache hit rate is increased, repeated reasoning of a large model is reduced, response delay is reduced, and therefore the requirements for high redundancy and high real-time performance of cache data reasoning are met.

The invention relates to the technical field of computers, in particular to an aging processing method and device in a cache and electronic equipment, and the method comprises the following steps: receiving a data query request; if it is determined that the target data requested by the data query request is located in the cache, obtaining the target data from a target cache table item in the cache, and returning the target data to a sender of the data query request; meanwhile, if the target cache table item is not aged, reducing the residual maximum hit times of the target cache table item, and updating the residual survival time length to the target survival time length; if the residual maximum hit frequency of the target cache table item is smaller than a first threshold value, or the residual survival duration is smaller than a second threshold value, determining that the target cache table item is aged; according to the method provided by the invention, the aging of the cache table item is restricted by the residual maximum hit times and the residual survival duration, so that the hit rate of the cache is improved.

The invention relates to the technical field of storage control, in particular to a memory control method, a storage device, a medium and computer equipment, a three-level index structure is arranged, only a first-level root table resides in a cache, an index is rapidly extracted through bit operation so as to reduce cache occupation and invalid IO overhead, the search range is narrowed through a hash bucket of a three-level page table, and the search efficiency is improved. Conflicts are solved through a linked list, and the mapping efficiency and the system stability can be improved through a fixed structure; and the second-level block table and the third-level page table are loaded in a non-blocking manner without occupying a CPU, so that the bandwidth is saved, and the concurrency performance and the cache hit rate are improved. Through the overall scheme of three-level index, chained hash table, loading on demand and bit operation, the delay of the whole process from logic address input to physical address output is remarkably reduced, and the low delay requirement of a high-performance storage system is met; the three-level index structure supports flexible adjustment of the sizes of a root table, a block table and a page table, adapts to storage devices with different capacities, and has good expandability.

The invention discloses a high-concurrency time series data processing method and device for an anti-unmanned aerial vehicle system. According to the method, a lock-free hash table is adopted in a cache space to cache time sequence data, lock overhead is reduced through lock-free cache design and CAS atomic operation, the batch writing throughput is high, the conflict probability is low, multi-thread high-concurrency reading and writing are supported, thread competition and blocking caused by a traditional cache mechanism are thoroughly eliminated, and the concurrent processing capacity is improved; meanwhile, the expenditure of dynamic memory allocation is reduced through batch writing, and the memory utilization rate is increased; through quick positioning of a compound key, priority query of a hotspot data index structure and a lock-free reading mechanism, the query delay of hotspot data is extremely low, the delay of related query in the future is further reduced through a data preloading mechanism, and the real-time response capability is improved; based on a dynamic elimination mechanism of an elimination score, hotspot / recent data can be reserved preferentially, the resource utilization rate is ensured, the cache hit rate is remarkably improved, and the access frequency of the TSDB is greatly reduced.

Apparatus and method for throttling ray tracing operations based on a cache hit rate. For example, one embodiment of a processor comprises: a cache subsystem comprising one or more caches; circuitry to track a plurality of hits and misses in the cache subsystem for data accesses associated with ray tracing operations; and thread management logic to dynamically control a working set size for the ray tracing operations based on the plurality of hits and misses.

This invention discloses a cache adjustment method, apparatus, device, and computer-readable storage medium, comprising: collecting access characteristic data from the client layer, object storage daemon layer, and device layer respectively; determining the object access mode of the system based on the access characteristic data; and adjusting the cache priority of each access task, the cache capacity ratio between the client layer and the object storage daemon layer, and the data residence time in the cache layer according to the object access mode; wherein the cache layer includes the client layer and the object storage daemon layer. This invention improves cache hit rate and overall read / write throughput performance, and reduces access latency.

A dynamic data loading method, a graphics processor, and a readable storage medium are disclosed. The dynamic data loading method includes: responding to a received data request, detecting the current operating mode of the caching module; the current operating mode is either a first mode or a second mode, where in the first mode the data request reads data in units of cache lines, and in the second mode the data request reads data in units of sub-blocks within a cache line; performing cache line miss detection based on the data request; in the current operating mode, determining whether to perform a cache line swapping operation based on the miss detection result; responding to the determination that a cache line swapping is triggered, performing the cache line swapping operation, and determining whether there is a need to switch operating modes; and determining whether to switch the current operating mode based on the number of times the cache line swapping operation is performed in the current operating mode and the number of times there is a need to switch operating modes. This solution can improve the performance and hit rate of the caching module.

The application discloses a kind of high concurrency time series data processing method and device for anti-UAV system.The application uses lock-free hash table to cache time series data in cache space, lock-free cache design and CAS atomic operation reduce lock overhead, batch write throughput is high, conflict probability is low, support multi-thread high concurrency read-write, completely eliminate thread competition and blocking caused by traditional cache mechanism, improve concurrent processing capability;At the same time, batch write reduces the overhead of dynamic memory allocation, improves memory utilization;Through composite key fast positioning, hot data index structure priority query and lock-free reading mechanism, the query delay of hot data is extremely low, data preloading mechanism further reduces the delay of future related query, improves real-time response capability;Based on the dynamic elimination mechanism of elimination score, hot data / recent data can be preferentially retained, ensure resource utilization, significantly improve cache hit rate, greatly reduce access frequency to TSDB.

The invention discloses a cache implementation method and device, and the method comprises the steps: setting a plurality of replacement algorithms for a cache, one of the replacement algorithms being a main replacement algorithm, and the rest being auxiliary replacement algorithms; and circularly executing the steps according to a preset frequency: counting an accumulated value of a cache miss number of each replacement algorithm; determining the cache hit rate of each replacement algorithm according to the accumulated value of the cache miss number; when the replacement algorithm corresponding to the highest cache hit rate is an auxiliary replacement algorithm, setting the auxiliary replacement algorithm corresponding to the highest cache hit rate as a new main replacement algorithm; wherein when the cache receives the access request, the cache miss number of each replacement algorithm is updated. The cache hit rate can be improved.

The invention belongs to the field of GPGPU chip design, and particularly relates to a GPGPU thread block scheduling method and system based on data space locality, and the method comprises the steps: carrying out the statistics of Bank feature information of access data of all thread blocks, and classifying the thread blocks accessing the same Bank into the same Bank group according to the Bank feature information; preferentially distributing the thread blocks of the same Bank group to the same programmable multiprocessor until the resources of the programmable multiprocessor are saturated; setting a private row cache space for each Bank in each programmable multiprocessor; when the data in the private line cache space is updated, synchronously updating the corresponding data in the L1 cache, the L2 cache and the DRAM; counting the hit rate of the private line cache in real time, and closing the private line cache function when the hit rate is lower than a preset threshold value. High-delay external storage access is reduced, the data reuse rate is improved, and the execution efficiency is remarkably improved.

This application relates to an instruction fetching method, apparatus, and computer device based on a reduced instruction set (RISC). The method includes: obtaining the address of the instruction to be executed; searching for the address of the instruction to be executed in a cache block group; if a first or second cache block is hit, reading and outputting the instruction to be executed corresponding to the address of the instruction to be executed from the corresponding hit cache block; if a third or fourth cache block is hit, writing the instruction currently stored in the corresponding hit cache block into a free cache block in the first or second cache block; and reading and outputting the instruction to be executed corresponding to the address of the instruction to be executed from the free cache block. The entire instruction fetching process, by adding a small-capacity static prediction cache and a victim cache, forms a four-cache group interconnected structure with the dual cache blocks that implement general read / write operations. This improves the cache hit rate and achieves fast instruction fetching even with limited instruction cache capacity.

The invention provides a CPU end DLRM reasoning dynamic optimization method based on load pre-calculation, and belongs to the technical field of deep learning and computer system optimization. Comprising the following steps: step 1, dynamic core distribution: S1, constructing a heterogeneous load quantification model of an embedded search task and a Bottom-MLP task, S2, carrying out load sampling and dynamic adjustment, and S3, modifying a deep learning framework thread pool mechanism, and distributing and binding a CPU physical core for the two types of tasks to realize parallel scheduling; and step 2, dynamic prefetching: T1, determining timeliness matching and accurate prefetching principles and defining a prefetching distance as a key parameter according to embedded search three-level indirect addressing characteristics, T2, recording an iteration period, separating and aggregating calculation time and DRAM access time in an embed dingbag operator core cycle, and T3, dynamically calculating the prefetching distance and inserting a prefetching instruction to execute prefetching. According to the method, the problems of unbalanced heterogeneous task core distribution and irregular memory access prefetching failure are effectively solved, the DLRM reasoning end-to-end delay is reduced, and the CPU resource utilization rate and the cache hit rate are improved.

The invention discloses a fair resource management method and system for multi-user shared large language model reasoning, and the method is characterized in that the method employs a cold data recognition elimination mechanism and a fair cache distribution mechanism, carries out the unified quantification of the resource consumption of a request in GPU calculation, decoding and cache transmission, and carries out the fair scheduling based on the accumulated CPI value of each user; the system comprises a user request management module, a CPI fair scheduling module, a model reasoning execution module, a resource monitoring module and a cache management module. Compared with the prior art, the method has the advantages that users can share computing and caching resources fairly, efficient operation performance of the system is guaranteed, users with high cache hit rate are prevented from excessively occupying GPU execution opportunities due to low apparent computing overhead, the overall reasoning efficiency and the cache hit rate of the system are remarkably improved, and the user experience is improved. Unified and fair distribution of computing resources and cache resources is achieved, the effect is prominent especially in a multi-tenant high-load scene, and the method has good application prospects and commercial development value.

The application relates to the field of computer architecture and parallel computing technology, and discloses an address mapping method and system for GPU convolution acceleration, which is applied to the process of performing convolution operation based on general matrix multiplication by a GPU, dynamically remaps the memory access address of a workspace matrix, and comprises the following steps: identifying a data repetition mode in the workspace matrix; based on the data repetition mode, remapping the original address of a current memory access request to a target address by using an address mapping strategy; performing a data access operation by using the target address; if the target data is cached in a cache level of the GPU, directly returning the same value in the original access address; otherwise, loading data from a global memory and storage. The application avoids frequent global memory and storage access caused by scattered storage of repeated data in a traditional method, significantly improves the cache hit rate, and further breaks through the bottleneck of low cache utilization of an existing strategy.

The application discloses a network-on-chip performance simulation method, device, system and storage medium, and belongs to the technical field of integrated circuit design. The method is suitable for being executed by an L2 Cache simulation model coupled between a network-on-chip design model and a memory subsystem model. The method comprises the following steps: receiving an access request forwarded by the network-on-chip design model from an upstream host device, wherein the access request comprises a hit-miss indication bit; analyzing the hit-miss indication bit; based on the analysis result, executing a corresponding response path; wherein if the hit-miss indication bit represents a hit state, a first response is generated and returned to the upstream host device via the network-on-chip design model; and if the hit-miss indication bit represents a miss state, the access request is proxied to the memory subsystem model, and a second response is returned to the upstream host device via the network-on-chip design model. The dependence on the RTL code of the L2 Cache is decoupled, and the hit rate can be actively controlled.

The embodiment of the invention discloses a self-adaptive frequency modulation method and device which are used for improving the frequency modulation accuracy of a processor. The method comprises the steps that the computing device collects real-time monitoring data of the processor, the real-time monitoring data comprises one or more events monitored by a performance monitoring unit (PMU) of the processor, and the events comprise one or more of the following items: the number of instruction execution times, the cache hit rate and the branch prediction error rate. And calculating a predicted performance change rate of the processor based on the real-time monitoring data, the predicted performance change rate being determined based on a current performance change rate of the processor, the current performance change rate being determined based on the real-time monitoring data, and the predicted performance change rate being used for indicating a ratio of a reference performance value to an operation performance value of the processor. And determining a target frequency of the processor based on the target performance loss rate and the predicted performance change rate, wherein the target performance loss rate is used for indicating the predicted performance change rate tolerable by the processor.

The invention discloses a strategy optimization method for cache garbage collection, relates to the technical field of computer cache management, and is used for solving the problem of low cache collection efficiency. According to the method, a cache storage area is abstracted into a plurality of cache partitions, a partition state vector composed of capacity, access and fragmentation degree operation indexes is constructed for each partition, and a strategy evaluation thread periodically reads the partition state vectors to calculate garbage recycling pressure and recycling priority. The upper limit of the number of scanning objects and the upper limit of time slices are distributed for the target cache partition by combining the idle time proportion of the processor, and a recycling execution thread conducts incremental recycling in a batch mode according to a strategy list and records the recycling progress; after recovery, index weight and scanning configuration are adjusted according to release capacity, fragment degree change, hit times and back-to-source request change, so that garbage recovery gives consideration to release space and overhead, the hit rate and the memory utilization rate are improved, and pause and invalid recovery caused by centralized recovery are reduced.

This invention relates to the field of high-performance computing and provides a method and apparatus for optimizing the performance of a graphics processing unit (GPU) aggregate communication operator. The method includes: allocating device-side storage space on the host side; copying the structured data to be communicated to the device-side storage space to obtain a target data pointer; adjusting the kernel function parameter of the aggregate communication operator from a structure type to a pointer type and passing the target data pointer; executing the kernel function on the GPU side, directly accessing the device-side storage space based on the target data pointer to complete the aggregate communication operation; and releasing the device-side storage space after the aggregate communication operation is completed. This invention solves the problem of excessive GPU register pressure and decreased cache hit rate caused by inefficient memory access instructions generated by the compiler to pass structure parameters in existing technologies. It avoids the generation of inefficient instructions, reduces register pressure, and improves execution efficiency.