71 results about "Hit ratio" patented technology

The invention discloses a cache space control method of a storage system, electronic equipment and a storage medium, and relates to the technical field of data caching, and the method comprises the steps of identifying dirty data, comparing the size relation between the hit rate of cached data in a cache space and a first hit rate threshold value and a second hit rate threshold value, and when the hit rate is greater than the first hit rate threshold value, controlling the cache space of the storage system. When the hit rate is smaller than a first hit rate threshold value, executing an expansion instruction to control the cache module to expand a cache space, and when the hit rate is smaller than a second hit rate threshold value, controlling the cache module to brush dirty data down to a rear-end memory, executing a reduction instruction to control the cache module to reduce the cache space, and performing a cache hit rate judgment mechanism and a linkage telescopic cache mechanism in a full random access scene. The problems of performance reduction, high delay of data access, high power consumption and the like of the cache system due to the fact that the cache space is fixed and cannot adapt to different application scenes and changes of data loads in the prior art are solved, performance self-adaption of the IO model is achieved, the response speed is increased, and the performance of the cache system in multiple storage scenes is improved.

The invention belongs to the field of integrated circuits and computer system structures, and provides a caching method and system for a memory access unit of a superscalar processor, and the method comprises the steps: receiving a plurality of memory access instructions in the same period, and determining a corresponding Bank in a to-be-accessed cache through the memory access instructions; after the memory access instruction obtains a cache access permission, if cache line missing occurs, generating a missing request, merging all the missing requests, and performing parallel prefetching training on the merged missing requests by utilizing a mode of fusing a constant step length prefetching mode and a complex step length prefetching mode to obtain a prefetching request and a prefetching cache address corresponding to the prefetching request; requesting a missing cache line from the first-level cache to the second-level cache based on the missing queue, and writing the missing cache line back to the cache line of the corresponding data cache in the first-level cache; and storing the bus consistency request by using the sniffing queue, judging whether the data in the multi-core cache are consistent or not by using the consistency request, and performing consistency modification according to a judgment result. The cache hit rate and the bandwidth utilization rate are improved.

The invention provides a large model reasoning method and system based on a multi-level cache mechanism, electronic equipment and a storage medium, and relates to the technical field of computers. The large model reasoning method based on the multi-level cache mechanism comprises the following steps: receiving an original request; based on the key value cache distribution and the load condition of each instance, determining a pre-filling instance and a decoding instance corresponding to the original request; determining a plurality of key value cache blocks of the original request based on a three-level cache mechanism through a pre-filling instance; based on the key value cache block, executing calculation in a pre-filling stage to obtain a first output mark in the pre-filling stage; performing calculation in a decoding stage based on the first output mark in the pre-filling stage and the key value cache block through the decoding instance to obtain a complete output mark corresponding to the original request; and determining a reasoning result of the original request based on the complete output mark. Therefore, the hit rate of the prefix cache can be greatly improved, and the problem of load imbalance is effectively relieved.

The invention provides a cache elimination method and device, equipment, a medium and a program product, relates to the technical field of data processing, and aims to solve the problem of low cache hit rate. The method comprises the following steps: monitoring a key operation event of a Redis node based on an EBPF program to obtain a first key name of a first key value pair operated by the key operation event and access information corresponding to the key operation event; inputting a first service label corresponding to the first key name and the access information into a preset machine learning model for judgment to obtain a target cache elimination strategy of the first key value pair; updating a preset strategy mapping table based on the first service label and the target cache elimination strategy to obtain a first updated strategy mapping table; and performing cache elimination on the key value pair of the Redis node based on the first updating strategy mapping table. The cache hit rate can be improved.

The invention discloses a filter type cache replacement method based on reinforcement learning and a related device, and the method employs a dynamic learning rate technology, pays attention to the cache miss condition during the operation of the method at any time, quantifies the miss condition into a loss value, and adjusts the learning rate in the direction of reducing the loss value through a gradient descent method. And the cache replacement method is always in better performance. A cache replacement problem is abstracted into a dobby machine problem, and cache replacement is guided by using an LRU algorithm and an LFU algorithm with weight values. The weight of the expert algorithm is adjusted according to the cache miss condition so as to adaptively change the workload. For a cache structure, the cache structure is logically divided into three layers, the first two layers of the cache quickly filter non-frequent access data, and the third layer of the cache retains data which are frequently accessed in the future as much as possible, so that the hit rate of a cache replacement algorithm is increased. In addition, the method can dynamically adjust the cache structure, and the adaptive capacity of the method is further enhanced.

The invention provides a distributed large model reasoning method and system based on modular cache, and the method comprises the steps: disassembling a prompt into modules which can be independently reused through modular division driven by Schema, and calculating the attention Key-Value state of each module in advance before reasoning for caching. When a user request arrives, the system firstly performs module analysis and cache assembly on a prompt, loads a cache as required and supplements a missed part, and then injects a complete KV state into a reasoning process. Meanwhile, a layered and distributed scheduling mechanism is designed, a global scheduler selects a target GPU based on a cache hit rate and a node load, and a local scheduler manages cache copying and elimination in a node, so that load balancing and cache multiplexing are both considered. According to the scheme, repeated calculation can be remarkably reduced, the first Token delay is reduced, the system throughput rate is increased, and the method is suitable for high-repetition prompt scenes such as code generation and long text question and answer.

The invention relates to a distributed storage cache hotspot prediction method and system based on an access mode. The method comprises the following steps: acquiring an access log to obtain original data; the gradient change of the access frequency between the adjacent data blocks is analyzed in real time to divide the hot spot range and identify continuous hot spot data blocks; constructing a time sequence based on the historical access record, and pre-judging an access hotspot in a future time period by applying a time sequence trend prediction algorithm; constructing a multi-dimensional feature vector and inputting the multi-dimensional feature vector into a machine learning prediction model to output a hotspot probability of the corresponding data object in a future time period; according to the hotspot probability and a popularity threshold value, scheduling the data predicted as the hotspot into a cache, and allocating corresponding cache levels or storage paths for the data objects with different popularity at the same time; and periodically updating the prediction model and the popularity statistics to obtain dynamic prediction and cache optimization of the hotspot data. According to the invention, the prediction of the hotspot data is realized, and the hit rate and the system performance of the distributed cache are improved.

The embodiment of the invention provides a multi-level cache access method, system and device and a storage medium, and relates to the technical field of data cache access, and the method comprises the following steps: obtaining a key field in a user auditing request, and converting the key field into a feature vector; searching whether a reasoning result corresponding to the feature vector exists in a first preset cache or not; if yes, a reasoning result is returned to the user; if not, a corresponding semantic partition is found in a second preset cache, and a historical cache vector with the highest similarity and a corresponding reasoning result are determined from the semantic partition; and determining a reasoning result corresponding to the feature vector according to the feature vector, a historical cache vector and a reasoning result corresponding to the historical cache vector. In this way, a two-stage cache mechanism based on the feature vector is constructed, a corresponding reasoning result is found in combination with semantic partition, the cache hit rate is increased, repeated reasoning of a large model is reduced, response delay is reduced, and therefore the requirements for high redundancy and high real-time performance of cache data reasoning are met.

The invention relates to the technical field of storage control, in particular to a memory control method, a storage device, a medium and computer equipment, a three-level index structure is arranged, only a first-level root table resides in a cache, an index is rapidly extracted through bit operation so as to reduce cache occupation and invalid IO overhead, the search range is narrowed through a hash bucket of a three-level page table, and the search efficiency is improved. Conflicts are solved through a linked list, and the mapping efficiency and the system stability can be improved through a fixed structure; and the second-level block table and the third-level page table are loaded in a non-blocking manner without occupying a CPU, so that the bandwidth is saved, and the concurrency performance and the cache hit rate are improved. Through the overall scheme of three-level index, chained hash table, loading on demand and bit operation, the delay of the whole process from logic address input to physical address output is remarkably reduced, and the low delay requirement of a high-performance storage system is met; the three-level index structure supports flexible adjustment of the sizes of a root table, a block table and a page table, adapts to storage devices with different capacities, and has good expandability.

The invention discloses a high-concurrency time series data processing method and device for an anti-unmanned aerial vehicle system. According to the method, a lock-free hash table is adopted in a cache space to cache time sequence data, lock overhead is reduced through lock-free cache design and CAS atomic operation, the batch writing throughput is high, the conflict probability is low, multi-thread high-concurrency reading and writing are supported, thread competition and blocking caused by a traditional cache mechanism are thoroughly eliminated, and the concurrent processing capacity is improved; meanwhile, the expenditure of dynamic memory allocation is reduced through batch writing, and the memory utilization rate is increased; through quick positioning of a compound key, priority query of a hotspot data index structure and a lock-free reading mechanism, the query delay of hotspot data is extremely low, the delay of related query in the future is further reduced through a data preloading mechanism, and the real-time response capability is improved; based on a dynamic elimination mechanism of an elimination score, hotspot / recent data can be reserved preferentially, the resource utilization rate is ensured, the cache hit rate is remarkably improved, and the access frequency of the TSDB is greatly reduced.

Apparatus and method for throttling ray tracing operations based on a cache hit rate. For example, one embodiment of a processor comprises: a cache subsystem comprising one or more caches; circuitry to track a plurality of hits and misses in the cache subsystem for data accesses associated with ray tracing operations; and thread management logic to dynamically control a working set size for the ray tracing operations based on the plurality of hits and misses.

This invention discloses a cache adjustment method, apparatus, device, and computer-readable storage medium, comprising: collecting access characteristic data from the client layer, object storage daemon layer, and device layer respectively; determining the object access mode of the system based on the access characteristic data; and adjusting the cache priority of each access task, the cache capacity ratio between the client layer and the object storage daemon layer, and the data residence time in the cache layer according to the object access mode; wherein the cache layer includes the client layer and the object storage daemon layer. This invention improves cache hit rate and overall read / write throughput performance, and reduces access latency.

A dynamic data loading method, a graphics processor, and a readable storage medium are disclosed. The dynamic data loading method includes: responding to a received data request, detecting the current operating mode of the caching module; the current operating mode is either a first mode or a second mode, where in the first mode the data request reads data in units of cache lines, and in the second mode the data request reads data in units of sub-blocks within a cache line; performing cache line miss detection based on the data request; in the current operating mode, determining whether to perform a cache line swapping operation based on the miss detection result; responding to the determination that a cache line swapping is triggered, performing the cache line swapping operation, and determining whether there is a need to switch operating modes; and determining whether to switch the current operating mode based on the number of times the cache line swapping operation is performed in the current operating mode and the number of times there is a need to switch operating modes. This solution can improve the performance and hit rate of the caching module.

The application discloses a kind of high concurrency time series data processing method and device for anti-UAV system.The application uses lock-free hash table to cache time series data in cache space, lock-free cache design and CAS atomic operation reduce lock overhead, batch write throughput is high, conflict probability is low, support multi-thread high concurrency read-write, completely eliminate thread competition and blocking caused by traditional cache mechanism, improve concurrent processing capability;At the same time, batch write reduces the overhead of dynamic memory allocation, improves memory utilization;Through composite key fast positioning, hot data index structure priority query and lock-free reading mechanism, the query delay of hot data is extremely low, data preloading mechanism further reduces the delay of future related query, improves real-time response capability;Based on the dynamic elimination mechanism of elimination score, hot data / recent data can be preferentially retained, ensure resource utilization, significantly improve cache hit rate, greatly reduce access frequency to TSDB.

The invention discloses a cache implementation method and device, and the method comprises the steps: setting a plurality of replacement algorithms for a cache, one of the replacement algorithms being a main replacement algorithm, and the rest being auxiliary replacement algorithms; and circularly executing the steps according to a preset frequency: counting an accumulated value of a cache miss number of each replacement algorithm; determining the cache hit rate of each replacement algorithm according to the accumulated value of the cache miss number; when the replacement algorithm corresponding to the highest cache hit rate is an auxiliary replacement algorithm, setting the auxiliary replacement algorithm corresponding to the highest cache hit rate as a new main replacement algorithm; wherein when the cache receives the access request, the cache miss number of each replacement algorithm is updated. The cache hit rate can be improved.

The invention belongs to the field of GPGPU chip design, and particularly relates to a GPGPU thread block scheduling method and system based on data space locality, and the method comprises the steps: carrying out the statistics of Bank feature information of access data of all thread blocks, and classifying the thread blocks accessing the same Bank into the same Bank group according to the Bank feature information; preferentially distributing the thread blocks of the same Bank group to the same programmable multiprocessor until the resources of the programmable multiprocessor are saturated; setting a private row cache space for each Bank in each programmable multiprocessor; when the data in the private line cache space is updated, synchronously updating the corresponding data in the L1 cache, the L2 cache and the DRAM; counting the hit rate of the private line cache in real time, and closing the private line cache function when the hit rate is lower than a preset threshold value. High-delay external storage access is reduced, the data reuse rate is improved, and the execution efficiency is remarkably improved.

This application relates to an instruction fetching method, apparatus, and computer device based on a reduced instruction set (RISC). The method includes: obtaining the address of the instruction to be executed; searching for the address of the instruction to be executed in a cache block group; if a first or second cache block is hit, reading and outputting the instruction to be executed corresponding to the address of the instruction to be executed from the corresponding hit cache block; if a third or fourth cache block is hit, writing the instruction currently stored in the corresponding hit cache block into a free cache block in the first or second cache block; and reading and outputting the instruction to be executed corresponding to the address of the instruction to be executed from the free cache block. The entire instruction fetching process, by adding a small-capacity static prediction cache and a victim cache, forms a four-cache group interconnected structure with the dual cache blocks that implement general read / write operations. This improves the cache hit rate and achieves fast instruction fetching even with limited instruction cache capacity.

The invention provides a CPU end DLRM reasoning dynamic optimization method based on load pre-calculation, and belongs to the technical field of deep learning and computer system optimization. Comprising the following steps: step 1, dynamic core distribution: S1, constructing a heterogeneous load quantification model of an embedded search task and a Bottom-MLP task, S2, carrying out load sampling and dynamic adjustment, and S3, modifying a deep learning framework thread pool mechanism, and distributing and binding a CPU physical core for the two types of tasks to realize parallel scheduling; and step 2, dynamic prefetching: T1, determining timeliness matching and accurate prefetching principles and defining a prefetching distance as a key parameter according to embedded search three-level indirect addressing characteristics, T2, recording an iteration period, separating and aggregating calculation time and DRAM access time in an embed dingbag operator core cycle, and T3, dynamically calculating the prefetching distance and inserting a prefetching instruction to execute prefetching. According to the method, the problems of unbalanced heterogeneous task core distribution and irregular memory access prefetching failure are effectively solved, the DLRM reasoning end-to-end delay is reduced, and the CPU resource utilization rate and the cache hit rate are improved.

The invention discloses a fair resource management method and system for multi-user shared large language model reasoning, and the method is characterized in that the method employs a cold data recognition elimination mechanism and a fair cache distribution mechanism, carries out the unified quantification of the resource consumption of a request in GPU calculation, decoding and cache transmission, and carries out the fair scheduling based on the accumulated CPI value of each user; the system comprises a user request management module, a CPI fair scheduling module, a model reasoning execution module, a resource monitoring module and a cache management module. Compared with the prior art, the method has the advantages that users can share computing and caching resources fairly, efficient operation performance of the system is guaranteed, users with high cache hit rate are prevented from excessively occupying GPU execution opportunities due to low apparent computing overhead, the overall reasoning efficiency and the cache hit rate of the system are remarkably improved, and the user experience is improved. Unified and fair distribution of computing resources and cache resources is achieved, the effect is prominent especially in a multi-tenant high-load scene, and the method has good application prospects and commercial development value.

The application relates to the field of computer architecture and parallel computing technology, and discloses an address mapping method and system for GPU convolution acceleration, which is applied to the process of performing convolution operation based on general matrix multiplication by a GPU, dynamically remaps the memory access address of a workspace matrix, and comprises the following steps: identifying a data repetition mode in the workspace matrix; based on the data repetition mode, remapping the original address of a current memory access request to a target address by using an address mapping strategy; performing a data access operation by using the target address; if the target data is cached in a cache level of the GPU, directly returning the same value in the original access address; otherwise, loading data from a global memory and storage. The application avoids frequent global memory and storage access caused by scattered storage of repeated data in a traditional method, significantly improves the cache hit rate, and further breaks through the bottleneck of low cache utilization of an existing strategy.

The application discloses a network-on-chip performance simulation method, device, system and storage medium, and belongs to the technical field of integrated circuit design. The method is suitable for being executed by an L2 Cache simulation model coupled between a network-on-chip design model and a memory subsystem model. The method comprises the following steps: receiving an access request forwarded by the network-on-chip design model from an upstream host device, wherein the access request comprises a hit-miss indication bit; analyzing the hit-miss indication bit; based on the analysis result, executing a corresponding response path; wherein if the hit-miss indication bit represents a hit state, a first response is generated and returned to the upstream host device via the network-on-chip design model; and if the hit-miss indication bit represents a miss state, the access request is proxied to the memory subsystem model, and a second response is returned to the upstream host device via the network-on-chip design model. The dependence on the RTL code of the L2 Cache is decoupled, and the hit rate can be actively controlled.

The invention discloses a strategy optimization method for cache garbage collection, relates to the technical field of computer cache management, and is used for solving the problem of low cache collection efficiency. According to the method, a cache storage area is abstracted into a plurality of cache partitions, a partition state vector composed of capacity, access and fragmentation degree operation indexes is constructed for each partition, and a strategy evaluation thread periodically reads the partition state vectors to calculate garbage recycling pressure and recycling priority. The upper limit of the number of scanning objects and the upper limit of time slices are distributed for the target cache partition by combining the idle time proportion of the processor, and a recycling execution thread conducts incremental recycling in a batch mode according to a strategy list and records the recycling progress; after recovery, index weight and scanning configuration are adjusted according to release capacity, fragment degree change, hit times and back-to-source request change, so that garbage recovery gives consideration to release space and overhead, the hit rate and the memory utilization rate are improved, and pause and invalid recovery caused by centralized recovery are reduced.

This invention relates to the field of high-performance computing and provides a method and apparatus for optimizing the performance of a graphics processing unit (GPU) aggregate communication operator. The method includes: allocating device-side storage space on the host side; copying the structured data to be communicated to the device-side storage space to obtain a target data pointer; adjusting the kernel function parameter of the aggregate communication operator from a structure type to a pointer type and passing the target data pointer; executing the kernel function on the GPU side, directly accessing the device-side storage space based on the target data pointer to complete the aggregate communication operation; and releasing the device-side storage space after the aggregate communication operation is completed. This invention solves the problem of excessive GPU register pressure and decreased cache hit rate caused by inefficient memory access instructions generated by the compiler to pass structure parameters in existing technologies. It avoids the generation of inefficient instructions, reduces register pressure, and improves execution efficiency.

Embodiments of the present application disclose a data processing method and related products, the method comprising: obtaining target data; detecting that a remaining storage space of a preset storage area cannot meet a storage requirement of the target data; determining an importance level of each data in the preset storage area according to a preset rule, the importance level being used to indicate a possibility of reuse of the data; deleting first data in the preset storage area, the remaining storage space of the preset storage area meeting the storage requirement of the target data after the first data is deleted, the importance level of the first data being lower than importance levels of other data in the preset storage area; and storing the target data in the preset storage area. The present application is beneficial to meeting the storage requirement of the target data while ensuring a hit rate of stored data in the storage area.

The invention relates to a time cost determination method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a runtime cache access track and a performance parameter of a target convolution operator, and extracting a target convolution shape parameter and a target partitioning parameter from the runtime cache access track and the performance parameter; inputting the target convolution shape parameter and the target partitioning parameter into a pre-training model to obtain a hit rate predicted value output by the pre-training model, the pre-training model being a machine learning model obtained by training convolution shape parameters, partitioning parameters and hit rate actual values of different convolution operators; determining calculation time by using the target convolution shape parameter and a preset hardware configuration parameter, and determining memory time by using the hit rate predicted value and the hardware configuration parameter; and determining the time cost of the target convolution operator according to the calculation time and the memory time. By adopting the method, the calculation precision of the time cost can be greatly improved.

The invention provides a consistency data monitoring request processing method and device. The consistency data monitoring request processing method comprises the steps of obtaining a consistency request, a consistency monitoring request and a processing load state of a consistency data monitoring stash request, and judging a processing mode of the stash request. When the load of the cache unit is low, a standard processing mode is adopted to execute directional cache operation, high-frequency data is stored in the specified cache unit, a data access path is shortened, and the cache hit rate is increased; and when the load of the cache unit is relatively high, a degradation processing mode is adopted, so that stash operation is prevented from preempting resources, conventional data read-write and cache line state maintenance are prevented from being blocked, and the performance balance of the multi-core consistency cache system is realized. On the basis of the existing request processing logic of the cache unit, the consistency data monitoring stash request in the degradation processing mode is multiplexed to the processing flow of the consistency monitoring request, so that the occupation of hardware resources is reduced.

The application discloses a data management method and device, medium and product, relates to the technical field of computers, and when data of a program running on a certain processor core needs to be inserted into a shared cache, the data belonging to different application programs is placed in positions of different priorities according to the categories of the application programs. The shortest time for ensuring that the data stays in a private cache is the time for the data to go from a highest priority position matched with a processor core to which the data belongs to a lowest priority position, and the intermediate process is not accessed again. Further, the data block with a higher hit rate is stored in the cache system for a longer time, so that the overall hit rate of the cache system is improved.

The embodiment of the invention provides a self-adaptive cache management method and system. The method comprises the following steps: firstly, configuring basic expiration time, an updating strategy and a hotspot flow threshold value by utilizing intelligent annotation, and generating a caching strategy basic instance bound with a service method; then, flow indexes such as the query rate per second, the cache hit rate, the data updating frequency and the response delay are collected in real time by means of a dynamic flow sensing module, a fluctuation coefficient is determined through a sliding time window algorithm, and flow changes are sensed in real time; when the fluctuation coefficient is greater than a preset threshold value, dynamically updating the cache strategy among inert updating, active preloading and expired data permitting strategies according to a preset rule, and associating the current instance; and finally, on the basis of the current instance of the cache strategy, utilizing a distributed lock coordination mechanism to guarantee the atomicity of cache updating operation. According to the process, the database pressure caused by premature failure of the cache in the peak period and the resource waste in the valley period are reduced through dynamic adjustment of the cache strategy and distributed coordination, so that the resource utilization rate can be remarkably improved.

This invention discloses a native parallel data packing method for 64-bit CPUs, belonging to the field of computer data representation and processing technology. This method packs eight 8-bit raw data columns into a standard 64-bit register, with each data column including 6 data bits and 2 carry bits. The raw data is expanded into a 64-bit data column through a bit extension step, and then packed into the 64-bit register using bit manipulation instructions to form 64-bit packed data. Inline functions are used to achieve zero function call overhead. The compiler automatically optimizes and generates efficient machine code, requiring no SIMD instruction set support. A lookup table method is used to achieve zero computational overhead packing; the lookup table contains 4096 entries. 64-byte cache row alignment is used to improve cache hit rate. Compared to traditional methods, packing and unpacking overhead is reduced by 6.6 times, from 11.3 ms to 1.7 ms. L1 cache hit rate is increased from 65% to 95%, memory access latency is reduced from 45 cycles to 12 cycles, and memory performance is improved by 3.75 times.

The present invention discloses a dynamic dual-partition cache mapping method and system based on multidimensional feature classification. The method comprises cache partitions including a set-associative master region (SAMR), a direct-mapped region (DMR), and a fully associative region (FAR), and sets the initial capacity of each partition in the cache partition; collects multidimensional feature data of data blocks in real time, the multidimensional feature data including access frequency f, spatial locality s, prefetch benefit p, and temporal volatility t; inputs the multidimensional feature data into a weighted classifier module to calculate a comprehensive score Fscore of the data block; generates a migration trigger instruction based on the comprehensive score Fscore, and migrates the data block to the direct-mapped region (DMR) or the fully associative region (FAR) between dynamic regions. The present invention improves cache hit rate, reduces hardware overhead of fully associative mapping, optimizes cache efficiency by dynamically adjusting rules, and improves the accuracy of data migration decisions.