403 results about "Cache management" patented technology

A system for adaptively caching network communication protocols enhances efficiency across heterogeneous device environments through a multi-level cache architecture with device-capability-based tiers. The system collects endpoint telemetry data including device capabilities and operational constraints to classify endpoints and generate context-aware protocol variants optimized for specific device types. Protocol optimization opportunities are determined through structural analysis of message patterns and state transitions. The system performs protocol deduplication by identifying functionally equivalent variants and maintaining canonical representations to reduce cache redundancy. Cache synchronization across distributed nodes uses enhanced Merkle tree structures with protocol normalization processing. The system predicts communication needs based on historical patterns, network context, and endpoint constraints, enabling proactive cache management tailored to device capabilities. Integration with event-driven data communication systems enables seamless protocol selection and translation while maintaining compatibility between diverse endpoint types, from high-performance servers to resource-constrained IoT devices.

The invention provides a cache management method, a cache management device, a computer storage medium and electronic equipment, and relates to the technical field of computers. The method comprises the steps of receiving a reasoning task request and distributing the reasoning task request to a target storage page; key value cache information of the first round of reasoning task is stored in a hard disk cache, and when the second round of reasoning task is executed, key value cache information generated before the second round of reasoning task is preloaded layer by layer from the hard disk cache; when the last round of reasoning task is received, storing first target key value cache information correspondingly generated by the last round of reasoning task into the matched target physical block; and performing hybrid grouping compression on key cache information and value cache information in the first target key value cache information to obtain second target key value cache information after quantization compression. According to the invention, triple balance of video memory-calculation performance-precision can be realized.

The invention discloses a method for improving long text processing efficiency and accuracy, and relates to the technical field of natural language processing and large language models.According to the method, text word segmentation embedding, sliding block preprocessing, YaRN position code injection, dynamic sparse attention calculation, multi-level attention fusion, graded KV cache management and output generation are sequentially executed; position drift is inhibited through logarithmic scaling, and key contexts are adaptively screened according to the attention activeness, so that the attention calculation complexity is close to linearity; in million-level Token reasoning, the video memory occupation of the method is reduced, the remote dependency recall rate is improved, and the method is suitable for scenes such as document analysis, code auditing and multi-mode streaming understanding.

Techniques for cache management for LLM processing are described. Example embodiments include a signal hashing model that generates a key for particular context data. An LLM output corresponding to the context data is stored in a cache along with the key. For a user input received by the system, a cache lookup is performed using a key for context data corresponding to the received user input. For a cache hit, the stored output is used to respond to the user input. For a cache miss, a LLM processes the context data and the user input to generate an output within a first timeout. If the LLM is unable to generate an output within the first timeout, then in some cases, the LLM is allowed to continue processing until a second timeout, and a final or partial output from the LLM is stored in the cache.

The invention discloses a dynamic script compiling and hot updating system based on Grouping and an implementation method of the dynamic script compiling and hot updating system. The system comprises a script source code management module, a dynamic compiling engine, an intelligent cache manager, an exception handling and recovery system and a hot update controller. And the script source code management module receives and preprocesses the Grouping script to generate a unique identifier for subsequent management. And the dynamic compilation engine executes compilation operation according to the identifier to ensure that a compilation result accords with a specification. And the intelligent cache manager efficiently stores and retrieves the compiling result, so that the repeated compiling cost is reduced. And the exception handling and recovery system monitors the execution state in real time, and automatically starts a recovery process in case of exception to guarantee stable operation of the system. The hot update controller realizes seamless update and version switching of scripts, and ensures service continuity. Through cooperative work of the modules, high-reliability, high-performance and zero-shutdown deployment is realized.

A metadata access method, apparatus, and computer-readable storage medium for efficient metadata retrieval through cache management. The method receives metadata query requests including target index information from processes and performs matching operations on a global cache file containing records with index information and slot identifiers. Each cache description array corresponds to memory blocks caching metadata. Upon successful matching, the target cache description array is accessed using the target slot identifier. The data state of target metadata is determined from the cache description array, and target address information indicating the location of the target memory block in shared memory is obtained and returned to the requesting process, enabling efficient shared memory-based metadata access.

Implementations of this specification provide key-value cache management, model reasoning, and data processing methods and apparatuses for large language models. In an implementation, a method comprises allocating a virtual memory block in a virtual address slot to newly-added token key-value data of a model reasoning request, in response to determining that a scheduling result of the model reasoning request indicates the model reasoning request is scheduled for execution, maintaining a mapping relationship between an occupied virtual address slot and a physical graphics memory block allocated to the model reasoning request, and copying the newly-added token key-value data to the physical graphics memory block.

The embodiment of the invention provides a model reasoning cache management method based on software and hardware collaboration, which comprises the following steps: a user side sends a model access request, executes reasoning calculation, and stores a newly generated key value cache in a video memory cache layer; in the video memory cache layer, updating the access popularity corresponding to each unit-level key value cache, and sorting according to the access popularity from high to low to form a first popularity sequence; and updating the hierarchical access popularity corresponding to each hierarchical key value cache, and sorting according to the hierarchical access popularity from high to low to form a second popularity sequence. And when the space utilization rate of the video memory cache layer reaches a set threshold value, determining and evicting a unit-level key value cache and / or a hierarchical key value cache to be evicted based on the first heat sequence and / or the second heat sequence. The cross-layer data migration efficiency is optimized by utilizing the characteristics of a multi-level memory and through mechanisms such as cache elimination prediction and the like, and efficient utilization of a video memory, reduction of reasoning delay and improvement of the throughput capacity of a system are realized.

The invention relates to the technical field of artificial intelligence and automatic testing, and particularly discloses an AI-based automatic testing method and device. The method comprises the following steps: capturing and recording complete execution historical information in a process of executing a test task by an AI agent, and serializing and storing the complete execution historical information as reusable cache data; before the test is executed, the validity of the cache is judged through a multi-dimensional verification mechanism including data integrity, timeliness, functionality and the like; when the cache verification is passed, a result is quickly obtained based on the cache data playback test process; when cache verification is not passed or execution fails, triggering an AI agent to perform new reasoning and steps to generate and update cache data; and meanwhile, a caching strategy is dynamically adjusted according to test case changes and resource conditions. The device comprises a test executor, an AI agent manager, an intelligent cache manager and a dynamic cache management module, and all the modules work cooperatively to improve the test efficiency, reduce the calling cost of a large language model and enhance the stability of a test result. The method is suitable for Web / APP automatic testing, cross-platform testing, CI / CD process testing and other scenes, the execution efficiency can be remarkably improved, the cost is reduced, and resource utilization is optimized.

The application provides a cache manager and a control component. The cache manager comprises an interface, an allocation module and a memory. The interface receives a cache allocation request and sends the cache allocation request to the allocation module. The interface receives first cache index information returned by the allocation module and outputs the first cache index information. In response to the cache allocation request, the allocation module allocates N idle memory blocks and generates the first cache index information. The memory blocks are located in a cache, N is a positive integer, and the first cache index information comprises indexes of the N memory blocks. The memory stores indexes of at least N memory blocks. The cache manager implemented by hardware replaces the software management cache in the prior art, thereby reducing the interaction times between software and hardware when allocating a cache in the control component, improving the information interaction speed inside the control component, and achieving the technical effect of reducing the data transmission time between the host and the flash memory chip.

The invention provides a hierarchical storage read cache management method and system, a storage medium and a product, and relates to the field of data storage, the method comprises the following steps: dividing all data blocks into three data hierarchies of hot data, warm data and cold data for respective storage according to access popularity, the data storage positions, the storage data proportions and the information recording precision of the three data levels are different; when the cold data is accessed, the accessed target cold data is upgraded to the temperature data, and the temperature data is updated; when the hot data is accessed, the complete metadata information of the accessed target hot data is cached in the memory, and the hot data is sorted and updated again; and when the temperature data is accessed, determining whether to replace the target hot data with the target temperature data or not according to the size relationship between the first historical access time of the accessed target temperature data and the second historical access time of the target hot data with the lowest access heat. By implementing the method, the occupation of memory resources can be reduced when the popularity of mass data is tracked.

The invention discloses a structured data storage method based on natural language transformation, which comprises the following steps of: a system initialization configuration stage: deploying a protocol adapter in a local memory of a PC (Personal Computer) client, and loading natural language processing pipeline configuration parameters; a heterogeneous data acquisition stage: capturing a multi-source text data stream through the protocol adapter, uniformly converting the multi-source text data stream into a standardized data packet, and sending the standardized data packet to a message queue theme; a text cleaning stage: a named entity recognition stage: inputting the pure text data into an NER module deployed with a language model loader; in the conditional feature extraction stage, feature vectors are generated for texts meeting preset conditions on the basis of entity type tags in the entity recognition result; and a consistent storage stage: inserting the entity identification results into a relational database in batches, and updating the entity mapping relationship in the cache. According to the method, the intelligent level of cache management is remarkably improved, and the access fluency of the key data of the user is guaranteed.

The invention provides a key value cache scheduling method and device, a data processing method and device, a server and a medium. Relates to the technical field of data processing. The key value cache scheduling method is applied to a cache manager in a server for deploying a large model, and is used for uniformly scheduling cache resources when a heterogeneous accelerator executes large model reasoning. The method comprises the steps of obtaining a first request sent by a heterogeneous accelerator based on to-be-reasoned data, and determining the number of required cache blocks; selecting a plurality of target cache blocks from a preset memory cache region according to the cache block occupation state; and aggregating the logic addresses to generate a key value cache address and sending the key value cache address to the heterogeneous accelerator for accessing the key value data. According to the method, through a multi-level cache division and state awareness scheduling mechanism, the cache resource utilization rate and the cross-platform compatibility are effectively improved, the problems that an existing method is low in resource allocation efficiency and poor in compatibility are solved, and the large model reasoning efficiency and the system stability are remarkably improved.

The invention discloses a data pre-reading method and device, equipment and a storage medium, and relates to the field of electronic computers.In the data pre-reading method, data heat, a load mode and a historical read command proportion are determined by analyzing historical access information, cache operation is judged based on the data heat, and a target hard disk area is predicted based on the load mode; and adjusting the size of a pre-read cache space based on the historical read command proportion to realize pre-read and cache management. By optimizing the data pre-reading strategy and cache resource allocation, the pre-reading precision of the data can be improved.

The invention relates to a three-dimensional stacking storage and calculation integrated artificial intelligence acceleration system and inference method, and the system comprises a three-dimensional stacking storage module which comprises a plurality of vertically stacked DRAM layers which carry out the communication through a high-density vertical interconnection structure; the calculation unit array is in direct communication coupling with at least one layer of the three-dimensional stacked storage module through a three-dimensional integration technology and is configured to execute at least partial reasoning calculation of a large language model or a multi-mode large language model; and the pre-stored key value cache management module is configured to pre-store key value caches generated by pre-filling and calculating predefined system cue words in one or more specified physical areas of the three-dimensional stack storage module, and the pre-stored key value caches are stored in one or more specified physical areas of the three-dimensional stack storage module. The computing unit array is further configured to access a pre-stored key value cache and combine it with data generated from dynamic user input when the inference computations are performed, thereby avoiding repeated pre-fill computations on system cues.

The invention discloses a cache maintenance system of a heterogeneous computing system and electronic equipment, and relates to the technical field of data processing, a first directory controller manages the consistency state of a host end, and a second directory controller manages the consistency state of an equipment end. The consistency maintenance operation is firstly efficiently processed by the corresponding directory controller in the storage domain, cross-device frequent coordination communication is reduced, and when cross-domain data access occurs, the two directory controllers cooperate through an established communication coupling mechanism and execute global consistency maintenance as required. The method not only can adapt to a high-burst and high-parallel access mode of a device end, but also can avoid direct conflicts with a cache management mechanism of a host end, and realizes global consistency through distributed collaboration. The technical problem of performance bottleneck of cache consistency in the heterogeneous system is solved, and the technical effect that the overall data access efficiency of the heterogeneous system is improved while the correctness is maintained is achieved.

The invention discloses an efficiency self-adaptive optimization method and system for data retrieval, relates to the technical field of data management, and discloses an efficiency self-adaptive optimization system for data retrieval. Comprising a mode acquisition module, a task evaluation module, a cost prediction module, a strategy selection module, a consensus retrieval module, an execution decision module, a cache management module and a model updating module. Risk perception support is provided for strategy selection and execution modes; according to the method, resource consumption, execution time delay and path complexity under a plurality of standard retrieval strategies are predicted by using a query cost prediction model, and a confidence score is output, so that prospective evaluation of a strategy execution effect is realized.

The invention discloses a KV Cache compression and hierarchical management method and device for RAG acceleration and a medium, and belongs to the technical field of large model reasoning acceleration. In order to solve the problems that in an existing RAG KV Cache management technology, storage occupation is too large, and long-term importance and instant accessibility of data cannot be considered at the same time, the invention provides a hierarchical management strategy, and according to the method, a multi-dimensional popularity vector containing global popularity, session popularity and context popularity is defined for a KV Cache block. The method is characterized in that the compression precision of a cache block is independently determined by using global popularity so as to balance fidelity and performance; and meanwhile, the session heat and the context heat are used for independently determining the placement positions in heterogeneous hierarchies such as a GPU (Graphics Processing Unit), a CPU (Central Processing Unit) fixed memory and a CPU paging memory. According to the method, placement is guided through the real-time dimension, compression is guided through the importance dimension, context intelligent prefetching is achieved, and the reasoning performance of an RAG system, especially in a session and in a context pursuit scene is remarkably improved.

The invention discloses a front-end cache management method, system and device for a lightweight large-model dialogue state and a medium, belongs to the technical field of front-end cache management of a large-model dialogue system, and aims at solving the technical problem of how to overcome the defects that in a traditional scheme, long context cache is low in efficiency, storage redundancy and insufficient in dynamic semantic adaptation capacity, and the large-model dialogue state cannot be managed easily. In order to realize dialogue context volume compression, improve semantic similar request hit rate and reduce cross-end synchronization delay, the adopted technical scheme is as follows: data acquisition and preprocessing: capturing user interaction behaviors in real time through front-end burying points, and performing preprocessing operation on the acquired user behavior data; semantic normalization processing: performing embedded vector conversion and semantic clustering on the text input by the user to generate a unique semantic identifier and a context vector; querying and updating the multi-level cache; and dynamic collaborative updating: dynamically adjusting the cache based on the cache hit rate, the response delay and the user feedback, and optimizing the cache effect in real time.

The invention relates to the technical field of front-end development, in particular to a method and a system for generating a multi-end table data display page. Comprising a table definition module, a table rendering module, a data processing module, a function extension module and a state management module, the table definition module is used for finely defining the structure and behavior of a table, the table rendering module is used for converting the configured table definition into a multi-terminal adaptive visual interface and optimizing rendering performance and interactive experience, and the data processing module is used for efficiently processing table data and carrying out format conversion, cache management and paging loading. The function expansion module provides flexible function expansion capability and meets interaction requirements of complex business scenes, the state management module uniformly manages interaction states of tables and ensures consistency and persistence of multi-terminal operation, and by means of the mode, the problems that table components are limited in function, insufficient in performance and poor in expansibility are solved, and the user experience is improved. Therefore, the table component has good multi-terminal adaptation capability and can meet complex business requirements.

The invention relates to the technical field of front-end Web development, and particularly provides a multi-label page component cache management method and system, a terminal and a medium, and the method comprises the steps: setting a cache level mark for each routing node in routing configuration, and generating a unique component identifier by intercepting a routing event; recursively screening target components based on the depth of the cache hierarchy, and constructing a precise cache list; and in response to the change of the global state, directionally updating the field of the influenced component by adopting a difference comparison mechanism. According to the method, efficient management of multi-label page cache is realized, and the memory utilization rate and the page rendering performance are improved.

The invention provides a distributed cache management method, system and device, a storage medium and a program product, and is applied to a distributed cache management system which comprises a business application node, a database node and a cache data service node. Each running business application process in the business application nodes comprises a cache software development tool SDK and a local cache component, and the method comprises the steps that when data in a database node is changed, a first cache SDK obtains a data change event in a cache data service node; the first cache SDK obtains change data in the database node according to the data change event; and the first cache SDK updates cache data in the first local cache component according to the change data. According to the scheme, the local cache in the distributed system can be effectively managed.

The invention provides a cache space management method and device, equipment, a storage medium and a program product, and relates to the field of cache management. Comprising the steps of initializing a to-be-managed cache space, and obtaining file data; calculating a first real-time utilization rate according to the keyword data, triggering and obtaining a collection data set according to the collection instruction, and obtaining a reading data set according to the reading instruction; according to the first real-time utilization rate, the collected data set and the read data set, performing three-source data verification to obtain a second real-time utilization rate, comparing the second real-time utilization rate with a preset trimming threshold set, determining a trimming state, and calculating to obtain a target reserved space and a space alignment release amount; therefore, the to-be-pruned task queue is pruned, and the pruned cache space is obtained. The technical problem that in the prior art, an existing cache management strategy lacks dynamic adaptability, cannot dynamically adapt to service loads and is difficult to meet complex scene requirements, so that cache space management efficiency is low is solved.

The invention discloses a virtual machine monitor system for a resource-limited persistent memory server, which comprises a configuration file module, a CPU (Central Processing Unit) management module, a memory management module, a cache management module, an interrupt management module, a cross-partition communication module and a virtual machine management module. The CPU management module is used for binding a virtual CPU to a physical CPU core, so that computing resources of the virtual machines are dedicated, and meanwhile, the virtual machines are prevented from competing for the CPU; the memory management module is used for dividing a physical memory region according to a configuration file and realizing address space isolation between virtual machines by utilizing two-stage address translation; the cache management module is used for performing isolation control on cache access of a plurality of virtual machines through a coloring mechanism of the last-level cache; and the interrupt management module is used for simulating the universal interrupt controller, and the virtual machine monitor re-injects the interrupt into the target virtual machine according to the configuration file after capturing the hardware interrupt.

The invention relates to the technical field of artificial intelligence, in particular to a cache management method, device, equipment, medium and product, and the method comprises the steps: evaluating the importance scores of different attention heads for a current reasoning task in real time, and dynamically allocating different levels of storage resources for the key value cache of each attention head according to the importance scores, comprising a storage position and numerical precision, so that the memory is saved to the greatest extent on the premise of ensuring the model output quality and particularly not losing key information, and the reasoning speed is further improved.

The invention provides a data reading method, a cache management system and electronic equipment. The data reading method comprises the following steps: receiving a reading request sent by a host; the read request is used for requesting target data with preset first granularity as a unit in a storage medium; identifying a hit state of the read request in a read cache; wherein the read cache takes a preset second granularity as a unit; when the read request is in a preset miss state in the read cache, initiating a read operation to a storage medium based on the read request; when the read request is in a preset partial hit state in the read cache, determining that a read operation corresponding to the read request is in a to-be-completed state; and after the reading operation is completed, storing the target data block returned by the storage medium to the read cache, and returning the target data to the host. In the mode, the situation that repeated and redundant reading operation is initiated to the storage medium due to granularity mismatching can be avoided, so that the reading delay is reduced, the reading interference phenomenon is inhibited, and the overall reading performance and data reliability of the storage system are improved.

The invention discloses a three-level cache replacement method based on dirty page perception and adaptive capacity, and belongs to the technical field of computer memory cache management. The method comprises the following steps of: constructing a short-time-efficiency queue, a medium-time-efficiency queue and a global queue in a CXL extended memory equipment side cache; executing object insertion, migration and metadata updating according to the hit condition of the access request in each queue; when the cache space is insufficient, the access count, the dirty page mark and the dirty page revival mark are combined to execute eviction judgment; and calculating a return visit ratio based on global queue identifier insertion times and return visit times, and dynamically adjusting short-time-efficiency and medium-time-efficiency queue capacities. According to the method, the problems of high dirty page write-back cost and inflexible cache space distribution in the CXL extended memory are solved, and the redundant write-back overhead is reduced, the utilization efficiency of the cache space is improved and the access delay is reduced through collaborative optimization of the access characteristics, the dirty page state and the elimination return visit behavior.

The invention discloses a large model reasoning hardware accelerator based on data flow execution, and belongs to the technical field of large model reasoning hardware acceleration. Each node comprises a linear calculation kernel used for sequentially completing query / key / value projection and attention output projection in multi-head attention calculation, dimension raising projection and dimension reduction projection calculation in a feedforward neural network layer, and inverse quantization operation after each projection is output; the multi-head attention calculation kernel is used for completing loading and quantization of query / key / value vectors, multi-head division and cache management, attention score calculation, Softmax normalization, context vector generation and KV cache updating; the RLA kernel is used for completing residual addition, layer normalization, quantization / inverse quantization and nonlinear activation calculation of each layer; the three cores in the same node are located in the same SLR area, and data transmission is carried out through a hardware queue and an HLS blocking mechanism. According to the invention, high hardware utilization rate and low-delay reasoning can be realized.

The invention provides a metadata processing acceleration method, electronic equipment and a storage medium, and relates to the technical field of distributed storage. According to the technical scheme, the acceleration card obtains the metadata request, the metadata is cached to the cache module of the hybrid storage architecture based on the dynamic cache management strategy, and the acceleration card directly queries the cache and controls the network card to directly construct the response packet to return when the cache is hit, so that the data processing efficiency is improved. According to the method and the device, the problems of high delay and CPU resource scrambling and the like caused by the fact that a metadata request depends on a CPU to analyze a network protocol and inquire software cache in the prior art can be solved, so that the technical effects of remarkably reducing metadata processing delay, improving metadata throughput, reducing CPU occupancy rate, optimizing hardware resource utilization rate and reducing cross-node synchronous bandwidth occupancy can be achieved.

Cache, cache management method, and electronic device. The cache includes: multiple cache lines, a first read request queue and a second read request queue; a first read request queue is configured for storing and sending a first read request to a memory controller; the first read request is configured for requesting data from memory and storing the data in the memory controller; the number of the first read requests stored in the first read request queue is greater than that of the multiple cache lines; the second read request queue is configured for storing and sending a second read request to memory controller; and the second read request corresponds one-to-one with the first read request, and the second read request is used to request data corresponding to the first read request from the memory controller when a cache line corresponding to the first read request is idle