694 results about "Cache management" patented technology

A system and method for implementing a convergent intelligence fabric (CIF) for distributed artificial intelligence operations. The CIF architecture integrates tensor-theoretic foundations, probabilistic cache management, precision-aware memory operations, quantum-resistant security, and neural-based optimization within a unified framework. The system orchestrates asynchronous, multi-hop data flow among computational resources while maintaining data security through per-block encryption and identity-based access control. Key components include a universal multi-model KV cache subsystem, agent-parallel disaggregation pipelines, reinforcement learning-based orchestration, and neuromorphic memory integration. Advanced implementations incorporate graphon-enhanced memory for sparse graph sequences, multi-modal cognitive persistent memory, and quantum-resistant asynchronous multi-domain trust protocols. The system enables efficient cross-agent collaboration, sophisticated knowledge sharing, and secure cross-domain operations while optimizing computational resources and maintaining strict privacy guarantees across distributed AI deployments.

A scalable platform for orchestrating networks of collaborative AI agents utilizing modular hybrid computing architecture. The platform integrates classical, quantum, and neuromorphic computing paradigms through hardware-accelerated translation layers and cross-paradigm coordination mechanisms. A central orchestration engine manages interactions between domain-specific AI agents, dynamically distributing workloads across heterogeneous computing cores based on task complexity, computational requirements, and resource availability. The platform employs hardware-accelerated translation between paradigms, enabling efficient cross-paradigm information exchange while maintaining semantic consistency and computation integrity across different architectures. Specialized monitoring and optimization systems continuously adjust resource allocation and fine-tune performance across computing paradigms. Advanced cache management and fault tolerance mechanisms ensure reliable operation, while privacy-preservation techniques enable secure collaboration. The platform's modular architecture supports integration of different computational approaches, enabling complex multi-domain problem solving that leverages the unique advantages of each paradigm while maintaining system-wide efficiency, scalability, and coherence.

The invention discloses an AI model automatic deployment platform based on a containerization technology, which relates to the technical field of automatic deployment of an artificial intelligence model, and comprises a container construction and intelligent configuration module, a transmission and cache management module, a security and multi-version warehouse module, a resource scheduling and optimization module and a deployment and interface management module, the container construction and intelligent configuration module adopts a three-layer mirror image construction strategy of a base layer, a framework layer and a model layer. According to the invention, all the modules cooperate to form a closed loop, after the container construction module generates an incremental packet and the incremental packet is subjected to security signature verification, the transmission module distributes the incremental packet according to network quality, the resource scheduling module dynamically adjusts bandwidth and quota, and the deployment module starts the container and monitors the container in real time. The problems of efficiency, safety and resource optimization of model deployment in an edge environment are solved, and cross-platform compatibility and service continuity are improved.

The invention discloses a memory access optimization method based on intelligent cache management, and relates to the technical field of computer storage. According to the method, spatial-temporal characteristics and semantic association data of memory access requests are collected in real time, a dynamic heat matrix is constructed, and a multi-dimensional access rule is fused to improve modeling precision. And inputting the dynamic popularity matrix into a hybrid prediction model, predicting a future access probability by using a time convolutional network, analyzing a competition relationship between data blocks through a graph attention network, generating a conflict pre-judgment weight and a corrected popularity ranking, and effectively reducing the cache jitter risk. On the basis of popularity ranking and conflict weight, a fragmented reinforcement learning algorithm is adopted to divide logic sub-regions, a differential reward function is designed to dynamically decide cache operation, and performance and energy efficiency requirements are balanced; and finally, through an online learning mechanism, combining real-time feedback to dynamically adjust a prediction model weight and strategy parameters, forming a closed-loop optimization link, and realizing adaptive stability under long-term load fluctuation.

The invention discloses a big data stream-oriented memory cache storage system, relates to the technical field of memory cache storage, and is used for solving the problem of poor memory cache storage management. According to the method, the access frequency, the timeliness and the service importance of the data are evaluated, the priority ranking model is constructed, the cache strategy is optimized based on the ranking result, the data storage and scheduling efficiency is improved, the write-in and read operation of the memory cache is adjusted in real time in combination with the flow prediction model, memory overload is effectively avoided, and the data storage and scheduling efficiency is improved. According to the method, non-emergency data writing is delayed by adopting a delayed writing and lazy loading strategy, data is loaded as required, meanwhile, a cache elimination mechanism is designed according to data priority, access frequency, survival time and size, data migration is carried out when a cache space is insufficient, reasonable resource allocation is ensured, and through a multi-level and dynamic cache management scheme, the cache management efficiency is improved. Therefore, the system performance, the resource utilization rate and the response speed are remarkably improved, the data processing efficiency is optimized, and cache conflicts are reduced.

The invention discloses an industrial scene-oriented data acquisition system and an industrial scene-oriented data acquisition method. According to the invention, through the multi-level dynamic optimization design, the data acquisition efficiency and the system reliability in a complex environment are significantly improved. The system dynamically allocates thread resources based on the real-time communication state of the equipment, realizes stable throughput under a high-concurrency scene in combination with core binding and polling scheduling strategies, and ensures that millisecond-level response is still maintained when 10000-level equipment is accessed. The memory preloading and protocol template technology eliminates jitter during operation, the hierarchical resource isolation mechanism provides deterministic guarantee for key control instructions, and interruption of the production process due to data delay or resource competition is avoided. The data flow delay is further reduced through multi-protocol efficient analysis and intelligent cache management, seamless compatibility of heterogeneous equipment in a hybrid networking scene is supported, and the strict requirements of continuous production industries such as steel and chemical engineering for real-time performance and stability are met.

The invention relates to the technical field of data storage and processing, in particular to an RAID card static cache management method and device based on data popularity, and the method comprises the steps that the physical position of a data block needing to be accessed in an SSD array is acquired according to a read-write request; historical data access information of the RAID card is collected, a historical data set is generated, and a neural network model is trained by using the historical data set to obtain a cache management model; predicting and outputting data blocks which are possibly accessed in the future and the access probability of the data blocks through the cache management model; the RAID controller dynamically adjusts a cache strategy in combination with an LRU strategy according to a prediction result output by the cache management model and a read-write request of a file system, and optimizes a storage position and an updating mechanism of a data block in a cache space; and when the hit rate or the data access efficiency does not reach the preset value, adjusting the parameters of the cache strategy and updating the cache strategy. According to the method, cache resources can be more effectively distributed, the cache hit rate is improved, the cache replacement overhead is reduced, and therefore the performance of a storage system is optimized.

A system for adaptively caching network communication protocols provides improved efficiency and performance through a multi-level cache architecture. The system monitors performance metrics and synchronizes cache contents across distributed nodes using a hierarchical structure. Protocol cache requirements are predicted based on usage patterns, enabling proactive cache management. The system compresses cached protocols using existing codebooks to optimize storage and transmission efficiency. Integration with event-driven data communication systems enables seamless protocol selection and translation. The caching system works in conjunction with transaction managers and protocol predictors to enhance network communication performance. The system maintains cache coherency across local, regional, and global cache levels while minimizing network overhead and optimizing protocol availability. By predicting and pre-caching likely needed protocols, the system reduces protocol negotiation latency and improves overall network communication efficiency.

A system for adaptively caching network communication protocols enhances efficiency across heterogeneous device environments through a multi-level cache architecture with device-capability-based tiers. The system collects endpoint telemetry data including device capabilities and operational constraints to classify endpoints and generate context-aware protocol variants optimized for specific device types. Protocol optimization opportunities are determined through structural analysis of message patterns and state transitions. The system performs protocol deduplication by identifying functionally equivalent variants and maintaining canonical representations to reduce cache redundancy. Cache synchronization across distributed nodes uses enhanced Merkle tree structures with protocol normalization processing. The system predicts communication needs based on historical patterns, network context, and endpoint constraints, enabling proactive cache management tailored to device capabilities. Integration with event-driven data communication systems enables seamless protocol selection and translation while maintaining compatibility between diverse endpoint types, from high-performance servers to resource-constrained IoT devices.

The invention discloses a hybrid expert model reasoning method based on cooperation of a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), and belongs to the field of deep learning. According to the method, a CPU-GPU computing framework of a hybrid expert model is constructed, heterogeneous computing resource loads are effectively balanced, and the hardware utilization rate is remarkably increased; an intelligent cache management mechanism based on dynamic priority scores is provided, high-demand experts are reserved preferentially, and the transmission overhead caused by cache missing is reduced; through pipeline parallel design for separating calculation and transmission tasks, CPU calculation and PCIe transmission are overlapped in the GPU execution period, and delay is effectively hidden. In addition, in combination with a multi-layer expert activation prediction prospective prefetching mechanism, the expert cache hit rate is improved. The method is compatible with hybrid expert models of different scales and structures, and stable and efficient reasoning acceleration is realized on a resource-limited heterogeneous platform.

Techniques for cache management for reducing latency in LLM inferencing are described. In some embodiments, a system caches encoded data of portions of a prompt so that the encoded data is available for use by the LLM across dialog turns of a dialog session. Within a dialog session, a portion of the LLM prompt may be the same across dialog turns, and instead of recomputing the attention / encodings for such portions, the cached encodings can be used by the LLM during processing. In some embodiments, user inputs for the dialog session may be routed to the same LLM container and encoded data for the dialog session may be stored at the same cache associated with the LLM. In some embodiments, the system enables asynchronous prompt encoding while performing ASR processing.

The invention provides a cache management system during large model reasoning, and relates to the technical field of large-scale artificial intelligence reasoning optimization. The system comprises a cache module, a cache manager, a retrieval enhancement generation controller, a reasoning time controller, a large language model reasoning engine, a process reward model reasoning engine, a result reward model reasoning engine, a Monte Carlo tree search scheduler and a runtime scheduler, the cache module is used for caching required key value pair data in a pre-filling stage, and the cache manager is used for storing the key value pair data in the pre-filling stage. The key-value pair data is used for storing context information of the preorder lexical elements, and the key-value pair data is also used for being reused in the decoding stage. According to the embodiment of the invention, in the pre-filling stage, the required key value pair data is generated and cached, and the cached key value pair data is used for storing the context information of the preorder lexical elements. In the decoding stage, the caches are reused, repeated calculation is avoided, and therefore the GPU memory is saved.

A key-value cache management system processes inference requests of a generative model. An inference request requests a starting virtual memory address and a number of layers in the generative model. In response to receiving an inference request, contiguous virtual memory space is reserved for the key-value cache in accordance with the inference request by assigning the starting virtual memory address in the key-value cache and calculating memory pointers for each layer in the model and each block so that the one or more blocks are written sequentially. The generated memory pointers are outputted to the generative model so that each self-attention layer writes computed key-value pairs at physical addresses in the key-value cache specified by the memory pointers.

The invention relates to the technical field of page rendering, and discloses a progressive page rendering method and system based on streaming data, and the method comprises the steps: carrying out the data segmentation through obtaining a data stream, user interaction data and equipment performance parameters, and obtaining data blocks; then, performing cache management in combination with the data, and constructing a multi-level cache pool; thirdly, performing priority grading and sorting on the data blocks to form a rendering task queue; and according to the equipment performance parameters, optimizing a task sequence and obtaining an optimized task sequence. Next, combining the optimized task sequence and user interaction data, predicting data about to enter a viewport, and generating a viewport pre-rendering task; and finally, according to the viewport pre-rendering task, the multi-level cache pool and the equipment performance parameters, performing rendering strategy optimization to obtain a dynamically adjusted rendering task flow. The method can realize dynamic resource scheduling.

The invention discloses a solid state disk pre-reading method and device based on host I / O software and a medium. According to the method, through deep collaboration between the host I / O software and the solid state disk, multi-dimensional context information such as application program running, user operation and system resources is accurately perceived, and then the pre-reading strategy of the solid state disk is dynamically optimized. Various kinds of context information are collected through host I / O software and then transmitted to the solid state disk, the solid state disk analyzes and processes the context information by means of a machine learning technology so as to predict subsequent pre-access related parameters, so that a pre-reading strategy is correspondingly and dynamically adjusted, a cache management mechanism is matched, pre-read data better meet actual requirements, and the pre-read data are more accurate. The hit rate of pre-reading is improved, the overall data reading efficiency is improved, and the data reading speed is remarkably increased.

The invention provides a cache management method, a cache management device, a computer storage medium and electronic equipment, and relates to the technical field of computers. The method comprises the steps of receiving a reasoning task request and distributing the reasoning task request to a target storage page; key value cache information of the first round of reasoning task is stored in a hard disk cache, and when the second round of reasoning task is executed, key value cache information generated before the second round of reasoning task is preloaded layer by layer from the hard disk cache; when the last round of reasoning task is received, storing first target key value cache information correspondingly generated by the last round of reasoning task into the matched target physical block; and performing hybrid grouping compression on key cache information and value cache information in the first target key value cache information to obtain second target key value cache information after quantization compression. According to the invention, triple balance of video memory-calculation performance-precision can be realized.

The invention provides a sentence semantic perception-based data management method and related equipment, and relates to the technical field of cache management, and the method comprises the following steps: segmenting a long text to obtain a plurality of sentences and a sorting sequence of the sentences; calculating importance scores of the tokens corresponding to the sentences based on the sentences and the sorting sequence of the sentences; key tokens in the sentences are screened out according to the importance scores of the tokens corresponding to the sentences and the dynamic budget corresponding to the sentences, the key tokens are stored in a reserved set of the corresponding sentences, and then the local weighted attention score of each token is calculated; performing weighted fusion based on the local weighted attention score of each token in the sentence and the fusion weight of a preset attention head to obtain a sentence-level semantic vector; and constructing semantic index cache and key value pair (KV) content cache based on the sentence-level semantic vectors of the sentences and the corresponding reserved sets. Therefore, in combination with a KV cache management method of context-dependent dynamic loading, a more efficient and more intelligent reasoning process is realized.

The invention discloses a cache management method and device for memory access, equipment and a storage medium, and relates to the technical field of storage, and the method comprises the following steps: obtaining a data read-write message of a central processing unit; detecting a data popularity type of an access address of the data read-write message; searching cache data corresponding to the access address from a target cache region; if the cache data is not found in the target cache region, sending a corresponding read-write request to a memory so as to find memory data corresponding to the access address from the memory; by setting the preset cache regions corresponding to the cold data and the hot data respectively, the cache can be partitioned according to the heat of the data, so that cache resources are reasonably allocated, the cost is reduced, the memory access efficiency and speed are improved, finer consistency control can be realized, and the cache hit rate and the system performance are improved; therefore, the required data is searched in the corresponding preset cache region according to the heat of the data, and the memory access speed is increased.

The invention relates to the technical field of audio optimization, and discloses an audio data offset detection and compensation method and system, a storage medium and equipment, and the method comprises the steps: building a timestamp synchronization mechanism with nanosecond precision, and distributing a unique time identifier for each floating-point audio sampling point; detecting time sequence offsets of input and output signals through a self-adaptive sliding window, and positioning and determining an offset value based on a cross-correlation function extreme value; a compensation strategy is dynamically selected according to different intervals where the offset absolute value is located, and meanwhile a dynamic cache management mechanism is established to support real-time offset compensation operation; constructing a time sequence relation model through inter-channel signal correlation analysis in the multi-channel system, and implementing coordinated offset compensation based on a correlation analysis result and the compensation strategy; a performance monitoring index system is established to evaluate the offset compensation effect in real time, compensation parameters are adaptively adjusted according to audio content characteristics and system performance feedback, and accurate time sequence control over audio data is achieved through the method.

The invention discloses an SSD (Solid State Disk) cache management method, which relates to the technical field of SSDs, and comprises the following steps: step 1, cache space division: the cache space of the SSD is divided into a plurality of different types of sub-cache regions, and each sub-cache region at least comprises a hot data cache region, a cold data cache region and an intermediate data cache region; 2, data popularity monitoring: monitoring the access frequency of the data stored in the SSD in real time, judging the popularity level of the data according to the access frequency, judging the data with the access frequency higher than a set threshold as hot data, judging the data with the access frequency lower than another set threshold as cold data, and judging the data between the hot data and the cold data as intermediate data; 3, data cache allocation: according to the heat level of the data, storing hot data into a hot data cache region, storing cold data into a cold data cache region, and storing intermediate data into an intermediate data cache region; the cache utilization rate and the read-write performance are comprehensively improved, and the delay is reduced.

The invention discloses a method for improving long text processing efficiency and accuracy, and relates to the technical field of natural language processing and large language models.According to the method, text word segmentation embedding, sliding block preprocessing, YaRN position code injection, dynamic sparse attention calculation, multi-level attention fusion, graded KV cache management and output generation are sequentially executed; position drift is inhibited through logarithmic scaling, and key contexts are adaptively screened according to the attention activeness, so that the attention calculation complexity is close to linearity; in million-level Token reasoning, the video memory occupation of the method is reduced, the remote dependency recall rate is improved, and the method is suitable for scenes such as document analysis, code auditing and multi-mode streaming understanding.

The invention discloses a multi-modal data sensing processing method and system for a sound intelligent film, and relates to the technical field of data processing. By constructing the modal state transition diagram and the spatial position matrix, the abnormal state and the interference area can be accurately identified, and the processing strategy is dynamically adjusted, so that the response speed and the processing flexibility of the system to the abnormal modal data are effectively improved. Particularly, under the condition that interference data is frequently transmitted or bandwidth resources are limited, data transmission and calculation burdens are greatly reduced through a modal compression and delay decoding mechanism, and continuity and stability of a main processing stream are guaranteed. Meanwhile, the method has high adaptive capacity, cache management and path priority can be dynamically adjusted according to real-time state changes, and data congestion and delay diffusion are effectively prevented.

Techniques for cache management for LLM processing are described. Example embodiments include a signal hashing model that generates a key for particular context data. An LLM output corresponding to the context data is stored in a cache along with the key. For a user input received by the system, a cache lookup is performed using a key for context data corresponding to the received user input. For a cache hit, the stored output is used to respond to the user input. For a cache miss, a LLM processes the context data and the user input to generate an output within a first timeout. If the LLM is unable to generate an output within the first timeout, then in some cases, the LLM is allowed to continue processing until a second timeout, and a final or partial output from the LLM is stored in the cache.

The invention provides a multi-tenant large model reasoning service architecture and an operation method thereof, and the architecture comprises a multi-level cache module which is used for managing the storage of a LoRA model in a video memory, an internal memory, a disk and a remote memory according to the priority; the route registration module is used for recording the model cache position and the load state of each server; the dynamic routing module is used for allocating requests according to the tenant identifiers and the server states; and the parameter fusion module is used for superposing the LoRA model and the basic model to generate a personalized inference model. According to the method, efficient resource utilization and low-delay service are realized through multi-stage cache management, a dynamic routing strategy and a load balancing mechanism. The framework supports on-demand loading and parameter fusion of the tenant personalized LoRA model, and the reasoning service efficiency in a multi-tenant scene is remarkably improved.

The invention relates to the related technical field of information transmission processing, in particular to a docking station adaptive data transmission optimization system and method, and the method comprises the steps: multi-module collaborative optimization docking station transmission: dynamically collecting port data, intelligently analyzing a configuration strategy, correcting cache congestion in real time, generating a control instruction, and feeding back an adjustment protocol; the technical problems that the bandwidth allocation strategy cannot adapt to the connection state of the port equipment and the sudden change of the data transmission requirement, the power distribution of the power supply is not matched with the equipment load, and the stability and reliability of the data transmission are limited are solved, the bandwidth allocation proportion of each port is dynamically adjusted, the bandwidth waste and congestion phenomena are reduced, and the data transmission efficiency is improved. The technical effects of improving the smoothness and the real-time performance of data transmission, improving the energy efficiency by cooperatively adjusting the distribution of power supply power based on a power supply power distribution strategy of an equipment compatibility thermodynamic diagram, reducing data packet loss and transmission errors by adopting a congestion risk control and cache management strategy, and improving the stability and the reliability of data transmission are achieved.

The invention discloses a dynamic script compiling and hot updating system based on Grouping and an implementation method of the dynamic script compiling and hot updating system. The system comprises a script source code management module, a dynamic compiling engine, an intelligent cache manager, an exception handling and recovery system and a hot update controller. And the script source code management module receives and preprocesses the Grouping script to generate a unique identifier for subsequent management. And the dynamic compilation engine executes compilation operation according to the identifier to ensure that a compilation result accords with a specification. And the intelligent cache manager efficiently stores and retrieves the compiling result, so that the repeated compiling cost is reduced. And the exception handling and recovery system monitors the execution state in real time, and automatically starts a recovery process in case of exception to guarantee stable operation of the system. The hot update controller realizes seamless update and version switching of scripts, and ensures service continuity. Through cooperative work of the modules, high-reliability, high-performance and zero-shutdown deployment is realized.

A metadata access method, apparatus, and computer-readable storage medium for efficient metadata retrieval through cache management. The method receives metadata query requests including target index information from processes and performs matching operations on a global cache file containing records with index information and slot identifiers. Each cache description array corresponds to memory blocks caching metadata. Upon successful matching, the target cache description array is accessed using the target slot identifier. The data state of target metadata is determined from the cache description array, and target address information indicating the location of the target memory block in shared memory is obtained and returned to the requesting process, enabling efficient shared memory-based metadata access.

Implementations of this specification provide key-value cache management, model reasoning, and data processing methods and apparatuses for large language models. In an implementation, a method comprises allocating a virtual memory block in a virtual address slot to newly-added token key-value data of a model reasoning request, in response to determining that a scheduling result of the model reasoning request indicates the model reasoning request is scheduled for execution, maintaining a mapping relationship between an occupied virtual address slot and a physical graphics memory block allocated to the model reasoning request, and copying the newly-added token key-value data to the physical graphics memory block.

The invention discloses a multi-model dynamic image analysis system, and relates to the technical field of artificial intelligence. The system adopts a heterogeneous computing architecture and comprises a preprocessing model, a multi-modal analysis model, an anisotropic correction model, a large language model, a task knowledge base embedding model and a KV cache management system. An incongruous task reconstruction mechanism based on hardware isomerism is constructed, a large language reasoning model is nested for analysis, and efficient reuse of computing resources is achieved by using a cross-model KV cache optimization technology. The system is provided with multiple graphics cards and a double-channel NUMA, and a multi-mode large language model carries out unsupervised or weak supervised training on a target detection model in the operation of the system. According to the system, through a dynamic load scheduling strategy, a mechanism for reasoning by using a processor and a memory enhanced display card in a full-load mode is innovatively designed, and three-section real-time streaming enhanced output is achieved. According to the system, through a dynamic load scheduling strategy, a mechanism for reasoning by using a processor and a memory enhanced graphics card in a full-load mode is innovatively designed.