836 results about "Cache management" patented technology

A system and method for implementing a convergent intelligence fabric (CIF) for distributed artificial intelligence operations. The CIF architecture integrates tensor-theoretic foundations, probabilistic cache management, precision-aware memory operations, quantum-resistant security, and neural-based optimization within a unified framework. The system orchestrates asynchronous, multi-hop data flow among computational resources while maintaining data security through per-block encryption and identity-based access control. Key components include a universal multi-model KV cache subsystem, agent-parallel disaggregation pipelines, reinforcement learning-based orchestration, and neuromorphic memory integration. Advanced implementations incorporate graphon-enhanced memory for sparse graph sequences, multi-modal cognitive persistent memory, and quantum-resistant asynchronous multi-domain trust protocols. The system enables efficient cross-agent collaboration, sophisticated knowledge sharing, and secure cross-domain operations while optimizing computational resources and maintaining strict privacy guarantees across distributed AI deployments.

A scalable platform for orchestrating networks of collaborative AI agents utilizing modular hybrid computing architecture. The platform integrates classical, quantum, and neuromorphic computing paradigms through hardware-accelerated translation layers and cross-paradigm coordination mechanisms. A central orchestration engine manages interactions between domain-specific AI agents, dynamically distributing workloads across heterogeneous computing cores based on task complexity, computational requirements, and resource availability. The platform employs hardware-accelerated translation between paradigms, enabling efficient cross-paradigm information exchange while maintaining semantic consistency and computation integrity across different architectures. Specialized monitoring and optimization systems continuously adjust resource allocation and fine-tune performance across computing paradigms. Advanced cache management and fault tolerance mechanisms ensure reliable operation, while privacy-preservation techniques enable secure collaboration. The platform's modular architecture supports integration of different computational approaches, enabling complex multi-domain problem solving that leverages the unique advantages of each paradigm while maintaining system-wide efficiency, scalability, and coherence.

The invention discloses an AI model automatic deployment platform based on a containerization technology, which relates to the technical field of automatic deployment of an artificial intelligence model, and comprises a container construction and intelligent configuration module, a transmission and cache management module, a security and multi-version warehouse module, a resource scheduling and optimization module and a deployment and interface management module, the container construction and intelligent configuration module adopts a three-layer mirror image construction strategy of a base layer, a framework layer and a model layer. According to the invention, all the modules cooperate to form a closed loop, after the container construction module generates an incremental packet and the incremental packet is subjected to security signature verification, the transmission module distributes the incremental packet according to network quality, the resource scheduling module dynamically adjusts bandwidth and quota, and the deployment module starts the container and monitors the container in real time. The problems of efficiency, safety and resource optimization of model deployment in an edge environment are solved, and cross-platform compatibility and service continuity are improved.

The invention discloses a memory access optimization method based on intelligent cache management, and relates to the technical field of computer storage. According to the method, spatial-temporal characteristics and semantic association data of memory access requests are collected in real time, a dynamic heat matrix is constructed, and a multi-dimensional access rule is fused to improve modeling precision. And inputting the dynamic popularity matrix into a hybrid prediction model, predicting a future access probability by using a time convolutional network, analyzing a competition relationship between data blocks through a graph attention network, generating a conflict pre-judgment weight and a corrected popularity ranking, and effectively reducing the cache jitter risk. On the basis of popularity ranking and conflict weight, a fragmented reinforcement learning algorithm is adopted to divide logic sub-regions, a differential reward function is designed to dynamically decide cache operation, and performance and energy efficiency requirements are balanced; and finally, through an online learning mechanism, combining real-time feedback to dynamically adjust a prediction model weight and strategy parameters, forming a closed-loop optimization link, and realizing adaptive stability under long-term load fluctuation.

The invention discloses a big data stream-oriented memory cache storage system, relates to the technical field of memory cache storage, and is used for solving the problem of poor memory cache storage management. According to the method, the access frequency, the timeliness and the service importance of the data are evaluated, the priority ranking model is constructed, the cache strategy is optimized based on the ranking result, the data storage and scheduling efficiency is improved, the write-in and read operation of the memory cache is adjusted in real time in combination with the flow prediction model, memory overload is effectively avoided, and the data storage and scheduling efficiency is improved. According to the method, non-emergency data writing is delayed by adopting a delayed writing and lazy loading strategy, data is loaded as required, meanwhile, a cache elimination mechanism is designed according to data priority, access frequency, survival time and size, data migration is carried out when a cache space is insufficient, reasonable resource allocation is ensured, and through a multi-level and dynamic cache management scheme, the cache management efficiency is improved. Therefore, the system performance, the resource utilization rate and the response speed are remarkably improved, the data processing efficiency is optimized, and cache conflicts are reduced.

The invention discloses an industrial scene-oriented data acquisition system and an industrial scene-oriented data acquisition method. According to the invention, through the multi-level dynamic optimization design, the data acquisition efficiency and the system reliability in a complex environment are significantly improved. The system dynamically allocates thread resources based on the real-time communication state of the equipment, realizes stable throughput under a high-concurrency scene in combination with core binding and polling scheduling strategies, and ensures that millisecond-level response is still maintained when 10000-level equipment is accessed. The memory preloading and protocol template technology eliminates jitter during operation, the hierarchical resource isolation mechanism provides deterministic guarantee for key control instructions, and interruption of the production process due to data delay or resource competition is avoided. The data flow delay is further reduced through multi-protocol efficient analysis and intelligent cache management, seamless compatibility of heterogeneous equipment in a hybrid networking scene is supported, and the strict requirements of continuous production industries such as steel and chemical engineering for real-time performance and stability are met.

The invention relates to the technical field of data storage and processing, in particular to an RAID card static cache management method and device based on data popularity, and the method comprises the steps that the physical position of a data block needing to be accessed in an SSD array is acquired according to a read-write request; historical data access information of the RAID card is collected, a historical data set is generated, and a neural network model is trained by using the historical data set to obtain a cache management model; predicting and outputting data blocks which are possibly accessed in the future and the access probability of the data blocks through the cache management model; the RAID controller dynamically adjusts a cache strategy in combination with an LRU strategy according to a prediction result output by the cache management model and a read-write request of a file system, and optimizes a storage position and an updating mechanism of a data block in a cache space; and when the hit rate or the data access efficiency does not reach the preset value, adjusting the parameters of the cache strategy and updating the cache strategy. According to the method, cache resources can be more effectively distributed, the cache hit rate is improved, the cache replacement overhead is reduced, and therefore the performance of a storage system is optimized.

A system for adaptively caching network communication protocols provides improved efficiency and performance through a multi-level cache architecture. The system monitors performance metrics and synchronizes cache contents across distributed nodes using a hierarchical structure. Protocol cache requirements are predicted based on usage patterns, enabling proactive cache management. The system compresses cached protocols using existing codebooks to optimize storage and transmission efficiency. Integration with event-driven data communication systems enables seamless protocol selection and translation. The caching system works in conjunction with transaction managers and protocol predictors to enhance network communication performance. The system maintains cache coherency across local, regional, and global cache levels while minimizing network overhead and optimizing protocol availability. By predicting and pre-caching likely needed protocols, the system reduces protocol negotiation latency and improves overall network communication efficiency.

A system for adaptively caching network communication protocols enhances efficiency across heterogeneous device environments through a multi-level cache architecture with device-capability-based tiers. The system collects endpoint telemetry data including device capabilities and operational constraints to classify endpoints and generate context-aware protocol variants optimized for specific device types. Protocol optimization opportunities are determined through structural analysis of message patterns and state transitions. The system performs protocol deduplication by identifying functionally equivalent variants and maintaining canonical representations to reduce cache redundancy. Cache synchronization across distributed nodes uses enhanced Merkle tree structures with protocol normalization processing. The system predicts communication needs based on historical patterns, network context, and endpoint constraints, enabling proactive cache management tailored to device capabilities. Integration with event-driven data communication systems enables seamless protocol selection and translation while maintaining compatibility between diverse endpoint types, from high-performance servers to resource-constrained IoT devices.

The invention discloses a hybrid expert model reasoning method based on cooperation of a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), and belongs to the field of deep learning. According to the method, a CPU-GPU computing framework of a hybrid expert model is constructed, heterogeneous computing resource loads are effectively balanced, and the hardware utilization rate is remarkably increased; an intelligent cache management mechanism based on dynamic priority scores is provided, high-demand experts are reserved preferentially, and the transmission overhead caused by cache missing is reduced; through pipeline parallel design for separating calculation and transmission tasks, CPU calculation and PCIe transmission are overlapped in the GPU execution period, and delay is effectively hidden. In addition, in combination with a multi-layer expert activation prediction prospective prefetching mechanism, the expert cache hit rate is improved. The method is compatible with hybrid expert models of different scales and structures, and stable and efficient reasoning acceleration is realized on a resource-limited heterogeneous platform.

Techniques for cache management for reducing latency in LLM inferencing are described. In some embodiments, a system caches encoded data of portions of a prompt so that the encoded data is available for use by the LLM across dialog turns of a dialog session. Within a dialog session, a portion of the LLM prompt may be the same across dialog turns, and instead of recomputing the attention / encodings for such portions, the cached encodings can be used by the LLM during processing. In some embodiments, user inputs for the dialog session may be routed to the same LLM container and encoded data for the dialog session may be stored at the same cache associated with the LLM. In some embodiments, the system enables asynchronous prompt encoding while performing ASR processing.

The invention provides a cache management system during large model reasoning, and relates to the technical field of large-scale artificial intelligence reasoning optimization. The system comprises a cache module, a cache manager, a retrieval enhancement generation controller, a reasoning time controller, a large language model reasoning engine, a process reward model reasoning engine, a result reward model reasoning engine, a Monte Carlo tree search scheduler and a runtime scheduler, the cache module is used for caching required key value pair data in a pre-filling stage, and the cache manager is used for storing the key value pair data in the pre-filling stage. The key-value pair data is used for storing context information of the preorder lexical elements, and the key-value pair data is also used for being reused in the decoding stage. According to the embodiment of the invention, in the pre-filling stage, the required key value pair data is generated and cached, and the cached key value pair data is used for storing the context information of the preorder lexical elements. In the decoding stage, the caches are reused, repeated calculation is avoided, and therefore the GPU memory is saved.

A key-value cache management system processes inference requests of a generative model. An inference request requests a starting virtual memory address and a number of layers in the generative model. In response to receiving an inference request, contiguous virtual memory space is reserved for the key-value cache in accordance with the inference request by assigning the starting virtual memory address in the key-value cache and calculating memory pointers for each layer in the model and each block so that the one or more blocks are written sequentially. The generated memory pointers are outputted to the generative model so that each self-attention layer writes computed key-value pairs at physical addresses in the key-value cache specified by the memory pointers.

The invention relates to the technical field of page rendering, and discloses a progressive page rendering method and system based on streaming data, and the method comprises the steps: carrying out the data segmentation through obtaining a data stream, user interaction data and equipment performance parameters, and obtaining data blocks; then, performing cache management in combination with the data, and constructing a multi-level cache pool; thirdly, performing priority grading and sorting on the data blocks to form a rendering task queue; and according to the equipment performance parameters, optimizing a task sequence and obtaining an optimized task sequence. Next, combining the optimized task sequence and user interaction data, predicting data about to enter a viewport, and generating a viewport pre-rendering task; and finally, according to the viewport pre-rendering task, the multi-level cache pool and the equipment performance parameters, performing rendering strategy optimization to obtain a dynamically adjusted rendering task flow. The method can realize dynamic resource scheduling.

The invention provides a front-end performance optimization method based on cache management, and the method comprises the steps: S1, collecting and processing user behavior data, and obtaining target data; s2, analyzing a user behavior based on the target data, constructing a user behavior model, and predicting an access demand of a user by using a prediction algorithm to obtain a prediction model; s3, generating a caching strategy based on the user behavior model and the prediction model; and S4, storing resources on the client, the server and the content distribution network node based on the caching strategy. The cache hit rate is improved; the caching strategy is dynamically adjusted according to the actual use condition of the user, and the user requirement is more accurately met; the cache preloading mechanism can shorten the response time of the user request, can adapt to different user behaviors, and improves the user experience. The system and the electronic equipment provided by the invention have corresponding advantages.

The invention relates to the technical field of data transmission and updating, in particular to a data transmission and updating system and method of a cloud mobile phone. According to the technical scheme, the system comprises an intelligent network monitoring module, a dynamic bandwidth allocation module, a server load balancing module, an intelligent data synchronization module and a cache management module, and the intelligent network monitoring module monitors network conditions, including network bandwidth, network congestion degree and network stability, between a user terminal and a cloud server in real time. Through the synergistic effect of the intelligent network monitoring module, the dynamic bandwidth allocation module, the server load balancing module, the intelligent data synchronization module, the cache management module and the like, the network condition is accurately monitored, the bandwidth is reasonably allocated, the server load is optimized, the data synchronization and completeness are ensured, and the cache efficiency is improved; stable and reliable network connection, preferential key data transmission and rapid data synchronization and loading are brought to users, and the use experience of the cloud mobile phone and the overall performance of the system are greatly improved.

The invention discloses a solid state disk pre-reading method and device based on host I / O software and a medium. According to the method, through deep collaboration between the host I / O software and the solid state disk, multi-dimensional context information such as application program running, user operation and system resources is accurately perceived, and then the pre-reading strategy of the solid state disk is dynamically optimized. Various kinds of context information are collected through host I / O software and then transmitted to the solid state disk, the solid state disk analyzes and processes the context information by means of a machine learning technology so as to predict subsequent pre-access related parameters, so that a pre-reading strategy is correspondingly and dynamically adjusted, a cache management mechanism is matched, pre-read data better meet actual requirements, and the pre-read data are more accurate. The hit rate of pre-reading is improved, the overall data reading efficiency is improved, and the data reading speed is remarkably increased.

The invention provides a cache management method, a cache management device, a computer storage medium and electronic equipment, and relates to the technical field of computers. The method comprises the steps of receiving a reasoning task request and distributing the reasoning task request to a target storage page; key value cache information of the first round of reasoning task is stored in a hard disk cache, and when the second round of reasoning task is executed, key value cache information generated before the second round of reasoning task is preloaded layer by layer from the hard disk cache; when the last round of reasoning task is received, storing first target key value cache information correspondingly generated by the last round of reasoning task into the matched target physical block; and performing hybrid grouping compression on key cache information and value cache information in the first target key value cache information to obtain second target key value cache information after quantization compression. According to the invention, triple balance of video memory-calculation performance-precision can be realized.

The invention provides a sentence semantic perception-based data management method and related equipment, and relates to the technical field of cache management, and the method comprises the following steps: segmenting a long text to obtain a plurality of sentences and a sorting sequence of the sentences; calculating importance scores of the tokens corresponding to the sentences based on the sentences and the sorting sequence of the sentences; key tokens in the sentences are screened out according to the importance scores of the tokens corresponding to the sentences and the dynamic budget corresponding to the sentences, the key tokens are stored in a reserved set of the corresponding sentences, and then the local weighted attention score of each token is calculated; performing weighted fusion based on the local weighted attention score of each token in the sentence and the fusion weight of a preset attention head to obtain a sentence-level semantic vector; and constructing semantic index cache and key value pair (KV) content cache based on the sentence-level semantic vectors of the sentences and the corresponding reserved sets. Therefore, in combination with a KV cache management method of context-dependent dynamic loading, a more efficient and more intelligent reasoning process is realized.

The invention discloses a big language model-oriented inference operation acceleration method for processing an extremely long text, which belongs to the technical field of large-scale pre-training language model inference optimization, and specifically comprises the following steps of: selecting a big language model, starting a sparse attention mode, inputting different texts, recording execution time of each attention head and an attention mode type in different layers, and calculating the inference operation speed of the large language model; establishing a statistical database; solving a multi-head attention load balanced distribution strategy of the large language model under the actual text; establishing a weight index table by splitting the weight matrix; retrieving a weight sub-matrix corresponding to each attention head, and loading the weight sub-matrix to a corresponding GPU device; by asynchronously preloading the MHA calculation weight and the MLP calculation weight of the adjacent layers and combining KV cache management, load balancing in the reasoning process is realized. According to the method, dynamic segmentation, recombination and scheduling of the attention head are carried out in the reasoning stage, so that the problems of multi-GPU load imbalance and resource idling are effectively avoided, and the system throughput of long sequence processing is remarkably improved.

The invention discloses a cache management method and device for memory access, equipment and a storage medium, and relates to the technical field of storage, and the method comprises the following steps: obtaining a data read-write message of a central processing unit; detecting a data popularity type of an access address of the data read-write message; searching cache data corresponding to the access address from a target cache region; if the cache data is not found in the target cache region, sending a corresponding read-write request to a memory so as to find memory data corresponding to the access address from the memory; by setting the preset cache regions corresponding to the cold data and the hot data respectively, the cache can be partitioned according to the heat of the data, so that cache resources are reasonably allocated, the cost is reduced, the memory access efficiency and speed are improved, finer consistency control can be realized, and the cache hit rate and the system performance are improved; therefore, the required data is searched in the corresponding preset cache region according to the heat of the data, and the memory access speed is increased.

The invention relates to the technical field of audio optimization, and discloses an audio data offset detection and compensation method and system, a storage medium and equipment, and the method comprises the steps: building a timestamp synchronization mechanism with nanosecond precision, and distributing a unique time identifier for each floating-point audio sampling point; detecting time sequence offsets of input and output signals through a self-adaptive sliding window, and positioning and determining an offset value based on a cross-correlation function extreme value; a compensation strategy is dynamically selected according to different intervals where the offset absolute value is located, and meanwhile a dynamic cache management mechanism is established to support real-time offset compensation operation; constructing a time sequence relation model through inter-channel signal correlation analysis in the multi-channel system, and implementing coordinated offset compensation based on a correlation analysis result and the compensation strategy; a performance monitoring index system is established to evaluate the offset compensation effect in real time, compensation parameters are adaptively adjusted according to audio content characteristics and system performance feedback, and accurate time sequence control over audio data is achieved through the method.

The present invention relates to a hardware acceleration method, device, equipment and medium. The method comprises: obtaining a read-write command issued by a host; parsing the read-write command to obtain a parsing command; judging whether the parsing command is a read command; if it is a read command, judging whether the parsing command is a sequential read command; if it is a sequential read command, applying for a buffer from a read cache management engine, and constructing a pre-read command, carrying a buffer address and an LBA address; according to the pre-read command, caching data in a read cache buffer; submitting the parsing command to the read cache management engine for processing; judging whether the parsing command hits the cache data of the current read cache buffer; if it hits, directly returning the corresponding data from the read cache buffer to the host, and marking the command completion. The present invention improves the pertinence and efficiency of processing, shortens the response time of commands and the transmission delay of data, and improves the overall performance of the system.

The invention discloses an SSD (Solid State Disk) cache management method, which relates to the technical field of SSDs, and comprises the following steps: step 1, cache space division: the cache space of the SSD is divided into a plurality of different types of sub-cache regions, and each sub-cache region at least comprises a hot data cache region, a cold data cache region and an intermediate data cache region; 2, data popularity monitoring: monitoring the access frequency of the data stored in the SSD in real time, judging the popularity level of the data according to the access frequency, judging the data with the access frequency higher than a set threshold as hot data, judging the data with the access frequency lower than another set threshold as cold data, and judging the data between the hot data and the cold data as intermediate data; 3, data cache allocation: according to the heat level of the data, storing hot data into a hot data cache region, storing cold data into a cold data cache region, and storing intermediate data into an intermediate data cache region; the cache utilization rate and the read-write performance are comprehensively improved, and the delay is reduced.

The invention discloses a method for improving long text processing efficiency and accuracy, and relates to the technical field of natural language processing and large language models.According to the method, text word segmentation embedding, sliding block preprocessing, YaRN position code injection, dynamic sparse attention calculation, multi-level attention fusion, graded KV cache management and output generation are sequentially executed; position drift is inhibited through logarithmic scaling, and key contexts are adaptively screened according to the attention activeness, so that the attention calculation complexity is close to linearity; in million-level Token reasoning, the video memory occupation of the method is reduced, the remote dependency recall rate is improved, and the method is suitable for scenes such as document analysis, code auditing and multi-mode streaming understanding.

The invention discloses a multi-modal data sensing processing method and system for a sound intelligent film, and relates to the technical field of data processing. By constructing the modal state transition diagram and the spatial position matrix, the abnormal state and the interference area can be accurately identified, and the processing strategy is dynamically adjusted, so that the response speed and the processing flexibility of the system to the abnormal modal data are effectively improved. Particularly, under the condition that interference data is frequently transmitted or bandwidth resources are limited, data transmission and calculation burdens are greatly reduced through a modal compression and delay decoding mechanism, and continuity and stability of a main processing stream are guaranteed. Meanwhile, the method has high adaptive capacity, cache management and path priority can be dynamically adjusted according to real-time state changes, and data congestion and delay diffusion are effectively prevented.

The invention discloses an operation and maintenance platform data acquisition system based on big data, and relates to the technical field of big data, and the system comprises the steps: capturing a data packet from a network flow of an operation and maintenance platform, and storing the data packet in a buffer area; performing preliminary screening on the data packets in the buffer area, and allocating priorities for the data packets based on a dynamic weight scheduling mechanism to complete preliminary preprocessing of the data packets; based on a circular sorting and block optimization technology, distributing the plurality of decomposed analysis tasks to a plurality of threads for parallel processing; formatting the analysis result, and performing batch parallel processing according to the data scale of the analysis result; formatting the analyzed data, writing the formatted data into a time sequence database according to a time sequence, and generating an index based on the tag field; through a dynamic cache management mechanism, the cache size is dynamically adjusted according to the throughput and the maximum delay of the network flow, and the problems of data loss and processing delay are avoided.

Techniques for cache management for LLM processing are described. Example embodiments include a signal hashing model that generates a key for particular context data. An LLM output corresponding to the context data is stored in a cache along with the key. For a user input received by the system, a cache lookup is performed using a key for context data corresponding to the received user input. For a cache hit, the stored output is used to respond to the user input. For a cache miss, a LLM processes the context data and the user input to generate an output within a first timeout. If the LLM is unable to generate an output within the first timeout, then in some cases, the LLM is allowed to continue processing until a second timeout, and a final or partial output from the LLM is stored in the cache.

The invention provides a multi-tenant large model reasoning service architecture and an operation method thereof, and the architecture comprises a multi-level cache module which is used for managing the storage of a LoRA model in a video memory, an internal memory, a disk and a remote memory according to the priority; the route registration module is used for recording the model cache position and the load state of each server; the dynamic routing module is used for allocating requests according to the tenant identifiers and the server states; and the parameter fusion module is used for superposing the LoRA model and the basic model to generate a personalized inference model. According to the method, efficient resource utilization and low-delay service are realized through multi-stage cache management, a dynamic routing strategy and a load balancing mechanism. The framework supports on-demand loading and parameter fusion of the tenant personalized LoRA model, and the reasoning service efficiency in a multi-tenant scene is remarkably improved.

The invention discloses an extensible content interactive display method and system based on streaming media playing, the extensible content interactive display method based on streaming media playing belongs to the field of streaming media, and the method comprises the following steps: S1, dynamically adjusting the resolution and code rate of content under different network conditions; dynamic self-adaptive stream transmission is realized; s2, directly acquiring data through the nearest edge node when the content is requested, and caching high-frequency interaction data at the edge node; s3, the user request is routed to the optimal node; s4, adopting a predictive buffering and preloading mechanism; s5, predicting user interaction behaviors by using a deep reinforcement learning algorithm; and S6, adopting H.265, and combining the characteristics of low delay and high reliability of the QUIC protocol to carry out transmission. The extensible content interactive display system based on streaming media playing comprises a user behavior analysis module, a content management and distribution module, an interactive prediction and recommendation module, a preloading and cache management module, a network transmission and optimization module and the like.