263 results about "Cache optimization" patented technology

Systems and methods for improving cache efficiency and utilization are disclosed. In one embodiment, a graphics processor includes processing resources to perform graphics operations and a cache controller of a cache memory that is coupled to the processing resources. The cache controller is configured to set an initial aging policy using an aging field based on age of cache lines within the cache memory and to determine whether a hint or an instruction to indicate a level of aging has been received.

The invention discloses a large-scale language model KV Cache optimization method based on recent query attention information. According to the method, attention calculation is carried out on sub-words input into a large-scale language model; constructing a record table of attention information of recent r queries by taking the average attention score as a threshold value; performing OR operation on the recorded attention information of each query, and deleting the key-value cache which is determined to be unimportant; judging whether the number of all generated sub-words reaches a preset number or not; according to the method, the importance degree of information in the KV Cache is judged by recording the attention information of the Query in the recent reasoning step, the relatively less important KV Cache is deleted, the important cache part is reserved, and finally, the performance of the model can be kept not to be obviously influenced under the condition that more than 70% of the KV Cache can be optimized.

The invention provides an online map tile fast loading and caching optimization method, medium and system, and belongs to the technical field of map processing.The method includes the steps that a tile index database and an exchange priority matrix are established, a viewport moving trend vector is calculated according to user sliding operation, a future viewport range is predicted, and a preloaded tile list is screened; the method comprises the following steps: performing cache validity check on tiles, establishing a small change detection vector to judge whether updating is needed, establishing a tile hit rate statistical matrix, fusing a time decay factor and a spatial correlation factor to calculate a popularity score, and gathering high-popularity tiles in a quick access area by adopting a thermodynamics-imitating entropy reduction algorithm to realize cache ordering. And the next tile demand of the user is predicted, so that the technical problem that the overall loading response speed is reduced due to an overlong high-value tile access path caused by disordered distribution of cache resources in the online map tile loading process is solved.

The invention provides a task disassembly and multi-agent arrangement execution system and method based on a large language model, and belongs to the technical field of computers, and the system comprises an instruction analysis module, a DAG construction module, a scheduling execution module and a cache optimization module. Analysis is carried out according to the dependency relationship between the tasks, a task execution DAG is automatically constructed, and concurrent calling is carried out on the sub-modules without dependency; by caching an authentication result, a context reasoning result and the like, a universal module is executed in advance, the result is reused, and repeated calculation is reduced; the maintainability and the expandability of the system are improved through graph structure visualization and node element information injection; and performing context analysis and scheduling optimization in combination with the reasoning ability of the language model to realize an intelligent decision execution path.

The invention discloses a data production and application method based on index management, and the method comprises the steps: receiving an index query request containing a target index identifier, a dimension constraint condition and a time range parameter, carrying out the analysis and semantic verification of the target index identifier based on a global index asset library, and obtaining the index definition information; generating a standardized query statement according to the index definition information, and selecting an optimal data calculation engine to execute query; performing multi-level cache optimization and parallel computing acceleration on the query process to obtain an original data set; processing the original data set in real time according to a preset analysis model to generate structured index data containing trend analysis, anomaly detection or attribution inference results; and filtering and desensitizing the data based on a fine-grained permission control strategy, only returning contents in a permission range and recording an audit log. By means of the method, unified management, efficient query and intelligent analysis of the index data are achieved, the automation level and query performance of data production are improved, and meanwhile data safety and access controllability are guaranteed.

The invention discloses a multi-level cache optimization method and system based on data popularity self-adaptive adjustment, relates to the technical field of data storage, and aims to solve the problems that a cache mechanism in an existing desktop cloud system is static, initialization is slow, and cold and hot data are difficult to recognize. The adopted scheme comprises the steps that a three-level cache structure of a local cache L1, a distributed cache L2 and a database L3 is set; constructing a data popularity scoring model based on the access frequency, the time decay and the user priority; dividing the data into hot data, temperature data and cold data according to the popularity score, and dynamically determining migration and elimination strategies of the data in L1, L2 and L3; the hot data are loaded to the upper-level cache T in advance through a prediction mechanism, and container-level cache preheating is supported; a cache preheating strategy is integrated to a Kubernetes life cycle, and hot data loading and cold data degradation are achieved; visual strategy configuration and monitoring during operation are supported, and cache strategies are managed in real time. The method is suitable for an access optimization scene of a large-scale desktop cloud system.

A system and method for extending mobile-optimized multi-stage language model processing with federated persistent cognitive architecture. The system processes prompts through a first large language model to generate “thoughts,” which are cached and processed with the original prompt through a smaller language model. Building upon the three-tier thought caching, the system implements a federated multi-tier hierarchy with local device, domain-specific branch, and global collective caches. A federated cognitive orchestrator coordinates operations across multiple domain-specialized instances, managing thought routing, state synchronization, and cross-domain knowledge sharing while maintaining domain boundaries. During user inactivity, autonomous reasoning continues in cloud environments, generating insights from existing thoughts and interaction history. The system performs memory consolidation, thought cache optimization, and cross-domain pattern recognition without consuming mobile device resources, while maintaining privacy boundaries. This persistent cognitive architecture functions as an evolving reasoning partner rather than merely a responsive tool.

The invention provides a cold start optimization system and a cold start optimization method for a Python Serverless function. The system comprises a dependency analysis module, a module pre-compiling system, a hierarchical module loader, an inert import mechanism, a code cache optimizer and a self-adaptive optimization system. The dependency analysis module analyzes code dependency in a function deployment stage; the module pre-compiling system pre-compiles the source code into an optimized byte code; the hierarchical module loader loads modules of different hierarchies according to priorities; actual loading of an inert import mechanism delay module; the code cache optimizer caches a hotspot function and a module state; and the self-adaptive optimization system dynamically adjusts preloading and caching strategies. The system further comprises a JIT compiling intensifier, a state storage and quick recovery module, a memory management optimizer and a security mechanism. The method has the beneficial effects that the function cold start time is remarkably shortened, the function execution efficiency is improved, the resource utilization is optimized, and meanwhile, the safety and the expandability are ensured.

The invention provides a meta-universe scene distributed two-stage cache optimization method combining MADDPG and GCN, and the method comprises the steps: dividing a meta-universe scene in the horizontal and vertical directions, obtaining a preset number of sub-scenes, and carrying out the integration of resources needed by the rendering of each sub-scene; the sub-scenes are compressed through an edge network, and the similarity of user tracks is calculated and obtained through a fast dynamic time warping algorithm; based on the similarity of different track fragments, sub-scene popularity ranking is obtained by weighting, summarizing and sorting the number of access times of a GPU server, sub-scenes predicted to be frequently accessed are requested and cached in advance according to the popularity ranking of the sub-scenes, and a two-stage cache replacement framework combining GCN and MADDPG is adopted to adjust a cache strategy of the server. According to the method, intelligent cache distribution and adjustment can be realized according to sub-scene popularity prediction, the cache hit rate is improved, and the delay response of a user request is reduced.

The invention discloses a database query execution method and device based on dynamic query compiling cache optimization, and belongs to the field of database management systems. Receiving a query statement input by a user and constructing an abstract syntax tree; generating a corresponding identifier according to the abstract syntax tree, searching a matched machine code, and loading and executing the matched machine code which can be reused to obtain an execution result; generating a corresponding executable plan tree for the query statement input by the user without finding the matched machine code or the query statement input by the user with the matched machine code incapable of being reused, generating and optimizing the machine code through dynamic compiling, obtaining the optimized machine code, and loading and executing the optimized machine code to obtain an execution result; and then an execution result is sent to the user, and machine codes in the cache are cleaned regularly. According to the method, whether the machine codes in the cache are multiplexed or not is accurately determined, so that unnecessary compilation overhead is reduced, and query execution efficiency is improved.

The invention relates to the technical field of computers, in particular to a fault report generation method based on the size of a dynamic window in an edge cloud scene, which comprises the following steps of: S1, performing data processing according to acquired data to obtain preprocessed data, and inputting the preprocessed data into a pre-training model for training to obtain an adjustment model; s2, multi-modal input data generates a multi-modal KV cache according to the adjustment model, encoding is carried out to obtain an encoding set, KV key value pairs are screened according to the encoding set, decoding is carried out after the KV key value pairs are combined, KV cache compression is carried out on the decoded data through Page Attelation to obtain matched compressed data, and a window is dynamically adjusted according to the compressed data; and S3, receiving second real-time operation state data of the edge equipment, generating a second historical fault report, and judging whether the second historical fault report meets the model requirements or not. Through LOOK-M and paged attention technologies, KV cache of a large model is reduced, memory use and calculation efficiency are optimized, model fine tuning is introduced, and it is ensured that a generated fault analysis report has high accuracy and real-time performance.

The invention provides a large language model long text reasoning acceleration method and device based on a speculative key value cache sparse technology, a medium, a terminal and a program product, the method is applied to electronic equipment comprising a GPU and a CPU, and the method comprises the steps that an index list of most important historical tokens is generated based on a distillation language model according to an obtained context sequence; comparing the index list generated at the current moment with the index list at the previous moment, and calculating to obtain a difference set part; asynchronously prefetching the key value cache of the difference set part from the CPU to the GPU; according to the asynchronously prefetched key value cache, performing parallel execution based on a large language model to generate a new token; obtaining a new sequence length according to the generated new token, and judging whether the new sequence length exceeds a preset threshold value or not; and if the threshold value is exceeded, executing unloading operation. According to the method, the performance and the stability of processing long text reasoning by the large language model can be improved, and key value cache optimization in the long context reasoning process is ensured to be always effective.

The invention relates to an edge service cache optimization method and device, computer equipment and a storage medium. The method comprises the following steps: modeling an edge service cache optimization problem into an MDP model, and collecting state information of a current environment through an intelligent agent at each time step; based on the state information of the current environment, the intelligent agent adopts a depth deterministic strategy gradient algorithm to optimize a cache strategy, and generates a cache strategy through a strategy network; executing the caching strategy, calculating a service access delay after executing the caching strategy through a value network, and calculating a reward function according to the service access delay; and evaluating the execution effect of the cache strategy according to the reward function, and optimizing the strategy network to generate an optimal target network. According to the embodiment of the invention, through intelligent agent learning and cache strategy optimization, the storage resources of the edge server can be utilized more effectively, so that the cache hit rate, the resource utilization efficiency and the service quality are improved.

The invention discloses a cross-system full-link metadata processing method and device.According to the method, a separated metadata storage architecture is adopted, entity attributes are stored in a JSON document form, relationships between entities are stored in an independent relationship table in a quintuple structure, and efficient storage and rapid query are achieved; through a topology driving processing framework, depth-first traversal is carried out based on a hierarchical structure of a directed acyclic graph, and multi-thread concurrent processing is started at a mode level, so that the processing efficiency of large-scale heterogeneous metadata is improved; the entity life cycle event scheduler is used for monitoring entity change events, triggering index updating, relation maintenance and other operations, and real-time response of metadata change is ensured. Meanwhile, in combination with a search index synchronization mechanism, a hierarchical entity association model, a connector abstraction factory and a fingerprint cache optimization mechanism, efficient management, real-time synchronization and deep association analysis of cross-system metadata are realized. According to the method, the storage efficiency, the real-time performance, the concurrent processing capability and the expansibility are remarkably improved.

The invention discloses a multi-model dynamic image analysis system, and relates to the technical field of artificial intelligence. The system adopts a heterogeneous computing architecture and comprises a preprocessing model, a multi-modal analysis model, an anisotropic correction model, a large language model, a task knowledge base embedding model and a KV cache management system. An incongruous task reconstruction mechanism based on hardware isomerism is constructed, a large language reasoning model is nested for analysis, and efficient reuse of computing resources is achieved by using a cross-model KV cache optimization technology. The system is provided with multiple graphics cards and a double-channel NUMA, and a multi-mode large language model carries out unsupervised or weak supervised training on a target detection model in the operation of the system. According to the system, through a dynamic load scheduling strategy, a mechanism for reasoning by using a processor and a memory enhanced display card in a full-load mode is innovatively designed, and three-section real-time streaming enhanced output is achieved. According to the system, through a dynamic load scheduling strategy, a mechanism for reasoning by using a processor and a memory enhanced graphics card in a full-load mode is innovatively designed.

The invention discloses a big language model key value cache optimization method based on a cooperative game, which comprises the following steps: modeling all attention heads of a big language model as participants in a cooperative game model according to a reasoning task to be performed, and calculating cooperative contribution scores of different attention heads to the reasoning task; normalizing the collaborative contribution score of each attention head, allocating a preset total shared key value cache budget to the corresponding attention head in proportion according to the normalized score, and ensuring that all the attention heads reserve a basic local window cache; the key values in each attention head are subjected to importance sorting, and a plurality of most important key values are reserved according to the key value cache budget of the attention head, so that compression optimization of key value cache is realized. According to the method, the cooperative game theory is applied to large language model key value cache allocation optimization, so that the utilization rate of hardware computing resources is improved while the reasoning accuracy is ensured.

The invention discloses an input data sharing and cache optimization method in matrix multiplication calculation. The method comprises the steps of 1, segmenting and distributing an input data matrix and a weight matrix according to the number N of calculation cores on a chip; 2, sequentially connecting the plurality of calculation cores end to end to form a data transmission annular structure; step 3, calculating the distributed matrix multiplication by each calculation core, and transmitting the current input sub-matrix of the calculation core to the next calculation core; step 4, performing matrix multiplication operation on the transmitted input sub-matrix and the weight sub-matrix in the next calculation kernel; and 5, iterating transmission and calculation of the input sub-matrixes, and carrying out N rounds of matrix multiplication of the input sub-matrixes and the weight sub-matrixes to complete the whole operation process. The invention further discloses a system for implementing the method, and the system has wide application value.

The embodiment of the invention provides a cache optimization method in large model reasoning, which comprises the following steps: in a pre-filling stage, cache operation is carried out layer by layer for a plurality of attention layers in a large model, and the cache operation for any ith layer comprises the step of acquiring a target attention matrix of the ith layer. And determining a first index value and a second index value according to the distribution of the row data and the column data of the target attention matrix. And determining an ith preference score corresponding to the ith layer according to the first index value and the second index value. And according to the ith preference score, determining a target cache region allocated for the ith layer in the total cache region, and storing attention data of a target character in the input text in the target cache region. And according to the ith preference score, updating a previous cache region of each layer before the ith layer, and updating attention data of characters stored in the previous cache region.

The invention discloses a mobile edge computing unloading and service caching method and device based on deep reinforcement learning. The method comprises the following steps: constructing a mobile edge computing system model; constructing a service cache model and a decision model; constructing a cost calculation model; establishing a task unloading and service cache optimization problem according to the mobile edge computing system model, the service cache model and the decision model; and solving a task unloading and service cache optimization problem based on a deep reinforcement learning method by taking the minimum cost as an optimization objective function to obtain an unloading decision and a service cache decision. According to the method and the device, the calculation complexity in the MEC calculation unloading solving process can be reduced.

The invention relates to the field of database reading, and particularly discloses an efficient database reading method based on multi-level cache optimization and dynamic index fragment.The efficient database reading method comprises the steps that high-frequency query data is accurately recognized through a sliding window algorithm, a dynamic cache loading and preheating strategy is implemented, query delay is effectively shortened, and the database burden is relieved; intelligent distribution of index fragments is realized by applying a consistent Hash algorithm, and the query efficiency is greatly improved in cooperation with distributed query routing and transaction processing optimization; the query load balancing ensures stable and efficient operation of the system in a high-concurrency scene by means of weighted polling and a minimum connection strategy in combination with real-time monitoring and an elastic capacity expansion and contraction mechanism. According to the method, the database access efficiency and the dynamic adaptive capacity are remarkably improved, and the method is particularly suitable for a large-scale distributed data processing environment.

The invention discloses a distributed cache optimization method and device, a storage medium and a product, and relates to the technical field of distributed caches, and the method comprises the steps: determining the overall distribution type of a data access mode through a low-precision sorter; the overall distribution types are classified through a high-precision sorter; and determining an allocation strategy of the cache resources through the cache allocator according to the distribution parameter combination under each overall distribution type, and performing optimal allocation of the cache resources according to the allocation strategy. According to the method, the overall distribution type of data access is identified through the low-precision sorter, so that the consumption of computing resources is reduced; the distributed parameters are refined through the high-precision sorter, the accuracy of data feature capture is improved, the hierarchical processing mode not only improves the accuracy and efficiency of decision making, but also can adapt to different access modes and maintain efficient cache management, and the problem that various existing algorithms are insufficient in universality in specific scenes is solved.

The invention relates to a KV cache optimization method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: calculating a key vector and a value vector corresponding to each element in a text input sequence input into a large language model; through a multi-head potential attention mechanism, performing low-rank joint compression on the key vector and the value vector to obtain a potential vector, and storing the potential vector in a KV cache space; based on a scaling law, determining an optimal compression dimension, regenerating an adaptive potential vector and updating a KV cache space; for the same text input sequence, generating corresponding query vectors, and grouping the query vectors according to a preset grouping rule; calculating a semantic association weight between each group and the correspondingly called potential vector, and taking the semantic association weight as a group attention calculation result; in the reasoning process, potential vectors and grouping attention calculation results are calculated to calculate attention weights. By adopting the method, the storage requirement of the KV cache can be further reduced.

The invention relates to the technical field of project cost consultation management, and discloses a project cost consultation management method and system, and the method comprises the steps: building a material price database which is dynamically updated in real time, obtaining market material price data from the Internet, and updating the material price database; adding the material price database into the data warehouse set, executing a cluster cache strategy, and optimizing the data warehouse set by using cache; executing a load balancing strategy on the server network of the transmission data warehouse set; acquiring data features of the cost project, executing an intelligent matching strategy, and acquiring an optimal cost model of the cost project; calculating the cost of the cost project by using the optimal cost model; if the cost is higher than the cost threshold, executing a budget project adjustment strategy, and adjusting budget distribution of sub-projects of the cost project; and executing the block chain recording strategy, and storing the operation record in the log record. And the reliability and accuracy of project cost management are improved.

The embodiment of the invention provides a distributed function scheduling optimization method and device under a server-free architecture, and the method employs a dynamic priority scheduling strategy, dynamically adjusts the priority of a function task according to a predefined priority rule and real-time monitoring data, guarantees the priority processing of a key task, and improves the scheduling efficiency. And the system response speed and the resource utilization efficiency are improved. A distributed cache is adopted to store function tasks, execute environment snapshots and a dependency library, a preheating mechanism is supported, cold start delay is reduced, and function task execution efficiency is improved. Through a monitoring feedback mechanism, index data related to system operation are collected in real time, and according to the index data, a priority and a caching strategy are dynamically adjusted, so that adaptive optimization is realized, and the stability and robustness of the system are improved. And through dynamic priority scheduling and cache optimization, the system throughput and the resource utilization rate are improved.

The invention discloses a distributed high-concurrency real-time industrial data pushing method and system, relates to the technical field of high-concurrency data processing, and solves the problems that dynamic adjustment of traditional buried point logic is difficult, and decoupling with service data cannot be carried out. The industrial data pushing system comprises a gateway management and control layer, a message queue layer, a cache optimization layer, a data processing layer, a pushing service layer and a monitoring operation and maintenance layer, asynchronous decoupling is carried out on the client and the server through the message queue layer, and priority transmission and flow peak clipping are carried out on original industrial data; performing anomaly prediction on the original industrial data through a data processing layer according to an anomaly detection rule to obtain anomaly alarm data and an out-of-order anomaly event; according to the invention, the push service layer carries out connection management and abnormal alarm data and out-of-order abnormal event push on the client according to a scheduling push mechanism, and carries out data burying point judgment on a client request, thereby improving the dynamic flexibility of data burying points, and solving the problems of industrial data analysis and resource configuration optimization.

The invention relates to the technical field of cache optimization, and discloses a cache optimization method and device, equipment, a storage medium and a computer program product.The method comprises the steps that data information of all cache nodes in different cache hierarchies is obtained, a cache strategy is updated based on the data information, and an updated cache strategy is obtained, the different cache levels comprise a local cache level, a regional cache level and a global cache level; on the basis of the updated caching strategy, each caching hierarchy is optimized, and the optimized caching content is obtained; in response to a content request of a user, carrying out dynamic path selection at different cache hierarchies to obtain a target request path; and obtaining target cache content from the optimized cache content based on the target request path. The method can effectively adapt to data requests of different scales and types, efficient utilization of storage resources and optimization of access performance are achieved, the cache hit rate is effectively increased, and the content transmission cost is reduced.

The invention provides a data cache optimization method, device and equipment for a vehicle-specification MCU and a storage medium, and the method comprises the steps: building a multi-target cache decision through predicting the future data access probability distribution, and giving consideration to the real-time performance and the memory utilization rate. A cache is divided into logic partitions with different priorities, and the sizes of the partitions are dynamically adjusted, so that a self-adaptive cache layout is formed. In combination with access probability distribution and a partitioning strategy, a dynamic pre-fetching threshold value is set, data pre-fetching is executed, and elimination and updating are performed in a task execution period, so that the caching efficiency and stability of a vehicle gauge MCU in a high-temperature and strong-vibration environment are improved, and the method is suitable for a vehicle-mounted control system with strict requirements on safety and performance.

The invention discloses a hierarchical quantization and dynamic priority management-based model cache optimization method and device. The method comprises the following steps of: acquiring an importance score of each Token of a word sequence corresponding to a reasoning long text by utilizing a mask language model; constructing a Token importance scoring model, and dividing the cloud platform GPU into a head cache region, a middle cache region and a tail cache region; the service reasoning model stores the key value pairs of the first # imgabs0 # Token in a Token reasoning sequence corresponding to the reasoning long text in a head cache region in an FP16 format, stores the key value pairs of the first # imgabs1 # Token before the current processing Token in a tail cache region in the FP16 format according to an FIFO principle, and stores the key value pairs shifted out of the tail cache region in an intermediate cache region in an FP8 format; and after the intermediate cache region is full, updating the intermediate cache region by using the Token importance scoring model and the importance score. According to the method, the reasoning efficiency of the large language model in long text generation can be improved.

The invention discloses a label-based digital equipment model dynamic creation method and system, and relates to the field of complex data integration and equipment management technology application in the power industry, and the method comprises the following steps: predefining a general data label model, dynamically configuring a query model, and establishing a database; and constructing a digital equipment data entity and designing a technical process of parallel computing and asynchronous processing. Data standardization is realized, and the query efficiency is improved; through a dynamic configuration mechanism, the system can automatically generate the query model, and the flexibility is improved; through the data entity construction method, different business data can be associated through a standardized model, and the data consistency is improved; according to the method, parallel computing and cache optimization technologies are utilized, and data query and processing are accelerated; according to the method, an incremental updating mechanism is adopted, so that the real-time performance and consistency of data are ensured; according to the method, the creation efficiency, query performance and system flexibility of the equipment model can be remarkably improved, and better technical support is provided for digital transformation of the power industry.

The invention discloses a task scheduling and cache optimization method in a storage and calculation collaborative edge computing network, and aims to optimize the service delay cost of a user and the cache deployment cost of a service provider. The method comprises the following steps: modeling a reusable task scheduling and cache optimization problem in a storage-computing collaborative edge computing network, and decoupling the reusable task scheduling and cache optimization problem into two sub-problems: a task scheduling problem for minimizing service delay cost, and providing a dynamic task scheduling algorithm based on a merging mechanism for the task scheduling problem. The invention provides a calculation result caching algorithm combining dynamic planning and greedy, and further provides a forward-looking pre-caching strategy for solving the problem of minimizing the service caching cost. The task calculation scheduling and result caching method with reusability is provided for space-time correlation of task requests in a storage and calculation collaborative edge calculation network, strict performance guarantee is provided, the method has wide applicability, and the method is suitable for large-scale popularization and application. And challenges of resource limitation and high real-time requirements in a complex calculation scene can be efficiently dealt with.