30 results about "Cache miss" patented technology

Techniques are disclosed relating to generating user-specific time-to-live (TTL) values using machine learning. In various embodiments, a server system maintains a cache data store that is operable to store data for a plurality of users of a web service. In response to a cache miss for a request from a first one of the plurality of users, the server system may generate a user-specific TTL value for the first user. In various embodiments, generating the user-specific TTL value may include using a machine learning model to generate a predicted future access pattern for the first user that indicates a distribution of time periods during which the first user is expected to access the web service and, based on the predicted future access pattern, determining the user-specific TTL value for the first user.

A processor includes a time counter and issuing instructions and executing instructions at a future time, which is based on the time counter. The execution times are based on fixed latency times of instructions, with the exception of the load instruction, which is based on the data cache hit latency time. A data cache miss causes the load instruction to fetch data from the level 2 cache, wherein a time tracker unit adjusts the level 2 cache latency time based on a counter.

The application discloses an intelligent code analysis method and system based on a defect hash cache, which comprises the following steps: obtaining defect details of a code, generating semantic hash values of each defect according to rule identifiers and code segments, and taking the semantic hash values of the defects as cache keys; if the cache keys exist in a cache database, directly outputting analysis results corresponding to the cache keys in the cache database; grouping all defects that do not hit the cache according to rule identifiers and file paths to form a plurality of defect groups; for each defect group, retrieving multi-layer context information from an abstract syntax tree index; combining the context information and the defect details of each defect group into batch analysis prompt words, calling a large language model to obtain LLM analysis results; if the large language model is unavailable, adopting a three-level degradation strategy to obtain degraded analysis results. Through the hash cache mechanism and the batch grouping optimization algorithm, the number of LLM calls is reduced, and the analysis speed is improved.

The present application relates to a kind of transparent large page performance monitoring method and electronic equipment, belong to computer technology field, the method includes: in each sampling period, in kernel state, obtain memory alignment performance data by memory alignment start and end trace point, obtain direct memory recovery performance data by direct memory recovery start and end trace point, obtain large page split performance data by probe and carry out transparent large page split monitoring;In each sampling period, user state program collects and records page table cache miss performance data, records continuous free block performance data;The block size of free block is the integer multiple of the page size of transparent large page;After sampling period ends, user state program reads memory alignment performance data, direct memory recovery performance data and large page split performance data from mapping, and is associated with page table cache miss performance data and continuous free block performance data and is shown.This application realizes the accurate acquisition of transparent large page performance monitoring result.

Techniques for cache management for LLM processing are described. Example embodiments include a signal hashing model that generates a key for particular context data. An LLM output corresponding to the context data is stored in a cache along with the key. For a user input received by the system, a cache lookup is performed using a key for context data corresponding to the received user input. For a cache hit, the stored output is used to respond to the user input. For a cache miss, a LLM processes the context data and the user input to generate an output within a first timeout. If the LLM is unable to generate an output within the first timeout, then in some cases, the LLM is allowed to continue processing until a second timeout, and a final or partial output from the LLM is stored in the cache.

This invention provides a cache replacement method, apparatus, electronic device, storage medium, and program product, belonging to the field of artificial intelligence chip technology. The cache replacement method includes: associating attribute information with each cache line in the cache of a graphics processing unit (GPU); in response to an access request, upon detecting a cache miss, if a cache line with a status identifier of a second identifier exists in the GPU, a target cache line is selected from the cache lines with the second identifier for replacement based on replacement order information; if no cache line with the second identifier exists in the GPU, a target cache line is selected from the cache lines with the first identifier for replacement based on replacement order information. By introducing status identifiers to stratify the value of cache lines and combining replacement order information to perform replacement selection within each layer, the overall operating performance of the GPGPU is optimized.

The application discloses a load balancing method and device, electronic equipment and a storage medium, and belongs to the technical field of electronic equipment. The method comprises the following steps: in the case that the load of a central processing unit (CPU) core is unbalanced, determining a first CPU core and a second CPU core, wherein the load of the first CPU core is higher than a load threshold, and the load of the second CPU core is lower than the load threshold; determining cache sensitivity scores of at least one task corresponding to the first CPU core according to historical cache miss rates of the at least one task, wherein the historical cache miss rate of a task is negatively correlated with the cache sensitivity score; and migrating a target task to the second CPU core according to the cache sensitivity scores of the at least one task, wherein the target task is a task in the at least one task.

Methods and systems are presented for providing multi-tiered cache system that works with an artificial intelligence (AI)-based conversation system for facilitating a conversation with users and processing transactions for the users. The multi-tiered cache system includes multiple tiers of cache modules that use different structures for caching and / or querying data. As a new utterance is received, the cache system uses each of the cache modules in sequence to determine whether a cache hit occurs. If a cache miss occurs at a first cache module, the cache system determines if a cache hit occurs at a second cache module. When a response is obtained from one of the cache modules and / or the AI model, the cache system updates the cache modules using the response.

The application relates to the technical field of computer data processing, and provides a high-performance operation method and system of an analytical database based on LLVM, which solves the technical problems of low hardware resource utilization and limited operation performance. The method comprises the following steps: obtaining a structured query statement sent by a computing device, and collecting pipeline pause data and cache miss rate of the computing device during operation; performing syntax and semantic analysis on the structured query statement, and calculating a load pressure value representing the current hardware resource occupation degree; determining a segmentation threshold value according to the load pressure value, and obtaining a plurality of fusion operator subgraphs; in the process of generating an intermediate representation based on an LLVM compiler architecture, an LLVM intermediate representation sequence is generated; the LLVM intermediate representation sequence is compiled by using a compiler of an LLVM back end, target machine instructions are generated, and the target machine instructions are loaded to the computing device for operation processing, so that the operation efficiency of the analytical database and the hardware resource utilization can be improved.

The application discloses a large model intelligent question and answer acceleration method for a high concurrency scene, comprising the following steps: receiving a user input question and answer request, extracting core query keywords and semantic features, and generating a query vector; querying a high-frequency intention cache; if the high-frequency intention cache is not hit, querying a high-frequency question and answer pair cache; if the high-frequency question and answer pair cache is not hit, querying a historical question and answer result cache; if none of the three layers of caches is hit, entering a mixed semantic retrieval process; starting vector retrieval and keyword matching double-path parallel retrieval, and generating a final reference knowledge set after deduplication and integration; large model dynamic reasoning pruning: for a new request that is not hit in the cache, dynamic reasoning pruning optimization is performed, and a large model is called to generate a question and answer result. The application also discloses a large model intelligent question and answer acceleration system. The application solves the problems of high response delay, large resource consumption, low retrieval efficiency and insufficient service stability of an existing large model intelligent question and answer system in a high concurrency scene.

This invention provides a streaming data management method and apparatus for a reconfigurable processor multi-port cache, relating to the field of large-scale integrated circuit technology. The method includes: when processing a cell array accessing the cache, if a cache miss occurs, evicting a cache line with a target bit set to a first preset value using a polling arbitration method, and setting the target bit of subsequent cache lines entering the cache to the first preset value; when processing a cell array accessing the cache, if a cache hit occurs, setting the target bit of the currently hit cache line to a second preset value based on the interval between the previously hit cache line and the currently hit cache line. This invention can effectively reduce the probability of cache thrashing under the flushing of streaming data, thereby reducing the frequency of accessing off-chip memory, increasing the memory access bandwidth of the PEA, mitigating potential conflicts and misses caused by set-associations, and reducing the miss rate for streaming data.

A system and method for enhancing the performance of a Central Processing Unit (CPU) for artificial intelligence (AI) workloads. An orchestration engine logically partitions physical CPU cores into a plurality of “micro-containers,” which are isolated execution sandboxes. A workload profiler analyzes incoming AI tasks and provides performance metrics to an autoscaler. The autoscaler dynamically adjusts the number of active micro-containers on each core to optimize performance based on real-time hardware counter data, such as instructions-per-cycle or cache-miss rates. This architecture allows general-purpose CPUs to achieve performance comparable to specialized GPUs for parallel processing tasks, while reducing cost and power consumption.

This application provides a multi-level caching method, processor, and computer device. The method is applied to a processor, which includes a processor core and a multi-level cache system. The method includes: a control module generating a Level 1 cache lookup instruction based on the priority order of memory access requests; a Level 1 cache component generating a Level 1 cache miss signal when accessing the Level 1 cache data array based on the Level 1 cache lookup instruction fails; a Level 2 cache control module responding to the Level 1 cache miss signal, and a control context-aware prefetch module determining the target access source and the target prefetch policy instance based on address information and an access source identifier list, and determining the prefetch address list corresponding to the target prefetch policy instance; the access source identifier list being determined based on a historical access address sequence; a cache data extraction module performing memory access based on the prefetch address list to obtain the target data; and a response output module sending the target data to the processor core.

The present application relates to a kind of optimization method and system of timing diagram subgraph matching.The method includes: time block is carried out to original timing diagram, and the edge with same end point is aggregated and compressed in each time block to generate intra-block multi-edge object, to build block index in turn;Query graph with time constraint is received, with the edge in query graph as matching unit to execute search, and maintain current legal time window;In candidate generation stage, according to current to be matched data vertex pair and current legal time window, construct cache key, and carry out exploration in hotspot cache;If hotspot cache is not hit, then use current legal time window to locate target time block, and execute intersection operation in target time block to generate candidate multi-edge list;When current to be matched data vertex pair meets preset hotspot condition, the candidate multi-edge list is written into hotspot cache;Continue to execute search based on the candidate multi-edge list of hit or generated candidate multi-edge list, until outputting model matching result.

The present disclosure relates to a DRAM cache management device. The DRAM cache management device includes: a processor; a main memory; a DRAM cache memory; a DRAM cache management unit connected to one or more cache frames that partition the DRAM cache memory; and a memory management unit configured to generate cache-miss information and transmit the generated cache-miss information to the DRAM cache management unit, wherein the DRAM cache management unit may be configured to receive the cache-miss information from the memory management unit, allocate a free cache frame from among the one or more cache frames upon receiving the cache-miss information, generate a data replication command, and update cache-main memory mapping information between the DRAM cache frames and one or more main memory frames that partition the main memory.

The cache replacement policy is described. According to the described technology, when a request for data is received, the cache replacement policy controls how the controller responds to the request. Depending on whether the request is a cache miss requiring the evicting of at least one cache line, the cache replacement policy assigns each cache line a priority value indicating whether the cache line should be preserved over other cache lines. The cache replacement policy decrements the priority value until at least one cache line reaches the minimum priority value, at which point the cache line is evicted. The cache replacement policy designates certain cache lines as protected, either through a separate protected indicator or through the cache line priority value, thereby prioritizing the preservation of protected cache lines in the cache while selecting unprotected cache lines for evicting.

The embodiment of the specification discloses a kind of knowledge base document concurrent processing method, equipment and medium, it is related to distributed computing technical field, method includes: receiving multiple concurrent user requests by asynchronous network interface, generating unique request identifier for each concurrent user request, and return asynchronous acceptance response to client, concurrent user request and unique request identifier are published to message queue;Multiple user requests are batched pulled from message queue, determine request batch, based on each user request in request batch, determine the unique hash digest corresponding to each user, to unique hash digest, carry out query in preset multi-level cache system, determine cache query result;When cache query result is cache miss, request batch is dispatched to GPU computing unit to execute model inference calculation, determine model inference result data, to store model inference calculation result data to multi-level cache system, and it is returned to client by long connection protocol stream.

This invention discloses a two-layer collaborative caching method for remote sensing satellite networks, applied to a two-layer network composed of high-orbit and low-orbit satellite nodes. High-orbit satellite nodes serve as the global cache and backhaul backbone, while low-orbit satellite nodes act as edge cache access points directly serving ground users. When a low-orbit satellite node receives a user content request and its own cache misses, it executes the following core steps: First, based on the temporal and spatial attributes of the cached content, it calculates the content duplication rate with neighboring nodes and selects the node with the lowest duplication rate as the target query node. Second, it sends a query request to the target node to obtain a copy of the content. Finally, it distributes the obtained copy to the user. This invention significantly improves cache hit rate and content distribution efficiency through an intelligent collaborative query mechanism, effectively reducing transmission latency, and is particularly suitable for remote sensing satellite network environments with wide coverage and dynamic topology.

This invention discloses an address mapping method and apparatus based on a non-transparent bridge and a mapping lookup table cache, relating to the field of address mapping technology. The method includes: receiving an access request from a first bus domain through a non-transparent bridge module; decomposing the target address into a page number and an intra-page offset through an address resolution module, and extracting access attributes; querying a mapping lookup table cache module based on the page number to determine the extraction method for the page-level mapping item; verifying the extracted page-level mapping item and access attributes; if a cache miss occurs but the verification passes, cache backfilling is performed; generating a local target address based on the target page base address and intra-page offset in the page-level mapping item; and forwarding the access request to a second bus domain connected to the non-transparent bridge module through an access forwarding module. This invention, after the access request enters the non-transparent bridge module, extracts the page number based on the request address and obtains the target address information through a lookup table, enabling rapid redirection of the access address.

The present application relates to the technical field of distributed storage, and discloses a brand growth engine interaction system based on distributed storage. A client cache layer of the system constructs a directed acyclic graph of brand interaction behaviors, graph nodes correspond to data blocks, and graph edge weights are obtained by frequency statistics of clicks and conversion paths. A topology weight prefetch module is embedded in a bottom layer reading protocol. When a starting data block reading request is received, the outgoing edge weight with the starting node as the starting point is extracted, and an asynchronous prefetch instruction is issued to the physical node corresponding to the distributed storage cluster in descending order of the outgoing edge weight. The present application shortens the data preparation time of multi-hop interaction behavior aggregation calculation, avoids cache misses caused by the dispersion of prefetch data block physical addresses, reduces the overhead of repeated data transmission between nodes, and solves the network congestion and input / output blocking problems in the aggregation scenario of cross-type heterogeneous data blocks.

This application discloses a data access method, apparatus, cache unit, and computer system. The application obtains the access response of a cache-missed target request. The response is based on the status information in the target entry corresponding to the target request, indicating an unprepared state, and the absence of a cache line matching the address information of the target request. A corresponding target cache line is allocated based on a predetermined replacement algorithm, and the access response is stored in the target cache line. Therefore, this application employs a cache delay allocation strategy. Upon receiving the access response, it determines whether the page table range of a previously requested access response covers the current access response by using the status information in the corresponding target entry. Simultaneously, it checks whether a cache line with a matching address exists. Only after determining that there is no local access response in the current cache is a new cache line allocated. Based on this, the application effectively avoids the problem of duplicate page table storage, improves the available cache capacity, and avoids resource waste.

This invention specifically relates to a method, device, and storage medium for generating equipment status profiles based on substations. The method includes: a caller initiating a creation request, which must carry pre-set profile data; after successfully parsing the pre-set profile data, a dynamic cache manager creates a globally unique profile object for the profile, pre-sets the parsed profile data into the profile object's dedicated cache space, marks the pre-set data as "persistent," and establishes a cache index table; the profile service returns the profile ID and initialization result to the caller; after receiving the status acquisition request, the profile service queries the cache object in the dynamic cache manager by the profile ID, initiates a data read lock on the cache object, and uses a hash index stored by device ID for fast lookup to determine whether the target device status has been cached; if the cache is not found, a precise query request is sent to the real-time database; the profile service returns the processed complete data to the caller.

This invention relates to a method for accelerating the processing of hot data information in a database, comprising the following steps: collecting user-related feature data, including static and dynamic features; identifying hot data in the database based on global query logs; calculating the matching degree between user feature data and the hot data, and triggering preloading of the hot data to the user's local cache based on the matching degree result; managing the data in the local cache, including data updates and data eviction; and upon receiving a user query request, prioritizing matching data from the local cache and outputting it, and if the cache misses, initiating a query to the database. By combining user static and dynamic features to calculate the matching degree, indiscriminate preloading is avoided, thereby improving the cache hit rate and reducing server-side system resource consumption while improving query response speed.

The invention belongs to the field of embedded real-time systems, and particularly relates to a cache layout optimization-oriented data flow diagram construction and solving method, which is used for optimizing the data layout of a program. Compared with the prior art, the method has the advantages that a plurality of data structures are uniformly modeled by using the data flow diagram, a systematic, extensible and high-adaptability universal method is formed, and the data layout and the time sequence predictability are collaboratively optimized; on this basis, a more efficient optimization solution method is provided, the cache missing rate is effectively reduced, cross-scene performance fluctuation is reduced, and the predictability of program execution is improved.

An apparatus and method for an instruction cache miss throttler. An example method comprises: storing instructions in an instruction cache; predicting, by a branch predictor circuit, whether one or more branch instructions will result in taken branches and responsively providing IPs corresponding to the predicted taken branches; storing the IPs corresponding to the predicted taken branches, wherein the IPs are to be used to perform a lookup for corresponding instructions in the instruction cache; storing IPs of instructions from the fetch queue which missed in the instruction cache, the IPs to indicate instructions to be filled into the instruction cache from the memory subsystem; tracking a number of cachelines which resulted in instruction cache misses since a last predicted branch was taken; and throttling the branch predictor circuit in response to a threshold reached for the number of cachelines which resulted in instruction cache misses.

The application is mechanical and electrical equipment fault diagnosis system and method based on knowledge graph, relates to mechanical and electrical equipment fault diagnosis technical field, and includes: atlas certainty partition module; memory continuity allocation module; boundary edge pruning index module; certainty multi-hop query execution module.In the application, the certainty coding mechanism based on the equipment physical bill of materials tree maps the physical hierarchical assembly relationship of the mechanical and electrical equipment into the binary semantic identification of nodes, and the continuous memory partition and linear storage layout are realized on the basis, the random memory access of the fault diagnosis is completely converted into sequential memory access, the processor hardware prefetch characteristics are fully adapted, and the occurrence probability of cache miss, page table cache invalidation and page fault interruption is reduced.

The application discloses a design space exploration method for cache hierarchy in a multi-core system. The method aims to optimize the cache subsystem in the multi-core system by reasonably configuring the cache hierarchy and the inter-chiplet network topology, thereby improving the system performance. The specific steps include: 1) cache miss rate and network delay modeling: modeling the cache miss rate and the network delay as functions of the cache hierarchy and the inter-chiplet network parameters; 2) optimization problem definition: defining the optimization target under the cost and power consumption constraints, and minimizing the concurrent average memory access time (C-AMAT); and 3) solving by a double-layer optimization algorithm: performing the cache subsystem optimization and the inter-chiplet network optimization by the double-layer optimization algorithm. The method provides an effective solution for the cache optimization of the multi-core system, and has a wide application prospect.

This application provides a method and system for accelerating large-scale model inference. In the field of artificial intelligence technology, this application obtains the output feature vectors of the current Transformer layer and the previous Transformer layer, as well as the cache miss rate of the cache memory, during the long text inference process of a large model. It calculates the cosine of the angle between the output feature vectors to obtain the inter-layer similarity. When the cache miss rate is greater than a preset congestion threshold and the inter-layer similarity is in a redundant range, a low-rank weight matrix is ​​reconstructed. Each word in the input word sequence of the current Transformer layer is mapped to a low-dimensional subspace defined by the low-rank weight matrix through token merging to obtain the target word sequence. The target word sequence is then input into the current Transformer layer to accelerate the inference of the current Transformer layer, thus achieving efficient acceleration of large-scale model inference.