A method and system for data prefetching to cache

Abstract

This invention relates to the field of data prefetching and caching technology, and discloses a method and system for data prefetching and caching. The method includes: sorting an initial dataset by time and dividing it into subsets; extracting addresses and timestamps from the subsets to form access record items; mining association rules to remove anomalies and obtain final record items; clustering data blocks based on their one-dimensional coordinates; calculating correlation coefficients between core points to construct an association graph; and identifying time abrupt change points by comparing the singular value vectors of adjacent windows. Priority weights are adjusted for data blocks whose average access latency exceeds a threshold before and after the abrupt change point, and the blocks are rearranged to obtain a final priority queue. A predetermined number of data blocks are taken from this queue to form an initial cache queue. After multiple rounds of simulation to remove zero-hit blocks and adjusting the weights based on the hit rate, the queue is reordered to obtain the final cache queue. This method can dynamically adjust the caching strategy according to the actual request pattern.

Description

Technical Field

[0001] This invention relates to the field of data prefetching to cache technology, and in particular to a method and system for data prefetching to cache. Background Technology

[0002] Currently, in the field of industrial data storage, caching technology is of paramount importance as a key means to improve data access efficiency. Especially in modern computing environments, where the speed gap between processors and memory is widening, cache hit rate directly determines the overall system efficiency, becoming a crucial indicator for measuring system performance.

[0003] In current technologies, most caching strategies, such as LRU and LFU, are static strategies that rely solely on fixed rules to build the cache queue. However, in real-world data requests, especially under highly dynamic and volatile workloads, data request patterns frequently switch, and the distribution of hot data, access timing patterns, and access frequency all change significantly. In such cases, static caching strategies struggle to adapt to the actual data access patterns in real time, often leading to increased data hit rates, replacement overhead, and even frequent data churn.

[0004] Therefore, existing technologies have the drawback of not being able to dynamically adjust caching strategies based on actual request patterns. Summary of the Invention

[0005] This invention provides a method and system for data prefetching into a high-speed cache, which can solve the problem that existing technologies cannot dynamically adjust caching strategies according to actual request patterns.

[0006] In a first aspect, to solve the above-mentioned technical problems, the present invention provides a method for prefetching data into a cache, comprising: Obtain the initial dataset, sort the initial dataset in ascending order according to timestamps, and divide it into multiple sub-datasets; The accessed memory addresses and timestamps in the subset of data are extracted to form access record items. The address association rules of the access record items are mined to identify and delete abnormal records, and the final record items are obtained. Extract the one-dimensional coordinates of the data blocks in the final record item, and group the data blocks according to the one-dimensional coordinates to obtain a data block set; Calculate the correlation coefficient between each pair of core points in each group of data blocks in the data block set, and construct an association graph with the core points as graph vertices and the correlation coefficients as edge weights. Identify time abrupt change points by comparing the changes in the singular value vectors of the association graph under adjacent time windows. Obtain an initial priority queue, correct the priority weights of data blocks whose average access latency exceeds a preset latency threshold within the adjacent time window before and after the time mutation point, add the corrected weights to the initial priority queue and reorder them to obtain the final priority queue. A preset number of data blocks are taken from the final priority queue to form an initial cache queue. After multiple rounds of simulated access iterations, the hit rate of the data blocks is statistically analyzed. The cache weight of the initial cache queue is adjusted according to the hit rate, and the data blocks are reordered to obtain the final cache queue.

[0007] In one optional implementation, obtaining the initial dataset, sorting the initial dataset in ascending order according to timestamps, and dividing it into multiple sub-datasets includes: Obtain the initial dataset; The initial dataset is sorted in ascending order according to its timestamp to obtain an ordered dataset. The ordered dataset is divided into sub-datasets according to a preset time interval.

[0008] In one optional implementation, the step of extracting accessed memory addresses and timestamps from the subset of data to form access record items, mining address association rules of the access record items to determine and delete abnormal records, and obtaining the final record item includes: Extract the accessed memory addresses from the subset of data, and combine the memory addresses with the corresponding timestamps to form access record items; The address association rules for the access record entries are extracted using the Apriori algorithm. The access record items are judged to be abnormal according to the address association rules. The records that are judged to be abnormal are deleted to obtain the final record items.

[0009] In one optional implementation, the step of extracting the one-dimensional coordinates of data blocks from the final record item, and grouping the data blocks according to the one-dimensional coordinates to obtain a data block set, includes: Extract all accessed data blocks and one-dimensional coordinates from the final record item; The DBSCAN clustering algorithm is used to group the data blocks according to the one-dimensional coordinates to obtain a set of classified data blocks.

[0010] In one optional implementation, the step of calculating the correlation coefficient between each pair of core points in each group of data blocks in the data block set, constructing an association graph with the core points as vertices and the correlation coefficients as edge weights, and identifying time abrupt change points by comparing the changes in the singular value vectors of the association graph under adjacent time windows includes: Obtain the access sequence of the core points of each group of data blocks in the data block set, and divide the access sequence into multiple sub-sequences; Extract the access frequency vector within each subsequence, and calculate the correlation coefficient between each pair of core points based on the access frequency vector; Using the core points as graph vertices and the correlation coefficients as edge weights, a correlation graph is constructed. The singular value vector of the Laplacian matrix of the correlation graph is calculated, and the time abrupt change point is identified by comparing the changes of the singular value vector in adjacent time windows.

[0011] In one optional implementation, obtaining the initial priority queue, adjusting the priority weights of data blocks whose average access latency exceeds a preset latency threshold within adjacent time windows before and after the time abrupt change point, adding the adjusted weights to the initial priority queue and reordering them to obtain the final priority queue, includes: Obtain the initial priority queue for each data block in the data block set; The average access latency of all data blocks within the adjacent time window before and after the time abrupt change point is calculated, and the data blocks whose average access latency exceeds a preset latency threshold are marked as data blocks to be adjusted. Based on the average access latency, the priority weight of the data block to be adjusted is adjusted using a preset correction function to obtain the corrected priority weight; The corrected priority weights are added to the initial priority queue and the priority is reordered to obtain the final priority queue.

[0012] In one optional implementation, the step of retrieving a preset number of data blocks from the final priority queue to form an initial cache queue, iterating through multiple rounds of simulated access to statistically analyze the hit rate ratio of the data blocks, adjusting the cache weight of the initial cache queue based on the hit rate ratio, and reordering the data blocks to obtain the final cache queue includes: Take out a preset number of data blocks that are ranked first in the final priority queue and put them into the cache queue to obtain the initial cache queue; Simulate access requests and count the percentage of hits for each data block in the initial cache queue; If the hit rate is zero, the data block is removed from the initial cache queue, and subsequent data blocks are selected sequentially from the final priority queue to supplement it. The access request is simulated repeatedly and the hit rate is counted until the maximum number of repetitions is reached. The weights of the data blocks in the initial cache queue are adjusted and reordered based on the hit rate ratio to obtain the final cache queue.

[0013] In a second aspect, the present invention provides a system for prefetching data into a cache, comprising: The initial data acquisition module is used to acquire an initial dataset, sort the initial dataset in ascending order according to timestamps, and divide it into multiple sub-datasets. The record item generation module is used to extract the accessed memory addresses and timestamps from the subset of data to form access record items, mine the address association rules of the access record items to determine and delete abnormal records, and obtain the final record items; The data block classification module is used to extract the one-dimensional coordinates of the data blocks in the final record item, and group the data blocks according to the one-dimensional coordinates to obtain a data block set. The mutation point identification module is used to calculate the correlation coefficient between each pair of core points in each group of data blocks in the data block set, and construct an association graph with the core points as graph vertices and the correlation coefficients as edge weights. It identifies time mutation points by comparing the changes in the singular value vectors of the association graph under adjacent time windows. The priority queue generation module is used to obtain an initial priority queue, correct the priority weights of data blocks whose average access latency exceeds a preset latency threshold within the adjacent time window before and after the time mutation point, add the corrected weights to the initial priority queue and reorder them to obtain the final priority queue. The cache queue generation module is used to take a preset number of data blocks from the final priority queue to form an initial cache queue, and to calculate the hit rate of the data blocks through multiple rounds of simulated access iterations. Based on the hit rate, the cache weight of the initial cache queue is adjusted and reordered to obtain the final cache queue.

[0014] Thirdly, the present invention also provides an electronic device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement a method for prefetching data into a cache as described in any one of the preceding claims.

[0015] Fourthly, the present invention also provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform a method for prefetching data into a cache as described in any one of the preceding claims.

[0016] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention constructs structured access record items by combining addresses and corresponding timestamps in the dataset, and introduces the Apriori algorithm to mine frequent association rules between addresses. These rules are used to make anomaly judgments on access records, effectively identifying and deleting those that do not conform to the normal access pattern. Through this strategy, feature extraction in dynamic access patterns is completed, which not only eliminates abnormal access patterns in the original data, but also improves the reliability of subsequent caching decisions.

[0017] (2) In this invention, by constructing an association graph, the similarity of access behaviors between different data blocks is quantified, and by analyzing the changes in the singular value vectors of the Laplacian matrix corresponding to the association graph under adjacent time windows, the abrupt change points in access patterns can be sensitively detected. This method can quantitatively characterize the dynamic change process of access patterns and accurately locate the key time nodes where drastic changes occur, thereby providing a basis for subsequent dynamic priority adjustment.

[0018] (3) This invention dynamically adjusts the priority weight of data blocks and optimizes the priority queue based on the time mutation point and the delay value, which can adapt to changes in access mode and improve the real-time performance and rationality of hot data identification. Furthermore, by simulating access iterations to statistically analyze the hit rate and dynamically adjust the cache queue weight, the cache hit rate and resource utilization efficiency are significantly improved, ultimately forming a cache update strategy that adapts to access mode. Attached Figure Description

[0019] Figure 1 This is a schematic flowchart of a method for prefetching data into a cache according to the first embodiment of the present invention; Figure 2 This is a schematic diagram of a system structure for data prefetching to a cache provided in the second embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Reference Figure 1 The first embodiment of the present invention provides a schematic diagram of a method for prefetching data into a cache, including steps S11 to S16, as follows: S11, Obtain the initial dataset, sort the initial dataset in ascending order according to the timestamp, and divide it into multiple sub-datasets; S12, extract the accessed memory addresses and timestamps from the subset of data to form access record items, mine the address association rules of the access record items to determine and delete abnormal records, and obtain the final record items; S13, extract the one-dimensional coordinates of the data blocks in the final record item, and group the data blocks according to the one-dimensional coordinates to obtain a data block set; S14, calculate the correlation coefficient between each pair of core points in each group of data blocks in the data block set, and construct an association graph with the core points as graph vertices and the correlation coefficients as edge weights. Identify time change points by comparing the changes in the singular value vectors of the association graph under adjacent time windows. S15, obtain the initial priority queue, correct the priority weight of data blocks whose average access latency exceeds a preset latency threshold in the adjacent time window before and after the time mutation point, add the corrected weight to the initial priority queue and reorder them to obtain the final priority queue. S16, a preset number of data blocks are taken out from the final priority queue to form an initial cache queue. After multiple rounds of simulated access iterations, the hit rate of the data blocks is statistically analyzed. The cache weight of the initial cache queue is adjusted according to the hit rate and the data blocks are reordered to obtain the final cache queue. In step S11, an initial dataset is obtained, and the initial dataset is sorted in ascending order according to timestamps and divided into multiple sub-datasets, including: Obtain the initial dataset; The initial dataset is sorted in ascending order according to its timestamp to obtain an ordered dataset. The ordered dataset is divided into sub-datasets according to a preset time interval.

[0022] It's worth noting that pre-set monitoring tools, such as the Performance Monitoring Unit (PMU) provided by Intel processors, capture every data access request between the processor and memory data blocks in real time. The monitoring tool records key attributes of each request, including a timestamp, the physical address of the accessed data block, and the task ID (process or thread identifier) ​​that issued the request. To ensure data integrity, a circular buffer or lock-free queue is typically used to temporarily store the captured records, preventing data loss due to high-frequency access. Within a preset time period, such as 7 AM to 9 AM, all data access requests are collected to form the initial dataset.

[0023] It's worth noting that the initial collection order of records in the dataset may deviate from the actual chronological order. For example, during multi-core parallel collection, timestamps from different cores may be out of order. Therefore, all records are reordered by timestamp. The sorting is based on the timestamp field of each record in the initial dataset, arranged chronologically. Quicksort or mergesort is used as the sorting algorithm. During sorting, the address and task ID of each record move along with the timestamp as supplementary information. After sorting, an ordered dataset is obtained, sorted in ascending order by timestamp. This ordered dataset ensures the correctness of subsequent time window division. For example, in the initial dataset, record 1 has a timestamp of 1620099603, record 2 has a timestamp of 1620099601, and record 3 has a timestamp of 1620099602. After reordering, the order of the records is adjusted to record 2, record 3, record 1.

[0024] It is worth noting that the preset time interval The time interval is set to 5 seconds. This 5-second interval is based on the analysis requirements of time-series access behavior and was verified through a large amount of historical access data. It is determined that 95% of valid access associations in historical data occur within a 5-second time window; longer or shorter time intervals would lead to incomplete feature extraction or increased noise interference. The start time of the dataset is then determined. That is, the timestamp of the first record. Then, the entire ordered dataset is traversed in chronological order, and for each record, its timestamp is used to... Calculate the time period number to which it belongs. The formula for calculating the time period number is: Group all records with the same segment number into the same subset. After traversing all records, you will obtain a series of subsets arranged in chronological order.

[0025] For example, there is a start timestamp of an ordered dataset. The time interval is 1620099600. =5 seconds, timestamp of a certain record Substituting into the formula, we get the time period number as 2, and this record is assigned to the second subset. When the time interval is 0, it is assigned to the 0th sub-dataset by substituting it into the formula.

[0026] In step S12, the accessed memory addresses and timestamps in the subset of data are extracted to form access record items. Address association rules for these access record items are then mined to identify and delete abnormal records, resulting in the final record items, including: Extract the accessed memory addresses from the subset of data, and combine the memory addresses with the corresponding timestamps to form access record items; The address association rules for the access record entries are extracted using the Apriori algorithm. The access record items are judged to be abnormal according to the address association rules. The records that are judged to be abnormal are deleted to obtain the final record items.

[0027] It's worth noting that the process iterates through each access record in the subset, extracting all accessed memory addresses. These extracted memory addresses are then normalized, for example, aligned by page (4KB page) or by cache line (64 bytes). Addresses within the same page or cache line are merged into a single identifier. All unique normalized addresses within each subset are then integrated with their corresponding timestamps to form a single access record entry. The data format is timestamp followed by an address set. For example, subset 1 contains memory addresses 0x7FFF1010, 0x7FFF1030, and 0x7FFF2050. After aligning to 64-byte cache lines, the first two addresses belong to line 0x7FFF1000, and the third address to line 0x7FFF2040. After normalization, these are combined into a timestamp of 1620099600 and an address set {0x7FFF1000, 0x7FFF2040}, forming a single access record entry.

[0028] It's worth noting that the obtained set of access records is considered a transaction database, with each timestamp corresponding to a set of addresses constituting a transaction. The system first sets a minimum support threshold of 0.2, indicating that the address combination appears at least in 20% of the windows, and a minimum confidence threshold of 0.7. These minimum support and confidence thresholds (0.2 and 0.7) are determined based on the accuracy requirements for address access pattern mining, validated through extensive time-series samples, and are set because 95% of valid access associations in historical data fail to meet rule reliability requirements when support or confidence is below 0.7. The Apriori algorithm is used to traverse all transactions, counting the frequency of each single address, and filtering out addresses with support greater than or equal to the minimum support as frequent 1-itemsets. Candidate 2-itemsets (combinations of two addresses) are generated from these frequent 1-itemsets. The transaction database is scanned again to calculate the actual support of these candidate combinations, retaining the frequent 2-itemsets that meet the criteria. This process is repeated iteratively to generate candidate k-itemsets and prune them until no new frequent itemsets can be generated. Finally, association rules are generated based on frequent itemsets. For example, for a frequent itemset {A, B}, attempts are made to generate rules A→B and B→A. The confidence level is calculated as the number of transactions containing both A and B divided by the number of transactions containing A. Rules with a confidence level greater than or equal to the minimum confidence level are retained. The output is an address association rule, for example, {0x7FFF1000}→{0x7FFF2000}, which means that if address 0x7FFF1000 is accessed within a time window, then address 0x7FFF2000 is also highly likely to be accessed.

[0029] For example, in a subset of data, there are 50 time window transactions. The address 0x7FFF1000 appears 15 times, with a support of 0.3 ≥ 0.2, making it a frequent 1-itemset. The combination {0x7FFF1000, 0x7FFF2000} appears 11 times, with a support of 0.22 ≥ 0.2, making it a frequent 2-itemset. The confidence of the rule 0x7FFF1000→0x7FFF2000 is calculated to be 0.73 ≥ 0.7, and finally, the valid association rule {0x7FFF1000}→{0x7FFF2000} is generated.

[0030] It's worth noting that, based on address association rules, the system checks the address set of access record items in each subset. For each access record item's address set, the system applies known association rules one by one. For example, if rule A→B exists, and the window's address set contains A but not B, then the window is considered to have violated the rule, and the access record item is judged as abnormal. Abnormalities may arise from sudden atypical access patterns, behavioral mutations caused by task switching, or data acquisition errors. The system removes these abnormal records from the subset, and the remaining window records are the final records. For example, based on the association rule {0x7FFF1000}→{0x7FFF2000}, access record item 1's address set only contains 0x7FFF1000 and not 0x7FFF2000; therefore, this access record item is judged as abnormal and removed.

[0031] In step S13, the one-dimensional coordinates of the data blocks in the final record item are extracted, and the data blocks are grouped according to the one-dimensional coordinates to obtain a data block set, including: Extract all accessed data blocks and one-dimensional coordinates from the final record item; The DBSCAN clustering algorithm is used to group the data blocks according to the one-dimensional coordinates to obtain a set of classified data blocks.

[0032] It's worth noting that the data block size is 4KB. Based on the ratio of the physical address of each access record in the final record entry to the data block size, the block number corresponding to the physical address is deduced, and this block number is used as the one-dimensional coordinate of the data block. For example, if the physical address of an access record is 0x7FFF1234 and the data block size is 4KB, the corresponding data block number is calculated to be 524273; therefore, block number 524273 is used as the one-dimensional coordinate of this data block.

[0033] It is worth noting that all data blocks are sorted according to their one-dimensional coordinate values, and the DBSCAN clustering algorithm is used to classify all data blocks. Specifically, each coordinate is scanned sequentially, and for the current point, its neighborhood is found. , All points within ], where The value is typically set to 4. If the number of data blocks in the neighborhood (including itself) reaches the minimum number of points MinPts, which is typically set to 5, then the point is marked as a core point, and all unassigned points in its neighborhood are grouped into the same cluster. At the same time, the neighborhoods of these points are recursively expanded. If the number of points in the neighborhood is less than MinPts but the point falls within the neighborhood of a core point, then it is marked as a boundary point. Otherwise, it is marked as a noise point. The final output is a set of classified data blocks, that is, each data block is assigned a cluster label. Data blocks with the same label are clustered in physical space, usually corresponding to the same memory region or the same type of access pattern.

[0034] For example, the sorted data block coordinates are 524288, 524289, 524290, 524291, 524292, and 524300. (Setting...) =4, MinPts=5, the neighborhood of coordinate 524289 contains the first 5 consecutive blocks, which meets the minimum number of points, so it is marked as a core point and assigned to cluster 1; coordinate 524300 is far away from other blocks, and the number of points in its neighborhood is insufficient, so it is marked as a noise point.

[0035] It's worth noting that the pre-set access database records all access records for data blocks within the past month. Each record is formatted as a data block ID followed by an access timestamp. Therefore, by retrieving access records from the database between 8:00 AM and 9:00 AM on the current day, a list of data block IDs and timestamps is obtained. This list is then grouped by data block ID. Within each group, access timestamps are counted in 10-second intervals, sequentially counting the access counts for each data block within the intervals 8:00:00-8:00:10, 8:00:20-8:00:30, and up to 8:59:50-9:00:00. Finally, these counts are arranged chronologically to form a sequence of access counts per minute for each data block. Ultimately, each data block yields an access sequence containing 360 values, where the i-th value represents the access count within the i-th minute. The access sequence is segmented into two-minute time windows, starting from 8:00 and dividing into continuous time intervals from 8:00:00-8:02:00, 8:02:00-8:04:00, up to 8:58:00-9:00:00. The complete access sequence of each data block is then divided into multiple equal-length two-minute subsequences according to the corresponding time window, resulting in 30 access subsequences for each data block.

[0036] For example, data block ID 1001 is accessed 3 times in the interval 8:00:00-8:00:10, 5 times in the interval 8:00:10-8:00:20, and so on, counting 360 10-second intervals to form an access sequence of length 360; then it is divided into segments of 2 minutes each, and the first 12 values ​​of the access sequence constitute the first subsequence of 8:00-8:02, and so on, until 30 groups of 2-minute access subsequences are finally obtained.

[0037] In step S14, the correlation coefficients between each pair of core points in each group of data blocks in the data block set are calculated, and an association graph is constructed with the core points as vertices and the correlation coefficients as edge weights. Time abrupt changes are identified by comparing the singular value vector changes of the association graph under adjacent time windows, including: Obtain the access sequence of the core points of each group of data blocks in the data block set, and divide the access sequence into multiple sub-sequences; Extract the access frequency vector within each subsequence, and calculate the correlation coefficient between each pair of core points based on the access frequency vector; Using the core points as graph vertices and the correlation coefficients as edge weights, a correlation graph is constructed. The singular value vector of the Laplacian matrix of the correlation graph is calculated, and the time abrupt change point is identified by comparing the changes of the singular value vector in adjacent time windows.

[0038] It's worth noting that the pre-set access database records all access records for data blocks within the past month. Each record is formatted as a data block ID followed by an access timestamp. Therefore, by retrieving access records from the database between 8:00 AM and 9:00 AM on the current day, a list of data block IDs and timestamps is obtained. This list is then grouped by data block ID. Within each group, access timestamps are counted in 10-second intervals, sequentially counting the number of accesses for each data block within each interval from 8:00:00-8:00:10, 8:00:20-8:00:30, up to 8:59:50-9:00:00. Finally, these are organized chronologically to form a sequence of access counts per minute for each data block. Ultimately, each data block yields an access sequence containing 360 values, where the i-th value represents the number of accesses within the i-th minute. The access sequence is segmented into two-minute time windows, starting from 8:00 and dividing into continuous time intervals from 8:00:00-8:02:00, 8:02:00-8:04:00, up to 8:58:00-9:00:00. The complete access sequence of each data block is then divided into multiple equal-length two-minute subsequences according to the corresponding time window, resulting in 30 access subsequences for each data block.

[0039] For example, data block ID 1001 is accessed 3 times in the interval 8:00:00-8:00:10, 5 times in the interval 8:00:10-8:00:20, and so on, counting 360 10-second intervals to form an access sequence of length 360; then it is divided into segments of 2 minutes each, and the first 12 values ​​of the access sequence constitute the first subsequence of 8:00-8:02, and so on, until 30 groups of 2-minute access subsequences are finally obtained.

[0040] It's worth noting that for each 2-minute time window corresponding to an access subsequence, an access frequency vector with the same dimension as the subsequence duration is constructed, using the number of accesses per minute within the subsequence as a component. Within the same time period, the Pearson coefficient is calculated between the access frequency vectors of each data block, and the correlation coefficient between each pair of core point vectors is calculated one by one to quantify the degree of association between core points in access patterns. Finally, the correlation coefficient between core points every 2 minutes is obtained. For example, within 8:00:00-8:02:00, the access frequency vector of the core point corresponding to data block group A is [12,15,...], and the core point corresponding to data block group B is [11,14,...]. The correlation coefficient between them, calculated using the Pearson coefficient, is 0.98.

[0041] It's worth noting that each core point is treated as a vertex in the graph, and the correlation coefficients between any two core points within the same time period are used as the weights of the edges connecting those vertices. Based on the correspondence between the vertex set and the edge weight set, a correlation graph depicting the visitor relationships of the core points is constructed. Using the same method, 30 correlation graphs are constructed based on 30 sets of correlation coefficients. For example, using core points A, B, and C as three vertices, setting the edge weight between A and B to 0.98, A and C to 0.12, and B and C to 0.15, a correlation graph is constructed. The remaining 29 sets of correlation coefficients are processed sequentially using the same rules, ultimately generating 30 temporally continuous correlation graphs.

[0042] It is worth noting that for each association graph, its Laplacian matrix is ​​calculated, and then singular value decomposition is performed on the Laplacian matrix to obtain the singular values. These singular values ​​constitute the spectral feature vector of the association graph. Next, for two temporally adjacent association graphs, the cosine similarity between their singular value vectors is calculated to quantify the difference between them. A preset change threshold is typically set to 0.8. This similarity threshold of 0.8 is based on the accuracy requirements for identifying abrupt changes in data block access patterns and was determined through verification using a large number of historical access samples. It is set based on the fact that 95% of normal access periods in historical data have a cosine similarity greater than 0.8 between the singular value vectors of adjacent association graphs, while abrupt change periods are generally below this threshold. When the cosine similarity is below 0.8, a sudden change in the access coordination pattern is considered, and this is identified as a time abrupt change point. This may correspond to events such as task switching, load fluctuations, or cache failures. The timestamps of all abrupt change points are recorded for subsequent latency analysis and strategy adjustments.

[0043] For example, the spectral feature vector of the 5th association graph is [0.92, 0.75, 0.31], and the spectral feature vector of the adjacent 6th association graph is [0.45, 0.22, 0.08]. The cosine similarity between the two is calculated to be 0.65, which is lower than the threshold of 0.8. Therefore, this moment is determined to be a time change point, and the corresponding timestamp of 8:10-8:12 is recorded.

[0044] In step S15, an initial priority queue is obtained. The priority weights of data blocks whose average access latency exceeds a preset latency threshold within the adjacent time window before and after the time abrupt change point are corrected. The corrected weights are added to the initial priority queue and reordered to obtain the final priority queue, including: Obtain the initial priority queue for each data block in the data block set; The average access latency of all data blocks within the adjacent time window before and after the time abrupt change point is calculated, and the data blocks whose average access latency exceeds a preset latency threshold are marked as data blocks to be adjusted. Based on the average access latency, the priority weight of the data block to be adjusted is adjusted using a preset correction function to obtain the corrected priority weight; The corrected priority weights are added to the initial priority queue and the priority is reordered to obtain the final priority queue.

[0045] It is worth noting that the priority order of each data block in the data block set is obtained through a preset priority database. The preset priority database records the initial priority weight of each data block. All data blocks are sorted from high to low according to their initial priority weights and stored in an initial priority queue. For example, if the initial weights of data blocks 521001, 521002, 521003, and 521004 in the priority database are 9.2, 8.5, 7.8, and 6.3, after sorting them from high to low weights, the initial priority queue is [521001, 521002, 521003, 521004].

[0046] It's worth noting that for each detected time abrupt change, the system calculates the access latency of all data blocks within the preceding time window (e.g., 5 seconds before the abrupt change) and the following time window (5 seconds after the abrupt change). Access latency can be obtained by collecting the time consumed for each memory access using hardware performance counters such as Intel PEBS. The average latency of each data block within the window is the ratio of the sum of the latencies of all access requests for that block to the number of accesses. A preset latency threshold of 3 milliseconds is set. If the average latency of a data block exceeds this threshold, the data block is marked as needing adjustment. This 3-millisecond latency threshold is determined based on memory access performance and system response efficiency requirements, verified through extensive real-world testing. It is based on historical data showing that 95% of normal data block accesses have an average latency below 3 milliseconds, while the average latency is generally higher during access congestion, resource contention, or abnormal access.

[0047] For example, within a 5-second window before and after mutation point 1, the average latency of data block 521001 is 4.2 milliseconds, which exceeds the 3-millisecond threshold and is marked as a data block to be adjusted.

[0048] It's worth noting that for each data block to be adjusted, the priority correction amount is calculated based on the relationship between its average access latency and the latency threshold. This is achieved through a correction function. ,in This is the amplification factor, usually set to 0.5. The original priority weight is multiplied by the correction factor to obtain the new priority weight, which is the corrected priority weight. For example, if the average latency of data block 1001 to be adjusted is 4.2ms and the latency threshold is 3ms, the calculated correction factor is 1.6; its initial weight of 9.2 multiplied by 1.6 yields a corrected priority weight of 14.72.

[0049] It is worth noting that the calculated priority weights of each data block to be adjusted are used to replace the initial priority weights in the original priority queue. Then, all data blocks in the queue are re-sorted in descending order of priority weight to form the final priority queue, which stores the block number and final priority weight of the data blocks. For example, if the priority weight is adjusted to 14.72, the initial priority weight of data block 1001 is replaced with 9.2. The other data blocks do not need to be modified and their priority weights remain unchanged. After re-sorting, the final priority queue is [521001(14.72),521002(8.5),521003(7.8),521004(6.3)].

[0050] In step S16, a preset number of data blocks are taken from the final priority queue to form an initial cache queue. After multiple rounds of simulated access iterations, the hit rate of these data blocks is statistically analyzed. Based on this hit rate, the cache weight of the initial cache queue is adjusted and the data blocks are reordered to obtain the final cache queue. This includes: Take out a preset number of data blocks that are ranked first in the final priority queue and put them into the cache queue to obtain the initial cache queue; Simulate access requests and count the percentage of hits for each data block in the initial cache queue; If the hit rate is zero, the data block is removed from the initial cache queue, and subsequent data blocks are selected sequentially from the final priority queue to supplement it. The access request is simulated repeatedly and the hit rate is counted until the maximum number of repetitions is reached. The weights of the data blocks in the initial cache queue are adjusted and reordered based on the hit rate ratio to obtain the final cache queue.

[0051] It's worth noting that the preset quantity is 20, which is the preset cache queue capacity. The first 20 data blocks are taken from the final priority queue in descending order of priority weight, and these data blocks are sequentially stored in the cache queue. This list is the initial cache queue. The order of the data blocks in the queue is consistent with the priority weight order. For example, if the final priority queue contains 100 data blocks, arranged in descending order of weight, the first 20 data blocks are taken sequentially and arranged in the same order to form an initial cache queue with a capacity of 20 [552699,522334,526563,...].

[0052] It's worth noting that the Zipf model was used to simulate 1000 access request flows. When simulating access requests using the Zipf distribution model, firstly, based on a preset total of 100 data blocks and a skew parameter s=1, an access probability is assigned to each data block. Each time a request is generated, a data block ID is independently sampled according to this probability distribution. This simulation is repeated multiple times, and a counter is set for each data block in the queue. Whenever a simulated request arrives, if the requested data block exists in the initial cache queue, it is counted as a hit for that data block; otherwise, it is ignored. After the simulation, the hit count for each data block equals its counter value. Finally, the ratio of the hit count for each data block to the total number of requests is used as the hit percentage.

[0053] For example, in a simulated 1000 access requests, data block 552899 was hit 120 times, with a hit rate of 0.12; data block 520005 was hit 0 times, with a hit rate of 0; the remaining data blocks were hit 30 to 80 times respectively, corresponding to their respective hit rates.

[0054] It's worth noting that the process iterates through each data block in the initial cache queue, checking its hit percentage in simulated access requests. If a data block's hit percentage is 0 (meaning it has never been hit by any simulated request), it is removed from the queue. Then, from the remaining data blocks in the final priority queue, new data blocks are retrieved sequentially in descending order of priority and added to the end of the initial cache queue. This process is repeated 1000 times, checking for data blocks with a hit percentage of 0 after each simulation. If such blocks exist, this process continues until the hit percentage of all data blocks in the queue is greater than 0. If, after 5 consecutive iterations, there are still data blocks with a hit percentage of 0 in the queue, the iteration process terminates, and the remaining data blocks with a hit percentage of 0 are directly removed from the cache queue. The highest-priority remaining data block from the final priority queue is then added to the initial cache queue.

[0055] For example, traversing the initial cache queue, it finds that data block 521005 has a hit rate of 0, so it is removed, and data block 521021 is taken from the 21st position of the final priority queue and added to the queue; after simulating the request again, it is checked that the hit rate of all data blocks is greater than 0, so the iteration stops.

[0056] It is worth noting that for each data block in the initial cache queue, the formula is used... The weights are further adjusted based on the hit rate, allowing the cache queue to more closely reflect actual hit rates. Let w be the new weight, and w be the initial weight in the initial cache queue. To determine the hit rate percentage, after calculating the new weights for all data blocks, all data blocks in the initial cache queue are reordered from high to low according to the new weights. The resulting queue is the final cache queue.

[0057] For example, data block 521001 has an initial weight of 14.72 and a hit rate of 0.12. The new weight w' is calculated to be 16.4864. The new weight replaces the weight corresponding to data block 521001 in the initial cache queue 1. After reordering according to weight from high to low, the final cache queue 1 is formed.

[0058] In summary, this invention discloses a method for data prefetching into a high-speed cache. It completes the process of acquiring data access requests and constructing structured access record items. Anomaly detection is performed on these access records to effectively identify and delete those that do not conform to conventional access patterns. This strategy achieves feature extraction in dynamic access patterns, eliminating abnormal access patterns in the original data. By constructing a correlation graph and analyzing the changes in the singular value vectors of the Laplacian matrix corresponding to this correlation graph under adjacent time windows, it can sensitively detect time abrupt changes in access patterns. Based on these time abrupt changes and latency values, the priority weight of data blocks is dynamically adjusted and the priority queue is optimized, adapting to changes in access patterns and improving the real-time performance and rationality of hot data identification. Furthermore, by simulating access iterations to statistically analyze the hit rate and dynamically adjust the weight and order of the cache queue, the cache hit rate and resource utilization efficiency are significantly improved, ultimately forming a cache update strategy adapted to access patterns. This overcomes the shortcomings of existing technologies that cannot dynamically adjust caching strategies according to actual request patterns.

[0059] Reference Figure 2 The second embodiment of the present invention provides a system for data prefetching into a cache, comprising: The initial data acquisition module is used to acquire an initial dataset, sort the initial dataset in ascending order according to timestamps, and divide it into multiple sub-datasets. The record item generation module is used to extract the accessed memory addresses and timestamps from the subset of data to form access record items, mine the address association rules of the access record items to determine and delete abnormal records, and obtain the final record items; The data block classification module is used to extract the one-dimensional coordinates of the data blocks in the final record item, and group the data blocks according to the one-dimensional coordinates to obtain a data block set. The mutation point identification module is used to calculate the correlation coefficient between each pair of core points in each group of data blocks in the data block set, and construct an association graph with the core points as graph vertices and the correlation coefficients as edge weights. It identifies time mutation points by comparing the changes in the singular value vectors of the association graph under adjacent time windows. The priority queue generation module is used to obtain an initial priority queue, correct the priority weights of data blocks whose average access latency exceeds a preset latency threshold within the adjacent time window before and after the time mutation point, add the corrected weights to the initial priority queue and reorder them to obtain the final priority queue. The cache queue generation module is used to take a preset number of data blocks from the final priority queue to form an initial cache queue, and to calculate the hit rate of the data blocks through multiple rounds of simulated access iterations. Based on the hit rate, the cache weight of the initial cache queue is adjusted and reordered to obtain the final cache queue.

[0060] It should be noted that the system for data prefetching to a cache provided in this embodiment of the invention is used to execute all the process steps of the method for data prefetching to a cache described in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0061] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

[0062] This invention also provides an electronic device. The electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps described in the method embodiments above.

[0063] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0064] The electronic device may be a desktop computer, laptop, handheld computer, or smart tablet, etc. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. It may include more or fewer components than described above, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0065] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting all parts of the electronic device via various interfaces and lines.

[0066] The memory can be used to store the computer programs and modules. The processor implements various functions of the electronic device by running or executing the computer programs and modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0067] If the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or system capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0068] It should be noted that the system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0069] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for prefetching data into a cache, characterized in that, include: Obtain the initial dataset, sort the initial dataset in ascending order according to timestamps, and divide it into multiple sub-datasets; The accessed memory addresses and timestamps in the subset of data are extracted to form access record items. The address association rules of the access record items are mined to identify and delete abnormal records, and the final record items are obtained. Extract the one-dimensional coordinates of the data blocks in the final record item, and group the data blocks according to the one-dimensional coordinates to obtain a data block set; Calculate the correlation coefficient between each pair of core points in each group of data blocks in the data block set, and construct an association graph with the core points as graph vertices and the correlation coefficients as edge weights. Identify time abrupt change points by comparing the changes in the singular value vectors of the association graph under adjacent time windows. Obtain an initial priority queue, correct the priority weights of data blocks whose average access latency exceeds a preset latency threshold within the adjacent time window before and after the time mutation point, add the corrected weights to the initial priority queue and reorder them to obtain the final priority queue. A preset number of data blocks are taken from the final priority queue to form an initial cache queue. After multiple rounds of simulated access iterations, the hit rate of the data blocks is statistically analyzed. The cache weight of the initial cache queue is adjusted according to the hit rate, and the data blocks are reordered to obtain the final cache queue.

2. The method for prefetching data into a cache according to claim 1, characterized in that, The process of obtaining the initial dataset involves sorting the initial dataset in ascending order according to timestamps and dividing it into multiple sub-datasets, including: Obtain the initial dataset; The initial dataset is sorted in ascending order according to its timestamp to obtain an ordered dataset. The ordered dataset is divided into sub-datasets according to a preset time interval.

3. The method for prefetching data into a cache according to claim 1, characterized in that, The process involves extracting accessed memory addresses and timestamps from the subset of data to construct access record items, mining address association rules within these access record items to identify and delete abnormal records, and obtaining the final record items, including: Extract the accessed memory addresses from the subset of data, and combine the memory addresses with the corresponding timestamps to form access record items; The address association rules for the access record entries are extracted using the Apriori algorithm. The access record items are judged to be abnormal according to the address association rules. The records that are judged to be abnormal are deleted to obtain the final record items.

4. The method for prefetching data into a cache according to claim 1, characterized in that, The step involves extracting the one-dimensional coordinates of data blocks from the final record item, grouping the data blocks according to the one-dimensional coordinates, and obtaining a data block set, including: Extract all accessed data blocks and one-dimensional coordinates from the final record item; The DBSCAN clustering algorithm is used to group the data blocks according to the one-dimensional coordinates to obtain a set of classified data blocks.

5. The method for prefetching data into a cache according to claim 1, characterized in that, The calculation of the correlation coefficient between each pair of core points in each group of data blocks in the data block set, and the construction of an association graph with the core points as vertices and the correlation coefficients as edge weights, and the identification of time abrupt change points by comparing the changes in the singular value vectors of the association graph under adjacent time windows, includes: Obtain the access sequence of the core points of each group of data blocks in the data block set, and divide the access sequence into multiple sub-sequences; Extract the access frequency vector within each subsequence, and calculate the correlation coefficient between each pair of core points based on the access frequency vector; Using the core points as graph vertices and the correlation coefficients as edge weights, a correlation graph is constructed. The singular value vector of the Laplacian matrix of the correlation graph is calculated, and the time abrupt change point is identified by comparing the changes of the singular value vector in adjacent time windows.

6. The method for prefetching data into a cache according to claim 1, characterized in that, The process of obtaining the initial priority queue involves adjusting the priority weights of data blocks whose average access latency exceeds a preset latency threshold within adjacent time windows before and after the time abrupt change point, adding the adjusted weights to the initial priority queue, and reordering them to obtain the final priority queue. This includes: Obtain the initial priority queue for each data block in the data block set; The average access latency of all data blocks within the adjacent time window before and after the time abrupt change point is calculated, and the data blocks whose average access latency exceeds a preset latency threshold are marked as data blocks to be adjusted. Based on the average access latency, the priority weight of the data block to be adjusted is adjusted using a preset correction function to obtain the corrected priority weight; The corrected priority weights are added to the initial priority queue and the priority is reordered to obtain the final priority queue.

7. The method for prefetching data into a cache according to claim 1, characterized in that, The process of retrieving a preset number of data blocks from the final priority queue to form an initial cache queue, iterating through multiple rounds of simulated access to statistically analyze the hit rate of the data blocks, adjusting the cache weight of the initial cache queue based on the hit rate, and reordering the data blocks to obtain the final cache queue includes: Take out a preset number of data blocks that are ranked first in the final priority queue and put them into the cache queue to obtain the initial cache queue; Simulate access requests and count the percentage of hits for each data block in the initial cache queue; If the hit rate is zero, the data block is removed from the initial cache queue, and subsequent data blocks are selected sequentially from the final priority queue to supplement it. The access request is simulated repeatedly and the hit rate is counted until the maximum number of repetitions is reached. The weights of the data blocks in the initial cache queue are adjusted and reordered based on the hit rate ratio to obtain the final cache queue.

8. A system for prefetching data into a cache, characterized in that, include: The initial data acquisition module is used to acquire an initial dataset, sort the initial dataset in ascending order according to timestamps, and divide it into multiple sub-datasets. The record item generation module is used to extract the accessed memory addresses and timestamps from the subset of data to form access record items, mine the address association rules of the access record items to determine and delete abnormal records, and obtain the final record items; The data block classification module is used to extract the one-dimensional coordinates of the data blocks in the final record item, and group the data blocks according to the one-dimensional coordinates to obtain a data block set. The mutation point identification module is used to calculate the correlation coefficient between each pair of core points in each group of data blocks in the data block set, and construct an association graph with the core points as graph vertices and the correlation coefficients as edge weights. It identifies time mutation points by comparing the changes in the singular value vectors of the association graph under adjacent time windows. The priority queue generation module is used to obtain an initial priority queue, correct the priority weights of data blocks whose average access latency exceeds a preset latency threshold within the adjacent time window before and after the time mutation point, add the corrected weights to the initial priority queue and reorder them to obtain the final priority queue. The cache queue generation module is used to take a preset number of data blocks from the final priority queue to form an initial cache queue, and to calculate the hit rate of the data blocks through multiple rounds of simulated access iterations. Based on the hit rate, the cache weight of the initial cache queue is adjusted and reordered to obtain the final cache queue.

Images