Distributed storage cache hotspot prediction method and system based on access mode
By collecting access logs in cloud PC and virtual desktop scenarios for hotspot detection and time series prediction, and combining them with machine learning models, hotspot data is dynamically scheduled to the high-speed cache. This solves the problem that cache management strategies are difficult to accurately identify hotspots in cloud environments, and improves the hit rate and overall performance of the cache system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BANGYAN TECH
- Filing Date
- 2026-01-08
- Publication Date
- 2026-04-14
AI Technical Summary
Existing cache management strategies struggle to accurately capture the spatiotemporal locality and periodic changes in data access in cloud PC and virtual desktop scenarios, resulting in inaccurate identification of hot data, delayed response, low cache resource utilization efficiency, and a lack of proactive analysis and prediction capabilities for the evolution of access patterns.
By collecting access logs from a distributed caching system, hotspot detection and time-series trend prediction are performed. Multidimensional feature vectors are constructed using machine learning models to predict hot data for future periods and dynamically schedule it to the high-speed cache, thus optimizing cache space allocation.
It significantly improves cache hit rate and system performance, can predict hot data in advance, quickly respond to sudden access demands, optimize cache resource utilization, reduce underlying storage pressure, and improve the overall performance of cloud PCs and distributed storage environments.
Smart Images

Figure CN121858043A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method and system for predicting hot spots in distributed storage cache based on access patterns, belonging to the fields of cloud computing and distributed storage technology. Background Technology
[0002] In cloud PC and virtual desktop scenarios, user data access behavior exhibits inherent patterns, with hot data often displaying characteristics such as clustering, periodic fluctuations, or sudden increases. Current mainstream cache management solutions, such as LFU and LRU algorithms, often rely on single, static statistical indicators like access frequency and recent access time for their decisions. These methods assume that data access patterns are stable and regular, but the actual cloud environment is dynamically distributed, with complex and varied access patterns that may simultaneously include periodic, sudden, or even non-stationary characteristics.
[0003] Traditional strategies rely solely on fixed rules and historical statistics for judgment, making it difficult to accurately capture deeper information such as spatiotemporal locality and periodic changes in access behavior. This leads to inaccurate identification of hot data, especially when responding to sudden access demands. Furthermore, the cache eviction mechanism of these strategies is based on simple rules, resulting in significant response lag. This can easily allow cache space to be occupied by non-hot data or periodically cold data, drastically reducing the overall utilization efficiency of cache resources.
[0004] Meanwhile, existing methods lack the ability to proactively analyze and predict the evolution of access patterns, essentially belonging to passive, reactive management. They cannot adaptively adjust to changes in system load and access patterns, nor can they optimize the layout and access paths of cached content in advance. Therefore, in complex application scenarios such as cloud PCs, traditional caching strategies, limited by their static and passive core mechanisms, are unable to achieve efficient and accurate management of hot data. Summary of the Invention
[0005] This invention provides a method and system for predicting hot spots in distributed storage cache based on access patterns, aiming to solve at least one of the technical problems existing in the prior art.
[0006] The technical solution of this invention relates to a distributed storage cache hotspot prediction method based on access patterns. The method according to this invention includes the following steps: S100: Collect access logs from each cache node in the distributed caching system to obtain raw data; S200. Perform hot zone detection based on the original data. By analyzing the gradient change of access frequency between adjacent data blocks in real time, the hot zone range is adaptively divided, and continuous hot spot data blocks with spatial locality are identified. S300. Construct a time series based on the historical access records in the original data, and apply a time series trend prediction algorithm to predict access hotspots in future periods. S400: The spatial features obtained from the hotspot detection, the temporal features obtained from the time series prediction, and the context and statistical features extracted from the access log are fused together to construct a multi-dimensional feature vector and input into the machine learning prediction model to output the hotspot probability of the corresponding data object in the future period. Specifically, based on the probability of hotspots and the popularity threshold, data predicted as hotspots are scheduled to the high-speed cache, while corresponding cache levels or storage paths are allocated to data objects with different popularity levels; the prediction model and popularity statistics are periodically updated to obtain dynamic prediction and cache optimization of hotspot data.
[0007] Furthermore, in step S100, the information collected includes the unique identifier of the accessed object, the timestamp of the access, the specific access operation type, and whether the access was hit in the cache.
[0008] Furthermore, in step S200, access statistics are performed on cached objects according to data blocks or file ranges, and hot data blocks are identified based on the number of accesses and frequency.
[0009] Furthermore, in step S400, the model is trained by fusing multi-dimensional data including time features, access frequency, user behavior patterns, access intervals and session features, and combining multi-source information such as throughput patterns, user usage patterns and IO features.
[0010] Furthermore, in step S400, the model training process includes the following steps: Access logs reported by cache nodes are centrally stored as training data sources; time-window statistics are performed on the logs through feature engineering to extract time-series features, spatial features, and contextual information; LightGBM, LSTM, or Transformer models are selected for supervised training of hotspot prediction tasks; through model evaluation and version management, the accuracy, recall, F1 score, and mean squared error are evaluated using validation or test sets, and a new version is released after the model is deemed qualified.
[0011] Furthermore, in step S400, model deployment and online learning include the following steps: The cache node extracts features in real time and dynamically calculates the latest features of the object each time it processes an access request; it loads the published prediction model through the online model service and inputs the features into the model to return the corresponding hot spot score or prediction result; it makes real-time decisions on cache tiering and eviction strategies based on the prediction result, and records the actual cache hit rate and access latency as feedback information.
[0012] Furthermore, in step S400, the model update and online learning process includes the following steps: Incremental updates are achieved by continuously collecting newly generated access logs to incrementally train the deployed model, while the model is periodically retrained using complete cumulative historical data to update the prediction model parameters. The hotspot detection threshold used in prediction is dynamically adjusted based on the real-time monitored cache hit rate and system load to enable adaptive optimization of the model.
[0013] The present invention also relates to a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the above-described method.
[0014] The technical solution of the present invention also relates to a distributed storage cache hotspot prediction system based on access patterns, the system including a computer device that includes the aforementioned computer-readable storage medium.
[0015] Furthermore, the system includes: A Cloud PC or VDI client used to initiate an access request, the access request including read / write operations, virtual machine identifier, and user identifier; A distributed caching layer, which includes a primary memory cache, a secondary solid-state drive cache, and a cache controller for managing cache operations; The hotspot prediction module includes a hotspot detection unit, a time series prediction unit, a machine learning prediction engine, and a model management and online update unit connected in sequence. The hotspot prediction module receives access requests from clients and generates hotspot prediction results to dynamically guide the cache controller's strategy. The underlying storage includes cloud storage, distributed block devices, object storage, and a log system. The beneficial effects of this invention are as follows: This invention analyzes the clustering, periodicity, and burstiness of data access to predict hot data, improve the hit rate of distributed cache and system performance, and stably improve the response speed and overall service performance of the caching system in complex and ever-changing cloud PC and distributed storage environments.
[0016] The method of this invention can effectively identify and accurately capture clustered, periodic, and sudden hotspot patterns in data access, thereby significantly improving the hit rate of the caching system. Furthermore, by combining time series analysis with machine learning prediction, it achieves continuous adaptation and learning to dynamically changing access patterns, enabling the caching system to predict hot data in advance and respond promptly, quickly adapting to sudden access demands. At the same time, the prediction mechanism optimizes the allocation strategy of cache space, improves the overall utilization efficiency of cache resources, and reduces the access pressure on the underlying storage system, thereby effectively improving the overall performance and responsiveness of cloud PCs and distributed storage environments. Attached Figure Description
[0017] Figure 1 This is a basic flowchart of the method according to the present invention.
[0018] Figure 2 This is a structural framework diagram of the system according to the present invention.
[0019] Figure 3 This is a flowchart of the model training process according to an embodiment of the present invention.
[0020] Figure 4 This is a flowchart of model deployment and online learning according to an embodiment of the present invention. Detailed Implementation
[0021] The following will provide a clear and complete description of the concept, specific structure, and technical effects of the present invention in conjunction with the embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of the present invention.
[0022] It should be noted that, unless otherwise specified, when a feature is referred to as "fixed" or "connected" to another feature, it can be directly fixed or connected to the other feature, or indirectly fixed or connected to the other feature. The singular forms "a," "described," and "the" used herein are also intended to include the plural forms, unless the context clearly indicates otherwise. Furthermore, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing particular embodiments only and not for limiting the invention. The term "and / or" as used herein includes any combination of one or more of the associated listed items.
[0023] It should be understood that although the terms first, second, third, etc., may be used to describe various elements in this disclosure, these elements should not be limited to these terms. These terms are used only to distinguish elements of the same type from one another. For example, a first element may also be referred to as a second element without departing from the scope of this disclosure, and similarly, a second element may also be referred to as a first element. Any and all instances or exemplary language (“e.g.,” “such as,” etc.) provided herein are intended only to better illustrate embodiments of the invention and, unless otherwise required, do not impose a limitation on the scope of the invention.
[0024] See Figures 1 to 4 In some embodiments of the present invention, the distributed storage cache hotspot prediction method based on access patterns according to the present invention includes at least the following steps: S100: Collect access logs from each cache node in the distributed caching system to obtain raw data; S200. Perform hot zone detection based on the original data. By analyzing the gradient change of access frequency between adjacent data blocks in real time, the hot zone range is adaptively divided, and continuous hot spot data blocks with spatial locality are identified. S300. Construct a time series based on the historical access records in the original data, and apply a time series trend prediction algorithm to predict access hotspots in future periods. S400: The spatial features obtained from the hotspot detection, the temporal features obtained from the time series prediction, and the context and statistical features extracted from the access log are fused together to construct a multi-dimensional feature vector and input into the machine learning prediction model to output the hotspot probability of the corresponding data object in the future period. Specifically, based on the probability of hotspots and the popularity threshold, data predicted as hotspots are scheduled to the high-speed cache, while corresponding cache levels or storage paths are allocated to data objects with different popularity levels; the prediction model and popularity statistics are periodically updated to obtain dynamic prediction and cache optimization of hotspot data.
[0025] This invention provides a hotspot prediction method for cloud PC caching systems. By analyzing data access patterns, it predicts hotspot data to improve cache hit rate and system performance. This invention integrates hotspot identification results, time-series prediction output, and machine learning model inference to collaboratively achieve accurate prediction and optimized management of cache hotspots in distributed storage environments. Based on a three-layer prediction mechanism—hotspot detection, time-series prediction, and machine learning models—this invention proactively identifies hotspot data in cloud PC systems, significantly improving cache hit rate and overall performance. It represents a proactive and intelligent distributed cache optimization method.
[0026] This invention integrates access pattern analysis, time series prediction, and multiple machine learning models to comprehensively and accurately identify clustering, periodicity, and suddenness patterns in data access behavior, thereby enabling dynamic and forward-looking prediction of hot data. Simultaneously, by comprehensively incorporating spatial locality detection and temporal dependency modeling, it mines data access patterns from multiple dimensions, significantly improving the accuracy and comprehensiveness of prediction results. Furthermore, based on the prediction results, the system can pre-load potentially hot data into a high-speed cache and rationally adjust the storage location of non-hot data, optimizing cache space allocation and avoiding resource conflicts and redundant consumption.
[0027] This invention not only significantly improves cache hit rate and resource utilization efficiency, but also effectively alleviates access pressure on the underlying storage. At the same time, relying on the adaptive learning capability of the machine learning model, the system can dynamically optimize prediction strategies and caching decisions based on real-time access logs, continuously adapting to the dynamic changes in user behavior. Therefore, this method can stably improve the response speed and overall service performance of the caching system in complex and ever-changing cloud PC and distributed storage environments.
[0028] In some embodiments of the present invention, during the access log collection process of the method of the present invention, access logs from a distributed storage system are collected. These logs include access time, access frequency, data block number, and access node information. Specifically, each cache node continuously collects detailed information for each data access in real time. The collected information includes at least the unique identifier of the accessed object, the timestamp of the access, the specific access operation type, and whether the access hit the cache, thus providing a complete and reliable raw data foundation for subsequent access pattern analysis and hotspot prediction.
[0029] In some embodiments of the present invention, during the hotspot detection operation, access statistics are performed on cached objects according to data blocks or file ranges. Hotspot data blocks are identified based on the number and frequency of accesses. Furthermore, the proximity of data blocks can be combined to cache multiple consecutive hotspot blocks as a whole, thereby improving space utilization efficiency. It is understood that the present invention identifies hotspot areas based on access frequency and spatial proximity, and can aggregate accesses at the block / page level to form a spatial locality assessment.
[0030] This patent's hotspot intelligent detection mechanism based on spatial access patterns employs a continuous data block access clustering feature extraction algorithm. By detecting the gradient changes in access frequency between adjacent data blocks in real time, it adaptively divides the range of hot regions. Unlike traditional hotspot detection methods that rely on fixed data block sizes, this invention's hotspot detection mechanism can effectively identify the spatial locality characteristics behind continuous access and pre-cache the identified local hotspots as whole units, thereby significantly reducing fragmented access and effectively improving caching efficiency.
[0031] In some embodiments of the present invention, in the time series analysis and prediction operation of the present invention, a time series window containing recent access history is constructed based on the access time series, and at least one time series prediction algorithm is used to predict future access trends. For the time series constructed for a single object or hotspot, methods such as SMA and EWMA are used to predict access trends and capture periodic or sudden behaviors.
[0032] Specifically, a corresponding time series model is first constructed based on the historical access data collected by the system. Then, trend prediction methods are used to predict future hot access. For example, the simple moving average (SMA) algorithm is used to smooth the recent access frequency to obtain a stable prediction trend, or the exponentially weighted moving average (EWMA) algorithm is used to assign higher weights to more recent sudden access events. This enables the prediction model to more sensitively capture and quickly respond to the dynamic changes in access patterns. As a result, the system can effectively predict hot data based on access patterns in the time dimension.
[0033] The core of the time-series prediction mechanism introduced in this invention lies in achieving proactive prediction of hot data rather than relying on post-event statistics. By applying existing time-series prediction algorithms such as SMA / EWMA to cloud PC cache hotspot identification scenarios, it can effectively identify periodic access patterns such as daily fixed office behaviors, while effectively capturing sudden access trends. Furthermore, based on the algorithm's fast computation characteristics, the entire prediction process can run stably in the production environment with low system overhead.
[0034] In some embodiments of the present invention, in the machine learning prediction operation of the present invention, time series and contextual features are combined, a trained model is input, and a hot spot score or access probability for a future period of time is output. It uses historical access logs to train a dedicated prediction model to achieve dynamic and adaptive prediction of hot spot data.
[0035] Specifically, this invention inputs access logs into a machine learning model and calculates the hotspot probability of data objects using a trained access pattern prediction model. The model framework of this invention integrates multi-dimensional data including time features, access frequency, user behavior patterns, access intervals, and session features, and combines multi-source information such as throughput patterns, user usage patterns, and IO features for model training. Lightweight tree models such as LightGBM or XGBoost can be applied, or deep time-series models such as LSTM or Transformer can be used. The trained model can directly output the probability of data becoming a hotspot within a specific future time period, thus achieving accurate and efficient probabilistic prediction of cache hotspots. Furthermore, the multi-source feature fusion ML prediction framework constructed in this invention can select prediction model frameworks such as LightGBM, XGBoost, LSTM, and Transformer for hotspot prediction.
[0036] In some embodiments of the present invention, see Figure 3 The machine learning prediction process of this invention includes data collection and feature construction; model selection and training; model deployment and online prediction; model updating and online learning.
[0037] Specifically, each cache node first systematically collects detailed access logs, including data block or file identifiers, access timestamps, operation types, and cache hit status. Then, statistics are performed according to a preset time window, and a multi-dimensional feature vector is constructed for each object to be predicted. This multi-dimensional feature vector includes time-series features, spatial features, contextual features, and statistical features. The time-series features include the number of accesses over the past N windows, the rate of change in access frequency, and the average access interval. Spatial features include the hot zone to which the object belongs and the number of accesses to neighboring data blocks. Contextual features include cache layer status, object size, read / write ratio, and user or virtual machine ID. Statistical features include moving average (SMA), exponentially weighted moving average (EWMA), standard deviation, and peak value.
[0038] Furthermore, in the model selection stage, this invention can use gradient boosting tree models such as LightGBM or XGBoost to predict future access frequency or hotspot scores, or it can use time series deep learning models such as LSTM or Transformer to process sequence data and predict access probabilities.
[0039] Furthermore, during model training, a training set is first constructed, where each sample corresponds to the feature vector of an object in the past time window and the object's future access frequency or hotspot tags. The training process includes: adjusting the tree depth, learning rate, and number of leaf nodes for the gradient boosting tree model and fine-tuning using a validation set; and setting the number of network layers, hidden units, and time steps for the LSTM or Transformer model and training using regression or classification loss functions. Subsequently, the model performance is evaluated using a test set, with evaluation metrics including accuracy, recall, F1 score, and time series prediction errors such as mean squared error and mean absolute error.
[0040] Furthermore, the trained model is deployed on cache nodes or centralized scheduling nodes. The system inputs the access characteristics of objects and outputs the corresponding hot spot prediction scores or access frequencies. Then, the caching strategy is dynamically adjusted according to the prediction results. High-scoring objects are allocated to high-speed caches such as memory or solid-state drives, secondary hot objects enter the second-level cache layer, and non-hot objects follow the standard cache path or directly access the underlying storage.
[0041] Further, see Figure 4 The system continuously collects newly generated access logs to incrementally train the deployed model for incremental updates. At the same time, it regularly retrains the model using complete accumulated historical data to capture and adapt to long-term access trend changes. In addition, the system dynamically adjusts the hot spot judgment threshold used in prediction based on the real-time monitored cache hit rate and system load to achieve adaptive optimization, thereby enabling model updates and online learning.
[0042] In some specific embodiments of the present invention, see Figure 3 The model training process of this invention includes: first, storing the access logs reported by the cache nodes as training data sources; then, performing time window statistics on the logs through feature engineering to extract time series features, spatial features, and contextual information; then, selecting models such as LightGBM, LSTM, or Transformer for supervised training on hotspot prediction tasks; and finally, evaluating accuracy, recall, F1 score, and mean squared error using validation or test sets through model evaluation and version management, and releasing a new version after the model is qualified.
[0043] In some specific embodiments of the present invention, see Figure 4The present invention includes the following steps for model deployment and online learning: First, the cache node extracts features in real time and dynamically calculates the latest features of the object each time it processes an access request. Then, the published prediction model is loaded through the online model service and the features are input into the model to return the corresponding hot spot score or prediction result. Then, the system makes real-time decisions on cache tiering strategy and eviction strategy based on the prediction result, and records feedback information such as actual cache hit rate and access latency for subsequent optimization.
[0044] In some embodiments of the present invention, the method employs dynamic adjustment based on a caching strategy. According to the hotspot probability and a popularity threshold, data predicted as hotspots is scheduled to a high-speed cache. The prediction model and popularity statistics are periodically updated to obtain dynamic prediction and cache optimization of hotspot data. Specifically, firstly, based on the hotspot score output by the model, appropriate caching levels, including memory or solid-state drives, are selected for the data objects, or specific eviction policies are executed. Simultaneously, a feedback mechanism is formed between the prediction results and actual accessed data, and feedback information, including hit rate and system load, is continuously collected. This feedback information is used to dynamically adjust the hotspot determination threshold and update the prediction model parameters, thereby achieving adaptive optimization of the system.
[0045] See Figures 1 to 4 In some embodiments of the present invention, the distributed storage cache hotspot prediction system based on access patterns according to the present invention includes: a Cloud PC or VDI client for initiating access requests, the access requests including read / write operations, virtual machine identifiers, and user identifiers; a distributed cache layer, the distributed cache layer being configured with a first-level memory cache, a second-level solid-state drive cache, and a cache controller for managing cache operations; a hotspot prediction module, the hotspot prediction module serving as the decision core including a hotspot detection unit, a time series prediction unit, a machine learning prediction engine, and a model management and online update unit connected in sequence, the hotspot prediction module receiving access requests from clients and generating hotspot prediction results to dynamically guide the cache controller's strategy; and underlying storage, the underlying storage including cloud storage, distributed block devices, object storage, and a log system.
[0046] Specifically, see Figure 2The system of this invention first initiates a read / write request containing virtual machine identifier and user information from a Cloud PC or VDI client. This request first enters the distributed cache layer, which includes a first-level memory cache, a second-level solid-state drive cache, and cache control logic responsible for receiving access requests and querying existing caches. While processing the request, the cache layer sends the access information to the hotspot prediction module and requests the module to provide a hotspot score or prediction result. The hotspot prediction module includes hotspot detection, time series prediction, machine learning engine, and model management components, and obtains the result through comprehensive access pattern analysis. Subsequently, the cache layer decides whether to put the relevant data into the high-speed cache or the lower-level cache based on the prediction result. For requests that fail to hit the cache, they are finally sent to the underlying storage system, including object storage, block storage, or file storage system, for processing.
[0047] It should be understood that the method steps in the embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable storage medium. The method can use standard programming techniques. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if necessary, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. Furthermore, for this purpose, the program can run on a programmed application-specific integrated circuit (ASIC).
[0048] Furthermore, the procedures described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by the context. The procedures described herein (or variations and / or combinations thereof) may be executed under the control of one or more computer systems configured with executable instructions, and may be implemented by hardware or a combination thereof as code (e.g., executable instructions, one or more computer programs, or one or more applications) that commonly executes on one or more processors. The computer program comprises a plurality of instructions executable by one or more processors.
[0049] Furthermore, the method can be implemented in any suitable type of computing platform, including but not limited to personal computers, minicomputers, mainframes, workstations, networked or distributed computing environments, standalone or integrated computer platforms, or in communication with charged particle tools or other imaging devices, etc. Aspects of the invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optical read and / or write storage medium, RSM, ROM, etc., such that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein. Furthermore, the machine-readable code, or portions thereof, can be transmitted via wired or wireless networks. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media comprises instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. When programmed according to the methods and techniques described in the invention, the invention may also include the computer itself.
[0050] A computer program can be applied to input data to perform the functions described herein, thereby transforming the input data to generate output data stored in non-volatile memory. The output information can also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including specific visual depictions of physical and tangible objects generated on the display.
[0051] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention, as long as they achieve the technical effects of the present invention by the same means, should be included within the scope of protection of the present invention. Within the scope of protection of the present invention, the technical solutions and / or implementation methods can have various modifications and variations.
Claims
1. A distributed storage cache hotspot prediction method based on access patterns, characterized in that, The method includes the following steps: S100: Collect access logs from each cache node in the distributed caching system to obtain raw data; S200. Perform hot zone detection based on the original data. By analyzing the gradient change of access frequency between adjacent data blocks in real time, the hot zone range is adaptively divided, and continuous hot spot data blocks with spatial locality are identified. S300. Construct a time series based on the historical access records in the original data, and apply a time series trend prediction algorithm to predict access hotspots in future periods. S400: The spatial features obtained from the hotspot detection, the temporal features obtained from the time series prediction, and the context and statistical features extracted from the access log are fused together to construct a multi-dimensional feature vector and input into the machine learning prediction model to output the hotspot probability of the corresponding data object in the future period. Specifically, based on the probability of hotspots and the popularity threshold, data predicted as hotspots are scheduled to the high-speed cache, while corresponding cache levels or storage paths are allocated to data objects with different popularity levels; the prediction model and popularity statistics are periodically updated to obtain dynamic prediction and cache optimization of hotspot data.
2. The method according to claim 1, characterized in that, In step S100, The information collected includes the unique identifier of the accessed object, the timestamp of the access, the specific type of access operation, and whether the access was cached.
3. The method according to claim 2, characterized in that, In step S200 Access statistics are performed on cached objects according to data blocks or file ranges, and hot data blocks are identified based on the number of accesses and frequency.
4. The method according to claim 2, characterized in that, In step S400, the model is trained by fusing multi-dimensional data, including time features, access frequency, user behavior patterns, access intervals, and session features, and combining multi-source information such as throughput patterns, user usage patterns, and IO features.
5. The method according to claim 5, characterized in that, In step S400, the model training process includes the following steps: Access logs reported by cache nodes are centrally stored as training data sources; time-window statistics are performed on the logs through feature engineering to extract time-series features, spatial features, and contextual information; LightGBM, LSTM, or Transformer models are selected for supervised training of hotspot prediction tasks; through model evaluation and version management, the accuracy, recall, F1 score, and mean squared error are evaluated using validation or test sets, and a new version is released after the model is deemed qualified.
6. The method according to claim 5, characterized in that, In step S400, model deployment and online learning include the following steps: The cache nodes extract features in real time and dynamically calculate the latest features of the object each time an access request is processed; The published prediction model is loaded through an online model service, and the features are input into the model to return the corresponding hotspot score or prediction result. Based on the prediction result, a real-time decision is made on the cache tiering strategy and eviction strategy, while the actual cache hit rate and access latency are recorded as feedback information.
7. The method according to claim 1, characterized in that, In step S400, the model update and online learning process includes the following steps: Incremental updates are achieved by continuously collecting newly generated access logs to incrementally train the deployed model, while the model is periodically retrained using complete cumulative historical data to update the prediction model parameters. The hotspot detection threshold used in prediction is dynamically adjusted based on the real-time monitored cache hit rate and system load to enable adaptive optimization of the model.
8. A computer-readable storage medium having stored thereon program instructions that, when executed by a processor, perform the method as described in any one of claims 1 to 7.
9. A distributed storage cache hotspot prediction system based on access patterns, characterized in that, include: A computer device, the computer device comprising the computer-readable storage medium according to claim 9.
10. The system according to claim 9, characterized in that, The system includes: A Cloud PC or VDI client used to initiate an access request, the access request including read / write operations, virtual machine identifier, and user identifier; A distributed caching layer, which includes a primary memory cache, a secondary solid-state drive cache, and a cache controller for managing cache operations; The hotspot prediction module includes a hotspot detection unit, a time series prediction unit, a machine learning prediction engine, and a model management and online update unit connected in sequence. The hotspot prediction module receives access requests from clients and generates hotspot prediction results to dynamically guide the cache controller's strategy. The underlying storage includes cloud storage, distributed block devices, object storage, and log systems.
Citation Information
Cited By
Memory access prediction method and system, computing device, storage medium and product
CN122240525A