Data preloading method and device, electronic equipment and storage medium

Abstract

The invention provides a data preloading method and device, electronic equipment and a storage medium. The method comprises the steps that a prediction request carrying a preset time period is sent to a prediction function module; data position information returned by the prediction function module in response to the prediction request is received, and the data position information is the accessed position predicted by the prediction function module in the preset time period; loading data content corresponding to the data position information to a local cache region according to a preloading strategy; and when a user access request is received, obtaining the data content matched with the user access request from the local cache region for response. According to the preloading scheme based on time period prediction, the matching degree of cache data and actual requirements is remarkably improved by accurately predicting a future access mode, so that read operation delay is effectively reduced, and the problem of cache resource waste caused by inaccurate prediction in a traditional method is solved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data preloading method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of cloud computing technology, cloud storage services have become the preferred solution for enterprises and individual users to store data. As a typical cloud storage service, the performance of cloud disks directly impacts user experience. In practical applications, users have particularly stringent requirements for the read performance of cloud disks, especially in business scenarios that require frequent data access.

[0003] Currently, the main method to improve cloud disk read performance is to use caching mechanisms. Common implementations include cache replacement strategies based on the Least Recently Used (LRU) algorithm and hot data identification methods based on access frequency. These techniques reduce read latency by keeping potentially repeatedly accessed data in the cache, thus reducing direct access to backend storage.

[0004] However, the preloaded data in existing solutions does not match the actual user access needs well, the utilization rate of cache resources is low, and it cannot significantly reduce read latency. Summary of the Invention

[0005] This application provides a data preloading method, apparatus, electronic device, and storage medium to solve the problems in the prior art where the preloaded data does not match the actual user access needs, the cache resource utilization is low, and the read latency cannot be significantly reduced.

[0006] Firstly, this application provides a data preloading method, including: Send a prediction request carrying a preset time period to the prediction function module; The system receives data location information returned by the prediction function module in response to the prediction request, wherein the data location information is the location that the prediction function module predicted would be accessed during the preset time period. According to the preloading strategy, the data content corresponding to the data location information is loaded into the local cache. When a user access request is received, data content matching the user access request is retrieved from the local cache to respond.

[0007] In one possible implementation, loading the data content corresponding to the data location information into the local cache according to the preloading strategy includes: Determine the predicted number of visits corresponding to each location in the data location information; Filter out locations in the data location information whose predicted access count is less than or equal to a preset number to obtain candidate locations; Load the data content corresponding to the candidate position into the local cache.

[0008] In one possible implementation, loading the data content corresponding to the candidate position into the local cache includes: Determine the predicted access frequency value corresponding to each candidate location; According to the predicted access frequency values ​​from high to low, the corresponding data content is loaded into the local cache in sequence.

[0009] In one possible implementation, the method further includes: When multiple candidate locations have the same predicted access frequency value, the predicted access data volume corresponding to the multiple candidate locations is determined. According to the predicted access data volume in descending order, the data content corresponding to the multiple candidate positions is loaded into the local cache in sequence.

[0010] In one possible implementation, loading the data content corresponding to the candidate position into the local cache includes: Determine the predicted access frequency value and predicted access data volume for each candidate position; The predicted access frequency value and the predicted access data volume are weighted and summed according to a preset weight ratio to obtain a priority score. According to the priority scores from high to low, the corresponding data content is loaded into the local cache in sequence.

[0011] In one possible implementation, the method further includes: Receive the predicted write-intensive region, the number of predicted writes corresponding to the predicted write-intensive region, and the predicted write end time returned by the prediction function module; If the actual number of writes in the predicted write-intensive region is greater than or equal to the predicted number of writes, or if the current time is greater than or equal to the predicted end time of the write operation, a garbage collection operation is triggered on the predicted write-intensive region.

[0012] In one possible implementation, the method further includes: Obtain historical storage access data, which includes access timestamp, operation type, logical block address, and data block size; The historical storage access data is trained using machine learning algorithms to generate a predictive model that can predict storage access behavior within a preset time period. The prediction model is deployed as the prediction function module, which responds to prediction requests and outputs access prediction data containing the prediction operation type, prediction logic block address, and prediction data block size.

[0013] Secondly, this application provides a data preloading apparatus, comprising: The sending module is used to send a prediction request carrying a preset time period to the prediction function module; A receiving module is configured to receive data location information returned by the prediction function module in response to the prediction request, wherein the data location information is the location accessed during the preset time period predicted by the prediction function module; The loading module is used to load the data content corresponding to the data location information into the local cache according to the preloading strategy; The acquisition module is used to retrieve data content matching the user access request from the local cache and respond when a user access request is received.

[0014] In one possible implementation, the loading module is specifically used for: Determine the predicted number of visits corresponding to each location in the data location information; Filter out locations in the data location information whose predicted access count is less than or equal to a preset number to obtain candidate locations; Load the data content corresponding to the candidate position into the local cache.

[0015] In one possible implementation, the loading module is further configured to: Determine the predicted access frequency value corresponding to each candidate location; According to the predicted access frequency values ​​from high to low, the corresponding data content is loaded into the local cache in sequence.

[0016] In one possible implementation, the loading module is further configured to: When multiple candidate locations have the same predicted access frequency value, the predicted access data volume corresponding to the multiple candidate locations is determined. According to the predicted access data volume in descending order, the data content corresponding to the multiple candidate positions is loaded into the local cache in sequence.

[0017] In one possible implementation, the loading module is further configured to: Determine the predicted access frequency value and predicted access data volume for each candidate position; The predicted access frequency value and the predicted access data volume are weighted and summed according to a preset weight ratio to obtain a priority score. According to the priority scores from high to low, the corresponding data content is loaded into the local cache in sequence.

[0018] In one possible implementation, the device further includes a recycling module for: Receive the predicted write-intensive region, the number of predicted writes corresponding to the predicted write-intensive region, and the predicted write end time returned by the prediction function module; If the actual number of writes in the predicted write-intensive region is greater than or equal to the predicted number of writes, or if the current time is greater than or equal to the predicted end time of the write operation, a garbage collection operation is triggered on the predicted write-intensive region.

[0019] In one possible implementation, the device further includes a training module for: Obtain historical storage access data, which includes access timestamp, operation type, logical block address, and data block size; The historical storage access data is trained using machine learning algorithms to generate a predictive model that can predict storage access behavior within a preset time period. The prediction model is deployed as the prediction function module, which responds to prediction requests and outputs access prediction data containing the prediction operation type, prediction logic block address, and prediction data block size.

[0020] Thirdly, this application provides an apparatus comprising: a processor and a memory, the processor being configured to execute a data preloading program stored in the memory to implement the data preloading method described in any one of the first aspects.

[0021] Fourthly, this application provides a storage medium storing one or more programs that can be executed by one or more processors to implement the data preloading method described in any one aspect.

[0022] Compared with the prior art, the technical solution provided in this application has the following advantages: First, the method provided in this application obtains the predicted data access location within a specific time period through a prediction function module, breaking through the limitation of traditional caching strategies that rely solely on simple historical access patterns. Second, it actively loads data into the local cache based on the prediction results, realizing a shift from passive caching to active preloading. Finally, it directly responds to hit requests through a cache matching mechanism, forming a complete optimization loop. This time-period-based prediction preloading scheme significantly improves the matching degree between cached data and actual needs by accurately predicting future access patterns, thereby effectively reducing read operation latency and solving the problem of wasted cache resources caused by inaccurate predictions in traditional methods. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0026] Figure 1 A flowchart illustrating an embodiment of a data preloading method provided in this application; Figure 2 A flowchart illustrating an embodiment of another data preloading method provided in this application; Figure 3 A flowchart illustrating another embodiment of the data preloading method provided in this application; Figure 4 A flowchart illustrating an overall data preloading scheme provided in this application embodiment; Figure 5 A block diagram illustrating an embodiment of a data preloading device provided in this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0029] To address the technical problems in existing technologies, such as the low matching degree between preloaded data and actual user access needs, low cache resource utilization, and inability to significantly reduce read latency, this application provides a data preloading method that can significantly improve the matching degree between cached data and actual needs, thereby effectively reducing read operation latency and solving the problem of cache resource waste caused by inaccurate prediction in traditional methods.

[0030] Figure 1 This is a flowchart illustrating an embodiment of a data preloading method provided in this application. Figure 1 As shown, the method includes the following steps: Step 101: Send a prediction request carrying a preset time period to the prediction function module.

[0031] The prediction module is a machine learning service deployed in the cloud disk system. It trains the prediction model by analyzing historical IO (Input / Output) records (including timestamps accurate to the second, read / write operation types, logical block addresses, and 4KB aligned data block sizes).

[0032] Preset time period, which is one or more future time windows configured for the user (e.g., "20250724 11:00:00-12:00:00").

[0033] In this embodiment, the cloud storage client automatically sends a prediction request for a future period (e.g., 24 hours) at a fixed time each day (e.g., 00:00). The request message uses JSON format and includes start time (start_time) and end time (end_time) fields. This triggers the prediction module to predict the location of data that may be accessed within the specified time period. This step, by setting a clear time window, allows the prediction module to optimize prediction results for specific time periods, solving the problem of traditional caching strategies lacking time-dimensional planning.

[0034] Step 102: Receive the data location information returned by the prediction function module in response to the prediction request, wherein the data location information is the location that the prediction function module predicted would be accessed during the preset time period.

[0035] The data location information is a structured array generated by the prediction module through time-series analysis and pattern recognition of historical IO data. Each element contains: logical block address (8-byte integer, such as offset 10240), predicted access count (2-byte integer), and data block size (4-byte integer, such as 4096 bytes).

[0036] In this embodiment, the cloud disk client receives data location information returned by the prediction function module through a secure communication interface. First, it performs integrity verification on the data format, including verifying the legality of the address range and that the data block size meets the storage system alignment requirements. Then, it stores the verified prediction results into the preloading task queue, providing a data foundation for subsequent accurate preloading. This step, through a standardized data interaction protocol and a rigorous verification mechanism, ensures that the prediction results can be directly used for cache optimization, solving the preloading failure problem caused by non-standard prediction data formats in traditional solutions.

[0037] Step 103: Load the data content corresponding to the data location information into the local cache according to the preloading strategy.

[0038] The preloading strategy is a decision logic that includes filtering rules. It can be set by the user according to actual needs, such as the following three-level strategy: 1) Filter low-frequency data with a predicted access count of ≤1; 2) Sort by access frequency in descending order; 3) Load data with the same frequency in descending order by data block size.

[0039] In this embodiment, firstly, the cloud disk client parses the data location information returned by the prediction function module and extracts the logical block address list of the data to be preloaded; secondly, the client filters the data according to the filtering rules in the preloading strategy and removes low-frequency access data blocks; then, a preloading task queue is generated according to the priority order of access frequency and data block size; finally, data blocks are read in batches from the cloud disk backend storage system and written to the local cache area through asynchronous IO, while updating the cache index table to record the location information of the preloaded data.

[0040] In the application, the preloading strategy is implemented with the following optimizations: 1) The client monitors memory usage in real time, and automatically pauses the preloading task when it exceeds 80% threshold. This mechanism ensures that critical system services always have sufficient memory resources; 2) During loading, the remaining space in the cache is dynamically monitored. When the available space is lower than a set threshold, the LRU (Least Recently Used) algorithm is immediately triggered to evict old data, which improves cache space utilization; 3) A version tag is added to each preloaded data item, and access is only granted after it is verified to be consistent with the backend storage metadata. This design reduces the data consistency error rate. Through the above optimization measures, system stability and data reliability are effectively balanced while ensuring preloading efficiency.

[0041] Step 104: When a user access request is received, retrieve the data content matching the user access request from the local cache and respond accordingly.

[0042] User access requests are actual I / O requests forwarded by the kernel virtualization layer (e.g., read offset 40960).

[0043] In this embodiment, when the cloud disk client receives a user access request, it first queries the local cache index table for quick matching: if the data is found in the cache, the request is responded to immediately; if not, an asynchronous origin-fetching process is initiated to retrieve the data from the backend storage, while simultaneously triggering the real-time learning mechanism of the prediction model to update the prediction parameters. This process ensures optimal response latency through a two-level cache check (prioritizing SSD (Solid State Drive) cache, followed by a memory cache pool), and employs a mechanism combining asynchronous loading and online learning to continuously optimize subsequent prediction accuracy while ensuring immediate response capabilities, thus maintaining a stable improvement in the overall system read performance.

[0044] The technical solution provided in this application firstly obtains the predicted data access location results within a specific time period through a prediction function module, breaking through the limitation of traditional caching strategies that rely solely on simple historical access patterns; secondly, it proactively loads data into the local cache based on the prediction results, realizing a shift from passive caching to proactive preloading; and finally, it directly responds to hit requests through a cache matching mechanism, forming a complete optimization loop. This time-period prediction-based preloading scheme significantly improves the matching degree between cached data and actual needs by accurately predicting future access patterns, thereby effectively reducing read operation latency and solving the problem of wasted cache resources caused by inaccurate predictions in traditional methods.

[0045] Figure 2 A flowchart illustrating an embodiment of another data preloading method provided in this application. Figure 2 The process shown is in Figure 1 Based on the illustrated process, the following steps are included: Step 201: Determine the predicted number of visits corresponding to each location in the data location information.

[0046] Predicted access frequency refers to the estimated access frequency of each logical block address (i.e., location) within a preset time period by the prediction function module.

[0047] In this embodiment, the cloud disk client parses the received data location information, extracts the predicted access count value corresponding to each location, and establishes a "location-access count" mapping table. This provides a data foundation for subsequent filtering.

[0048] Step 202: Filter the locations in the data location information whose predicted access count is less than or equal to a preset number to obtain candidate locations.

[0049] The preset number of times is the access frequency threshold configured by the system (default value ≥ 2 times).

[0050] In this embodiment, the cloud client uses comparison operators to filter out positions where the predicted number of accesses is greater than a preset number, forming a candidate position set, thereby filtering out requests that are only read once. This step optimizes cache space utilization by eliminating low-frequency access data.

[0051] Step 203: Load the data content corresponding to the candidate position into the local cache.

[0052] In one embodiment, loading the data content corresponding to the candidate position into the local cache may include the following steps: determining the predicted access frequency value corresponding to each candidate position; and loading the corresponding data content into the local cache in descending order of the predicted access frequency value.

[0053] The predicted access frequency value refers to the number of visits estimated by the prediction function module for each candidate location per unit time. This value is calculated by analyzing the time distribution characteristics in historical input and output data.

[0054] In this scheme, the cloud client first determines the predicted access frequency value for each candidate position, and then uses a quicksort algorithm to sort them from high to low frequency values, generating a priority queue. During loading, the client retrieves the corresponding data blocks from the cloud disk backend sequentially according to this priority order via the Remote Direct Memory Access Protocol and writes them to the local cache. Simultaneously, it monitors cache space usage in real time, pausing loading and triggering a Least Recently Used (LRU) eviction mechanism when the remaining space is insufficient. This scheme ensures that frequently accessed data is cached first through frequency sorting, effectively improving the cache hit rate and reducing backend storage access latency, thereby significantly optimizing cloud disk read performance.

[0055] In another embodiment, the method further includes: when multiple candidate positions have the same predicted access frequency value, determining the predicted access data volume corresponding to the multiple candidate positions; and loading the data content corresponding to the multiple candidate positions into the local cache in descending order of the predicted access data volume.

[0056] Predicted access data volume refers to the estimated size of the data block to be accessed for each candidate location by the prediction function module.

[0057] In this scheme, when multiple candidate positions have the same predicted access frequency value, the cloud client performs the following processing flow for these candidate positions: First, it extracts the predicted access data volume parameter corresponding to each candidate position. Then, it uses a stable sorting algorithm (such as merge sort) to prioritize these candidate positions according to the data volume from largest to smallest. During loading, the client prioritizes processing candidate positions with larger data volumes, acquiring the corresponding data content in batches through the Remote Direct Memory Access protocol and writing it to the local cache. By optimizing the data loading order, this scheme ensures the priority caching of frequently accessed data and improves the efficiency of a single data loading operation, thereby significantly improving the overall cache resource utilization and reducing input / output latency.

[0058] In another embodiment, loading the data content corresponding to the candidate position into the local cache may further include the following steps: determining the predicted access frequency value and the predicted access data volume corresponding to each candidate position; performing a weighted summation operation on the predicted access frequency value and the predicted access data volume according to a preset weight ratio to obtain a priority score; and loading the corresponding data content into the local cache in descending order of the priority scores.

[0059] In this scheme, the cloud client first obtains two key parameters for each candidate location: the predicted access frequency (reflecting access frequency) and the predicted access data volume (reflecting data block size). Then, it performs a weighted summation calculation using preset weight ratios (e.g., frequency weight α∈[0.6,0.8], data volume weight β=1-α) to generate a priority score for each location. Finally, it sorts the locations in descending order of their scores using a heap sort algorithm and loads the data sequentially via the Remote Direct Memory Access Protocol (RDP). This scheme integrates both access frequency and data volume metrics. Through a configurable weight adjustment mechanism, it achieves both priority caching of frequently accessed data and efficient loading of large data blocks, resulting in optimal cache space allocation and effectively improving the overall storage system's throughput and resource utilization.

[0060] Figure 2 The process described first quantifies the popularity of each data location based on predicted access frequency, establishing a data foundation for intelligent filtering. Second, threshold filtering effectively eliminates low-frequency access data, preventing cache space from being occupied by non-hotspot data. Finally, candidate data is loaded in an optimized order, ensuring that frequently accessed content is cached first. This tiered processing mechanism not only significantly improves cache hit rate but also optimizes backend storage bandwidth utilization, thereby improving the overall read performance of the cloud disk system.

[0061] Figure 3 A flowchart illustrating another embodiment of the data preloading method provided in this application. Figure 3 The process shown is in Figure 1Based on the illustrated process, the following steps are included: Step 301: Receive the prediction write-intensive region, the prediction write count, and the prediction write end time corresponding to the prediction write-intensive region returned by the prediction function module.

[0062] Step 302: If the actual number of writes in the predicted write-intensive region is greater than or equal to the predicted number of writes, or if the current time is greater than or equal to the predicted end time of the write, trigger a garbage collection operation on the predicted write-intensive region.

[0063] For ease of understanding, steps 301-302 will be explained uniformly below: Predicted write-intensive regions refer to high-probability write data blocks identified by the prediction function module based on historical write patterns. They include three key parameters: logical address range, predicted write count, and write end time (i.e., predicted write end time).

[0064] In this embodiment, the cloud client continuously monitors the actual write activity. When it detects that the actual number of writes to the area reaches or exceeds the predicted value, or when the current time exceeds the predicted end time of the write operation, it automatically triggers a garbage collection operation for that area.

[0065] Figure 3 The process shown intelligently determines the optimal time for garbage collection by predicting write behavior, which avoids data duplication caused by premature collection and prevents storage space waste caused by delayed collection, effectively optimizing the garbage collection efficiency of the storage system.

[0066] In another embodiment of this application, the method may further include the following steps: acquiring historical storage access data, the historical storage access data including access timestamps, operation types, logical block addresses, and data block sizes; training the historical storage access data using a machine learning algorithm to generate a prediction model capable of predicting storage access behavior within a preset time period; and deploying the prediction model as the prediction function module to respond to prediction requests and output access prediction data containing prediction operation types, prediction logical block addresses, and prediction data block sizes through the prediction function module.

[0067] Historical storage access data refers to structured log data recorded by the cloud disk system, which includes timestamps, read / write operation types, logical block addresses, and data block sizes.

[0068] In this embodiment, the cloud disk client first performs data cleaning and feature engineering on the raw logs, including converting timestamps into periodic features, spatially clustering logical block addresses, and extracting operation type combination patterns. Then, machine learning algorithms (such as XGBoost (eXtreme Gradient Boosting) or LSTM (Long Short-Term Memory)) are used to train the processed data through supervised learning, constructing a temporal prediction model capable of predicting access patterns within future time periods. Finally, the trained model is deployed as a prediction function module. This module receives time range parameters and outputs structured prediction results containing the predicted operation type (read / write), logical block address, and data block size. This solution accurately captures the spatiotemporal characteristics of user access behavior through machine learning methods, providing reliable data support for optimization strategies such as preloading and garbage collection, thereby significantly improving the overall performance of the storage system.

[0069] Figure 4 This is an overall flowchart of a data preloading scheme provided in an embodiment of this application. Figure 4 As shown, it includes the following steps: Phase 1, Model Training Phase: The cloud disk client collects IO data generated by user operations, such as the user's read / write offset over a period of time, as shown in Table 1-1: Table 1-1

[0070] The structured data collected above is input into the machine learning module for analysis and training. After training, the prediction module can retrieve and use the model. As shown in Table 1-2, the model can predict user behavior data such as read / write, offset, and size based on the input time.

[0071] Table 1-2

[0072] The second stage, the model application stage: Cloud disk data preloading: The cloud disk client periodically sends requests to the prediction module, with the request content including a time point, such as 20250724 11:00:00-12:00:00. The prediction module returns the offset data as shown in Table 1-2.

[0073] The cloud disk client preloads data into the cache based on the user's IO offset data. The preloading strategy is as follows: filter out requests that are read only once; prioritize preloading requests that are read multiple times; and prioritize preloading requests that read large amounts of data.

[0074] The user issues read / write I / O requests, which are then forwarded to the cloud disk client after kernel virtualization and other processes.

[0075] When the cloud drive client receives a user request, it queries the cache. If the cache is hit, it returns the user data directly. If the cache is not hit, it continues the normal process and requests the data from the backend system.

[0076] Cloud drive junk data recycling: The index server sends a request to the index module, the request content of which includes a time point, such as 20250724 11:00:00-13:00:00. The prediction module returns the offset data as shown in Table 1-2.

[0077] The index server intelligently generates reasonable GC (Garbage Collection) tasks based on user read and write patterns. As shown in Table 1-1, the GC task corresponding to offset 10240 should be generated more reasonably after 11:11, so only one data migration is required.

[0078] The GC server accepts GC tasks and performs specific garbage data collection actions.

[0079] This solution achieves a comprehensive performance improvement of the storage system through a two-stage collaborative mechanism: In the first stage (model training), the cloud disk client collects structured IO data, including timestamps, operation types, logical block addresses, and data block sizes, and uses machine learning algorithms to build a time-series model that can predict future access patterns; In the second stage (model application), a prediction-driven preloading mechanism (prioritizing high-frequency, large-size read requests) significantly improves the cache hit rate, while a prediction-guided intelligent garbage collection strategy (precisely triggering GC tasks based on write end time) minimizes the number of data migrations.

[0080] Figure 5 This is a block diagram illustrating an embodiment of a data preloading device provided in this application. Figure 5 As shown, the device includes: The sending module 51 is used to send a prediction request carrying a preset time period to the prediction function module; The receiving module 52 is configured to receive data location information returned by the prediction function module in response to the prediction request, wherein the data location information is the location accessed during the preset time period predicted by the prediction function module; Loading module 53 is used to load the data content corresponding to the data location information into the local cache according to the preloading strategy; The acquisition module 54 is used to retrieve data content matching the user access request from the local cache and respond when a user access request is received.

[0081] In one possible implementation, the loading module is specifically used for: Determine the predicted number of visits corresponding to each location in the data location information; Filter out locations in the data location information whose predicted access count is less than or equal to a preset number to obtain candidate locations; Load the data content corresponding to the candidate position into the local cache.

[0082] In one possible implementation, the loading module is further configured to: Determine the predicted access frequency value corresponding to each candidate location; According to the predicted access frequency values ​​from high to low, the corresponding data content is loaded into the local cache in sequence.

[0083] In one possible implementation, the loading module is further configured to: When multiple candidate locations have the same predicted access frequency value, the predicted access data volume corresponding to the multiple candidate locations is determined. According to the predicted access data volume in descending order, the data content corresponding to the multiple candidate positions is loaded into the local cache in sequence.

[0084] In one possible implementation, the loading module is further configured to: Determine the predicted access frequency value and predicted access data volume for each candidate position; The predicted access frequency value and the predicted access data volume are weighted and summed according to a preset weight ratio to obtain a priority score. According to the priority scores from high to low, the corresponding data content is loaded into the local cache in sequence.

[0085] In one possible implementation, the device further includes a recycling module for: Receive the predicted write-intensive region, the number of predicted writes corresponding to the predicted write-intensive region, and the predicted write end time returned by the prediction function module; If the actual number of writes in the predicted write-intensive region is greater than or equal to the predicted number of writes, or if the current time is greater than or equal to the predicted end time of the write operation, a garbage collection operation is triggered on the predicted write-intensive region.

[0086] In one possible implementation, the device further includes a training module for: Obtain historical storage access data, which includes access timestamp, operation type, logical block address, and data block size; The historical storage access data is trained using machine learning algorithms to generate a predictive model that can predict storage access behavior within a preset time period. The prediction model is deployed as the prediction function module, which responds to prediction requests and outputs access prediction data containing the prediction operation type, prediction logic block address, and prediction data block size.

[0087] like Figure 6 As shown in the figure, this application provides a device including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. Memory 113 is used to store computer programs; In one embodiment of this application, when the processor 111 executes a program stored in the memory 113, it implements the data preloading method provided in any of the foregoing method embodiments, including: Send a prediction request carrying a preset time period to the prediction function module; The system receives data location information returned by the prediction function module in response to the prediction request, wherein the data location information is the location that the prediction function module predicted would be accessed during the preset time period. According to the preloading strategy, the data content corresponding to the data location information is loaded into the local cache. When a user access request is received, data content matching the user access request is retrieved from the local cache to respond.

[0088] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the data preloading method provided in any of the foregoing method embodiments.

[0089] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0090] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0091] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0092] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A data preloading method, characterized in that, The method includes: Send a prediction request carrying a preset time period to the prediction function module; The system receives data location information returned by the prediction function module in response to the prediction request, wherein the data location information is the location that the prediction function module predicted would be accessed during the preset time period. According to the preloading strategy, the data content corresponding to the data location information is loaded into the local cache. When a user access request is received, data content matching the user access request is retrieved from the local cache to respond.

2. The method according to claim 1, characterized in that, The step of loading the data content corresponding to the data location information into the local cache according to the preloading strategy includes: Determine the predicted number of visits corresponding to each location in the data location information; Filter out locations in the data location information whose predicted access count is less than or equal to a preset number to obtain candidate locations; Load the data content corresponding to the candidate position into the local cache.

3. The method according to claim 2, characterized in that, The step of loading the data content corresponding to the candidate position into the local cache includes: Determine the predicted access frequency value corresponding to each candidate location; According to the predicted access frequency values ​​from high to low, the corresponding data content is loaded into the local cache in sequence.

4. The method according to claim 3, characterized in that, The method further includes: When multiple candidate locations have the same predicted access frequency value, the predicted access data volume corresponding to the multiple candidate locations is determined. According to the predicted access data volume in descending order, the data content corresponding to the multiple candidate positions is loaded into the local cache in sequence.

5. The method according to claim 2, characterized in that, The step of loading the data content corresponding to the candidate position into the local cache includes: Determine the predicted access frequency value and predicted access data volume for each candidate position; The predicted access frequency value and the predicted access data volume are weighted and summed according to a preset weight ratio to obtain a priority score. According to the priority scores from high to low, the corresponding data content is loaded into the local cache in sequence.

6. The method according to claim 1, characterized in that, The method further includes: Receive the predicted write-intensive region, the number of predicted writes corresponding to the predicted write-intensive region, and the predicted write end time returned by the prediction function module; If the actual number of writes in the predicted write-intensive region is greater than or equal to the predicted number of writes, or if the current time is greater than or equal to the predicted end time of the write operation, a garbage collection operation is triggered on the predicted write-intensive region.

7. The method according to claim 1, characterized in that, The method further includes: Obtain historical storage access data, which includes access timestamp, operation type, logical block address, and data block size; The historical storage access data is trained using machine learning algorithms to generate a predictive model that can predict storage access behavior within a preset time period. The prediction model is deployed as the prediction function module, which responds to prediction requests and outputs access prediction data containing the prediction operation type, prediction logic block address, and prediction data block size.

8. A data preloading device, characterized in that, The device includes: The sending module is used to send a prediction request carrying a preset time period to the prediction function module; A receiving module is configured to receive data location information returned by the prediction function module in response to the prediction request, wherein the data location information is the location accessed during the preset time period predicted by the prediction function module; The loading module is used to load the data content corresponding to the data location information into the local cache according to the preloading strategy; The acquisition module is used to retrieve data content matching the user access request from the local cache and respond when a user access request is received.

9. An electronic device, characterized in that, include: A processor and a memory, the processor being configured to execute a data preloading program stored in the memory to implement the data preloading method according to any one of claims 1-7.

10. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the data preloading method according to any one of claims 1-7.

Images