A cloud-based intelligent data storage method

Abstract

This invention discloses a cloud-based intelligent data storage method, belonging to the field of data storage management technology, to solve the problem of lagging identification of hot and cold data. This invention identifies data objects with high access frequency and deep path hierarchy by constructing an access behavior graph and calculating concurrent access coupling weights. It dynamically generates structure adjustment instructions, reducing access link complexity and index hierarchy redundancy. Combining the response latency before and after structure adjustment with the hotness prediction error, it constructs combined features to update the hotness vector calculation weights, achieving closed-loop linkage of access behavior perception, structure reconstruction, and model optimization. This method improves the scheduling efficiency of hot data, enhances adaptability to changes in storage access patterns, and has good dynamic adjustment capabilities and structure response performance, making it more suitable for large-scale intelligent data management scenarios under cloud platforms.

Description

Technical Field

[0001] This invention relates to the field of data storage management technology, and more specifically, to a cloud platform-based intelligent data storage method. Background Technology

[0002] Currently, cloud platforms have become the mainstream infrastructure for centralized storage and unified management of large-scale heterogeneous data. They have the characteristics of horizontal scalability, high availability and multi-tenant isolation, and widely support scenarios with extremely high data access performance requirements, such as e-commerce platforms, financial systems and content delivery networks (CDNs). In order to reduce storage costs and improve retrieval efficiency, the existing common practice is to adopt a hot and cold data tiered management mechanism, which stores frequently accessed data (hot data) on high-performance media such as SSD nodes, and migrates low-access frequency data (cold data) to archive media or remote storage nodes. Combined with object indexing, distributed routing and other technologies, the overall throughput capacity is improved.

[0003] The shortcomings of existing technologies: In the current cloud platform's hot and cold data scheduling strategy, the determination of data popularity usually relies on the statistics of access frequency within a fixed time window. This mechanism lacks the ability to model the evolution trend of access behavior and cannot capture the drastic fluctuations in access popularity at the micro time scale. Especially for sudden access hotspots, which have a short formation time and unstable life cycle, they often cool down before reaching the popularity threshold, leading to misjudgment as cold data and failure to adjust their physical location or index path in time. This ultimately leads to a decrease in cache hit rate and access link congestion. The reason for this problem is that the existing system simplifies the popularity identification logic to a linear counting accumulation model and fails to transform access behavior into structured information objects such as time series graphs, access trajectories, or state transition models. As a result, when faced with complex access patterns, the system can only rely on rule triggers and cannot predict in advance. This causes originally hot data to be misplaced on low-performance nodes, further inducing increased read latency, increased network load, and delayed business response. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, the following solution is proposed to solve the problem of delayed identification of hot and cold data in the above-mentioned background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A cloud-based intelligent data storage method includes the following steps:

[0007] S1. Collect access logs of the data object within a specified time range, calibrate the timestamp of each access record according to a unified time benchmark, and construct a structured sequence of access behaviors;

[0008] S2. Extract the number of access hops, the rate of change of access frequency, and the path concentration from the access behavior sequence to generate a heat vector, which is used to describe the access status of the data object.

[0009] S3. Construct an access behavior graph based on the heat vector, calculate the number of times data objects are requested simultaneously within the same time window, and normalize the number of times into concurrent access coupling weight;

[0010] S4. Identify a set of data objects whose access frequency ranking is above a preset threshold and whose path level is greater than a specified level in the access behavior graph, and generate a data structure adjustment instruction containing the target object identifier and the structure adjustment action type.

[0011] S5. Execute the data structure adjustment instruction to migrate the physical storage location and adjust the index path hierarchy of the target object, thereby changing the position of the data object in the access link.

[0012] S6. Based on the changes in access delay before and after structural adjustment and the prediction error of the heat vector, calculate the adjustment parameters and update the calculation weights corresponding to each indicator in the heat vector.

[0013] In a preferred embodiment, step S1 specifically includes:

[0014] Set the time window boundaries for access log collection, and obtain the start and end times to limit the collection range;

[0015] Within the time window, collect access request information related to each data object, including request timestamp, request path, user identifier, and source node number;

[0016] Perform time synchronization processing on the original timestamps of the source node records, and align the timestamps of each access record based on the unified time reference signal of the cloud platform;

[0017] The calibrated access requests are categorized and organized according to data objects, and arranged in chronological order to construct a sequence of access behavior events for each data object.

[0018] The sequence of access behavior events is uniformly encoded into a structured format to form an access behavior data set that can be used for feature extraction.

[0019] In a preferred embodiment, step S2 specifically includes:

[0020] Within a specified time window, count the number of path layers accessed for each data object, calculate the number of hops between consecutive requests, and express the number of hops as a metric in the form of the difference in access path changes.

[0021] The number of visits to each data object per unit of time is statistically analyzed, the difference in the number of visits within adjacent time slices is calculated, and the result is standardized into an access frequency change rate index.

[0022] The number of unique request paths corresponding to each data object is counted, and the ratio of the number of unique paths to the total number of accesses is calculated to obtain the path concentration index.

[0023] The number of hops, the rate of change of access frequency, and the path concentration are uniformly normalized and formed into a heat vector triple, which is used as the input of access behavior features for graph structure construction.

[0024] In a preferred embodiment, step S3 specifically includes:

[0025] Define a fixed time window and analyze all access behavior sequences in each window;

[0026] Identify data object pairs requested simultaneously by users within the same time window, and count the co-occurrence frequency of each object pair;

[0027] Construct an adjacency matrix among data objects, using the number of co-occurrences as the initial edge weights;

[0028] Divide each edge weight in the adjacency matrix by the maximum number of co-occurrences within the current time window to generate a normalized concurrent access coupling weight, which is then used as the edge weight in the behavioral graph.

[0029] In a preferred embodiment, step S4 specifically includes:

[0030] Sort the data objects in descending order according to their access frequency values ​​within the current period to obtain the access frequency ranking.

[0031] Data objects are filtered based on access frequency ranking, and the filtered data objects are used as the first candidate set.

[0032] For each data object in the first candidate set, extract the number of path levels corresponding to it in the index structure;

[0033] Data objects with a path level number not exceeding a set threshold are filtered out to obtain the final set of data objects to be adjusted.

[0034] Assign a data identifier and structural adjustment type label to each object to be adjusted, and generate a set of structural adjustment instructions.

[0035] In a preferred embodiment, the preset threshold in step S4 is determined by statistically ranking the access frequency of all data objects within the current time window and setting a percentile ranking method.

[0036] In a preferred embodiment, step S5 specifically includes:

[0037] Based on the target object identifier in the structural adjustment instruction, locate its current physical storage node in the storage system;

[0038] The target storage node with an access response time within the access response threshold is reassigned to the target object to complete the physical migration operation of the data object.

[0039] Based on the structure adjustment type defined in the instruction, modify the path level position of the target object in the index structure and adjust the index pointer level;

[0040] Update the access path mapping table for the corresponding target object so that access requests can be quickly located based on the new path.

[0041] In a preferred embodiment, adjusting the index pointer level involves moving the pointer node position of the target data object up the index tree in the index structure, and the adjusted path depth is less than that before the adjustment.

[0042] In a preferred embodiment, step S6 specifically includes:

[0043] Record the actual access response time for the target object before and after the structural adjustment, and calculate the latency change value;

[0044] The error between the predicted popularity vector and the feature values ​​extracted from actual access behavior in the current period is statistically analyzed to generate a popularity vector prediction error vector.

[0045] Construct a combined feature of delay change value and prediction error vector, and use it to represent the degree of deviation in prediction performance;

[0046] Based on the combined characteristics, the calculation weights of the three indicators—number of hops, frequency change rate, and path concentration—are updated and used for the calculation in the next cycle.

[0047] The technical effects and advantages of the intelligent data storage method based on a cloud platform of the present invention are as follows:

[0048] This invention can dynamically perceive the behavioral characteristics of data objects during the access process without relying on traditional static rule configuration, and construct a heat vector based on multi-dimensional indicators such as access hop count, access frequency change rate and path concentration, so as to achieve fine-grained characterization of data access status.

[0049] By constructing an access behavior graph and calculating concurrent access coupling weights, this method can accurately identify high-frequency deep data objects, dynamically generate structure optimization instructions, and effectively reduce the complexity of data access links and index redundancy. Furthermore, by combining response latency and modeling errors before and after structure adjustment, combined features are constructed and used for adaptive updates of the heat vector weights, achieving closed-loop regulation between access behavior, structure adjustment, and model optimization. This not only improves the system's response efficiency to hot data but also significantly enhances the adaptability of the storage structure to changes in access patterns, exhibiting high performance, high flexibility, and strong self-adjustment capabilities. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating a cloud-based intelligent data storage method according to the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] In order to achieve the above objectives, Figure 1 A schematic diagram of a cloud-based intelligent data storage method according to the present invention is provided, which specifically includes the following steps;

[0053] S1. Collect access logs of the data object within a specified time range, calibrate the timestamp of each access record according to a unified time benchmark, and construct a structured sequence of access behaviors;

[0054] S2. Extract the number of access hops, the rate of change of access frequency, and the path concentration from the access behavior sequence to generate a heat vector, which is used to describe the access status of the data object.

[0055] S3. Construct an access behavior graph based on the heat vector, calculate the number of times data objects are requested simultaneously within the same time window, and normalize the number of times into concurrent access coupling weight;

[0056] S4. Identify a set of data objects whose access frequency ranking is above a preset threshold and whose path level is greater than a specified level in the access behavior graph, and generate a data structure adjustment instruction containing the target object identifier and the structure adjustment action type.

[0057] S5. Execute the data structure adjustment instruction to migrate the physical storage location and adjust the index path hierarchy of the target object, thereby changing the position of the data object in the access link.

[0058] S6. Based on the changes in access delay before and after structural adjustment and the prediction error of the heat vector, calculate the adjustment parameters and update the calculation weights corresponding to each indicator in the heat vector.

[0059] For the intelligent data storage system in the cloud platform, the first step is to collect and structure the access logs to build the foundation data for subsequent heat identification and behavior graph modeling.

[0060] Step S1 specifically includes:

[0061] Define the scope of access log collection. Specifically, define the collection period through the system control terminal or scheduling rules, setting the start and end times of the access logs in a uniform format to form clear time window boundaries. For example, a continuous 10-minute time window can be set. This time window limits the time range of pending access behaviors within the cloud platform system, ensuring subsequent data structure consistency. Within the aforementioned time window, collect access request information related to the data object. Each access request must contain at least four types of fields: 1) Request timestamp, recording the specific system time when the user initiated the access request; 2) Request path, indicating the logical or physical path of the data object in the current access; 3) User identifier, which can be user ID, access terminal identifier, or access account, used to distinguish different initiators; 4) Source node number, i.e., the specific request source of the current access behavior in the multi-node environment of the cloud platform, used for subsequent multi-node data alignment. Access request information can be captured in real time through a log collector or application layer proxy.

[0062] The original timestamps recorded by the source nodes are processed uniformly. Since different nodes may have system clock deviations, time synchronization calibration needs to be performed on the collected original timestamps. Specifically, the standard clock service provided by the cloud platform scheduling center is used as the time base, and the timestamps in all access records are corrected. The time alignment strategy can adopt a synchronization method similar to the NTP protocol to ensure that all request behaviors are comparable under a unified time base.

[0063] The calibrated access request data is categorized and organized according to the data object identifier. For each type of data object, all its related access requests are arranged in chronological order to form an access behavior event sequence. This sequence is a list of access events sorted by time series, and each element contains a calibrated timestamp, request path, user identifier, and node number, used to characterize the dynamic access trajectory of the object within the observation period.

[0064] Access behavior event sequences are encoded into a unified structured data format to form an access behavior data set. This set can be a JSON structure, a tabular key-value structure, or a batch record based on a columnar storage format, depending on the system's storage method. Each data object corresponds to a set of access behavior records, which are used in subsequent steps such as heat vector generation, access behavior graph construction, and concurrency coupling analysis.

[0065] In actual deployment, this step can be completed through a distributed data collection framework on the cloud platform (such as Fluentd, Logstash, etc.) in conjunction with an edge collection agent, ensuring the integrity, consistency and low latency of log collection. This process provides basic data support for all subsequent intelligent analysis and has strong engineering feasibility and universality.

[0066] Based on the structured access behavior dataset constructed in step S1, access features of each data object are extracted to generate a heat vector triplet, which serves as the key input for subsequent behavior graph construction and scheduling strategy identification.

[0067] Step S2 specifically includes the following four stages:

[0068] The first stage involves calculating the access hop count metric. Within a specified time window (e.g., the last 10 minutes or several periods), the access behavior event sequence for each data object is parsed one by one to obtain request path information. Request paths are typically represented as multi-level path identifiers, such as / bucketA / folderX / file123. The path string can be split using forward slashes ( / ) as separators, and the number of path levels is counted. For example, the above path contains 4 levels (dataset1 to projectA to file123.txt). The number of path levels is used as the basis for access level statistics, and the difference in the number of levels for each access path is used to represent the access hop count. For example, if the previous request accessed a level 3 path and the next request accessed a level 5 path, the hop count would be 2 levels. By calculating the difference in path levels between consecutive requests, the set of path hop amplitudes for the data object throughout the entire time window is statistically analyzed, and the access hop count metric for the object is described using an average value or distribution characteristics, serving as the first component of the popularity vector.

[0069] The second stage involves calculating the access frequency change rate index. The current total time window (e.g., 10 minutes) is divided into several equally wide time slices (e.g., one time slice per minute, for a total of 10 slices). For each data object, the number of times the object is accessed within each time slice is counted, forming an access frequency time series (e.g., 5 times in the 1st minute, 8 times in the 2nd minute, 2 times in the 3rd minute, etc.). The access frequency changes between adjacent time slices are calculated by difference to obtain the access frequency change series. For example, the difference between the 2nd minute and the 1st minute is 8-5=3, and the difference between the 3rd minute and the 2nd minute is 2-8=-6. The absolute values ​​of all differences are standardized to distribute them within a specified range (e.g., between 0 and 1) to improve the comparability between different objects. The standardized average change amplitude or fluctuation range is used as the access frequency change rate index for the data object, serving as the second component in the popularity vector.

[0070] The third stage involves calculating the path concentration index. Path concentration measures whether access requests are concentrated on a small number of paths, reflecting the consistency or dispersion of request paths. The specific calculation steps are as follows:

[0071] Iterate through all access events for the data object within the time window, extract the complete request path string from each record, deduplicate all extracted request paths, and count the total number of distinct (unique) request paths, denoted as the path type count. Count the total number of access records for the data object within the current time window, denoted as the total access count. The ratio of path type count to total access count is used for quantitative description. If all 100 accesses to a data object are completed through a single path, the concentration is 0.01, indicating high concentration; if each access uses a different path, the concentration approaches 1, indicating extreme dispersion. This ratio is used as the path concentration index and as the third component of the heat vector.

[0072] The fourth stage involves constructing a heat vector. This heat vector quantifies the access status characteristics of each data object within a specified time window and is a triplet vector composed of three dimensions: access hop count, access frequency change rate, and path concentration. During construction, the values ​​of these three indicators are normalized and mapped to a range of 0 to 1, thus forming a unified heat vector. For example, suppose a data object is accessed 100 times within a 5-minute time window, with an average path hop count of 2.5, an average change in access frequency between adjacent time periods of ±8, and 10 unique paths. After normalization, the access hop count is 0.42, the frequency change rate is 0.53, and the path concentration is 0.10, resulting in a heat vector of (0.42, 0.53, 0.10). This heat vector will serve as the core feature input in subsequent steps such as access behavior graph construction, hotspot identification, and structural adjustment.

[0073] It should be noted that the access hop count is the difference in path level changes between consecutive access requests, while the path level is the depth of the object in the static index structure. The two have different statistical sources and calculation purposes.

[0074] The implementation of step S3 includes:

[0075] After generating the heat vector, to further identify concurrent access relationships between data objects within the same time window, this step constructs an access behavior graph and calculates concurrent access coupling weights based on the set of data objects to which the heat vector belongs. First, a fixed-length time window is set (e.g., every 5 minutes is an analysis period). Within each window, the access behavior sequence is traversed sequentially, mapping each access record to its corresponding time window. Within each time window, combinations of data objects accessed by different users within the same window are identified, with particular attention paid to object pairs requested simultaneously by two or more users. The co-occurrence count of these object pairs is counted; the co-occurrence count refers to the cumulative number of times two data objects are requested in parallel by the same user group or multiple users within that time window. Then, an adjacency matrix is ​​constructed with data objects as graph nodes and object pairs as graph edges. The co-occurrence count is filled into the corresponding initial edge weight positions, representing the initial access coupling strength of the object pair.

[0076] To eliminate the influence of different time windows or data volumes on dimensionality, the initial weight value of each edge in the adjacency matrix is ​​normalized. Normalization is achieved by dividing the co-occurrence count of each edge by the maximum co-occurrence count within the current time window, thus scaling all edge weights to the range [0, 1], forming a standardized concurrent access coupling weight. This weight reflects the degree of parallel requests between any two data objects within the same time period; the closer the value is to 1, the tighter the coupling. Finally, using data objects as nodes and normalized coupling weights as edge values, a complete access behavior graph is constructed, serving as the input foundation for subsequent hotspot structure identification and adjustment instruction generation.

[0077] To achieve a quantitative expression of concurrent access relationships between data objects, the weight of each side in the adjacency matrix needs to be standardized to generate normalized concurrent access coupling weights.

[0078] Specifically, firstly, within each fixed time window, the co-occurrence count of all data object pairs is obtained; that is, the number of times two data objects are accessed simultaneously by the same user group or different users at approximately the same time point within that time period is counted. Then, the maximum value among all co-occurrence counts within that window is determined and recorded as the maximum co-occurrence baseline value within that window. Subsequently, for each object pair corresponding to an edge in the adjacency matrix, its original co-occurrence count is divided by the maximum co-occurrence baseline value, and the resulting ratio is the normalized concurrent access coupling weight of that object pair in the current window.

[0079] For example, if data objects A and B co-occur 12 times, and the highest co-occurrence count among all object pairs within the time window is 24, then the normalized coupling weight between A and B is 12 ÷ 24 = 0.5. After processing in this way, the weight of each edge is between [0, 1]. The closer the value is to 1, the higher the concurrent access intensity, reflecting that these two objects may have a stronger joint use or linked reading relationship in the current time period. This normalized weight will be used as the final weight value of the edge in the access behavior graph and participate in subsequent structural hotspot identification and adjustment decisions.

[0080] Step S4 specifically includes the following processing procedures:

[0081] Based on the heat vector constructed in step S2, the access frequency index of all data objects within the current access period (i.e., a fixed-length time window) is sorted. The access frequency index is obtained by counting the number of requests to data objects per unit time. The sorting process adopts a descending order to construct an access frequency ranking list. Then, according to a preset frequency threshold strategy (e.g., selecting the top 10% or objects with an access frequency greater than a certain absolute value), high-frequency access objects are selected from the ranking results to form a first candidate set. Next, for each data object in the first candidate set, its path location information in the current storage system index structure is queried, and its path level in the tree index structure is extracted. The path level refers to the path depth from the index root node to the index node of the data object, which is used to measure the access complexity and response latency potential of the object.

[0082] Subsequently, a threshold value for the number of path levels is set (e.g., more than 4 levels) to determine whether the structural complexity meets the adjustment standard. Data objects that do not meet the path level requirements are filtered out, ultimately forming a set of data objects to be restructured. Each data object in this set is assigned a unique data identifier (such as object ID or location index), and a structural adjustment type label is generated based on the difference between its current structural position and the target structural state, such as "path shift," "index pointer optimization," and "physical migration suggestion." Finally, the above identifiers and labels are combined to form a structural adjustment instruction set, providing precise targets and operational basis for subsequent structural adjustment steps (S5).

[0083] In step S4, to ensure that the identified data objects are representative and have optimization potential, the setting of the preset threshold and the determination mechanism of the number of path levels need to be further clarified and explained, as follows:

[0084] Before constructing the first candidate set, the system comprehensively counts the access frequency of all data objects within the current time window, obtaining the number of times each object is accessed per unit time. These access frequencies are then sorted from high to low to form a complete access frequency sequence. A percentile ranking analysis is performed on this sequence, converting the relative position of each data object's access frequency within the overall population into a percentile. For example, if the set filtering threshold is the top 20%, only the high-frequency accessed objects with a percentile ranking in the top 20% are selected as candidate targets for structural optimization. This method is more universal than absolute numerical judgment, automatically adapting to access distribution patterns under different storage scales and traffic intensities, thus improving the flexibility and fairness of data filtering.

[0085] For data objects already included in the first candidate set, their path complexity within the current index structure needs further evaluation. This path complexity is primarily measured by the number of path levels, which is the number of hops from the root node of the index tree along the index pointer to the target data object. For example, in a B+ tree structure, each intermediate node represents a level. A higher path level means more intermediate hops during the access process, potentially leading to greater response latency. The path level of each candidate object is extracted and statistically analyzed, and compared with a predefined level threshold, typically set as a constant (e.g., 4 or 5 levels), to determine if the index path is too deep. When the path level of a data object exceeds this threshold, it is considered a redundant structure with room for optimization and is retained in the final reconstruction candidate set; otherwise, it is excluded.

[0086] The two-level screening mechanism first performs high-frequency screening based on percentile ranking of access popularity, and then performs path complexity screening based on threshold judgment of structural depth. This effectively avoids unnecessary operational intervention on cold data or data objects with optimized structure, thereby ensuring the pertinence and effectiveness of structural adjustment.

[0087] It should be noted that the structural adjustment instructions involve target object identifiers. The source of these target object identifiers is, for example, a globally unique ID preset in the object storage system, which can be generated through the object hash value, storage path, or unified number in the metadata field. The structural adjustment action types are as follows: Type A: physical storage location migration, Type B: index path reconstruction, and Type C: simultaneous migration and reconstruction. Type A (coded as 01) indicates that only physical storage location migration is performed, which is suitable for objects with high access frequency but relatively reasonable path structure. This type of adjustment operation only involves reallocating data locations between different physical storage nodes to reduce network response latency or improve access speed. Type B (coded as 10) indicates that only index path reconstruction is performed, which is suitable for data objects with excessively deep index levels but insignificant access frequency. By adjusting the node level position of such objects in the index structure, the location path can be shortened and the logical index overhead can be reduced. Type C (coded as 11) indicates that both physical storage location migration and index path structure adjustment are performed simultaneously, which is suitable for critical objects with both high access frequency and high path level. This is the type of operation with the highest resource adjustment intensity and should be processed first within the idle window.

[0088] The key to this step is to jointly evaluate the location of access performance bottlenecks based on access frequency and path hierarchy, ensuring that optimization operations are only performed on data objects with high access frequency and poor structural response. This achieves effective local performance improvement while ensuring system stability. Each judgment criterion and threshold setting can be dynamically configured according to the actual deployed storage scale and access mode, and calibrated and updated through policy parameters during system initialization or online training to ensure that the method has good adaptability and feasibility in practical applications.

[0089] Step S5 specifically includes:

[0090] Based on the generated structure adjustment instruction set, perform structure optimization on the storage location and index path of the target data object;

[0091] Based on the target object identifier in the structural adjustment instruction, locate the physical storage node of the data object in the current storage system. This identifier is usually a unique object ID in the system, which can be mapped to the storage node location information index table. By looking up the binding record between the object ID and its physical node, determine the current storage node address of the data object and prepare to execute the corresponding structural migration operation.

[0092] After obtaining the original storage location, based on the storage response performance indicators, several candidate target storage nodes with access response times within the access response threshold range are selected from the pool of available storage nodes. The access response time is the average response time recorded during historical access, usually in milliseconds (ms). The access response threshold is a preset performance boundary condition used to limit the response capability requirements that the target node must meet. Nodes with lower network transmission latency or better I / O performance are selected from these nodes, and physical storage locations are reallocated to the target data object. The original data object content is then completely migrated to the new node. For example, by conducting concurrent response tests on available nodes, the average response time over the most recent three periods is calculated and sorted, and the optimal node that meets the access response threshold condition is selected as the migration target.

[0093] During the data migration process, data consistency verification must be performed, such as using hash check values ​​or CRC values ​​to ensure that the data content has not been tampered with or lost after the migration.

[0094] After completing the physical storage migration, the storage system can further optimize the path hierarchy of the target data object in the index structure according to the structure adjustment type specified in the structure adjustment instruction. Structure adjustment types typically include operations such as "moving the path hierarchy up," "merging index nodes," or "simplifying pointer paths." The aim is to enable the target object to be retrieved quickly via a shorter path. Taking a B+ tree as an example, moving the path hierarchy up means promoting the pointer node of the target object to a middle level closer to the root node, thus reducing the number of nodes that need to be traversed during a query. For example, if the original object is a leaf node at level six, it can be promoted to a middle node at level four or five, provided that index balance is satisfied, thereby reducing the average number of jumps.

[0095] Finally, the access path mapping table needs to be updated, replacing the original mapping records pointing to the old physical nodes and old index paths with the new node addresses and new path structures. This ensures that subsequent access requests can accurately hit the target object. The access path mapping table is a data object path lookup index maintained internally by the storage system. It is used to map logical access requests to actual physical addresses. After the update is completed, the access behavior takes effect immediately, allowing access commands to directly locate the target data without having to go through the original complex path. This improves access efficiency, reduces response latency, and provides optimized structural support for subsequent system heat assessment. The entire process logically forms a closed loop from structural adjustment command to target identification, then to physical migration, then to path reconstruction, and finally to mapping update.

[0096] In step S6, to implement the data popularity modeling and adjustment mechanism based on access effect feedback, it is necessary to quantitatively evaluate the system performance before and after structural optimization, and adjust the calculation weights of each index of the popularity vector accordingly. The specific implementation process is as follows:

[0097] For target data objects whose paths are migrated or restructured during the restructuring operation, their actual access response times before and after the restructuring are recorded. Access response time refers to the total time elapsed from the issuance of an access request to the successful reading of the target data object, measured in milliseconds (ms). It is typically calculated from the request timestamp and response completion timestamp recorded in the access log. The system averages multiple access samples to obtain the average response time of the target object before optimization and the average response time after optimization, and calculates the difference between the two, i.e., the latency change value. The latency change value can be used to reflect the degree of access performance improvement brought about by the restructuring optimization.

[0098] The system selects actual access behavior data related to the target object from the current access cycle and extracts three corresponding access indicators: access hops, access frequency change rate, and path concentration. Simultaneously, it retains the predicted values ​​of these three indicators from the heat vector predicted in the previous cycle. The actual values ​​are compared with the predicted values ​​item by item, and the deviation of each indicator is calculated to obtain the heat vector prediction error vector. This error vector reflects the degree of deviation in the accuracy of the current heat modeling. Each prediction error is obtained by subtracting the predicted value from the previous cycle from the actual observed value, normalizing the absolute value, and forming comparable vector components.

[0099] The delay change value and the above-mentioned heat vector prediction error vector are used to construct a combined feature to describe the deviation of the prediction performance of the target object after structural optimization. The combined feature is constructed by combining the delay change value and the three prediction error values ​​into a four-dimensional feature vector and performing uniform normalization to ensure scale consistency. This combined feature can be used to dynamically evaluate the influence of the current heat modeling parameters on the access status prediction results.

[0100] Based on the importance of each component in the combined feature, a weight update operation is performed. Specifically, if the latency change value is negative, it indicates an increase in access latency, and the weight of response time-related indicators in the current popularity vector will be appropriately reduced; if the deviation of a certain indicator in the prediction error vector remains large, it indicates that the indicator's contribution to the current state modeling is insufficient, and the weight of that indicator in the popularity calculation process of the next period needs to be appropriately increased. For example, if the prediction error of the access frequency change rate indicator is the largest, the coefficient of the access frequency change rate will be increased in the popularity vector calculation of the next period to improve the model's sensitivity to this feature.

[0101] Specifically, the combined features are used to characterize the deviation between the system response performance before and after structural adjustment and the heat prediction model. The specific construction method is as follows: First, obtain the access response time of the target data object before and after structural adjustment, and calculate the difference between the two to obtain the delay change value. This value reflects whether structural optimization has led to an improvement or deterioration in access performance. Simultaneously, based on the heat vector prediction results within the current period and the heat index values ​​extracted from actual access behavior (including access hops, access frequency change rate, and path concentration), calculate the residual values ​​between the three types of indicators to form a prediction error vector. This vector reflects the model's prediction accuracy for the current access behavior. Then, concatenate the delay change value with the three-dimensional prediction error vector to form a four-dimensional combined vector, where each dimension represents: the access performance change brought about by structural adjustment, the access hops prediction error, the access frequency change rate error, and the path concentration error. This combined vector is used to quantify the deviation between the current system structural response and the model prediction, and serves as a reference for weight adjustment, achieving adaptive adjustment in the heat vector generation process.

[0102] To ensure that the update process of the weights of each indicator in the heat vector is both dynamically adaptable and maintains the continuity of parameter evolution, an exponential moving average mechanism is used in step S6 to update the weights of the three indicators: visit hop count, visit frequency change rate, and path concentration. Specifically:

[0103] In each round of weight update, a combined feature vector consisting of delay change values ​​and popularity prediction errors is introduced. This combined feature is used to measure the actual impact of the current structural adjustment on access performance, and a corresponding performance deviation factor is generated based on this. A sliding update logic is then employed.

[0104] After each structural adjustment, the storage system extracts a performance deviation factor for the current period based on changes in access response time and hotspot prediction errors. This factor represents the degree of deviation between the adjusted model prediction and actual access performance. This performance deviation factor is considered a new weight adjustment signal. Subsequently, the storage system calculates weights for historically stored metrics and merges them with this performance deviation factor according to a set ratio. This set ratio is controlled by a decay factor, a real number between 0 and 1, used to balance the impact of historical stability and recent changes. For example, a decay factor of 0.8 indicates that 80% of the current weights come from the previous period, and 20% come from newly observed performance deviations.

[0105] The current weight of each indicator is obtained by weighted fusion of the historical weight of the previous period and the newly acquired performance deviation factor in the current period. The influence of the historical weight is set by the decay factor. For example, when it is 0.8, it means that 80% of the weight continues the state of the previous period and 20% is adjusted according to the current behavior. All updated weights are persistently stored and written into the configuration management system of the cloud platform. They are retrieved and loaded before the start of the next time period. This update mechanism is executed once in each round, forming a periodic weight iteration closed loop, which can effectively cope with the dynamic fluctuations of access patterns.

[0106] It should be noted that the threshold information in this embodiment was set in advance by professionals and will not be explained in detail here. Some parameters in the embodiment may have the same English letters, but they are explained with different meanings when used, and will not be explained one by one here.

[0107] This invention can dynamically perceive the behavioral characteristics of data objects during the access process without relying on traditional static rule configuration, and construct a heat vector based on multi-dimensional indicators such as access hop count, access frequency change rate and path concentration, so as to achieve fine-grained characterization of data access status.

[0108] By constructing an access behavior graph and calculating concurrent access coupling weights, this method can accurately identify high-frequency deep data objects, dynamically generate structure optimization instructions, and effectively reduce the complexity of data access links and index redundancy. Furthermore, by combining response latency and modeling errors before and after structure adjustment, combined features are constructed and used for adaptive updates of the heat vector weights, achieving closed-loop regulation between access behavior, structure adjustment, and model optimization. This not only improves the system's response efficiency to hot data but also significantly enhances the adaptability of the storage structure to changes in access patterns, exhibiting high performance, high flexibility, and strong self-adjustment capabilities.

[0109] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0110] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0111] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0112] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0113] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0114] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cloud platform-based intelligent data storage method, characterized in that: Includes the following steps: S1. Collect access logs of the data object within a specified time range, calibrate the timestamp of each access record according to a unified time benchmark, and construct a structured sequence of access behaviors; S2. Extract the number of access hops, the rate of change of access frequency, and the path concentration from the access behavior sequence to generate a heat vector, which is used to describe the access status of the data object. S3. Construct an access behavior graph based on the heat vector, calculate the number of times data objects are requested simultaneously within the same time window, and normalize the number of times into concurrent access coupling weight; S4. Identify a set of data objects whose access frequency ranking is above a preset threshold and whose path level is greater than a specified level in the access behavior graph, and generate a data structure adjustment instruction containing the target object identifier and the structure adjustment action type. S5. Execute the data structure adjustment instruction to migrate the physical storage location and adjust the index path hierarchy of the target object, thereby changing the position of the data object in the access link. S6. Based on the changes in access delay before and after structural adjustment and the prediction error of the heat vector, calculate the adjustment parameters and update the calculation weights corresponding to each indicator in the heat vector. Step S1 specifically includes: Set the time window boundaries for access log collection, and obtain the start and end times to limit the collection range; Within the time window, collect access request information related to each data object, including request timestamp, request path, user identifier, and source node number; Perform time synchronization processing on the original timestamps of the source node records, and align the timestamps of each access record based on the unified time reference signal of the cloud platform; The calibrated access requests are categorized and organized according to data objects, and arranged in chronological order to construct a sequence of access behavior events for each data object. The sequence of access behavior events is uniformly encoded into a structured format to form an access behavior data set that can be used for feature extraction; Step S2 specifically includes: Within a specified time window, count the number of path layers accessed for each data object, calculate the number of hops between consecutive requests, and express the number of hops as a metric in the form of the difference in access path changes. The number of visits to each data object per unit of time is statistically analyzed, the difference in the number of visits within adjacent time slices is calculated, and the result is standardized into an access frequency change rate index. The number of unique request paths corresponding to each data object is counted, and the ratio of the number of unique paths to the total number of accesses is calculated to obtain the path concentration index. The number of hops visited, the rate of change of visit frequency, and the path concentration are uniformly normalized and formed into a heat vector triple, which is used as the input of visit behavior features for graph structure construction. Step S3 specifically includes: Define a fixed time window and analyze all access behavior sequences in each window; Identify data object pairs requested simultaneously by users within the same time window, and count the co-occurrence frequency of each object pair; Construct an adjacency matrix among data objects, using the number of co-occurrences as the initial edge weights; Divide each edge weight in the adjacency matrix by the maximum number of co-occurrences within the current time window to generate a normalized concurrent access coupling weight, which is then used as the edge weight in the behavioral graph.

2. The intelligent data storage method based on a cloud platform according to claim 1, characterized in that: Step S4 specifically includes: Sort the data objects in descending order according to their access frequency values ​​within the current period to obtain the access frequency ranking. Data objects are filtered based on access frequency ranking, and the filtered data objects are used as the first candidate set. For each data object in the first candidate set, extract the number of path levels corresponding to it in the index structure; Data objects with a path level number not exceeding a set threshold are filtered out to obtain the final set of data objects to be adjusted. Assign a data identifier and structural adjustment type label to each object to be adjusted, and generate a set of structural adjustment instructions.

3. The intelligent data storage method based on a cloud platform according to claim 2, characterized in that: The preset threshold in step S4 is determined by statistically ranking the access frequency of all data objects within the current time window and setting a percentile ranking method.

4. The intelligent data storage method based on a cloud platform according to claim 3, characterized in that: In step S4, the number of path levels is calculated based on the depth of the path corresponding to the data object in the index structure, and the path with a level greater than a predefined constant value is used as the screening condition for reconstruction candidate objects.

5. The intelligent data storage method based on a cloud platform according to claim 4, characterized in that: Step S5 specifically includes: Based on the target object identifier in the structural adjustment instruction, locate its current physical storage node in the storage system; The target storage node with an access response time within the access response threshold is reassigned to the target object to complete the physical migration operation of the data object. Based on the structure adjustment type defined in the instruction, modify the path level position of the target object in the index structure and adjust the index pointer level; Update the access path mapping table for the corresponding target object so that access requests can be quickly located based on the new path.

6. The intelligent data storage method based on a cloud platform according to claim 5, characterized in that: Adjusting the index pointer level involves moving the pointer node position of the target data object in the index structure up to the index tree, and the path depth after adjustment is less than before adjustment.

7. The intelligent data storage method based on a cloud platform according to claim 6, characterized in that: Step S6 specifically includes: Record the actual access response time for the target object before and after the structural adjustment, and calculate the latency change value; The error between the predicted popularity vector and the feature values ​​extracted from actual access behavior in the current period is statistically analyzed to generate a popularity vector prediction error vector. Construct a combined feature of delay change value and prediction error vector, and use it to represent the degree of deviation in prediction performance; Based on the combined characteristics, the calculation weights of the three indicators—number of hops, frequency change rate, and path concentration—are updated and used for the calculation in the next cycle.

Images