Cold and hot data grouping method and system based on access pattern clustering

Abstract

The application provides a cold and hot data grouping method and system based on access pattern clustering, obtains a full access log record set of a to-be-grouped storage system in a preset historical observation period, extracts an access timing pattern feature of a corresponding accessed data object based on an accessed data object unique identifier and an access initiation time stamp marker recorded by each entry in the access log record set, generates a period pattern descriptor containing a complete period attribute, completes access pattern similarity clustering of full accessed data objects by relying on a clearly defined access interval sequence repetition period length feature, an interval distribution feature in a single period, and a cross-period interval fluctuation feature in the period pattern descriptor, obtains multiple access pattern clustering clusters, assigns a cold and hot level label based on a comparison result of a clustering cluster aggregated access heat duration feature and a preset heat duration threshold, and generates a final cold and hot data grouping result.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically, to a method and system for grouping hot and cold data based on access pattern clustering. Background Technology

[0002] In existing storage systems, mainstream solutions for hot and cold data partitioning often rely on single, static metrics such as the number of accesses within a fixed time window and the last access time. This approach completely ignores the long-term, periodic access patterns hidden behind data objects and fails to capture the inherent similarities in access timing patterns between different data sets. Traditional hot and cold grouping mechanisms often perform isolated hotness assessments for each data object, failing to uncover common characteristics of access behavior across multiple data sets and unable to adapt to the numerous periodic batch access scenarios present in storage systems. This easily leads to blurred boundaries between hot and cold data partitioning and frequent fluctuations in grouping results. This not only causes inaccurate storage resource scheduling strategies but also results in the unreasonable migration of high-frequency accessed data to low-speed storage media and the long-term occupation of high-performance storage resources by cold data, significantly reducing the overall IO response efficiency and resource utilization of the storage system. Currently, there is no complete technical path to extract access timing pattern characteristics from the entire dataset, further explore the periodic repetition structure of access interval sequences, and then complete global hot and cold data grouping based on similar access patterns. Such technical solutions remain largely unavailable. Summary of the Invention

[0003] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide a method for grouping hot and cold data based on access pattern clustering, the method comprising:

[0004] Obtain a set of access log records for data objects in the storage system to be grouped. The set of access log records includes multiple access log entries recorded within a preset historical observation period. Each access log entry contains a unique identifier of the accessed data object and a timestamp of the access initiation time.

[0005] Access time sequence pattern extraction is performed on the access log record set. Based on the unique identifier of the accessed data object, the timestamps of all access initiation times are aggregated to generate the access time sequence of each data object. Based on the access time sequence of each data object, the access interval sequence of each data object is generated. Based on the access time sequence of each data object, the first access time and the last access time of each data object are determined to obtain the access time sequence pattern feature set of each data object. The access time sequence pattern feature set includes the access interval sequence, the first access time, and the last access time.

[0006] Based on the access time sequence pattern feature set of all data objects, the similarity structure analysis of access interval sequence is performed. The access interval pattern structure with periodic repetition characteristics is extracted from the access interval sequence of each data object, and the periodic pattern descriptor of each data object is generated. The periodic pattern descriptor includes the repetition period length feature of the access interval sequence, the interval distribution feature of the access interval sequence in a single period, and the interval fluctuation feature of the access interval sequence between consecutive periods.

[0007] Based on the repeating period length characteristics, interval distribution characteristics of the access interval sequence within a single period, and interval fluctuation characteristics of the access interval sequence between consecutive periods in the periodic pattern descriptors of all data objects, access pattern similarity clustering is performed on all data objects. Data objects with similar periodic pattern descriptors are merged into the same access pattern group to obtain at least one access pattern cluster. Each access pattern cluster corresponds to a cluster identifier.

[0008] Based on the first and last access times in the access time sequence pattern feature set of all data objects within each access pattern cluster, the aggregated access heat duration feature of each access pattern cluster is calculated. According to the comparison result of the aggregated access heat duration feature of each access pattern cluster with the preset heat duration threshold, a heat level label is assigned to each access pattern cluster. The heat level label indicates whether the data group corresponding to the access pattern cluster belongs to a cold data group or a hot data group. A mapping relationship set between each cluster identifier and the heat level label is established, and cold and hot data grouping results are generated based on the mapping relationship set.

[0009] In another aspect, embodiments of the present invention also provide a hot and cold data grouping system based on access pattern clustering, including a processor and a machine-readable storage medium connected to the processor. The machine-readable storage medium is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the machine-readable storage medium to implement the above-described method.

[0010] Based on the above, this embodiment of the invention models the access time-series behavior characteristics of all data objects, and extracts periodic pattern features and performs similarity clustering on the access interval sequence of all data. This ensures that the grouping results of hot and cold data are no longer affected by the occasional single access behavior of individual data, and the grouping boundaries have stronger behavioral feature consistency and long-term stability. This guarantees the rationality of the hotness level label allocation, effectively avoids the storage resource migration overhead caused by frequent switching between hot and cold data, and allows data within the same hotness group to adapt to a unified storage scheduling strategy. This significantly improves the resource matching accuracy of the layered deployment of the storage system, and significantly optimizes the overall access performance and space utilization of the storage system without increasing additional hardware costs. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the execution flow of the cold and hot data grouping method based on access pattern clustering provided in an embodiment of the present invention.

[0012] Figure 2 This is a schematic diagram of exemplary hardware and software components of the cold and hot data grouping system based on access pattern clustering provided in an embodiment of the present invention. Detailed Implementation

[0013] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart illustrating a method for grouping hot and cold data based on access pattern clustering according to an embodiment of the present invention. The following is a detailed description of this method for grouping hot and cold data based on access pattern clustering.

[0014] Step S110: Obtain the set of access log records of data objects in the storage system to be grouped. The set of access log records contains multiple access log entries recorded within a preset historical observation period. Each access log entry contains a unique identifier of the accessed data object and a timestamp of the access initiation time.

[0015] In this embodiment, the access log collection module continuously collects metadata for each data access event within a preset historical observation period. The preset historical observation period depends on the business scenario's requirements for the timeliness of hot and cold data; for example, it can be set to the last 30 days, the last 7 days, or the last 90 days. This embodiment does not impose a specific numerical limitation. The access log collection module resides on each access gateway node of the storage system. Whenever a client initiates a read or write request for a data object, the gateway node generates an access log entry and appends it to the access log record set while processing the request. Each access log entry contains at least two core fields. The first field is the unique identifier of the accessed data object. This unique identifier uniquely identifies a data object globally within the storage system. It can be the object identifier (i.e., the object identifier) ​​of the data object, or a combination of the data object's path name and version number. In this embodiment, the object identifier is used as an example. The object identifier field in each access log entry records the globally unique name of the accessed data object in the storage system. The second field is a timestamp indicating the time the access request was initiated. This timestamp precisely records the moment the access request was received by the gateway node and processing began. The timestamp can be represented as a millisecond- or second-level timestamp integer using Coordinated Universal Time (UTC), or as a date and time format string with time zone information. In this embodiment, a millisecond-level UTC timestamp integer is used to ensure the accuracy and consistency of subsequent time interval calculations. The access log record set can be physically stored as a sequence of log files partitioned by date, or as a structured dataset formed after deduplication and cleaning of a continuously appended log stream. In the application scenario of this embodiment, the access log collection module collected a large number of access log entries within a preset historical observation period. These access log entries cover all data objects in the storage system that have been accessed.

[0016] Step S120: Extract access time sequence patterns from the access log record set. Aggregate the timestamps of all access initiation times corresponding to the unique identifier of the accessed data object to generate an access time sequence for each data object. Generate an access interval sequence for each data object based on the access time sequence of each data object. Determine the first access time and the last access time of each data object based on the access time sequence of each data object to obtain an access time sequence pattern feature set for each data object. The access time sequence pattern feature set includes the access interval sequence, the first access time, and the last access time.

[0017] In this embodiment, the access timing pattern extraction step is performed by an access feature construction module, which is responsible for converting the flat event sequence of the access log record set into structured access timing features for each data object.

[0018] Step S121: Traverse each access log entry in the access log record set, and use the unique identifier of the accessed data object as the aggregation key to merge access log entries with the same unique identifier into the same access record group. Each access record group corresponds to one data object.

[0019] The access feature construction module first reads each access log entry sequentially from the storage location of the access log record set. The reading process can be a sequential scan of the log file or a batch retrieval based on an index. For each access log entry read, the access feature construction module parses out the object identifier field and the timestamp field. Then, using the object identifier as the aggregation key, the access feature construction module maintains a key-value mapping structure in memory. The key of the key-value mapping structure is the object identifier, and the value is a dynamically growing list storing all timestamps associated with that object identifier. For the currently read access log entry, the access feature construction module checks if an entry with the object identifier as the key already exists in the key-value mapping structure. If it doesn't exist, a new key-value pair is created, with the current object identifier as the key and a list containing only the current timestamp as the value. If it already exists, the current timestamp is appended to the end of the existing list corresponding to that key. After all access log entries in the access log record set have been traversed and processed, the key-value mapping structure contains a complete access record group for each accessed data object. Each access record group is a list data structure that stores the original collected timestamp values ​​of all access times for the corresponding data object within a preset historical observation period. At this point, each access record group establishes a one-to-one correspondence with the data object through its corresponding object identifier.

[0020] Step S122: For each data object corresponding to the access record group, extract the timestamp of the access initiation time of all access log entries in the access record group, and sort them in ascending order according to the time sequence of the timestamps to generate the access time sequence of the data object.

[0021] For each key-value pair in the key-value mapping structure generated in step S121, the access feature construction module obtains the value part of the key-value pair, i.e., the timestamp list of the data object. Since the append order of access log entries may not be strictly chronological due to factors such as concurrent writing, network latency, or log merging during the collection process, an explicit sorting operation is required for this timestamp list. The sorting is based on the numerical value of each timestamp, using an ascending order, so that timestamps with smaller values ​​(earlier timestamps) are placed at the beginning of the sequence, and timestamps with larger values ​​(later timestamps) are placed at the end. A mature comparison sorting algorithm can be used. After sorting, the resulting sequence of timestamps arranged in ascending time order is the access time sequence of the data object. The access time sequence is a one-dimensional ordered vector, where each element is a timestamp value, and the length of the vector equals the total number of times the data object was accessed within a preset historical observation period. For example, if a data object was accessed several times, its access time sequence would be a one-dimensional vector containing several elements. The access feature building module associates and stores the access time series with the object identifier of the data object.

[0022] Step S123: For the access time series of each data object, calculate the time interval length between adjacent timestamps in sequence, arrange the time interval lengths between all adjacent timestamps in chronological order, and generate the access interval sequence of the data object.

[0023] After obtaining the access time series, the access feature construction module further processes it to extract the access interval sequence. For access time series with a length greater than or equal to two, the access feature construction module performs an adjacent difference operation. Specifically, starting from the second element of the access time series, for each position index, the timestamp value at that position is subtracted from the timestamp value at the previous position. The difference is the length of the time interval between adjacent accesses, and the unit of measurement for this time interval is consistent with the unit of measurement for the timestamp. Arranging all these calculated time interval lengths in chronological order of their corresponding access times forms a new sequence, which is the access interval sequence of the data object. The length of the access interval sequence is equal to the length of the access time series minus one. If the access time series length of a data object is less than two, meaning that the data object has only been accessed once or zero times within the preset historical observation period, then its access interval sequence is an empty sequence. The access interval sequence is also a one-dimensional vector, where each element physically represents the time elapsed between two consecutive accesses.

[0024] Step S124: For each data object's access time series, extract the moment with the smallest timestamp value in the timestamp marker sequence as the first access moment of the data object, and extract the moment with the largest timestamp value in the timestamp marker sequence as the last access moment of the data object.

[0025] The access feature construction module performs extreme value search on the access time series generated in step S122. Since the access time series has already been sorted in ascending order, the first element of the series represents the moment with the smallest timestamp value. The access feature construction module directly reads the value of the first element of the series, which is the first access time of the data object. The last element of the series represents the moment with the largest timestamp value. The access feature construction module directly reads the value of the last element of the series, which is the last access time of the data object. Both the first and last access times are single timestamp scalar values, and their units of measurement are consistent with the original timestamp markers.

[0026] Step S125: Combine the access interval sequence, first access time and last access time of each data object into the access time sequence pattern feature set of the data object, and establish a mapping relationship between the unique identifier of the data object and the access time sequence pattern feature set.

[0027] The access feature construction module combines the three features obtained in steps S123 and S124 into a structure, which is the access time-series pattern feature set. The access time-series pattern feature set is a multi-feature structure containing three fields: the first field is the access interval sequence field, whose value is a one-dimensional vector; the second field is the first access time field, whose value is a scalar timestamp; and the third field is the last access time field, whose value is also a scalar timestamp. The access feature construction module constructs a global mapping table, where the key is the unique identifier of the data object, and the value is the access time-series pattern feature set structure for that data object. Thus, for each data object, its complete access time-series behavioral features within a preset historical observation period can be quickly retrieved from the global mapping table using its object identifier.

[0028] Step S130: Perform access interval sequence similarity structure analysis based on the access time sequence feature set of all data objects, extract the access interval pattern structure with periodic repetition characteristics from the access interval sequence of each data object, and generate the periodic pattern descriptor of each data object. The periodic pattern descriptor includes the repetition period length feature of the access interval sequence, the interval distribution feature of the access interval sequence in a single period, and the interval fluctuation feature of the access interval sequence between consecutive periods.

[0029] In this embodiment, the periodic pattern extraction step is performed by a periodic structure analysis module. The periodic structure analysis module receives the global mapping table generated in step S125 as input, performs deep structure analysis on the access interval sequence of each data object, and aims to identify whether there is a stable periodic repetitive access pattern, and construct a structured periodic pattern descriptor for each data object.

[0030] Step S131: For the access interval sequence in the access time sequence feature set of each data object, perform interval value change trend analysis on the access interval sequence, identify the fluctuation change pattern of the interval value in the access interval sequence over time, and generate the fluctuation pattern descriptor of the access interval sequence.

[0031] The periodic structure analysis module first iterates through each data object entry in the global mapping table. For the currently processed data object, it extracts the access interval sequence field from its access time sequence pattern feature set. If the access interval sequence is empty or too short to perform meaningful periodic analysis, the periodic structure analysis module directly generates a default periodic pattern descriptor for the data object, indicating no significant periodic pattern. For access interval sequences whose length meets the analysis requirements, the periodic structure analysis module performs interval value change trend analysis. The purpose of this analysis step is to capture the overall change trend of access interval values ​​along the direction of increasing sequence index. Specifically, the access interval sequence is treated as a one-dimensional discrete signal, and local smoothing is performed on the signal to eliminate high-frequency random disturbances. The smoothing can use the moving average method, i.e., setting a smoothing window size, the window slides on the access interval sequence, and the median or arithmetic mean of all interval values ​​within the window is calculated as the smoothed value at the center of the window. After obtaining the smoothed access interval sequence, the periodic structure analysis module identifies the fluctuation pattern by detecting the positions of local maxima and local minima of the smoothed sequence. Local maxima are defined as points in a smoothed sequence where the value is greater than its immediate preceding or following value, and local minima are defined as points in a smoothed sequence where the value is less than its immediate preceding or following value. Sorting all identified local maxima and minima according to their position indices forms a set of peak and valley position sequences describing the fluctuations of the access interval sequence. The periodic structure analysis module also calculates the magnitude of numerical changes between adjacent maxima and minima, as well as the time span between adjacent extreme points of the same type. These extreme point position sequences, peak and valley change magnitude sequences, and adjacent extreme point span sequences together constitute the fluctuation pattern descriptor of the access interval sequence. The fluctuation pattern descriptor is a multi-dimensional feature structure that characterizes the alternating density pattern of the access intervals of the data object.

[0032] Step S132: Based on the fluctuation pattern descriptor of the access interval sequence, perform self-similarity detection on the access interval sequence, match the access interval sequence with itself at different time offsets, extract the repetitive interval pattern segments that appear in the access interval sequence, and generate a candidate repetitive segment set of the access interval sequence. The candidate repetitive segment set includes the starting position and segment length of each repetitive interval pattern segment.

[0033] After obtaining the fluctuation pattern descriptor, the periodic structure analysis module performs self-similarity detection on the access interval sequence to find possible repetitive structures within the sequence. The core idea of ​​self-similarity detection is to compare the access interval sequence with its own version after a certain offset. The original access interval sequence is denoted as sequence S. The periodic structure analysis module sets a series of candidate offsets, increasing from a minimum offset to a maximum offset. For each candidate offset, the periodic structure analysis module constructs an offset sequence, which is a truncated sequence obtained by removing the first offset elements from the starting position of the original sequence S. Then, the periodic structure analysis module calculates the similarity between the sequence segment consisting of the first few elements of the original sequence S and the offset sequence. Similarity calculation can use methods such as dynamic time-warped distance or Pearson correlation coefficient. During the calculation, it is necessary to ensure that the two sequence segments being compared have the same length, which is determined by the length parameter of the candidate segments. The periodic structure analysis module simultaneously traverses multiple candidate segment length values, and for each pair of offset and segment length combinations, calculates a similarity metric. When the similarity metric for a given offset and segment length exceeds a preset similarity threshold, the periodic structure analysis module considers a candidate repeating segment to have been found in the original access interval sequence. This repeating segment is defined by two parameters: a start position parameter, i.e., the starting index position of the segment in the original sequence S; and a segment length parameter, i.e., the number of interval values ​​contained in the segment. The periodic structure analysis module records the information of this candidate repeating segment, including its start position and segment length, and stores it in a candidate repeating segment set. The candidate repeating segment set is a list structure, where each element is a structure containing a start position field and a segment length field. Furthermore, during self-similarity detection, the periodic structure analysis module utilizes the extreme point location information in the fluctuation pattern descriptor to narrow the search range, prioritizing high-density similarity matching in sequence regions near extreme points to improve computational efficiency.

[0034] Step S133: Perform interval value deviation statistical analysis on the multiple occurrence instances of each repetitive interval pattern fragment in the candidate repetitive fragment set in the access interval sequence, calculate the degree of difference between the interval values ​​of the corresponding positions of each repetitive interval pattern fragment in different occurrence instances, and select stable repetitive fragments with a degree of difference less than a preset deviation tolerance threshold from the candidate repetitive fragment set according to the degree of difference, thereby obtaining a stable repetitive fragment set.

[0035] Each candidate repeating segment in the candidate repeating segment set only identifies the start position and length of a possible repeating pattern segment, but it has not yet been verified whether this segment also recurs stably at other positions in the access interval sequence, or whether the similarity between different occurrence instances is sufficiently high. The periodic structure analysis module verifies each candidate segment in the candidate repeating segment set. For a given candidate segment, the periodic structure analysis module first extracts a reference segment instance from the original access interval sequence, starting from the start position of the candidate segment, based on the segment length of the candidate segment. The reference segment instance is a one-dimensional vector with a length equal to the segment length. Then, using the segment length of the candidate segment as the period length, the periodic structure analysis module traverses the entire access interval sequence, starting from the start position of the access interval sequence and using the period length as the step size, to extract multiple segment instances. Each segment instance is a sub-vector of length equal to the segment length, truncated from the access interval sequence at a position equal to the period length multiplied by a non-negative integer plus the start position offset. These segment instances are then compared position-by-position with the reference segment instance. The position-by-position comparison method involves calculating the absolute value of the difference between the interval value of the reference segment instance and the interval value of the current segment instance at each position index within a segment. This absolute value reflects the interval deviation between the two instances at that position. For a segment instance, the interval deviations at all positions are summed or averaged to obtain a segment-level deviation. If the segment-level deviation of a segment instance is less than a preset deviation tolerance threshold, the segment instance is considered a stable repeating instance of the reference segment instance. The periodic structure analysis module counts the total number of segment instances that meet the deviation tolerance threshold condition. If the ratio of this total number to the maximum possible number of instances that the candidate segment can theoretically appear throughout the entire access interval sequence exceeds a preset coverage threshold, the periodic structure analysis module considers the candidate segment to be a stable repeating segment that appears stably throughout the entire sequence. The candidate segment is then removed from the candidate repeating segment set and stored in the stable repeating segment set. Each element in the set of stable repeating fragments contains, in addition to the starting position and length of the candidate fragment, a list of the starting positions of all stable repeating instances of that fragment found during the verification process, as well as the fragment-level deviation for each instance.

[0036] Step S134: Perform periodic integrity analysis on each stable repeating segment in the set of stable repeating segments to verify whether the stable repeating segment maintains stable periodic repetition throughout the entire time span of the access interval sequence. Extract the average repetition interval of the stable repeating segment as the repetition period length feature of the access interval sequence. Extract the statistical mean sequence of the interval values ​​at each position within the stable repeating segment as the interval distribution feature of the access interval sequence within a single period.

[0037] The purpose of periodic integrity analysis is to identify the segment from the set of stable repeating segments that best represents the core access periodic pattern of the data object, and to extract the attributes of this segment into comparable feature vectors. The periodic structure analysis module traverses the set of stable repeating segments. For the currently processed stable repeating segment, the periodic structure analysis module first calculates the index distance between the starting positions of two adjacent stable repeating instances based on the starting positions of all stable repeating instances recorded in step S133. Ideally, if the access sequence has strict periodicity, the index distance between the starting positions of adjacent instances should be equal to the segment length. However, in reality, due to random perturbations in access behavior, the index distance between the starting positions of adjacent instances may fluctuate around the segment length. The periodic structure analysis module collects all the index distances between the starting positions of adjacent instances, forming an index distance set. Statistical analysis is performed on this index distance set, calculating its central tendency (e.g., median) and dispersion (e.g., interquartile range). If the dispersion relative to the central tendency is lower than a preset periodicity threshold, it indicates that the recurrence of this stable repeating segment has good periodicity. The periodic structure analysis module selects the stable repeating segment from the set of stable repeating segments that performs best in terms of both periodicity and coverage as the main periodic pattern segment for the data object. Next, the module extracts two key features from this main periodic pattern segment. The first feature is the repeating cycle length of the access interval sequence. This feature is extracted by converting the central trend statistic of the distance set of starting position indices of adjacent instances into a time-dimensional length value. Specifically, the central trend statistic (index span) is multiplied by the mean of all interval values ​​within the segment; the product is the repeating cycle length in units of time. The second feature is the interval distribution of the access interval sequence within a single period. This feature is extracted by collecting the interval values ​​of all stable repeating instances at each position index within the main periodic pattern segment, forming a set of position interval values. The arithmetic mean or median of this set of position interval values ​​is calculated, and this statistic is used as the standard interval value for that position. The standard interval values ​​of all positions within the segment are arranged in positional order into a one-dimensional vector; this vector represents the interval distribution feature within a single period. The dimension of this vector is equal to the length of the fragment.

[0038] Step S135: Extract the statistical measure of the deviation of the corresponding position interval value of each stable repeating segment from the statistical mean sequence in instances of occurrence in different periods, and use it as the interval fluctuation feature of the access interval sequence between consecutive periods. Combine the repeating period length feature of the access interval sequence, the interval distribution feature of the access interval sequence in a single period, and the interval fluctuation feature of the access interval sequence between consecutive periods into the periodic pattern descriptor of the data object.

[0039] After obtaining the interval distribution characteristics, i.e., the statistical mean sequence, within a single period, the periodic structure analysis module further quantifies the volatility of interval values ​​across different periods. For each position index in the main periodic pattern segment, the periodic structure analysis module already possesses the standard interval value (statistical mean) at that position, as well as the actual interval value at that position in each stable recurring instance. For each position index, the periodic structure analysis module calculates the deviation of the actual interval value at that position relative to the standard interval value. The deviation can be calculated using absolute or relative deviation; in this embodiment, absolute deviation is used, i.e., the absolute value of the actual interval value minus the absolute value of the standard interval value. Then, the periodic structure analysis module aggregates the deviations generated at all positions and in all instances to form a set of deviations. A statistic reflecting the overall volatility level is calculated for this set of deviations. This statistic can be the arithmetic mean of the set of deviations, or the standard deviation or root mean square value of the set of deviations. This statistic represents the interval volatility characteristic of the access interval sequence between consecutive periods. The dimension of the interval volatility characteristic is the same as that of the interval value, both being units of time. At this point, the periodic structure analysis module has extracted three core features for the current data object: the repeating cycle length is a scalar value with the dimension of time; the interval distribution within a single cycle is a one-dimensional vector, with each component of the vector having the dimension of time; and the interval fluctuation between consecutive cycles is a scalar value with the dimension of time. The periodic structure analysis module encapsulates these three features into a structured object, which is the periodic pattern descriptor for the data object. If a data object is determined to be aperiodic due to an excessively short access interval sequence or the absence of a significant periodic pattern, a specially marked default periodic pattern descriptor is assigned to it, with all three feature values ​​set to the default values ​​representing aperiodic patterns. The periodic pattern descriptor is stored in association with its corresponding unique identifier for the data object, forming a new global periodic pattern mapping table.

[0040] Step S140: Based on the repeating period length feature, the interval distribution feature of the access interval sequence in a single period, and the interval fluctuation feature of the access interval sequence between consecutive periods in the periodic pattern descriptors of all data objects, perform access pattern similarity clustering on all data objects, merge data objects with similar periodic pattern descriptors into the same access pattern group, and obtain at least one access pattern cluster, with each access pattern cluster corresponding to a cluster identifier.

[0041] In this embodiment, the access pattern similarity clustering step is performed by a clustering analysis module. The core task of the clustering analysis module is to group data objects with similar access pattern structures together based on the periodic pattern descriptor obtained in step S135, forming data object groups with inherent homogeneity.

[0042] Step S141: Construct a similarity measurement space to measure the similarity of access patterns between different data objects. The similarity measurement space includes a distance measurement function in the dimension of repetition period length, a sequence morphology distance measurement function in the dimension of interval distribution, and a fluctuation amplitude distance measurement function in the dimension of interval fluctuation.

[0043] The clustering analysis module first needs to define a metric system that can comprehensively measure the differences between the periodic pattern descriptors of two data objects. Since the periodic pattern descriptor contains three distinct feature components, a dedicated metric function needs to be designed for each feature component. These metric functions together constitute the similarity metric space. For the distance metric function in the repeating period length dimension, its inputs are two scalar values, representing the repeating period length features of the two data objects. The function aims to quantify the relative difference between these two period lengths. The implementation logic of this function is to calculate the absolute value of the difference between the two repeating period length feature values, then divide this absolute value by the maximum value of the two repeating period length feature values ​​for normalization, resulting in a dimensionless relative difference ratio. This relative difference ratio ranges from zero to one, where zero indicates that the two period lengths are exactly the same, and a larger value indicates a greater difference in period lengths. For the sequence morphology distance metric function in the interval distribution dimension, its inputs are two one-dimensional vectors, representing the interval distribution feature vectors of the two data objects within a single period. Since the interval distribution feature vectors of different data objects may have different dimensions, i.e., the segment lengths of their main periodic patterns may differ, measurement methods requiring vectors of equal length, such as Euclidean distance, cannot be directly used. This embodiment employs the dynamic time warping distance algorithm as the sequence morphological distance metric. This algorithm allows for non-linear alignment matching between two sequences of different lengths, quantifying the morphological similarity between the two sequences by finding a curved path with the minimum total cumulative distance. The cumulative distance is calculated based on the absolute difference between corresponding elements in the sequences. The resulting dynamic time warping distance is a non-negative scalar, with the same dimension as the interval value, representing a unit of time. To avoid the distance scale being affected by sequence length, this distance value is normalized by dividing it by the larger of the two sequence lengths, resulting in a dimensionless distribution morphological difference. For the fluctuation amplitude distance metric function of the interval fluctuation dimension, its input consists of two scalar values, representing the interval fluctuation characteristics of the two data objects between consecutive periods. The function implementation logic is as follows: calculate the absolute value of the difference between two fluctuation characteristic values, then divide this absolute value by the maximum value of the two fluctuation characteristic values ​​plus a small constant to avoid division by zero, obtaining a dimensionless difference in fluctuation degree. Each of these three measurement functions outputs a dimensionless difference value.

[0044] Step S142: For any two data objects, calculate the difference in period length between the repeated period length features of the access interval sequences of the two data objects using the distance metric function of the repeated period length dimension; calculate the difference in distribution pattern between the access interval sequences of the two data objects within a single period using the sequence pattern distance metric function of the interval distribution dimension; and calculate the difference in the degree of fluctuation between the access interval sequences of the two data objects between consecutive periods using the fluctuation amplitude distance metric function of the interval fluctuation dimension.

[0045] The clustering analysis module needs to process all pairwise combinations in the data object set to construct a pairwise difference matrix between data objects. For any pair of data objects, denoted as data object A and data object B, the clustering analysis module extracts the periodic pattern descriptors of both from the periodic pattern mapping table. If either one has a default periodic pattern descriptor representing no significant periodic pattern, a preset maximum difference constant value can be used as the difference in each difference dimension, thus significantly distinguishing them from other data objects with periodic patterns. If both have valid periodic pattern descriptors, the clustering analysis module applies the three measurement functions constructed in step S141 one by one. First, it extracts the repeating period length feature values ​​of data object A and data object B, calls the distance measurement function of the repeating period length dimension, takes these two feature values ​​as input parameters, and calculates the period length difference output internally. This output is a scalar with one dimension. Next, it extracts the interval distribution feature vector of data object A and data object B, calls the sequence morphology distance measurement function of the interval distribution dimension, and passes these two vectors as parameters. Internally, this function executes a dynamic time warping algorithm to align and match two vectors, accumulating the minimum distance. After normalizing the accumulated distance to the sequence length, it outputs a distribution morphology difference, which is also a dimensionless scalar. Finally, it extracts the interval fluctuation characteristic scalars of data object A and data object B, calls the fluctuation amplitude distance metric function of the interval fluctuation dimension, and inputs these two scalar values. The function then calculates the fluctuation degree difference, which is also a dimensionless scalar. The clustering analysis module temporarily stores these three differences and associates them with the identifiers of data objects A and B.

[0046] Step S143: Perform weighted fusion on the difference in period length, the difference in distribution pattern, and the difference in fluctuation degree to generate a comprehensive access pattern difference metric between any two data objects. The weights used in the weighted fusion are allocated according to the discriminative contribution of each dimension of the difference in the clustering objective.

[0047] The clustering analysis module assigns weighting coefficients to the three differences, denoted as period length weight, distribution pattern weight, and volatility weight, respectively. These three weights can be pre-learned on historical datasets using feature importance assessment methods such as information entropy or analysis of variance, or configured by the storage system administrator based on business experience. The sum of the three weights is a fixed constant; in this embodiment, this fixed constant is set to one. The specific execution process of weighted fusion is as follows: the period length difference obtained in step S142 is multiplied by the period length weight to obtain the first weighting term; the distribution pattern difference is multiplied by the distribution pattern weight to obtain the second weighting term; and the volatility difference is multiplied by the volatility weight to obtain the third weighting term. Then, the first, second, and third weighting terms are summed to obtain the comprehensive access pattern difference metric. This comprehensive access pattern difference metric is a scalar with one dimension, and its value comprehensively reflects the overall degree of difference between data object A and data object B in their periodic access pattern structure. The smaller the value, the more similar the access patterns of the two data objects. The clustering analysis module fills the calculated comprehensive access pattern difference metric into the corresponding position of a pairwise difference matrix. This pairwise difference matrix is ​​a symmetric square matrix, where the row index and column index correspond to each data object in the data object set. The element value in the i-th row and j-th column of the matrix is ​​the comprehensive access pattern difference metric between the i-th data object and the j-th data object.

[0048] Step S144: Construct a similarity graph of access patterns between data objects based on the comprehensive access pattern difference metric between all pairs of data objects. In the similarity graph of access patterns between data objects, establish connection edges for data object pairs whose comprehensive access pattern difference metric is less than a preset similarity judgment threshold.

[0049] The clustering analysis module constructs an undirected graph based on the pairwise difference matrix, which represents the similarity graph of access patterns between data objects. In this graph, each node represents a data object, and the node's identifier is the unique identifier of the data object. The edge set of the graph is initially empty. The clustering analysis module traverses the upper triangular portion of the pairwise difference matrix, i.e., all elements whose row index is less than their column index. For each element traversed, it checks whether its comprehensive access pattern difference metric is less than a preset similarity threshold. The preset similarity threshold is an empirical parameter set according to the clustering granularity requirements. If the difference metric is less than this threshold, the clustering analysis module determines that the corresponding two data objects have a highly similar access pattern and adds an undirected connection edge between these two nodes in the graph. The edge weight can be set to the reciprocal or complement of the difference metric for use by subsequent possible graph clustering algorithms. If the difference metric is greater than or equal to the threshold, no connection edge is established between these two nodes. After traversing all pairs of data objects, a complete graph of access pattern similarity between data objects is obtained. In this diagram, data objects with similar cyclical access patterns will form connected regions with dense internal connections and sparse connections between each other through connecting edges.

[0050] Step S145: Perform connected component search in the access pattern similarity graph between data objects, merge data objects with direct connection edges or indirect connection paths into the same access pattern group to form at least one access pattern cluster, and assign a unique cluster identifier to each access pattern cluster. The cluster identifier is associated with the unique identifier of all data objects in the access pattern cluster.

[0051] The clustering analysis module performs a connected component search algorithm on the similarity graph of access patterns between data objects constructed in step S144. The connected component search algorithm can employ breadth-first search, depth-first search, or a disjoint-set data structure (Union-Find) algorithm. Taking breadth-first search as an example, the clustering analysis module initializes an access marker set to record whether each node in the graph has been visited. Then, the clustering analysis module traverses each node in the graph. For the current node, if it has not been visited, it indicates that a new connected component has been discovered, and the clustering analysis module creates a new empty cluster. Next, a breadth-first search is performed starting from this node, using a queue data structure to maintain the nodes to be explored. First, the current node is added to the queue and marked as visited. When the queue is not empty, a node is taken from the head of the queue, and the unique identifier of the data object corresponding to this node is added to the data object member list of the current cluster. Then, all neighboring nodes of this node are traversed. For each neighboring node, if it has not been visited, it is marked as visited and added to the tail of the queue. When the queue is empty, all nodes in the connected component have been assigned to the current cluster. The clustering analysis module generates a globally unique cluster identifier for each cluster. This identifier can be a sequentially incrementing integer value or a globally unique identifier string. This cluster identifier is then associated with a list of unique identifiers for all data objects contained within that cluster and stored in a clustering result mapping table. The process continues, iterating through the graph to the next unvisited node, repeating the above steps until all nodes have been visited. After this iteration, the clustering analysis module obtains several access pattern clusters, each containing several data objects with similar access patterns. For isolated nodes without any connecting edges, each node forms its own access pattern cluster containing only its own single data object.

[0052] For example, in step S140A: For each access pattern cluster obtained by clustering, extract the repeat period length feature of the access interval sequence of all data objects in the access pattern cluster, perform intra-cluster consistency analysis on the repeat period length feature of the access interval sequence of all data objects in the access pattern cluster, evaluate the degree of aggregation of the repeat period length feature of the access interval sequence of each data object in the access pattern cluster around the period length of the cluster center, and generate an intra-cluster period consistency index.

[0053] This step corresponds to the additional step in the dependent claims that optimizes and adjusts the clustering results. After obtaining the initial set of access pattern clusters, the cluster analysis module evaluates the cohesion within each cluster. For each access pattern cluster, the cluster analysis module extracts the repetition period length feature values ​​of all data objects with valid periodic pattern descriptors within the cluster, forming a set of period length values. If the number of elements in the set is less than a certain minimum sample size, a default high value representing high consistency is directly assigned to the intra-cluster periodic consistency index. If the number of elements is sufficient, the cluster analysis module first calculates the cluster center period length of the set. The method for calculating the cluster center period length can be to take the median or arithmetic mean of all period length values ​​in the set. Next, the cluster analysis module calculates the absolute deviation of each period length value in the set from the cluster center period length, forming a deviation set. Then, the cluster analysis module calculates the dispersion statistics of the deviation set, such as the median or arithmetic mean of the deviation set. To ensure the consistency index is between zero and one for easy comparison, the cluster analysis module divides the dispersion statistic by the period length of the cluster centers to obtain a dimensionless relative deviation measure. This relative deviation measure is then subtracted from a preset constant (e.g., the value one) to obtain the intra-cluster period consistency index. The closer this index is to one, the more consistent the period lengths of the data objects within the cluster, indicating better cluster cohesion; the closer it is to zero, the more dispersed the period length distribution.

[0054] Step S140B: Extract the cluster center period length and intra-cluster period consistency index of each access pattern cluster. Based on the degree of difference between the cluster center period lengths of each access pattern cluster and the intra-cluster period consistency index of each access pattern cluster, perform inter-cluster merging or intra-cluster splitting adjustment on the at least one access pattern cluster to optimize the partition boundary of the access pattern cluster and generate an optimized access pattern cluster set.

[0055] The clustering analysis module performs cluster boundary optimization on the preliminary clustering results. This optimization includes two types: inter-cluster merging and intra-cluster splitting. For inter-cluster merging, the module iterates through all possible cluster pairs. For each pair, it extracts the cluster center period length and intra-cluster period consistency index. If the difference between the two cluster center period lengths divided by the ratio of the larger cluster center period length is less than a preset merging period similarity threshold, and both clusters' intra-cluster period consistency indices are below a preset high consistency threshold, it indicates that the two clusters have similar center patterns and are relatively loosely structured, potentially indicating over-splitting. In this case, the module merges the two clusters into a larger cluster, assigns a new cluster identifier, and recalculates the new cluster's cluster center period length and intra-cluster period consistency index. Intra-cluster splitting is performed on clusters with low intra-cluster period consistency indices and a large number of members. For such clusters, the clustering analysis module performs fine-grained re-clustering on the repetition period length feature value of the data objects within the cluster. For example, it uses one-dimensional mean shift or hierarchical clustering algorithms to split the original cluster into several new, smaller clusters, assigning a new cluster identifier to each. This merging and splitting process can be iterative until there are no more clusters to merge or split. The final set of clusters generated is the optimized access pattern cluster set.

[0056] Step S140C: Use the optimized access pattern cluster set as the input object for the subsequent steps of calculating the duration of access popularity features and assigning popularity level labels, and update the mapping relationship between the cluster identifier corresponding to each access pattern cluster and the unique identifier of the data object contained within it.

[0057] The cluster analysis module refreshes the optimized access pattern cluster set information obtained in step S147 to the cluster result mapping table, overwriting the preliminary clustering results. All subsequent steps will be processed based on this optimized cluster set.

[0058] Step S150: Based on the first access time and last access time in the access time sequence pattern feature set of all data objects in each access pattern cluster, calculate the aggregated access heat duration feature of each access pattern cluster, and assign heat level labels to each access pattern cluster according to the comparison result of the aggregated access heat duration feature of each access pattern cluster with the preset heat duration threshold. The heat level label indicates whether the data group corresponding to the access pattern cluster belongs to the cold data group or the hot data group, and establish a mapping relationship set between each cluster identifier and the heat level label. Based on the mapping relationship set, generate cold and hot data grouping results.

[0059] In this embodiment, the hot / cold label assignment step is performed by a hot / cold determination module. Based on the optimized access pattern cluster set and the access time sequence pattern feature set mapping table established in step S125, the hot / cold determination module performs aggregated hotness evaluation on each cluster and assigns hot / cold labels.

[0060] Step S151: For each access pattern cluster, obtain the access time sequence pattern feature set of all data objects in the access pattern cluster, and extract the first access time and the last access time of the data object from the access time sequence pattern feature set of each data object.

[0061] The hot / cold data determination module traverses each cluster in the optimized access pattern clustering cluster set. For the current cluster, the hot / cold data determination module reads the data object member list of that cluster, which contains unique identifiers for all data objects within the cluster. For each unique identifier of a data object in the list, the hot / cold data determination module uses this identifier to retrieve the corresponding access time sequence pattern feature set structure from the global mapping table established in step S125. From this structure, the hot / cold data determination module extracts the values ​​of the first access time field and the last access time field. Both of these values ​​are scalar timestamp values, with the same unit as the original timestamp. The hot / cold data determination module collects both timestamp values ​​for all data objects within the cluster.

[0062] Step S152: For each data object, calculate the time span between the last access time and the first access time of the data object to obtain the individual access activity duration of the data object.

[0063] For each data object traversed in step S151, the hot / cold data determination module performs calculations based on the extracted first and last access times. The calculation method is to subtract the first access timestamp from the last access timestamp; the difference is the individual access activity duration for that data object. The unit of measurement for the individual access activity duration is time. If the original timestamp is in milliseconds, then the unit of this duration is also milliseconds; it can be uniformly converted to units such as days or hours later. This duration reflects the total length of time that the data object has spanned from its initial recording to its most recent access.

[0064] Step S153: Aggregate and statistically analyze the individual access activity duration of all data objects within each access pattern cluster, and extract the distribution center trend characterization value of the individual access activity duration within the access pattern cluster as the aggregated access popularity duration feature of the access pattern cluster. The distribution center trend characterization value reflects the concentration level of the access activity duration of most data objects within the access pattern cluster.

[0065] Step S1531: The individual active duration of all data objects within the access pattern cluster is used to form a duration set to be analyzed. The duration values ​​in the duration set to be analyzed are subjected to numerical distribution pattern analysis to determine the concentrated distribution area and discrete distribution area of ​​the duration values ​​in the duration set to be analyzed.

[0066] The hot / cold index determination module performs distribution pattern analysis on a set of all collected individual visit duration values ​​within a cluster. First, the module sorts the values ​​in the duration set to be analyzed in ascending order, forming an ordered sequence. To analyze its distribution pattern, the module calculates several key quantiles of this ordered sequence, such as the first quartile, the second quartile (median), and the third quartile. The interquartile range is the third quartile minus the first quartile. By comparing the relative position of the median within the interquartile range and the distances from the upper and lower tails to the nearest quartile, it can be determined whether the distribution pattern is symmetrical or skewed. Specifically, if the median is approximately located between the first and third quartiles, and the distance from the maximum value of the ordered sequence to the third quartile is approximately equal to the distance from the minimum value of the ordered sequence to the first quartile, then the distribution pattern can be determined to be symmetrical. If the median is significantly skewed to one side of the first quartile, or if the distance from the maximum value to the third quartile is significantly greater than the distance from the minimum value to the first quartile, then the distribution pattern can be determined to be skewed, with the skew direction to the right, meaning the long tail is on the side of the larger value. The hot / cold determination module uses this to determine the overall distribution pattern of the numerical values ​​in the set of durations to be analyzed.

[0067] Step S1532: Based on the distribution morphology characteristics of the concentrated distribution area and the discrete distribution area, select a central trend extraction strategy that matches the distribution morphology characteristics. The central trend extraction strategy adopts the symmetrical distribution center extraction method when the distribution of duration values ​​in the set of durations to be analyzed presents a symmetrical shape, and adopts the skewed distribution center extraction method when the distribution of duration values ​​in the set of durations to be analyzed presents a skewed shape.

[0068] The hot / cold distribution determination module selects the appropriate central trend extraction strategy based on the distribution pattern characteristics determined in step S1531. If the distribution is determined to be symmetrical, the symmetrical distribution central trend extraction method is used. In this method, the distribution central trend representation value is the arithmetic mean of the set of durations to be analyzed. The arithmetic mean is calculated by summing all duration values ​​in the set and then dividing the sum by the number of elements in the set; the quotient is the arithmetic mean. If the distribution is determined to be skewed, the skewed distribution central trend extraction method is used. To improve the robustness of the central trend to extreme values, the distribution central trend representation value is the median of the set of durations to be analyzed. The median extraction process is as follows: from the sorted ordered sequence, find the element located in the middle position of the sequence. If the sequence length is odd, the value of the element in the middle position is the median; if the sequence length is even, the arithmetic mean of the two middle positions is taken as the median. Alternatively, the tail-cutting mean can be used as a method for extracting the center of the skewed distribution. That is, after removing a certain proportion of extreme values ​​at both ends of the ordered sequence, the arithmetic mean of the remaining part is calculated.

[0069] Step S1533: Extract the duration set to be analyzed using the selected central trend extraction strategy, extract statistics that can characterize the central location of the main part of the duration set to be analyzed from the duration values ​​of the duration set to be analyzed, and use the extracted statistics as the aggregated access popularity duration feature of the access pattern cluster.

[0070] The hot / cold index determination module executes the selected extraction strategy and calculates a scalar statistical value, which represents the duration of aggregated access popularity for that access pattern cluster. The dimension of this feature is consistent with the dimension of the duration of individual active access, both being units of time. This feature comprehensively describes the typical active time span of data objects within the cluster.

[0071] Step S1534: Establish an association between the cluster identifier of each access pattern cluster and the aggregated access popularity duration feature of the access pattern cluster.

[0072] The hot / cold determination module pairs the cluster identifier of the access pattern cluster with the aggregated access heat duration feature value that was just calculated, and stores it in a heat feature table.

[0073] Step S154: Compare the aggregated access popularity duration feature of each access pattern cluster with the preset popularity duration threshold. If the aggregated access popularity duration feature of an access pattern cluster is greater than or equal to the preset popularity duration threshold, then assign a popularity level label indicating hot data groups to the access pattern cluster. If the aggregated access popularity duration feature of an access pattern cluster is less than the preset popularity duration threshold, then assign a popularity level label indicating cold data groups to the access pattern cluster.

[0074] The hot / cold data determination module reads a preset hot data duration threshold. This threshold is a time length value configured by the system administrator based on business experience, representing the critical duration for distinguishing between long-term and short-term active data. The module iterates through each entry in the hot data feature table, specifically the aggregated access hot data duration feature for each access pattern cluster. For each access pattern cluster, the module compares the aggregated access hot data duration feature value with the preset hot data duration threshold. The comparison logic is as follows: if the aggregated access hot data duration feature value is greater than or equal to the threshold, it indicates that the data objects within the cluster generally have a long active history, meeting the characteristics of hot data. The module assigns a hot data level label to this cluster, identifying it as a hot data group. If the aggregated access hot data duration feature value is less than the threshold, it indicates that the data objects within the cluster generally have a short active history or have not been accessed for a long time, meeting the characteristics of cold data. The module assigns a hot data level label to this cluster, identifying it as a cold data group. The popularity level label can be an enumerated string value, such as "HOT" and "COLD".

[0075] Step S155: Establish a correspondence between the cluster identifier of each access pattern cluster and the assigned heat level label, summarize all the correspondence entries to form the mapping relationship set, and attach the corresponding heat level label to the cluster identifier of each data group based on the mapping relationship set to generate a cold and hot data grouping result containing the unique identifier of each data object and its assigned heat level label.

[0076] The hot / cold data determination module generates a mapping record for each access pattern cluster. This record includes a cluster identifier field and a popularity level label field. All cluster mapping records are aggregated into a list or mapping table, forming a mapping relationship set. Then, based on this mapping relationship set, the hot / cold data determination module iterates through the cluster result mapping table established in step S145 or S148. For each data object's unique identifier in the table, it searches for the corresponding popularity level label using its cluster identifier. After finding the popularity level label, it appends it to the data object's metadata. Finally, the hot / cold data determination module outputs a complete list of hot / cold data grouping results. Each row of this list contains a data object's unique identifier and its determined popularity level label. The storage system's data lifecycle management module can use this result list to migrate data objects labeled as cold data groups to the cold storage layer, and retain data objects labeled as hot data groups in the hot storage layer or prefetch them into the cache layer.

[0077] Step S150A: Obtain the access interval sequence from the access time sequence feature set of all data objects in each access pattern cluster, perform intra-cluster access frequency statistics on the access interval sequence of all data objects in each access pattern cluster, and extract the cluster average access interval feature corresponding to each access pattern cluster.

[0078] This step corresponds to the step in the dependent claims that refines the popularity by combining access frequency. This step is performed by a composite popularity assessment module. Building upon the work of the hot / cold determination module, the composite popularity assessment module further utilizes the access frequency information contained in the access interval sequence to perform a more refined hierarchical division of popularity.

[0079] Step S150A-1: For each access pattern cluster, traverse the access time sequence pattern feature set of all data objects in the access pattern cluster, extract the access interval sequence of the data object from the access time sequence pattern feature set of each data object, and merge all the extracted access interval sequences into a cluster-level interval sequence pool.

[0080] The composite popularity assessment module first traverses each access pattern cluster. For the current access pattern cluster, it obtains a list of unique identifiers for all data objects it contains. It then traverses each identifier in the list, retrieving the corresponding access time sequence pattern feature set structure from the global mapping table established in step S125. The value of the access interval sequence field is extracted from this structure. This access interval sequence is a one-dimensional vector containing several time interval values. The composite popularity assessment module extracts all elements from this access interval sequence one by one and appends them to a list container called the cluster-level interval sequence pool, maintained for this cluster. After traversing all data objects within the cluster, the cluster-level interval sequence pool contains the time interval records between all adjacent accesses for all data objects within the cluster. The cluster-level interval sequence pool is a dataset that aggregates fine-grained access frequency information for all data objects within the cluster; the unit of measurement for each element is consistent with the original access interval value, which is a unit of time length.

[0081] Step S150A-2: Perform a holistic statistical analysis on all access interval values ​​in the cluster-level interval sequence pool, extract a central trend statistic from the access interval values ​​in the cluster-level interval sequence pool that can characterize the typical interval length of data objects being accessed within the access pattern cluster, and use the central trend statistic as the cluster average access interval feature of the access pattern cluster.

[0082] The composite heat assessment module performs a holistic statistical analysis on the cluster-level interval sequence pool generated in step S161. The analysis first calculates the basic distribution characteristics of all interval values ​​in the cluster-level interval sequence pool. To extract a central tendency statistic that represents the typical interval length, the composite heat assessment module needs to evaluate the distribution pattern of the values ​​in the cluster-level interval sequence pool. The analysis method for the distribution pattern is similar to that in step S1531, i.e., determining symmetry or skewness by calculating quantiles and comparing the upper and lower tails. If the distribution is symmetrical, the arithmetic mean is used as the central tendency statistic. If the distribution is significantly skewed, i.e., a small number of extremely long interval values ​​lengthen the right tail of the distribution, the median or shortened mean is used as the central tendency statistic to more accurately reflect the length of most access intervals, rather than being affected by extremely long intervals. Based on the morphological analysis results, the composite heat assessment module selects and calculates a specific value, which is the cluster average access interval characteristic of the cluster for this access pattern. The dimension of the cluster average access interval characteristic is consistent with the dimension of the access interval value, both being time units. This feature reflects the typical time interval at which data objects within the cluster are accessed. The smaller the average cluster access interval feature, the more frequently the data within the cluster is accessed; the larger the value, the sparser the access.

[0083] Step S150A-3: Establish the correspondence between the cluster identifier of each access pattern cluster and the cluster average access interval feature of the access pattern cluster.

[0084] The composite heat assessment module pairs the cluster identifier of the access pattern cluster with the calculated average access interval feature of the cluster and stores them in a frequency feature table.

[0085] Step S150B: Perform joint heat analysis on the aggregated access heat duration characteristics of each access pattern cluster and the cluster average access interval characteristics of each access pattern cluster to obtain the composite heat evaluation characteristics of each access pattern cluster. The composite heat evaluation characteristics simultaneously reflect the access duration and access frequency density of the access pattern cluster.

[0086] After obtaining the aggregation access duration characteristics and average access interval characteristics of each cluster, the composite heat assessment module organically integrates the two to generate a composite feature that can more comprehensively measure the hotness or coldness of the data.

[0087] Step S150B-1: For each access pattern cluster, the aggregated access popularity duration feature and the cluster average access interval feature of the access pattern cluster are fused under a unified composite popularity evaluation dimension to construct a composite popularity evaluation vector that simultaneously represents the popularity duration capability and popularity frequency density of the access pattern cluster. The first dimension of the composite popularity evaluation vector corresponds to the aggregated access popularity duration feature, and the second dimension of the composite popularity evaluation vector corresponds to the cluster average access interval feature.

[0088] The composite popularity assessment module constructs a two-dimensional composite popularity assessment vector for each access pattern cluster. The first dimension feature value of this vector is directly read from the popularity feature table, which represents the duration of aggregated access popularity for that cluster, and is used as the value of the first dimension. The second dimension feature value of the composite popularity assessment vector is directly read from the frequency feature table, which represents the average access interval for that cluster, and is used as the value of the second dimension. Since the first and second dimensions have different dimensions—one representing the duration of time and the other representing the interval—the composite popularity assessment module standardizes these two dimensions before proceeding to the subsequent fusion steps to eliminate the influence of the difference in dimensions. Standardization can be performed using zero-mean unit variance standardization, i.e., Z-score standardization. Specifically, the aggregated access popularity duration feature values ​​of all access pattern clusters are collected, and the mean and standard deviation of these values ​​are calculated. For each cluster's feature value, the mean is subtracted, and then the result is divided by the standard deviation to obtain the standardized first dimension value. Similarly, the cluster mean visit interval feature values ​​of all visit pattern clusters are collected, and their mean and standard deviation are calculated. The same operation of subtracting the mean and dividing by the standard deviation is performed on this feature value for each cluster to obtain the standardized second dimension value. After standardization, the values ​​of both dimensions are converted into dimensionless relative values ​​with consistent distribution characteristics. This standardized two-dimensional vector is the constructed composite heat assessment vector.

[0089] Step S150B-2: Model the inter-dimensional interaction relationship of the composite popularity evaluation vector, capture the mutual influence pattern between the aggregated access popularity duration feature and the cluster average access interval feature on the popularity evaluation result, and generate dimensional interaction correction parameters. The dimensional interaction correction parameters adjust the contribution ratio of each dimensional feature in the composite popularity evaluation vector to the final composite popularity evaluation feature.

[0090] To capture the nonlinear impact of different combinations of "long duration and short access interval" and "short duration and long access interval" on the meaning of popularity, the composite popularity assessment module introduces a step for modeling inter-dimensional interaction relationships. This embodiment uses a shallow artificial neural network sub-model to implement this interaction relationship modeling; this sub-model is a miniature multilayer perceptron. The input layer of the multilayer perceptron contains two neurons, receiving the first and second dimension values ​​of the standardized composite popularity assessment vector output from step S171, respectively. A hidden layer is then connected to the input layer, containing several neurons. Each hidden layer node is fully connected to the two neurons in the input layer and a bias node. The computation logic for each hidden layer node is as follows: it receives the output values ​​of the two nodes in the input layer, multiplies them by the two weight parameters maintained by the hidden layer node, sums these two multiplications, and adds the bias parameter of the hidden layer node to obtain a net input. A nonlinear activation function, such as a hyperbolic tangent function or a linear rectified function, is applied to this net input to obtain the activation output value of the hidden layer node. Multiple nodes in the hidden layer compute in parallel, producing a hidden layer output vector. A hidden layer is followed by an output layer containing two neurons, corresponding to the first and second correction coefficients in the dimensionality interaction correction parameters, respectively. The two nodes in the output layer are fully connected to every node in the hidden layer and a bias node. The computation logic for each node in the output layer is similar to that in the hidden layer: the output values ​​of each node in the hidden layer are weighted, summed, and then the bias is added before being processed by an activation function to obtain the final output. To ensure that the correction coefficients output by the output layer are positive and within a reasonable range, an exponential function or a soft-addition function can be used as the activation function in the output layer. All weight and bias parameters of this miniature multilayer perceptron are pre-trained offline using a set of training samples with known composite popularity assessment features (either manually labeled or simulated), with the goal of minimizing the mean squared error loss function, through backpropagation and gradient descent optimization algorithms. The trained sub-model is then deployed to the composite popularity assessment module. During the inference phase, for a given standardized composite popularity assessment vector, it is fed into the multilayer perceptron, and after forward propagation, two correction coefficients are output. This set of dimensional interaction correction parameters automatically adjusts the importance of the first and second dimension features. The magnitude of the correction coefficient reflects the interaction influence pattern of the two original features on the final popularity assessment under the current data distribution.

[0091] Step S150B-3: The composite popularity evaluation vector is weighted and synthesized by the dimension interaction correction parameters. The aggregate access popularity duration feature of the first dimension and the cluster average access interval feature of the second dimension are fused into a scalar composite popularity evaluation feature according to the corrected contribution ratio. The composite popularity evaluation feature is positively correlated with the time span of continuous access and the frequency density of frequent access of data within the access pattern cluster.

[0092] After obtaining the dimensional interaction correction parameters, the composite popularity assessment module combines them with the original standardized composite popularity assessment vector. First, the standardized first dimension value is multiplied by the first maintenance positive coefficient to obtain the adjusted first contribution term. Then, the standardized second dimension value is multiplied by the second maintenance positive coefficient to obtain the adjusted second contribution term. Next, the adjusted first contribution term and the adjusted second contribution term are added to obtain the weighted synthesis result. Since the first dimension feature, i.e., the duration of aggregated visit popularity, is positively correlated with popularity, but the second dimension feature, i.e., the cluster average visit interval, is negatively correlated with popularity (i.e., the smaller the interval, the higher the popularity), to ensure that the final composite popularity assessment feature value has an intuitive positive correlation with popularity, the second contribution term can be added using its reciprocal form or its negative value during weighted synthesis. In this embodiment, the strategy is to calculate the adjusted first contribution term minus the adjusted second contribution term; the difference obtained is the final scalar form of the composite popularity assessment feature. If the difference is theoretically possible to be negative, it can be mapped to a positive numerical range through an affine transformation of translation and scaling. This composite popularity assessment feature is a dimensionless continuous value; the larger the value, the higher the overall popularity of the data in that cluster, i.e., the longer the duration of continuous activity and the shorter the access interval.

[0093] Step S150B-4: Establish a mapping relationship between the cluster identifier of each access pattern cluster and the corresponding composite heat evaluation feature.

[0094] The composite heat assessment module binds the calculated composite heat assessment feature value of each cluster to the cluster identifier of that cluster and stores it in a composite heat mapping table.

[0095] Step S150C: Based on the comparison results between the composite heat evaluation characteristics of each access pattern cluster and the preset composite heat classification threshold, the heat level labels of each access pattern cluster are refined and adjusted. The original heat level labels are expanded from a two-level division of cold data groups and hot data groups to a multi-level heat level label system that includes more heat level granularity.

[0096] The composite popularity assessment module divides each cluster into multiple levels based on a pre-set set of ordered composite popularity grading thresholds. These pre-set composite popularity grading thresholds can be an incremental set of values, such as threshold A, threshold B, and threshold C. These thresholds divide the numerical range of the composite popularity assessment features into several intervals. The composite popularity assessment module traverses each entry in the composite popularity mapping table to obtain the composite popularity assessment feature value for that cluster. This feature value is then compared with each threshold. For example, if the feature value is greater than threshold C, the highest popularity level label is assigned, such as "extremely hot data"; if the feature value is between threshold B and threshold C, the second highest popularity level label is assigned, such as "hot data"; if the feature value is between threshold A and threshold B, the "warm data" label is assigned; and if the feature value is less than threshold A, the "cold data" label is assigned. If the original system design only had two levels (hot and cold), this multi-level popularity grading label system refines and expands the original two-level division.

[0097] Step S150D: Update the mapping relationship set based on the multi-level heat level label system, and generate a refined result of cold and hot data grouping containing multi-level heat level labels.

[0098] The composite heat assessment module iterates through the mapping relationship set generated in step S155, using the cluster identifier as the association key, and replaces the original hot and cold level labels of each cluster with the multi-level heat level labels newly generated in step S180. Alternatively, it generates a new mapping relationship set, where each entry contains a cluster identifier and a corresponding multi-level heat level label. Finally, based on this updated mapping relationship, it backtracks to each data object, attaches its own multi-level heat level label to each data object, and generates a refined cold and hot data grouping result containing multi-level heat level labels. This refined result can guide the storage system to execute more granular data lifecycle management strategies, such as retaining "extremely hot data" in the high-end all-flash storage layer, migrating "warm data" to the hybrid storage layer, and archiving "cold data" and "extremely cold data" to tape libraries or cloud archiving services of different cost levels, respectively. This concludes the description of all steps in the cold and hot data grouping method based on access pattern clustering in this embodiment.

[0099] Figure 2 The illustration shows exemplary hardware and software components of a hot and cold data grouping system 100 based on access pattern clustering, which can implement the ideas of this application, according to some embodiments of this application. For example, a processor 120 can be used on the hot and cold data grouping system 100 based on access pattern clustering and to perform the functions in this application.

[0100] The hot and cold data grouping system 100 based on access pattern clustering can be a general-purpose server or a special-purpose server; both can be used to implement the hot and cold data grouping method based on access pattern clustering of this application. Although only one server is shown in this application, for convenience, the functions described in this application can be implemented in a distributed manner on multiple similar platforms to balance the load.

[0101] For example, the hot and cold data packet system 100 based on access pattern clustering may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and various forms of storage media 140, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, the hot and cold data packet system 100 based on access pattern clustering may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The methods of this application can be implemented according to these program instructions. The hot and cold data packet system 100 based on access pattern clustering also includes an I / O interface 150 between the computer and other input / output devices.

[0102] For ease of explanation, only one processor is described in the access pattern clustering-based hot and cold data grouping system 100. However, it should be noted that the access pattern clustering-based hot and cold data grouping system 100 of this application may also include multiple processors, and therefore the steps performed by one processor as described in this application may also be performed jointly or individually by multiple processors. For example, if the processor of the access pattern clustering-based hot and cold data grouping system 100 performs steps A and B, it should be understood that steps A and B may also be performed jointly by two different processors or individually by one processor. For example, the first processor performs step A, the second processor performs step B, or the first processor and the second processor jointly perform steps A and B.

[0103] Furthermore, this embodiment of the invention also provides a readable storage medium, wherein computer-executable instructions are preset in the readable storage medium, and when the processor executes the computer-executable instructions, the above-mentioned method for grouping hot and cold data based on access pattern clustering is implemented.

[0104] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. A method for grouping hot and cold data based on access pattern clustering, characterized in that, The method includes: Obtain a set of access log records for data objects in the storage system to be grouped. The set of access log records includes multiple access log entries recorded within a preset historical observation period. Each access log entry contains a unique identifier of the accessed data object and a timestamp of the access initiation time. Access time sequence pattern extraction is performed on the access log record set. Based on the unique identifier of the accessed data object, the timestamps of all access initiation times are aggregated to generate the access time sequence of each data object. Based on the access time sequence of each data object, the access interval sequence of each data object is generated. Based on the access time sequence of each data object, the first access time and the last access time of each data object are determined to obtain the access time sequence pattern feature set of each data object. The access time sequence pattern feature set includes the access interval sequence, the first access time, and the last access time. Based on the access time sequence pattern feature set of all data objects, the similarity structure analysis of access interval sequence is performed. The access interval pattern structure with periodic repetition characteristics is extracted from the access interval sequence of each data object, and the periodic pattern descriptor of each data object is generated. The periodic pattern descriptor includes the repetition period length feature of the access interval sequence, the interval distribution feature of the access interval sequence in a single period, and the interval fluctuation feature of the access interval sequence between consecutive periods. Based on the repeating period length characteristics, interval distribution characteristics of the access interval sequence within a single period, and interval fluctuation characteristics of the access interval sequence between consecutive periods in the periodic pattern descriptors of all data objects, access pattern similarity clustering is performed on all data objects. Data objects with similar periodic pattern descriptors are merged into the same access pattern group to obtain at least one access pattern cluster. Each access pattern cluster corresponds to a cluster identifier. Based on the first and last access times in the access time sequence pattern feature set of all data objects within each access pattern cluster, the aggregated access heat duration feature of each access pattern cluster is calculated. According to the comparison result of the aggregated access heat duration feature of each access pattern cluster with the preset heat duration threshold, a heat level label is assigned to each access pattern cluster. The heat level label indicates whether the data group corresponding to the access pattern cluster belongs to a cold data group or a hot data group. A mapping relationship set between each cluster identifier and the heat level label is established, and cold and hot data grouping results are generated based on the mapping relationship set.

2. The method for grouping hot and cold data based on access pattern clustering according to claim 1, characterized in that, The access time sequence pattern extraction of the access log record set involves aggregating the timestamps of all access initiation times corresponding to the unique identifier of the accessed data object, generating an access time sequence for each data object, generating an access interval sequence for each data object based on the access time sequence, and determining the first and last access times of each data object based on the access time sequence, thus obtaining an access time sequence pattern feature set for each data object, including: Iterate through each access log entry in the access log record set, and use the unique identifier of the accessed data object as the aggregation key to merge access log entries with the same unique identifier into the same access record group. Each access record group corresponds to one data object. For each data object corresponding to the access record group, extract the timestamp of the access initiation time of all access log entries in the access record group, sort them in ascending order according to the time sequence of the timestamps, and generate the access time sequence of the data object. For each data object's access time series, the time interval length between adjacent timestamps is calculated sequentially, and the time interval lengths between all adjacent timestamps are arranged in chronological order to generate the data object's access interval sequence. For each data object's access time series, the moment with the smallest timestamp value in the timestamp marker sequence is extracted as the first access moment of the data object, and the moment with the largest timestamp value in the timestamp marker sequence is extracted as the last access moment of the data object. The access interval sequence, first access time, and last access time of each data object are combined into an access time sequence pattern feature set of the data object, and a mapping relationship is established between the unique identifier of the data object and the access time sequence pattern feature set.

3. The method for grouping hot and cold data based on access pattern clustering according to claim 1, characterized in that, The step involves performing access interval sequence similarity structure analysis based on the access time sequence feature set of all data objects, extracting access interval pattern structures with periodic repetition characteristics from the access interval sequences of each data object, and generating periodic pattern descriptors for each data object, including: For each data object, the access interval sequence in the access time sequence feature set is analyzed to identify the fluctuation pattern of the interval value over time, and a fluctuation pattern descriptor of the access interval sequence is generated. Based on the fluctuation pattern descriptor of the access interval sequence, self-similarity detection is performed on the access interval sequence. The access interval sequence is matched with itself at different time offsets to extract the repetitive interval pattern segments that appear in the access interval sequence, and a candidate repetitive segment set of the access interval sequence is generated. The candidate repetitive segment set contains the starting position and segment length of each repetitive interval pattern segment. For each repetitive interval pattern fragment in the candidate repetitive fragment set, perform interval value deviation statistical analysis on multiple occurrence instances in the access interval sequence, calculate the degree of difference between the interval values ​​at corresponding positions of each repetitive interval pattern fragment in different occurrence instances, and select stable repetitive fragments with a degree of difference less than a preset deviation tolerance threshold from the candidate repetitive fragment set based on the degree of difference, thereby obtaining a stable repetitive fragment set. Periodic integrity analysis is performed on each stable repeating segment in the set of stable repeating segments to verify whether the stable repeating segment maintains stable periodic repetition throughout the entire time span of the access interval sequence. The average repetition interval of the stable repeating segment is extracted as the repetition period length feature of the access interval sequence. The statistical mean sequence of the interval values ​​at each position within the stable repeating segment is extracted as the interval distribution feature of the access interval sequence within a single period. Extract the statistical measure of the deviation of the corresponding positional interval value of each stable repeating segment from the statistical mean sequence in instances of occurrence in different periods, and use it as the interval fluctuation feature of the access interval sequence between consecutive periods. Combine the repeating period length feature of the access interval sequence, the interval distribution feature of the access interval sequence in a single period, and the interval fluctuation feature of the access interval sequence between consecutive periods into the periodic pattern descriptor of the data object.

4. The method for grouping hot and cold data based on access pattern clustering according to claim 1, characterized in that, The access pattern similarity clustering is performed on all data objects based on the repeating period length feature, the interval distribution feature of the access interval sequence within a single period, and the interval fluctuation feature of the access interval sequence between consecutive periods in the periodic pattern descriptors of all data objects. Data objects with similar periodic pattern descriptors are grouped into the same access pattern group to obtain at least one access pattern cluster, including: A similarity measurement space is constructed to measure the similarity of access patterns between different data objects. The similarity measurement space includes a distance measurement function in the dimension of repetition period length, a sequence morphology distance measurement function in the dimension of interval distribution, and a fluctuation amplitude distance measurement function in the dimension of interval fluctuation. For any two data objects, the difference in period length between the repeated period length features of the access interval sequences of the two data objects is calculated using the distance metric function of the repeated period length dimension; the difference in distribution pattern between the access interval sequences of the two data objects within a single period is calculated using the sequence pattern distance metric function of the interval distribution dimension; and the difference in the degree of fluctuation between the access interval sequences of the two data objects between consecutive periods is calculated using the fluctuation amplitude distance metric function of the interval fluctuation dimension. The differences in period length, distribution pattern, and fluctuation are weighted and fused to generate a comprehensive access pattern difference metric between any two data objects. The weights used in the weighted fusion are allocated according to the discriminative contribution of each dimension of the difference in the clustering objective. A similarity graph of access patterns between data objects is constructed based on the comprehensive access pattern difference metric between all pairs of data objects. In the similarity graph of access patterns between data objects, a connection edge is established between pairs of data objects whose comprehensive access pattern difference metric is less than a preset similarity judgment threshold. A connected component search is performed in the access pattern similarity graph between the data objects to merge data objects with direct connection edges or indirect connection paths into the same access pattern group, forming at least one access pattern cluster. A unique cluster identifier is assigned to each access pattern cluster, and the cluster identifier is associated with the unique identifier of all data objects within the access pattern cluster.

5. The method for grouping hot and cold data based on access pattern clustering according to claim 1, characterized in that, The method calculates the aggregated access popularity duration feature of each access pattern cluster based on the first and last access times of the access time sequence feature set of all data objects within each access pattern cluster. Then, based on the comparison between the aggregated access popularity duration feature of each access pattern cluster and a preset popularity duration threshold, it assigns a popularity level label to each access pattern cluster, including: For each access pattern cluster, obtain the access time sequence pattern feature set of all data objects within the access pattern cluster, and extract the first access time and the last access time of the data object from the access time sequence pattern feature set of each data object; For each data object, calculate the time span between the last access time and the first access time of the data object to obtain the individual access activity duration of the data object; The individual access activity duration of all data objects within each access pattern cluster is aggregated and statistically analyzed. The distribution center trend characterization value of the individual access activity duration within the access pattern cluster is extracted as the aggregate access popularity duration feature of the access pattern cluster. The distribution center trend characterization value reflects the concentration level of the access activity duration of most data objects within the access pattern cluster. The aggregated access popularity duration feature of each access pattern cluster is compared with the preset popularity duration threshold. If the aggregated access popularity duration feature of an access pattern cluster is greater than or equal to the preset popularity duration threshold, a popularity level label indicating hot data groups is assigned to the access pattern cluster. If the aggregated access popularity duration feature of an access pattern cluster is less than the preset popularity duration threshold, a popularity level label indicating cold data groups is assigned to the access pattern cluster. Establish a correspondence between the cluster identifier of each access pattern cluster and the assigned heat level label. Summarize all correspondence entries to form the mapping relationship set. Based on the mapping relationship set, attach the corresponding heat level label to the cluster identifier of each data group to generate a cold and hot data grouping result containing the unique identifier of each data object and its assigned heat level label.

6. The method for grouping hot and cold data based on access pattern clustering according to claim 5, characterized in that, The step of extracting the distribution center trend representation value of the duration of active access for individuals within the access pattern cluster as the aggregated access popularity duration feature of the access pattern cluster includes: The individual active duration of all data objects within the access pattern cluster is used to form a duration set to be analyzed. The duration values ​​in the duration set to be analyzed are subjected to numerical distribution pattern analysis to determine the concentrated distribution area and discrete distribution area of ​​the duration values ​​in the duration set to be analyzed. Based on the distribution morphology characteristics of the concentrated and discrete distribution areas, a central trend extraction strategy matching the distribution morphology characteristics is selected. The central trend extraction strategy adopts the symmetrical distribution center extraction method when the distribution of duration values ​​in the set of durations to be analyzed presents a symmetrical shape, and adopts the skewed distribution center extraction method when the distribution of duration values ​​in the set of durations to be analyzed presents a skewed shape. The selected central trend extraction strategy is used to extract the duration set to be analyzed. From the duration value of the duration set to be analyzed, a statistical quantity that can characterize the concentrated position of the main part of the duration set to be analyzed is extracted. The extracted statistical quantity is used as the aggregated access popularity duration feature of the access pattern cluster. The cluster identifier of each access pattern cluster is associated with the duration of aggregated access popularity of the access pattern cluster and stored accordingly.

7. The method for grouping hot and cold data based on access pattern clustering according to claim 1, characterized in that, After calculating the aggregated access popularity duration feature of each access pattern cluster based on the first access time and last access time of the access time sequence pattern feature set of all data objects within each access pattern cluster, the method further includes: Obtain the access interval sequence from the access time sequence feature set of all data objects in each access pattern cluster, perform intra-cluster access frequency statistics on the access interval sequence of all data objects in each access pattern cluster, and extract the cluster average access interval feature corresponding to each access pattern cluster. By performing a joint heat analysis on the duration of aggregated access heat of each access pattern cluster and the average access interval of each access pattern cluster, a composite heat evaluation feature for each access pattern cluster is obtained. The composite heat evaluation feature reflects both the duration of access and the density of access frequency of the access pattern cluster. Based on the comparison results between the composite heat evaluation characteristics of each access pattern cluster and the preset composite heat classification threshold, the heat level labels of each access pattern cluster are refined and adjusted. The original heat level labels are expanded from a two-level division of cold data group and hot data group to a multi-level heat level label system that includes more heat level granularity. The mapping relationship set is updated based on the multi-level popularity level tag system, and a refined result of cold and hot data grouping containing multi-level popularity level tags is generated.

8. The method for grouping hot and cold data based on access pattern clustering according to claim 7, characterized in that, The method involves performing a joint heat analysis on the duration of aggregated access popularity of each access pattern cluster and the average access interval of each access pattern cluster to obtain the composite heat evaluation characteristics of each access pattern cluster, including: For each access pattern cluster, the aggregated access popularity duration feature and the cluster average access interval feature of the access pattern cluster are fused under a unified composite popularity evaluation dimension to construct a composite popularity evaluation vector that simultaneously represents the popularity duration capability and popularity frequency density of the access pattern cluster. The first dimension of the composite popularity evaluation vector corresponds to the aggregated access popularity duration feature, and the second dimension of the composite popularity evaluation vector corresponds to the cluster average access interval feature. Model the inter-dimensional interaction relationship of the composite popularity evaluation vector, capture the mutual influence pattern between the aggregated access popularity duration feature and the cluster average access interval feature on the popularity evaluation result, and generate dimensional interaction correction parameters. The dimensional interaction correction parameters adjust the contribution ratio of each dimensional feature in the composite popularity evaluation vector to the final composite popularity evaluation feature. The composite popularity evaluation vector is weighted and synthesized by the dimension interaction correction parameters. The aggregate access popularity duration feature of the first dimension and the cluster average access interval feature of the second dimension are fused into a scalar composite popularity evaluation feature according to the corrected contribution ratio. The composite popularity evaluation feature is positively correlated with the time span of continuous access and the frequency density of frequent access of data within the access pattern cluster. Establish a mapping relationship between the cluster identifiers of each access pattern cluster and the corresponding composite heat evaluation features.

9. The method for grouping hot and cold data based on access pattern clustering according to claim 7, characterized in that, The step of obtaining the access interval sequence from the access time sequence feature set of all data objects within each access pattern cluster, performing intra-cluster access frequency statistics on the access interval sequence of all data objects within each access pattern cluster, and extracting the cluster average access interval feature corresponding to each access pattern cluster includes: For each access pattern cluster, traverse the access time sequence pattern feature set of all data objects in the access pattern cluster, extract the access interval sequence of the data object from the access time sequence pattern feature set of each data object, and merge all the extracted access interval sequences into a cluster-level interval sequence pool. A holistic statistical analysis is performed on all access interval values ​​in the cluster-level interval sequence pool. A central trend statistic that can characterize the typical interval length of data objects being accessed within the access pattern cluster is extracted from the access interval values ​​in the cluster-level interval sequence pool. The central trend statistic is used as the cluster average access interval feature of the access pattern cluster. Establish a correspondence between the cluster identifier of each access pattern cluster and the cluster average access interval feature of the access pattern cluster.

10. A hot and cold data grouping system based on access pattern clustering, characterized in that, The hot and cold data grouping system based on access pattern clustering includes a processor and a memory, the memory and the processor being connected. The memory is used to store programs, instructions or code, and the processor is used to execute the programs, instructions or code in the memory to implement the hot and cold data grouping method based on access pattern clustering as described in any one of claims 1-9.

Images