Data hierarchical storage method and device based on access characterization, equipment and medium

Abstract

The invention relates to the technical field of data processing, can be applied to business scenes of financial science and technology, medical treatment and health and the like, and discloses a data hierarchical storage method, device, equipment and medium based on access characterization. Writing the data into a matched first target storage device according to a preset device performance corresponding relation; after the data are stored, continuously monitoring subsequent access characteristics of the data, and dynamically judging whether the heat state is changed or not; and if the popularity state is changed and a specific migration condition is met, redistributing the second target storage device and completing data migration operation so as to realize automatic hierarchical storage management of cold and hot data. By acquiring the data access representation and dynamically identifying the popularity state, hierarchical migration of data with different popularity among storage devices with different performances is realized, the resource use efficiency and access performance of a storage system are improved, and the storage cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data hierarchical storage method, device, equipment and storage medium based on access representation. Background Art

[0002] In today's data-driven enterprise environment, with the increasing complexity of business systems and the continuous expansion of data volumes, enterprises are placing higher demands on efficient and intelligent storage systems. Traditional network-attached storage (NAS) systems typically use fixed-configuration storage media, such as HDDs or SSDs, and apply the same storage strategy regardless of data access frequency. This "one-size-fits-all" storage approach ignores the variability of data access behavior, resulting in large amounts of cold data occupying high-performance storage resources for long periods of time. This not only adds unnecessary costs but also reduces the access efficiency of frequently accessed data, limiting the overall performance of the storage system.

[0003] In the fintech sector, institutions often need to process hot data such as highly concurrent transactions, risk control model logs, and user interactions in real time. At the same time, they also need to retain large amounts of cold data with extremely low access frequencies, such as historical vouchers and audit records. Existing storage systems often fail to effectively identify and differentiate the access characteristics of this data, resulting in both core transaction data and audit archives being written to high-performance storage media. This causes performance congestion, wastes resources, and impacts real-time risk control and customer response efficiency.

[0004] In the healthcare sector, electronic medical records, imaging data, and monitoring data constitute a large, multi-faceted medical data set. This includes both hot data such as diagnostic reports and vital sign curves, which require frequent short-term access, and cold data such as long-term archived case records and follow-up documents. Traditional storage systems lack dynamic scheduling and hierarchical management capabilities for this mixed storage of hot and cold data. This leads to performance bottlenecks such as slow loading and access conflicts in key applications like image retrieval and medical record analysis, severely impacting clinical work efficiency and data service quality.

[0005] Furthermore, while some existing technologies incorporate rule engines to configure data tiered storage, these often rely on manually set access frequency thresholds and static policy matching. These methods are unable to adapt to the fluctuations in data access characteristics brought about by rapidly changing business models, and lack the ability to analyze and predict data access behavior in real time. These static policies struggle to respond promptly to shifts in data popularity, often resulting in cold data being stored on high-performance media for extended periods, or hot data being stored on low-speed media, limiting the intelligence and adaptability of storage systems.

[0006] Therefore, existing technologies still have obvious shortcomings in data access feature identification, dynamic tiered management of hot and cold data, and optimized allocation of storage resources. More intelligent and refined solutions are urgently needed to improve the overall storage efficiency and service performance of the system. Summary of the Invention

[0007] The main purpose of the present invention is to provide a data tiered storage method, device, equipment and storage medium based on access representation, aiming to solve the technical problem that the existing technology cannot dynamically identify the data heat status based on data access behavior and automatically adjust the storage medium allocation, thereby resulting in mixed storage of hot and cold data and low resource utilization efficiency.

[0008] To achieve the above object, the present invention provides a data hierarchical storage method based on access representation, comprising:

[0009] Obtaining an initial data access representation of the raw data;

[0010] Based on the initial data access representation, identifying data access characteristics of the original data through a first data analysis process to generate an initial data heat state of the original data;

[0011] Allocating a first target storage device to the original data according to the initial data heat state and a preset storage device performance correspondence, and storing the original data in the first target storage device;

[0012] monitoring a subsequent data access characteristic of the stored data in the first target storage device;

[0013] determining, based on the subsequent data access characterization, through a second data analysis process, whether a data heat state of the stored data has changed compared to the initial data heat state;

[0014] When the data heat state of the stored data changes and meets the preset data movement condition, a second target storage device is allocated for the stored data, and the stored data is moved from the first target storage device to the second target storage device.

[0015] Furthermore, to achieve the above-mentioned object, the present invention provides a data tiered storage device based on access representation, comprising:

[0016] An access representation acquisition module, used to obtain initial data access representation of raw data;

[0017] an access characteristic identification module, configured to identify data access characteristics of the original data through a first data analysis process based on the initial data access representation, and generate an initial data heat state of the original data;

[0018] an initial write decision module, configured to allocate a first target storage device to the original data according to the initial data heat state and a preset storage device performance correspondence, and store the original data in the first target storage device;

[0019] an access behavior monitoring module, configured to monitor subsequent data access characteristics of the data stored in the first target storage device;

[0020] a heat state determination module, configured to determine, based on the subsequent data access representation, through a second data analysis process, whether the data heat state of the stored data has changed compared to the initial data heat state;

[0021] The storage migration execution module is used to allocate a second target storage device to the stored data and move the stored data from the first target storage device to the second target storage device when the data heat state of the stored data changes and the preset data movement conditions are met.

[0022] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer device, which includes a memory, a processor, and a data tiered storage program based on access representation stored in the memory and executable on the processor, wherein the data tiered storage program based on access representation, when executed by the processor, implements the steps of the data tiered storage method based on access representation as described above.

[0023] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a data tiered storage program based on access representation is stored. When the data tiered storage program based on access representation is executed by a processor, the steps of the data tiered storage method based on access representation as described above are implemented.

[0024] Beneficial effects: The present invention relates to the field of data processing technology and can be applied to business scenarios such as financial technology and medical health. A data hierarchical storage method, apparatus, equipment and medium based on access representation are disclosed, including: obtaining the initial data access representation of the original data, identifying the initial data heat state of the original data based on the access representation, matching the corresponding first target storage device according to the initial data heat state and storing the original data; continuously monitoring the subsequent data access representation of the data stored in the first target storage device, and judging whether the heat state has changed; when the heat state has changed and the preset data movement conditions are met, reallocating the second target storage device and migrating the stored data from the first target storage device to the second target storage device. The present invention realizes hierarchical migration of data of different heat levels between storage devices of different performances by obtaining data access representations and dynamically identifying heat states, thereby improving the resource utilization efficiency and access performance of the storage system and reducing storage costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0026] Figure 1 A schematic diagram of an application environment of a data hierarchical storage method based on access representation in an embodiment of the present invention;

[0027] Figure 2 This is a flow chart of an embodiment of a data hierarchical storage method based on access representation according to the present invention;

[0028] Figure 3 A schematic diagram of functional modules of a preferred embodiment of a data tiered storage device based on access representation according to the present invention;

[0029] Figure 4 A schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0030] Figure 5 FIG. 2 is another structural diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0032] The data hierarchical storage method based on access representation provided by the embodiment of the present invention can be applied in the following situations: Figure 1In an application environment, the user terminal communicates with the server terminal through a network. The server terminal can obtain the initial data access representation of the original data through the user terminal, identify the initial data heat state of the original data based on the access representation, match the corresponding first target storage device according to the initial data heat state and store the original data; continuously monitor the subsequent data access representation of the data stored in the first target storage device, and judge whether the heat state changes; when the heat state changes and meets the preset data movement conditions, reallocate the second target storage device and migrate the stored data from the first target storage device to the second target storage device. The present invention realizes the hierarchical migration of different heat data between storage devices with different performances by obtaining data access representations and dynamically identifying heat states, thereby improving the resource utilization efficiency and access performance of the storage system and reducing storage costs. Among them, the user terminal can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server terminal can be implemented with an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0033] See also Figure 2 , Figure 2 This is a flow chart of an embodiment of the data hierarchical storage method based on access representation provided by the present invention. It should be noted that although the flow chart shows a logical order, in some cases, the steps shown or described may be performed in a different order than here.

[0034] like Figure 2 As shown, the data hierarchical storage method based on access representation proposed by the present invention includes the following steps:

[0035] S10, obtaining an initial data access representation of the original data;

[0036] In this embodiment, the initial data access representation of raw data is obtained primarily by collecting and processing various types of information recording the access behavior of the raw data to form a data set used to characterize the data access characteristics. The raw data may include structured data, unstructured data, semi-structured data, or log data, and may come from sources such as databases, distributed file systems, and data lake platforms. The "access representation" in the initial data access representation is a composite feature set designed to express the access patterns of the raw data in terms of time, frequency, method, and interval.

[0037] To implement this process, we first need to set a time period during which we continuously collect access log information for the raw data. This access log records the specific actions taken when the data is read and written, including access types such as read, write, modify, and delete. The access log should also record the timestamp of the access operation, the identifier of the access subject, the source IP address of the access, and the success or failure status of the access, for subsequent calculations.

[0038] Next, based on the collected log data, the total number of times the raw data is accessed within a unit time window is calculated to form an access frequency feature. The unit time window can be set based on specific business circumstances, for example, 5 minutes, 1 hour, or 24 hours. The access frequency feature reflects the activity level of the raw data within a certain time period and its utilization of computing resources.

[0039] In addition to access frequency, it's also necessary to assess the intervals between consecutive accesses of the original data. This is done by calculating the time difference between two consecutive accesses and taking the average to generate a time interval feature. This time interval feature reveals the concentration and regularity of data access. A smaller average value typically indicates high-frequency and continuous data access.

[0040] Furthermore, statistics are needed on the distribution ratios of different access operation types, e.g., read operations account for 70%, write operations account for 20%, and delete operations account for 10%. The distribution ratio of operation types is a key dimension in access characterization, used to reflect differences in data usage patterns. It is particularly valuable for decision-making during data migration and hot / cold data classification.

[0041] Finally, the access frequency, time interval, and operation type distribution characteristics are integrated to construct a complete initial data access representation. This representation can be in the form of a vector, structured record, or multi-dimensional tensor, which is used for subsequent data analysis and popularity determination.

[0042] In practice, access log collection can be accomplished by a data access monitoring agent deployed in the storage system or business platform. This agent can be injected into the business call chain through a hook mechanism or non-intrusive collection can be achieved through the operating system file system event capture interface. Data access records are written to a log database or message queue, and then parsed in real time by a log processing engine (such as Flink or Spark).

[0043] The access frequency calculation module can be implemented based on a sliding or fixed time window, using a counter to accumulate and calculate the total number of accesses after the window ends. Time intervals can be calculated by performing differential and mean calculations based on a chronologically sorted list of access timestamps. The distribution statistics of access operation types can be aggregated using the operation type field using MapReduce.

[0044] The data feature integration module can uniformly call the calculation results of three types of features: access frequency, time interval and operation type in the middle layer, and construct them in JSON format or vectorized representation and store them in the metadata system.

[0045] Example: In the healthcare sector, electronic medical records, examination image data, and patient behavior logs generated by hospital information systems are often stored in a unified system. By collecting access behavior for this data across departments and time periods, and building access frequency and access interval features, we can accurately distinguish between hot data related to real-time diagnosis and treatment and cold data for archiving and retention, supporting subsequent automatic archiving or expedited service.

[0046] In the fintech sector, access characteristics for customer transaction records, risk control model logs, and compliance audit data differ significantly. Obtaining initial access representations for raw data can help identify hot data nodes in high-frequency transactions, optimize their distributed storage structure, improve data access efficiency, and implement cold storage for infrequently accessed audit data, reducing resource consumption.

[0047] By acquiring the initial data access representation of raw data, this embodiment can comprehensively capture the access behavior patterns of raw data in actual operating environments. Without relying on manual labels or preset rules, it provides a quantifiable input basis for subsequent popularity status identification and storage strategy optimization. Especially in distributed data environments, this approach avoids strong coupling with specific business logic and provides high-quality feature input for the subsequent introduction of AI models to process access behavior.

[0048] S20, based on the initial data access representation, identifying data access characteristics of the original data through a first data analysis process to generate an initial data heat state of the original data;

[0049] In this embodiment, the initial data access characterization is used to analyze, identify the access characteristics of the original data, and generate the initial data heat state, which is a core action that connects the previous and next steps in the data processing chain. The initial data access characterization has aggregated access frequency characteristics, time interval characteristics, and access operation type distribution characteristics. The analysis and processing here requires extracting the access pattern of the original data based on multi-dimensional information to make it classifiable and infer the heat state through the model.

[0050] First, data cleansing is required for the initial data access representation. Since data access log collection may include missing access counts, abnormal access timestamps, and non-standard values ​​for the operation type field, missing value filling, outlier filtering, and standardization are necessary. Missing value filling can be achieved using methods such as sliding mean and median replacement; abnormal access operation types can be removed using a whitelist mechanism; and standardization typically utilizes z-score normalization to ensure that all feature dimensions are of equal scale, facilitating subsequent model convergence.

[0051] After data preprocessing is complete, the results are fed into a pretrained model for classification inference. This model is a multi-classification model trained on labeled historical access representations and popularity state mapping samples. It can employ algorithms such as random forests, gradient boosting trees, and neural networks. The model output is a multi-class probability distribution, indicating the confidence level that the original data belongs to different access popularity levels. For example, hot data is actively and frequently accessed data, cold data is rarely accessed or has not been accessed for a long time, and warm data falls in between, exhibiting periodic or bursty access characteristics.

[0052] Finally, the raw data is classified into one of three categories based on the maximum confidence level, and its popularity status is marked. To support subsequent data scheduling operations, a unique identification value is generated from the popularity status result and written into the corresponding metadata description, forming a callable field in the data management system.

[0053] In actual deployment, the data cleansing module is typically implemented using Spark or Pandas. Its processing flow can be defined within the data access pipeline as a standard operation chain. Abnormal operation types can be automatically identified based on a manually defined set of valid operations, combined with regular expression matching and filtering mechanisms. The normalization module can set global feature statistics to maintain consistency across different batches of data.

[0054] Classification models can be trained during system initialization or periodic retraining. Training samples are derived from a cross-record of access behavior and storage distribution in the historical storage system. During deployment, the model is compiled into the ONNX format and loaded into the inference service using TensorRT or another inference engine. The model inference service connects to the main system via a gRPC interface, supporting batch access to representation inputs.

[0055] The classification probability vector output by the model is then used by the decision function to categorize the data with the highest probability. If the classification confidence is too low, the data is marked as uncertain and re-identified after multiple cycles of data collection. The classification results are injected into the global metadata index through the RESTful interface or the metadata management engine and bound to the original data's identity.

[0056] Example: In healthcare data scenarios, different patients have distinct access patterns for their medical records, imaging data, and vital sign monitoring logs. By analyzing their initial data access characteristics, we can identify hot data currently under treatment and cold data from archived cases. By properly classifying their popularity, we can improve online access performance and reduce backend storage burdens.

[0057] In the financial sector, customer behavior logs, account change records, and transaction flow data exhibit significant access heterogeneity. By using the aforementioned processing method to identify initial popularity, the system can accurately distinguish high-frequency data during peak trading periods from static risk control parameter tables, providing high-performance support for real-time response requests while migrating less-popular data to more cost-effective storage resources.

[0058] This embodiment converts access representations into categorizable popularity states, not only modeling the usage behavior of raw data but also providing quantitative criteria for subsequent tiered storage scheduling. This approach improves the accuracy and robustness of popularity identification, avoids the limitations of relying on static rules that cannot adapt to dynamic access changes, ensures that data under different access modes is properly classified, and establishes the foundation for subsequent resource matching strategies.

[0059] S30, allocating a first target storage device to the original data according to the initial data heat state and a preset storage device performance correspondence, and storing the original data in the first target storage device;

[0060] In this embodiment, after identifying the data access heat state, matching the original data with the appropriate storage device is determined based on the coupling relationship between data characteristics and device capabilities. Data heat state here reflects comprehensive characteristics such as access frequency, time interval, and operational behavior, while storage device performance indicators include read and write speed, access latency, I / O concurrency, unit storage cost, and energy consumption. This matching process aims to optimize resource utilization by matching hot data with high-speed devices and cold data with low-cost devices.

[0061] To implement this, you first need to establish a mapping between storage device performance. This mapping can be obtained through a pre-set configuration file or resource-aware mechanism, creating a matrix that describes the matching of heat levels with device performance attributes. For example, high heat levels can be mapped to NVMe SSDs, medium heat levels to SATA SSDs, and low heat levels to HDDs or distributed archiving devices. This matrix can be established by combining device benchmark performance test results with cost assessment parameters and cached in the system as a rule set.

[0062] Once raw data is assigned a hot status, the system uses that status to filter all candidate devices in the storage resource pool that meet the requirements. The device resource pool is a dynamically managed collection of devices that contains metadata such as each device's real-time available space, current load rate, health status, and category.

[0063] Next, a priority is selected from the candidate set. This selection logic is typically based on a weighted scoring mechanism, taking into account the device's unit storage cost, read / write performance score, and current load balancing factor to calculate a comprehensive priority score. The scoring function can incorporate device aging factors and hot / cold migration cost factors as fine-tuning parameters. The device with the highest score is selected as the primary target storage device for the original data.

[0064] After allocation is complete, a secure data transmission link is established to ensure data is not tampered with, leaked, or lost during network transmission. Link security is typically achieved through TLS, IPSec, or a channel assurance mechanism based on the SDP architecture. Data transmission uses block-level segmentation and parallel writing to match the throughput capabilities of the target device.

[0065] After the data is successfully stored, the system synchronously updates the global metadata index and maps the original data identifier to the first target storage device identifier. This operation ensures data addressability and supports subsequent data access scheduling, status monitoring, and migration operations.

[0066] By deploying a set of device scheduling service nodes, we can periodically synchronize the operating status and performance parameters of each storage device and maintain a unified device capability database. This database provides a query interface that supports searching for candidate device sets by popularity category. Data popularity status is transmitted from the upstream identification module to the device scheduling module, triggering device screening and allocation.

[0067] The optimization algorithm dynamically adjusts weight parameters based on custom policy configurations, such as increasing the weight of read and write rates in high-concurrency scenarios and increasing the impact factor of unit capacity cost in cost-sensitive businesses. The optimization function can be deployed at the load balancing layer and calculated in real time by the system's built-in scoring model.

[0068] During data transmission, the message-based middleware sends a write task instruction to the target device management process. The write task is executed asynchronously and concurrently. A hash digest is recorded during the transmission process, and data integrity is verified on the storage device. Upon receiving a successful write signal, the metadata management engine is triggered to record the device mapping relationship and annotate the status in the data visibility module.

[0069] Policy constraints can also be added during the target device selection process, such as specifying that multiple raw data with the same data classification or business tag be written to the same physical location to reduce cross-device scheduling costs.

[0070] Example: In financial business scenarios, high-frequency user transaction logs, risk model iteration parameters, and strategy testing data are subject to high access pressure and can be matched to high-speed write-type SSD devices based on their popularity. Long-term archived contract scans and historical transaction data can be allocated to low-power cold storage devices, thus ensuring a balance between response speed and cost.

[0071] In healthcare business scenarios, real-time collected vital signs data and online image analysis caches need to be deployed to low-latency storage nodes to meet second-level response requirements. Long-term access data, such as patient historical imaging archives and surgical records, are written to large-capacity storage arrays after heat recognition, effectively alleviating primary storage pressure and reducing operating costs.

[0072] This embodiment precisely matches data popularity with device capabilities, ensuring that the storage location of raw data is highly consistent with its access patterns. This effectively mitigates the impact of high-frequency data on low-speed devices and prevents cold data from occupying high-performance resources, significantly improving the rationality and efficiency of storage resource allocation. Furthermore, a selection mechanism automates device selection, reducing manual intervention and enabling the implementation of an intelligent tiered storage strategy.

[0073] S40, monitoring subsequent data access characteristics of the stored data in the first target storage device;

[0074] In this embodiment, continuous monitoring of subsequent access to stored data is a prerequisite for dynamically adjusting the data storage location. The essence of this process is to build a behavioral analysis system that changes over time to determine whether data access patterns have significantly changed, thereby providing a basis for determining data migration.

[0075] First, you need to set a dynamic monitoring time window for each data object. This time window is not fixed but dynamically adjusted based on the current popularity of the data. For example, for hot data, a shorter monitoring period is used to ensure high responsiveness and capture fine-grained behavioral changes; for cold data, a longer monitoring period can be set to reduce unnecessary system load. The time window adjustment mechanism can be implemented using a sliding window strategy or an exponential decay function.

[0076] During each dynamic monitoring window, the system must accurately record data access events, primarily read and write operations. The event record structure includes fields such as access timestamp, access type, and access initiator identifier, generating real-time access operation records. This record data structure can be stored in a lightweight in-memory cache, with performance and stability maintained through periodic flushing to disk.

[0077] Next, we extract key access statistics from these access operation records. The number of read and write operation events reflects the frequency of use during the window, forming a dynamic access frequency metric. The average time difference between two consecutive accesses reflects the concentration and continuity of data use, forming a dynamic time interval metric. These statistical metrics provide the foundation for subsequent popularity assessment and trend prediction.

[0078] To capture the evolving direction of access trends, it's necessary to compare the current dynamic access frequency metric with historical access frequency characteristics. The difference can be calculated using methods such as differential algorithms, sliding average rate of change, or the Z-score method to generate an access trend change factor. This factor reflects the degree of deviation in data access frequency from historical performance and provides a reference for decision-making in the next stage of data popularity identification.

[0079] Ultimately, by integrating dynamic access frequency, dynamic time interval, and trend change factors, a multi-dimensional representation vector of subsequent access is formed. This representation not only quantifies the current access behavior of the data but also constructs an access trend model, serving as a key input for subsequent time series analysis and popularity prediction.

[0080] An access listener can be deployed at the proxy layer to transparently capture and record all access operations after data is written to the device. The access listener can be integrated into the storage system's IO path or deployed on access-layer middleware nodes, providing high-performance event filtering and data collection capabilities.

[0081] Dynamic configuration of monitoring time windows is achieved through metadata tagging. The heat label of each data item is associated with its monitoring period parameters, and the scheduling frequency of monitoring tasks is adjusted synchronously with updates. Monitoring scheduling is managed by an event-driven task scheduler, which controls the number of concurrent monitoring tasks through system load feedback mechanisms.

[0082] Access event records are cached using an in-memory database or ring buffer. Thresholds are set to trigger persistence, avoiding frequent disk writes. Access frequency and time interval metrics are updated in real time by a sliding statistics module. Trend factors are calculated using an exponential moving average compared to a periodic baseline.

[0083] Subsequent access characterization vectors can also be transmitted in real time to a centralized analysis node, enabling cross-device data state aggregation and providing a unified input source for global storage optimization. For important data, cross-dimensional access behavior analysis can also be achieved by combining access subject permission information or access source system tags.

[0084] Example: In the fintech sector, the access frequency of customer behavior logs, risk control parameter caches, and trade matching data can fluctuate dramatically with market activity. By setting up short-term dynamic monitoring windows to capture these access fluctuations in real time, we can effectively identify periods of high activity and low activity, enabling synchronized adjustments to data storage strategies.

[0085] In the healthcare sector, real-time monitoring of vital signs, imaging analysis results, and historical case access behavior is crucial for identifying active and follow-up phases of patients. Monitoring cycle parameters can be set for different types of health data, dynamically adjusting the granularity of data access trend assessments to rationally allocate medical data storage resources and improve the responsiveness and data availability of remote diagnosis and treatment systems.

[0086] By combining dynamic time windows with multi-dimensional access metrics, this system accurately captures changes in data usage across different lifecycle stages, effectively avoiding the sluggish response of traditional static classification models to data state changes. Furthermore, by incorporating fluctuating trends in access behavior into the analysis model, the system enhances its ability to predict future access changes, providing a more reliable data foundation for subsequent identification of popularity states and migration strategies.

[0087] S50, determining, based on the subsequent data access characterization, through a second data analysis process, whether the data heat state of the stored data has changed compared to the initial data heat state;

[0088] In this embodiment, to identify dynamic changes in data access patterns during the storage lifecycle, a secondary analysis and processing mechanism is introduced to quantify trends in existing data access behavior, thereby identifying changes in data popularity. The key is to map subsequent access representation vectors to the probability space of popularity state changes and use this to determine whether they exceed the fluctuation threshold of the original popularity.

[0089] The first step in the analysis is to normalize the subsequent data access representations so that data from different time windows and access frequencies are comparable on the same scale. This normalization typically uses Z-score normalization, min-max compression, or baseline difference calculation to eliminate scale effects or periodic errors in access behavior.

[0090] The processed, standardized access representations are fed into a time series analysis model trained on historical access sequences and corresponding annotations of popularity state changes. The input sequence structure consists of access frequencies and time intervals within multiple dynamic monitoring time windows, capturing the continuity, mutation, and trend patterns of access behavior. The annotation structure characterizes the popularity change trend within each time period, such as from hot to warm, warm to cold, or stable.

[0091] Time series models can be constructed using long short-term memory (LSTM), variational autoencoders (VAE), or ARIMA (autoregressive moving average) models to predict the fluctuation range of access frequency and time intervals in future time periods. The predicted results are used to construct the future distribution of access behavior and compare the deviation from the current popularity threshold.

[0092] Deviation is quantified by calculating the distance between the predicted value and the corresponding threshold of the current popularity status, forming frequency fluctuation deviation and interval fluctuation deviation respectively. These two are weighted to obtain a comprehensive deviation index. When this value exceeds the set threshold, it can be determined that the data popularity status has changed significantly.

[0093] This change judgment is not only reflected in the numerical value exceeding the limit, but also by generating data heat state transition instructions and binding them to metadata, forming a logical trigger point for subsequent migration, routing adjustment and other operations, and realizing intelligent adaptation of the storage structure.

[0094] A pre-trained LSTM time series model can be used to receive standardized access representations as input, and a lightweight inference engine deployed on cloud storage nodes can be used to output future trend ranges in real time. Based on the large number of access records collected during the training phase, this model automatically learns fluctuation patterns under different popularity states and generalizes modeling based on different device loads and data types.

[0095] Deviation thresholds can be set using K-fold cross-validation to select optimal parameters for accuracy, or dynamically adjusted based on empirical statistics in the actual operating environment. Weighting factors can be set based on parameters such as the current device read / write pressure and the migration cost between target popularity categories to accommodate migration constraints between different storage tiers.

[0096] Unsupervised clustering can also be used to automatically identify state change boundaries based on aggregated changes in access behavior. This approach is suitable for scenarios with insufficient data labels or historical data. Furthermore, to improve processing efficiency, the normalization and deviation calculation modules can be deployed in parallel on edge nodes, with asynchronous threads performing calculations and decisions in memory.

[0097] Example: In the fintech sector, data such as risk score caches or payment behavior logs has strong time-bound and cyclical characteristics. By predicting access frequency fluctuations, peak and low-activity periods can be identified in advance, enabling dynamic data migration between high-performance SSDs and economical HDDs, ensuring an optimal balance between performance and cost in the funds clearing process.

[0098] In the healthcare sector, patient medical data is frequently accessed during hospitalization and gradually archived for long-term storage after discharge. Trend analysis of subsequent access patterns can identify which cases will enter the cold data area, freeing up high-performance storage space in a timely manner while ensuring the availability and consistency of historical access, thereby optimizing medical information system resource allocation.

[0099] This embodiment, through a time-series-based trend assessment and threshold comparison mechanism, effectively avoids errors in static snapshot-based assessments and enables dynamic understanding of the evolution of access behavior. By predicting future access states and establishing a baseline for comparison with initial popularity, it not only improves the accuracy of popularity identification but also enables precise control of data migration timing, thereby enhancing the resource adaptation efficiency and responsiveness of the entire tiered storage system.

[0100] S60: When the data heat state of the stored data changes and meets the preset data movement condition, allocate a second target storage device for the stored data, and move the stored data from the first target storage device to the second target storage device.

[0101] In this embodiment, when data access behavior changes significantly over a certain period, and this change has been identified as a heat state transition by the pre-process analysis model, a cross-storage media data migration process should be initiated to prevent mismatched storage resources from continuing to be occupied. The goal of this section is to achieve real-time dynamic adjustment based on data heat, accurately matching resources with data usage needs to reduce storage costs or improve performance.

[0102] The system first retrieves the generated data heat state transition instructions and reads the metadata associated with the target data, particularly the current storage location mapping. By analyzing this metadata structure, it can determine which physical storage device the data is currently located on, further verifying whether to trigger the migration.

[0103] Preset data movement conditions are used to avoid frequent migration operations that waste system resources. They can include two aspects: one is the current load status of the storage device, such as whether the I / O pressure, CPU utilization, and the number of concurrent tasks exceed the threshold; the other is whether the data residence period reaches the maximum period set by the system. In particular, if the long-term residence characteristics of cold data in the hot and cold data hierarchical structure do not meet the requirements, the storage level should be adjusted.

[0104] After determining that the above conditions are met, the system matches the new hotness status with an appropriate storage device. This is done by reviewing the performance parameters, load status, and cost configuration of each storage device in the resource pool and selecting a subset of storage devices that corresponds to the target hotness category. Furthermore, the system calculates a comprehensive priority based on metrics such as read and write latency, throughput, and cost, and selects the optimal device as the second target storage device.

[0105] The data migration process requires the establishment of a highly reliable and secure data transmission channel to ensure that the data content is not tampered with or lost during the migration process. After the transfer is completed, the data cleanup process on the source device is triggered, and the global metadata index library is updated. The new location mapping information is written and synchronized to the metadata service to ensure the correct routing of subsequent access requests.

[0106] To prevent the historical popularity status from misleading subsequent access strategies, the current popularity status should be updated to the metadata management module simultaneously with the mapping information update, as a reference for routing adjustments and subsequent popularity judgments.

[0107] A metadata index can be maintained based on a bidirectional linked list structure, so that the popularity status, historical access records, and current storage path of each data object can be accessed in O(1) time. The migration trigger module is deployed in the data scheduling layer, and the output of the subsequent access analysis module is used as input to schedule migration tasks according to priority in a queue.

[0108] If the target storage device is from a different manufacturer or protocol platform, a protocol adaptation layer can be introduced to standardize the data transfer format, such as transcoding between S3 and POSIX, to ensure cross-platform consistency. Data migration can use snapshot compression and transfer to improve data throughput and stability during transfer.

[0109] In a high-concurrency environment, multiple data migration tasks can be executed in parallel based on file or block granularity. The number of concurrency is controlled through an asynchronous thread pool, and a failure retransmission mechanism is set to ensure reliability.

[0110] You can also set migration window priority policies, such as prioritizing data that is about to reach its retention threshold or that is cooling down, to optimize resource release for high-performance devices. Furthermore, the access prediction module can be used to delay temporarily cooling data that may cool down briefly, avoiding repeated migrations.

[0111] Example: In the healthcare business, a large number of high-resolution medical images generated during patient examinations are initially in a high-frequency usage stage. Doctors and AI-assisted diagnosis systems need to frequently access these image files to support rapid diagnosis and multidisciplinary consultations. Based on access frequency and time interval analysis, the system identifies them as hot data and allocates them to high-performance storage devices. However, after the diagnosis cycle, the access frequency of these images drops significantly. After the system detects changes in access patterns and determines that the migration conditions are met based on the current storage resource load status, it migrates the image files from the high-speed RAID disk to the object storage system and updates the access path mapping table so that subsequent access instructions are automatically routed to the new device to ensure that access consistency is not interrupted.

[0112] In the fintech business sector, intelligent risk control systems frequently access user transaction logs during risk analysis cycles and generate real-time modeling. This data is initially identified as hot data and stored on low-latency SSD devices to support high-concurrency reads. As the risk assessment is completed, data access frequency rapidly decreases. The system detects a change in access popularity and a high load on the SSD devices, thus meeting the migration criteria. The platform then automatically compresses and migrates this batch of log data to hybrid cloud cold storage, cleans up historical copies on the source devices, and frees up high-performance resources for the next round of risk model loading, optimizing resource circulation and processing efficiency.

[0113] This embodiment effectively prevents low-frequency data from occupying high-performance resources by migrating data with significantly changing access popularity from the current storage environment and re-assigning it to a more appropriate target device, thereby optimizing overall system performance and unit storage cost. Through precise judgment and controlled triggering, a dynamic balance between performance and cost is achieved, while simultaneously reducing data access latency, improving system response efficiency, and enhancing user experience.

[0114] The present invention relates to the field of data processing technology and can be applied to business scenarios such as financial technology and medical health. A data hierarchical storage method, apparatus, equipment and medium based on access representation are disclosed, including: obtaining an initial data access representation of the original data, identifying the initial data heat state of the original data based on the access representation, matching the corresponding first target storage device according to the initial data heat state and storing the original data; continuously monitoring the subsequent data access representation of the data stored in the first target storage device, and judging whether the heat state has changed; when the heat state has changed and the preset data movement conditions are met, reallocating the second target storage device and migrating the stored data from the first target storage device to the second target storage device. The present invention realizes hierarchical migration of data of different heat levels between storage devices of different performances by obtaining data access representations and dynamically identifying heat states, thereby improving the resource utilization efficiency and access performance of the storage system and reducing storage costs.

[0115] In one embodiment, the above step S10 includes:

[0116] S101, collecting data access logs of original data within a preset time period, wherein the data access logs include access operation types of the original data;

[0117] S102, determining the total number of accesses to the original data within a unit time window based on the data access log, and generating an access frequency feature;

[0118] S103, determining an average time difference between two adjacent access operations to the original data based on the data access log, and generating a time interval feature;

[0119] S104 , integrating the access frequency feature, the time interval feature, and the distribution ratio of the access operation type to generate an initial data access representation of the original data.

[0120] In this embodiment, the process of collecting data access logs of raw data within a preset time period is intended to build a behavior-based data feature input foundation. Access logs are structured event data generated in real time by a storage system or access control system, which records every read, write, modify, move, copy, and other operations on a data object. The preset time period can be flexibly set according to business needs, such as 1 hour, 6 hours, or 1 day. Its selection will affect the representativeness of the log and the stability of the access pattern. The collection of access operation types should ensure that a variety of common access behaviors are covered. For example, read operations are marked as READ, write operations are marked as WRITE, and modification operations are marked as UPDATE. The system can classify these operations in the form of identifiers as input variables for subsequent statistics and analysis.

[0121] After obtaining access logs, access counts must be performed on each raw data object within a time window to calculate the total number of accesses within the unit time window and generate an access frequency feature. The unit time window is an important parameter for classifying time series behavior, and its granularity must be set to balance accuracy and computational resource overhead. For example, the time period can be subdivided into 5-minute, 15-minute, or 30-minute windows. The number of access events within each window is counted to form a time-series access frequency sequence, and the average or maximum value is calculated as the frequency feature. A higher access frequency generally indicates that the data is active.

[0122] Next, we further calculate the time interval characteristic based on the same data access log, namely the average time difference between two consecutive access operations. By extracting the timestamp field, calculating the time series of access intervals, and then calculating the average, we can obtain a rhythmic indicator of data access. Data that is frequent but has regular intervals tends to have a higher popularity status. Data that has multiple accesses but extremely uneven interval distribution may be caused by batch or asynchronous calls, and requires further identification based on the operation type.

[0123] Building on the aforementioned frequency and time interval characteristics, the distribution ratio of access operation types is further introduced to reveal usage patterns of raw data in scenarios such as high reads and low writes, high writes and low reads, high-frequency updates, or read-only archiving. Combining these three features forms a vectorized initial access representation, which serves as model input for the heat identification stage. Integration methods can include feature concatenation, weighted combination, principal component compression, or discrete encoding, enabling access behavior to be standardized and represented in a unified structure, facilitating downstream analysis and storage strategy development.

[0124] This embodiment, through a combined modeling approach based on access frequency, access interval, and operation type distribution, comprehensively characterizes the dynamic access behavior of raw data during business operations. This allows subsequent determination of data heat status to rely not only on single access statistics but also incorporates multi-dimensional behavioral attributes, helping to more accurately identify the boundary between active and cold data. It can automatically adapt to the differences in access characteristics under different business models, thereby laying a more stable and generalizable data foundation for subsequent data heat classification and storage media allocation, achieving efficient matching of access behavior and storage resources.

[0125] In one embodiment, the above step S20 includes:

[0126] S201, performing data preprocessing on the initial data access representation to generate a preprocessed initial data access representation;

[0127] S202: Input the pre-processed initial data access representation into a classification model generated by training based on historical data access representation and corresponding data heat state labels;

[0128] S203, performing pattern matching on the access characteristics of the original data using the classification model to generate a classification probability distribution of the original data;

[0129] S204, classifying the original data into one of hot data, warm data, or cold data according to the classification probability distribution, and generating the initial data heat state;

[0130] S205: Associating the initial data heat state with the metadata description file of the original data in the form of a unique identifier.

[0131] In this embodiment, the main purpose of data preprocessing for the initial data access representation is to ensure that the feature data of the subsequent input classification model has structural integrity, consistency and numerical comparability. Missing value filling is used to deal with vacancies in fields such as access frequency or time interval due to acquisition delays or system jitter. Methods such as sliding average, global mean, and nearest neighbor interpolation can be used; abnormal access operation type filtering is to exclude access records that may be introduced by system testing, erroneous calls or malicious operations. Filtering rules may include operation type whitelists, frequency upper limit restrictions, or operation timing logic verification; data normalization is used to unify the numerical scales of various access features so that the model can avoid deviations due to differences in the dimensionality of feature values. Maximum and minimum normalization, Z-score standardization, or logarithmic mapping can be used. After data preprocessing, the access behavior of the original data will be converted into a standardized feature set with consistent structure and noise control.

[0132] The purpose of inputting the preprocessed initial data access representation into the classification model is to leverage existing experience to identify access patterns. This classification model is trained using historical data access representations and their corresponding data popularity status labels. The model structure can be a support vector machine, decision tree, random forest, logistic regression, neural network, or an integrated form. The training goal is to establish a mapping relationship between access features and popularity categories. The model training process includes feature selection, cross-validation, and hyperparameter tuning to ensure generalization across different business contexts.

[0133] After receiving new access features, the classification model performs pattern matching and outputs a classification probability distribution corresponding to each popularity category. This classification probability distribution reflects the likelihood of the raw data under different popularity states. For example, a probability of 0.8 for hot data, 0.15 for warm data, and 0.05 for cold data indicates that the data is highly active. After the classification probability distribution is generated, the system classifies the raw data into specific popularity categories based on the maximum probability principle or a multi-threshold strategy. Common categories include hot data, warm data, and cold data, and these can be further expanded or subdivided based on business scenarios.

[0134] Ultimately, the identified initial data popularity status is structured and expressed as a unique identifier, establishing a one-to-one correspondence with the metadata description file of the original data. Metadata description files typically store all data management information in a key-value structure, including data name, generation time, data source path, and permissions. By embedding popularity identifiers in metadata files, subsequent storage strategy selection, data migration management, and popularity change monitoring can be automated.

[0135] This embodiment improves the stability and expression accuracy of access features through a data preprocessing mechanism, and achieves efficient identification of raw data access characteristics in conjunction with a classification model trained based on historical access behavior. It can dynamically determine data popularity based on multi-dimensional behavior patterns and achieve refined data classification results. Furthermore, by structurally binding the popularity status to metadata information, it establishes a bridge between data access behavior and storage policy execution, significantly improving the response efficiency and accuracy of subsequent popularity perception migration.

[0136] In one embodiment, the above step S30 includes:

[0137] S301, querying preset storage device resource pool information according to the initial data heat state to match corresponding storage device performance requirements, and screening out a set of candidate storage devices that meet the storage device performance requirements from the storage device resource pool information;

[0138] S302, determining a comprehensive priority score based on the read / write performance indicators and unit storage cost of each storage device in the candidate storage device set;

[0139] S303, selecting the storage device with the highest comprehensive priority score as the first target storage device;

[0140] S304: Establish a secure data transmission channel with the first target storage device, and transmit the original data to the storage space of the first target storage device through the secure data transmission channel;

[0141] S305 , when the original data is stored in the first target storage device, updating the global metadata index library to record the storage mapping relationship between the original data and the first target storage device.

[0142] In this embodiment, the preset storage device resource pool information is queried based on the initial data heat status, and the core is to realize the matching process of data heat and storage device capabilities. The initial data heat status is the result obtained by access characteristic identification in the previous stage, which can be marked as hot data, warm data or cold data, and has enumerable characteristics. The storage device resource pool information is a structured data set, which contains meta-attributes such as performance indicators, storage types, current load status, storage paths, available capacity, energy consumption levels, etc. of multiple storage devices. The query process establishes a mapping relationship between the heat status and the device performance configuration table defined in the resource pool, such as hot data corresponds to high IOPS and low latency devices (such as NVMe SSD), and cold data corresponds to large-capacity and low-cost devices (such as near-line SATA HDD). The screening operation forms a set of candidate storage devices that meet the performance requirements through a multi-level link of heat status → performance requirements → device capabilities → screening set.

[0143] Based on the read and write performance indicators and unit storage cost of each device in the candidate storage device set, a comprehensive priority score is calculated to further optimize the resource allocation strategy while meeting the requirements of heat adaptation. Read and write performance indicators mainly include parameters such as random read and write IOPS, sequential throughput rate, write latency, and concurrent processing capacity. Unit storage cost includes equipment procurement cost, operating energy consumption, maintenance frequency, etc. After normalizing the performance indicators and cost factors, a weighted scoring function is constructed to achieve a comprehensive priority score. The scoring function can set weight parameters based on application preferences to achieve performance-oriented or cost-oriented strategy transformation. For example, it can prioritize improving performance response in high-frequency trading business or compressing budget consumption in archiving and backup scenarios.

[0144] The device with the highest overall priority score is selected as the first target storage device for the initial flush of raw data. This device not only meets the quality of service requirements for the current data popularity but also offers the best price-performance ratio and scheduling weight within the resource pool. This decision-making process can utilize static rule mapping or incorporate a dynamic scoring model to adjust priority weights in real time, ensuring timely and flexible resource scheduling.

[0145] Establishing a secure data transmission channel with the primary target storage device is fundamental to ensuring the reliability of the data writing process. This secure data transmission channel is typically built using mechanisms such as TLS encryption, VPN tunnels, SDN isolation, and segment verification to prevent unauthorized access, data leakage, and tampering attacks during data transmission from the processing node to the target device. This channel can also incorporate load-aware mechanisms to dynamically allocate bandwidth resources to avoid transmission congestion.

[0146] After the original data is successfully written to the first target storage device, the system automatically updates the global metadata index, marking the mapping relationship between the data and the device. The global metadata index is a unified management structure across devices and popularity levels, recording each piece of data's unique identifier, storage location, popularity category, storage timestamp, and access permissions. This update ensures that the system can quickly locate the target data during subsequent popularity changes, data migration, and access scheduling, ensuring the consistency and integrity of data lifecycle management.

[0147] This embodiment achieves refined allocation of original data in the initial writing stage by constructing a heat-driven storage device screening mechanism, combining device performance indicators with a cost trade-off model. Dynamically adapt device capabilities based on access characteristics, and establish a unified metadata index structure to form a tightly coupled relationship between data resources and hardware resources. This method can increase the access speed of high-frequency data and reduce the storage cost of cold data. At the same time, it introduces an encrypted transmission mechanism in the data writing process to improve system security, providing a solid foundation for subsequent data migration, scheduling, and heat management. By comprehensively considering performance, cost, and security factors, the overall storage system's utilization efficiency and sustainability can be significantly improved.

[0148] In one embodiment, the above step S40 includes:

[0149] S401, setting a dynamic monitoring time window for the data stored in the first target storage device, which is dynamically adjusted according to the initial data heat status;

[0150] S402, capturing each read operation event and write operation event of the stored data within the dynamic monitoring time window, and generating a real-time access operation record;

[0151] S403, counting the number of read operation events and the number of write operation events of the stored data within the dynamic monitoring time window to generate a dynamic access frequency index of the current monitoring window;

[0152] S404, determining an average of time differences between two adjacent access operations to the stored data within the dynamic monitoring time window, and generating a dynamic time interval indicator for the current monitoring window;

[0153] S405, analyzing the difference between the dynamic access frequency index and the historical access frequency characteristic to generate an access trend change factor;

[0154] S406 , integrating the dynamic access frequency indicator, the dynamic time interval indicator, and the access trend change factor to generate a subsequent data access representation.

[0155] In this embodiment, setting a dynamic monitoring time window for the data stored in the first target storage device is a key pre-operation in data heat lifecycle management. The dynamic nature of the time window is reflected in the setting of different monitoring frequencies and durations according to the initial data heat state. Hot data corresponds to a shorter window period in order to respond to heat changes in a timely manner, while cold data is configured with a longer period to reduce resource consumption. The time window can be set as a fixed-length window or a sliding window, which is generated through a rule table based on heat classification or a dynamic policy mapping mechanism, and is bound to the metadata control policy of the corresponding data object after the data is stored on the disk, so as to realize personalized access monitoring scheduling.

[0156] During the dynamic monitoring time window, the system captures all read and write operations on stored data. This capture process is based on real-time monitoring of the I / O request queue, typically implemented through file system hooks at the operating system level, the log playback interface of the distributed storage engine, or the access logging module at the block storage level. It supports low-latency event collection under high-frequency data streams. Each access event record includes elements such as the access type, timestamp, request size, request source, and session ID, forming a structured, real-time access operation record. This record serves as the core input source for subsequent access analysis, directly reflecting the actual behavioral status of data usage.

[0157] Based on access records, the system counts read and write operations within the current time window to calculate a dynamic access frequency metric. This metric uses the number of visits per unit time as its unit of measurement. It supports dimensional statistics by access type and can also perform read-write weighting to reflect the interaction density and usage activity of the data within the current cycle. Unlike initial access frequency, dynamic metrics are phased and real-time, supporting trend analysis and identifying shifts in popularity.

[0158] In addition to access frequency, the dynamics of the time interval between operations must also be evaluated. Specifically, this involves calculating the average time difference between two consecutive access operations to generate a dynamic time interval indicator for the current monitoring window. This indicator reflects the consistency or suddenness of user access behavior and is crucial for analyzing data from hot states to cold states. A significantly shorter time interval may indicate data reactivation; a significantly longer time interval may indicate the data has entered a cooling phase.

[0159] Based on a comparison of current and historical access frequency metrics, the system generates an access trend change factor. This factor, calculated by calculating the difference between the current and initial frequencies, the slope of change, or the weighted fluctuation coefficient, quantifies the direction and intensity of the trend in data access behavior. For example, a sustained decline indicates a cooling trend, while a fluctuating increase indicates a short-term hotspot. This factor provides intermediate features for subsequent determination of popularity shifts.

[0160] Finally, the dynamic access frequency and time interval metrics are integrated with the access trend change factor to form a subsequent data access representation. This representation structure maintains the same dimensions as the initial access representation, facilitating subsequent model calls for comparative analysis and supporting fine-grained time series trend modeling or classifier input. This representation not only serves to identify hot spots but also provides data support for potential future data migration, cache adjustments, and pre-reading strategies.

[0161] This embodiment constructs an access behavior collection framework through a dynamic time window mechanism, which realizes real-time tracking of data heat evolution over time, avoiding the problem that the static monitoring mechanism cannot identify heat changes in a timely manner. The combined indicator of access frequency and operation interval improves the sensitivity and accuracy of behavior recognition, and at the same time introduces access trend change factors to enhance the system's perception of behavior trends. In the entire processing flow, the monitoring operation covers the complete path of behavior collection, feature generation, trend evaluation and structured output, providing a real-time, explainable and behavioral contextual input basis for subsequent data heat judgment, migration decision-making and resource allocation. It improves the responsiveness to the dynamics of storage resource allocation and improves the system's recognition efficiency and processing accuracy of data life cycle evolution.

[0162] In one embodiment, the above step S50 includes:

[0163] S501, performing standardization processing on the subsequent data access representation to generate a standardized subsequent data access representation;

[0164] S502: Input the standardized subsequent data access representation into a time series analysis model generated by training based on the historical data access representation time series and the corresponding data heat state transition labels;

[0165] S503, generating a predicted access frequency fluctuation range and a predicted time interval fluctuation range of the stored data within a future preset time window through the time series analysis model;

[0166] S504, comparing the predicted access frequency fluctuation range with the reference access frequency threshold corresponding to the initial data heat state to generate a frequency fluctuation deviation;

[0167] S505, comparing the predicted time interval fluctuation range with a reference time interval threshold corresponding to the initial data heat state to generate an interval fluctuation deviation;

[0168] S506: Determine whether the data popularity state of the stored data has changed based on the weighted sum of the frequency fluctuation deviation and the interval fluctuation deviation;

[0169] S507, when the weighted sum exceeds a preset deviation threshold, determining that the data heat state has changed, generating a data heat state change instruction, and associating the data heat state change instruction with the metadata description file of the stored data.

[0170] In this embodiment, in the process of determining whether the data heat state of the stored data has changed compared to the initial data heat state, the subsequent data access representation needs to be standardized first. This standardization process is intended to eliminate the numerical dimension differences of different feature dimensions, so that the subsequent analysis model can more accurately identify the trend of feature changes. Standardization methods may include z-score standardization, minimum-maximum normalization or quantile scaling. Through a unified standard scale, indicators such as access frequency and time interval can be compared and fused within a unified interval, reducing the impact of feature skewness on the analysis results.

[0171] Inputting standardized subsequent data access representations into the time series analysis model is based on learning and predicting the changing trends of continuous access behavior. The historical data access representation time series used in the model training process is composed of a combination of access frequency indicators and time interval indicators in multiple continuous dynamic monitoring time windows, ensuring that the analysis model has the ability to identify periodic, sudden or trend patterns. The corresponding heat state transition label indicates whether the data changes from hot data to cold data or from cold data to hot data between adjacent time windows. It serves as the output label in the supervised learning process and is used to guide the model to establish a mapping relationship between data behavior changes and heat evolution.

[0172] Time series analysis models can employ deep learning architectures such as long short-term memory (LSTM), gated recurrent units (GRU), and variational autoencoders (VAE), as well as traditional forecasting models such as ARIMA and exponential smoothing. The model outputs the predicted access frequency and time interval fluctuations within a preset future time window. This provides a range of changes in data usage behavior over a forward-looking period, serving as an important basis for determining whether popularity is shifting.

[0173] Next, the predicted access frequency fluctuation range is compared with the baseline access frequency threshold corresponding to the initial data popularity state to calculate the frequency fluctuation deviation. This deviation is typically measured by the maximum deviation rate, mean deviation rate, or the degree of overlap between fluctuation intervals, reflecting the extent to which current usage behavior deviates from the original popularity expectations. Similarly, the predicted time interval fluctuation range is compared with the time interval benchmark under the initial popularity state to generate the interval fluctuation deviation, which assesses whether the temporal continuity of data usage and access intervals have changed significantly.

[0174] These two deviation metrics are weighted and combined to form a weighted sum used to determine whether the data's popularity has changed. The weighting coefficients can be adjusted based on preset settings for how frequency and interval influence the business scenario, adapting to the characteristics of different data types and their sensitivity to timeliness and access density. By setting a preset deviation threshold as the decision boundary, when the weighted sum exceeds this threshold, the data's popularity status is determined to have changed.

[0175] Ultimately, the system generates a data heat state transition instruction to drive subsequent data migration, policy adjustments, and other processes. This instruction is written into the metadata description file of the original data as structured metadata, ensuring that the system can complete actions such as storage reallocation based on this instruction when subsequently called, supporting intelligent management of the storage lifecycle.

[0176] This embodiment uses a prediction mechanism based on time series modeling, so that the judgment of the popularity status not only depends on the current access behavior, but also combines historical trends and future evolution directions to achieve forward-looking perception of popularity changes. By standardizing the unified feature dimensions, the model generalization ability and judgment accuracy are improved. The weighted fusion of frequency deviation and time interval deviation ensures a comprehensive evaluation of multi-dimensional access behavior and reduces the possibility of misjudgment caused by a single indicator. The final generated popularity transition instruction provides a decision-making basis for subsequent data storage strategies, effectively improving the system's responsiveness to changes in the data life cycle and the dynamic adaptability of resource scheduling.

[0177] In one embodiment, the above step S60 includes:

[0178] S601, when the data heat state of the stored data changes, obtaining a data heat state change instruction and an associated metadata description file of the stored data;

[0179] S602: Verify whether a preset data movement condition is met based on the storage mapping relationship of the first target storage device recorded in the metadata description file;

[0180] S603, when the preset data movement condition is met, according to the target data heat state indicated in the data heat state transition instruction, selecting a second target storage device that matches the target data heat state from the storage device resource pool information;

[0181] S604: Send a data migration instruction to the first target storage device to transfer the data stored in the first target storage device to the storage space of the second target storage device through a secure data transmission channel;

[0182] S605: After the migration of the stored data is completed, the stored data in the first target storage device is deleted, and the storage mapping relationship in the global metadata index library is updated to record the mapping information between the stored data and the second target storage device;

[0183] S606: Adjust a routing strategy for subsequent data access requests according to the read and write performance indicators of the second target storage device.

[0184] In this embodiment, when a data's popularity status changes, the system first retrieves the generated data popularity status transition instruction and the metadata description file associated with the data. The metadata description file typically contains information such as the data's unique identifier in the system, its current storage location, historical access characteristics, and policy identification. It comprehensively reflects the data's status and its mapping relationship to the infrastructure, ensuring accurate positioning and consistent information during subsequent migration.

[0185] The system verifies the preset data migration conditions based on the mapping relationship between the first target storage device and the stored data recorded in the metadata description file. This set of conditions includes, but is not limited to, whether the current load rate of the first target storage device exceeds a preset load threshold, and whether the actual residence time of the stored data exceeds a preset cycle threshold. The current load rate can be calculated from I / O operation density, CPU utilization, or bandwidth utilization, and the residence cycle can be calculated from the data write time and the current system time. If any of these conditions are met, the data is determined to meet the migration requirements.

[0186] Once the data migration conditions are met, the system selects devices whose performance indicators match the target data's heat state, as the second target storage device, based on the target data's heat state specified in the migration instruction and the storage device resource pool information. Resource pool information typically includes each device's response latency, throughput, reliability level, and cost-effectiveness. The matching strategy may employ weighted comprehensive scoring, priority sorting, or heat range mapping.

[0187] After selecting a target device, the system issues a data migration command to the first target storage device, transferring the data via a secure data transmission channel. This channel can be established using SSL / TLS protocols, IPsec tunnels, or dedicated storage network channels (such as iSCSI and Fibre Channel) to protect data from leakage and tampering during the migration process. During the migration process, data can be segmented and synchronized at a block, file, or object level to improve concurrent migration efficiency and fault tolerance.

[0188] After the data is completely migrated to the second target storage device, the system confirms the consistency of the replica and deletes the original data, freeing up storage space on the first target storage device. The system then updates the global metadata index, replacing the mapping between the original data and the new device. It also records the change time, migration status, and target device ID, ensuring traceability and accuracy for subsequent system scheduling and queries.

[0189] To ensure efficient access after data migration, the system must also adjust the routing strategy for subsequent access requests based on the read and write performance indicators of the second target storage device. This adjustment involves optimizing the access path based on the new storage location within the load balancer or file system's I / O scheduling logic, ensuring that requests reach the new target device directly, avoiding redundant jumps or pointer failures. This ensures stable system performance and consistent response.

[0190] Example: In the fintech sector, a large bank's data management platform needs to implement hierarchical management and intelligent scheduling of historical transaction data, customer behavior logs, and risk control model input data to reduce high-frequency storage resource usage and improve the efficiency of critical data retrieval. The platform deploys a distributed intelligent storage system with hot and cold tiered management capabilities. The system first obtains an initial data access representation of the raw data (such as user transaction records and credit application logs). To do this, the platform collects access logs from the past month, extracts the access operation type (such as query, update), access frequency per unit time, and the average interval between adjacent accesses for each data entry, and integrates these to generate access frequency features, time interval features, and operation type distribution ratios as the initial data access representation. The system then performs a first data analysis on the initial data access representation. Using a classification model trained on access samples labeled with data popularity categories over the past two years, the system predicts the access characteristics and popularity trends of the current raw data. For example, the model identifies customer repayment records and active transaction user behavior logs frequently accessed within the past week as "hot data," while scanned copies of existing loan contracts that have not been accessed for a long time are "cold data." Based on the initial heat state of this data, the system queries the defined storage device performance-heat mapping (e.g., NVMe SSD for hot data, hybrid HDD for warm data, and SATA HDD for cold data) and matches storage devices in the device resource pool that meet the performance requirements. Taking into account the device's read and write performance and storage cost, the system prioritizes low-latency, high-IOPS devices as the first target storage device for hot data. The system writes the original data to the target device via an SSL channel and records the mapping in the global metadata index. As the system runs, the monitoring module dynamically sets a monitoring window (e.g., 7 days), records read and write events for each data block on the target device, and calculates access frequency and operation intervals in real time, comparing them with historical characteristics to generate access trend change factors and subsequent access representations. In the second data analysis process, the system uses a time series model to analyze the changing trends of data access patterns, predict the access fluctuation range in future windows, and calculate the deviation based on the threshold range of the original heat state. If the deviation significantly exceeds the preset range and detects that the current device load is high or the data has exceeded the set storage period, a heat state transition judgment and migration condition judgment are triggered. The system then selects a new target storage device based on the changed popularity status. For example, a repayment log file, deemed to have decreased popularity, will be migrated from a high-performance SSD to a capacity-based HDD. During the migration process, the system performs a full data synchronization via a secure channel, then deletes the original copy, updates the global index, and adjusts data query and task scheduling strategies in real time based on the performance of the target device, achieving intelligent heat management for financial business data.

[0191] In the healthcare sector, a regional medical center's data platform needs to process massive amounts of patient medical records, medical images, laboratory reports, and follow-up records. Faced with data growth and limited storage resources, the center deployed an AI-driven intelligent data platform capable of identifying and migrating hot and cold data. The system first collects raw data, such as newly uploaded CT images, historical hospitalization records, and patient medication logs. It then extracts access logs within a set timeframe, calculates the number of accesses per unit time, the mean interval between adjacent accesses, and the distribution of operation types, and generates a complete initial data access representation. The platform then uses an AI-trained heat classification model to perform pattern recognition on these data access characteristics, identifying CT data in continuous image reading requests as "hot data" and electronic medical records that have not been accessed for a long time as "cold data." The platform then generates corresponding initial heat states. Based on these heat states, the system selects appropriate devices from a multi-tiered storage pool: for example, high-speed image servers for hot CT images and standard NAS storage devices for cold medical records. After comprehensively weighting the data, the system selects the primary target storage device and securely writes and maps the data. During operation, the system continuously monitors data access behavior, sets monitoring cycles based on popularity categories (for example, emergency-related imaging is monitored daily, and follow-up reports are monitored weekly), and generates real-time access frequency indicators and trend change factors. During the analysis process, the platform uses historical medical behavior models and access timing models to predict access trends for the next cycle and determine whether a popularity shift has occurred. If access to a batch of CT data has decreased recently and the system detects that the current image server load is too high, the system will determine that the migration conditions are met. At this time, the platform will migrate these CT data to the mid-range HDD storage array, and update the mapping information to the index library, while simultaneously adjusting the access routing configuration of the doctor's workstation to ensure that it can still be accessed in a timely manner but does not occupy high-speed resources. This series of operations not only realizes the intelligent management of the patient data life cycle, but also greatly alleviates the storage resource pressure faced by medical institutions while ensuring the timeliness of medical services.

[0192] This embodiment realizes the dynamic storage redistribution of data at different life cycle stages through a data migration mechanism driven by heat state transitions, so that hot data is matched with high-performance devices and cold data is migrated to low-cost devices, thereby significantly improving resource utilization efficiency. The introduction of load rate and residence period as judgment criteria avoids unnecessary data migration triggered by a single heat state transition, thereby enhancing system stability and economy. With the help of metadata indexing and secure channel management, precise positioning and high-security transmission of the data migration process are achieved, avoiding data consistency or information leakage problems caused by migration operations. At the same time, by updating the access routing strategy, the immediate adjustment of the system read and write paths after migration is ensured, ensuring that access requests are seamlessly switched to the new target device at the logical level, and improving the adaptability and scalability of the overall system.

[0193] In one embodiment, a data tiered storage device based on access representation is provided, and the data tiered storage device based on access representation corresponds to the data tiered storage method based on access representation in the above embodiment. Figure 3 , Figure 3 This is a functional module diagram of a preferred embodiment of a data tiered storage device based on access representations according to the present invention. It includes an access representation acquisition module 10, an access characteristics identification module 20, an initial write decision module 30, an access behavior monitoring module 40, a heat state determination module 50, and a storage migration execution module 60. Each functional module is described in detail below:

[0194] An access representation acquisition module 10 is used to obtain an initial data access representation of the original data;

[0195] An access characteristic identification module 20 is configured to identify data access characteristics of the original data through a first data analysis process based on the initial data access representation, and generate an initial data heat state of the original data;

[0196] an initial write decision module 30, configured to allocate a first target storage device to the original data according to the initial data heat state and a preset storage device performance correspondence, and store the original data in the first target storage device;

[0197] An access behavior monitoring module 40 is configured to monitor subsequent data access characteristics of the stored data in the first target storage device;

[0198] a heat state determination module 50 for determining, based on the subsequent data access representation, whether the data heat state of the stored data has changed compared to the initial data heat state through a second data analysis process;

[0199] The storage migration execution module 60 is used to allocate a second target storage device to the stored data and move the stored data from the first target storage device to the second target storage device when the data heat state of the stored data changes and meets the preset data movement conditions.

[0200] In one embodiment, the access characterization acquisition module 10 is specifically configured to:

[0201] Collecting data access logs of the original data within a preset time period, wherein the data access logs include access operation types of the original data;

[0202] Based on the data access log, determining the total number of accesses to the original data within a unit time window, and generating an access frequency feature;

[0203] Based on the data access log, determining an average time difference between two adjacent access operations to the original data, and generating a time interval feature;

[0204] The access frequency feature, the time interval feature, and the distribution ratio of the access operation type are integrated to generate an initial data access representation of the original data.

[0205] In one embodiment, the access characteristic identification module 20 is specifically configured to:

[0206] Performing data preprocessing on the initial data access representation to generate a preprocessed initial data access representation;

[0207] Input the preprocessed initial data access representation into a classification model generated by training based on historical data access representation and corresponding data heat state labels;

[0208] Performing pattern matching on the access characteristics of the original data using the classification model to generate a classification probability distribution of the original data;

[0209] Classifying the original data into one of hot data, warm data, or cold data according to the classification probability distribution to generate the initial data heat state;

[0210] The initial data heat state is associated with the metadata description file of the original data in the form of a unique identifier.

[0211] In one embodiment, the initial write decision module 30 is specifically configured to:

[0212] Querying preset storage device resource pool information according to the initial data heat state to match corresponding storage device performance requirements, and screening out a set of candidate storage devices that meet the storage device performance requirements from the storage device resource pool information;

[0213] Determining a comprehensive priority score based on the read and write performance indicators and unit storage cost of each storage device in the set of candidate storage devices;

[0214] Selecting the storage device with the highest comprehensive priority score as the first target storage device;

[0215] Establishing a secure data transmission channel with the first target storage device, and transmitting the original data to the storage space of the first target storage device through the secure data transmission channel;

[0216] When the original data is stored in the first target storage device, the global metadata index library is updated to record the storage mapping relationship between the original data and the first target storage device.

[0217] In one embodiment, the access behavior monitoring module 40 is specifically configured to:

[0218] Setting a dynamic monitoring time window for the data stored in the first target storage device, which is dynamically adjusted according to the initial data heat state;

[0219] Within the dynamic monitoring time window, capturing each read operation event and write operation event of the stored data to generate a real-time access operation record;

[0220] Counting the number of read operation events and the number of write operation events of the stored data within the dynamic monitoring time window to generate a dynamic access frequency index of the current monitoring window;

[0221] Determine the average time difference between two adjacent access operations of the stored data within the dynamic monitoring time window, and generate a dynamic time interval indicator of the current monitoring window;

[0222] Analyzing the difference between the dynamic access frequency index and the historical access frequency characteristics to generate an access trend change factor;

[0223] The dynamic access frequency indicator, the dynamic time interval indicator, and the access trend change factor are integrated to generate a subsequent data access representation.

[0224] In one embodiment, the heat status determination module 50 is specifically configured to:

[0225] performing standardization processing on the subsequent data access representation to generate a standardized subsequent data access representation;

[0226] Input the standardized subsequent data access representation into a time series analysis model generated by training based on the historical data access representation time series and the corresponding data heat state transition labels;

[0227] Generate a predicted access frequency fluctuation range and a predicted time interval fluctuation range of the stored data within a future preset time window through the time series analysis model;

[0228] Comparing the predicted access frequency fluctuation range with the reference access frequency threshold corresponding to the initial data heat state to generate a frequency fluctuation deviation;

[0229] Comparing the predicted time interval fluctuation range with a reference time interval threshold corresponding to the initial data heat state to generate an interval fluctuation deviation;

[0230] determining whether a data heat state of the stored data has changed based on a weighted sum of the frequency fluctuation deviation and the interval fluctuation deviation;

[0231] When the weighted sum exceeds a preset deviation threshold, it is determined that the data heat state has changed, and a data heat state change instruction is generated, and the data heat state change instruction is associated with the metadata description file of the stored data.

[0232] In one embodiment, the storage migration execution module 60 is specifically configured to:

[0233] When the data heat state of the stored data changes, obtaining a data heat state change instruction and an associated metadata description file of the stored data;

[0234] Verifying whether a preset data movement condition is met based on the storage mapping relationship of the first target storage device recorded in the metadata description file;

[0235] When the preset data movement condition is met, according to the target data heat state indicated in the data heat state transition instruction, selecting a second target storage device that matches the target data heat state from the storage device resource pool information;

[0236] Sending a data migration instruction to the first target storage device to transfer the data stored in the first target storage device to the storage space of the second target storage device through a secure data transmission channel;

[0237] After the migration of the stored data is completed, deleting the stored data in the first target storage device, and updating the storage mapping relationship in the global metadata index library to record the mapping information between the stored data and the second target storage device;

[0238] Adjusting a routing strategy for subsequent data access requests based on the read and write performance indicators of the second target storage device.

[0239] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide determination and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external user terminal via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the service side of a data hierarchical storage method based on access representation.

[0240] In one embodiment, a computer device is provided. The computer device may be a user terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, memory, network interface, display screen and input device connected via a system bus. The processor of the computer device is used to provide determination and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the user side of a data hierarchical storage method based on access representation.

[0241] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0242] Obtaining an initial data access representation of the raw data;

[0243] Based on the initial data access representation, identifying data access characteristics of the original data through a first data analysis process to generate an initial data heat state of the original data;

[0244] Allocating a first target storage device to the original data according to the initial data heat state and a preset storage device performance correspondence, and storing the original data in the first target storage device;

[0245] monitoring a subsequent data access characteristic of the stored data in the first target storage device;

[0246] determining, based on the subsequent data access characterization, through a second data analysis process, whether a data heat state of the stored data has changed compared to the initial data heat state;

[0247] When the data heat state of the stored data changes and meets the preset data movement condition, a second target storage device is allocated for the stored data, and the stored data is moved from the first target storage device to the second target storage device.

[0248] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0249] Obtaining an initial data access representation of the raw data;

[0250] Based on the initial data access representation, identifying data access characteristics of the original data through a first data analysis process to generate an initial data heat state of the original data;

[0251] Allocating a first target storage device to the original data according to the initial data heat state and a preset storage device performance correspondence, and storing the original data in the first target storage device;

[0252] monitoring a subsequent data access characteristic of the stored data in the first target storage device;

[0253] determining, based on the subsequent data access characterization, through a second data analysis process, whether a data heat state of the stored data has changed compared to the initial data heat state;

[0254] When the data heat state of the stored data changes and meets the preset data movement condition, a second target storage device is allocated for the stored data, and the stored data is moved from the first target storage device to the second target storage device.

[0255] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the user side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0256] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0257] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0258] It should be noted that if any software tools or components other than those of the Company appear in the embodiments of this application, they are merely for illustration and do not represent actual use. The above embodiments are intended only to illustrate the technical solutions of the present invention, not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some of the technical features therein with equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A data hierarchical storage method based on access representation, characterized in that: The following steps are involved: Obtaining an initial data access representation of the raw data; Based on the initial data access representation, identifying data access characteristics of the original data through a first data analysis process to generate an initial data heat state of the original data; Allocating a first target storage device to the original data according to the initial data heat state and a preset storage device performance correspondence, and storing the original data in the first target storage device; monitoring a subsequent data access characteristic of the stored data in the first target storage device; determining, based on the subsequent data access characterization, through a second data analysis process, whether a data heat state of the stored data has changed compared to the initial data heat state; When the data heat state of the stored data changes and meets the preset data movement condition, a second target storage device is allocated for the stored data, and the stored data is moved from the first target storage device to the second target storage device.

2. The data hierarchical storage method based on access representation according to claim 1, characterized in that: Get initial data access representation of the raw data, including: Collecting data access logs of the original data within a preset time period, wherein the data access logs include access operation types of the original data; Based on the data access log, determining the total number of accesses to the original data within a unit time window, and generating an access frequency feature; Based on the data access log, determining an average time difference between two adjacent access operations to the original data, and generating a time interval feature; The access frequency feature, the time interval feature, and the distribution ratio of the access operation type are integrated to generate an initial data access representation of the original data.

3. The data hierarchical storage method based on access representation according to claim 1, characterized in that: Based on the initial data access representation, identifying data access characteristics of the original data through a first data analysis process to generate an initial data heat state of the original data includes: Performing data preprocessing on the initial data access representation to generate a preprocessed initial data access representation; Input the preprocessed initial data access representation into a classification model generated by training based on historical data access representation and corresponding data heat state labels; Performing pattern matching on the access characteristics of the original data using the classification model to generate a classification probability distribution of the original data; Classifying the original data into one of hot data, warm data, or cold data according to the classification probability distribution to generate the initial data heat state; The initial data heat state is associated with the metadata description file of the original data in the form of a unique identifier.

4. The data hierarchical storage method based on access representation according to claim 1, characterized in that: Allocating a first target storage device to the original data according to the initial data heat state and a preset storage device performance correspondence, and storing the original data in the first target storage device, includes: Querying preset storage device resource pool information according to the initial data heat state to match corresponding storage device performance requirements, and screening out a set of candidate storage devices that meet the storage device performance requirements from the storage device resource pool information; Determining a comprehensive priority score based on the read and write performance indicators and unit storage cost of each storage device in the set of candidate storage devices; Selecting the storage device with the highest comprehensive priority score as the first target storage device; Establishing a secure data transmission channel with the first target storage device, and transmitting the original data to the storage space of the first target storage device through the secure data transmission channel; When the original data is stored in the first target storage device, the global metadata index library is updated to record the storage mapping relationship between the original data and the first target storage device.

5. The data hierarchical storage method based on access representation according to claim 1, characterized in that: Monitoring subsequent data access characteristics of the stored data in the first target storage device includes: Setting a dynamic monitoring time window for the data stored in the first target storage device, which is dynamically adjusted according to the initial data heat state; Within the dynamic monitoring time window, capturing each read operation event and write operation event of the stored data to generate a real-time access operation record; Counting the number of read operation events and the number of write operation events of the stored data within the dynamic monitoring time window to generate a dynamic access frequency index of the current monitoring window; Determine the average time difference between two adjacent access operations of the stored data within the dynamic monitoring time window, and generate a dynamic time interval indicator of the current monitoring window; Analyzing the difference between the dynamic access frequency index and the historical access frequency characteristics to generate an access trend change factor; The dynamic access frequency indicator, the dynamic time interval indicator, and the access trend change factor are integrated to generate a subsequent data access representation.

6. The data hierarchical storage method based on access representation according to claim 1, characterized in that: Determining, based on the subsequent data access characterization, through a second data analysis process, whether the data heat state of the stored data has changed compared to the initial data heat state includes: performing standardization processing on the subsequent data access representation to generate a standardized subsequent data access representation; Input the standardized subsequent data access representation into a time series analysis model generated by training based on the historical data access representation time series and the corresponding data heat state transition labels; Generate a predicted access frequency fluctuation range and a predicted time interval fluctuation range of the stored data within a future preset time window through the time series analysis model; Comparing the predicted access frequency fluctuation range with the reference access frequency threshold corresponding to the initial data heat state to generate a frequency fluctuation deviation; Comparing the predicted time interval fluctuation range with a reference time interval threshold corresponding to the initial data heat state to generate an interval fluctuation deviation; determining whether a data heat state of the stored data has changed based on a weighted sum of the frequency fluctuation deviation and the interval fluctuation deviation; When the weighted sum exceeds a preset deviation threshold, it is determined that the data heat state has changed, and a data heat state change instruction is generated, and the data heat state change instruction is associated with the metadata description file of the stored data.

7. The data hierarchical storage method based on access representation according to claim 1, characterized in that: When the data heat state of the stored data changes and a preset data movement condition is met, allocating a second target storage device to the stored data and moving the stored data from the first target storage device to the second target storage device includes: When the data heat state of the stored data changes, obtaining a data heat state change instruction and an associated metadata description file of the stored data; Verifying whether a preset data movement condition is met based on the storage mapping relationship of the first target storage device recorded in the metadata description file; When the preset data movement condition is met, according to the target data heat state indicated in the data heat state transition instruction, selecting a second target storage device that matches the target data heat state from the storage device resource pool information; Sending a data migration instruction to the first target storage device to transfer the data stored in the first target storage device to the storage space of the second target storage device through a secure data transmission channel; After the migration of the stored data is completed, deleting the stored data in the first target storage device, and updating the storage mapping relationship in the global metadata index library to record the mapping information between the stored data and the second target storage device; Adjusting a routing strategy for subsequent data access requests based on the read and write performance indicators of the second target storage device.

8. A data tiered storage device based on access representation, characterized in that: The data tiered storage device based on access representation includes: An access representation acquisition module, used to obtain initial data access representation of raw data; an access characteristic identification module, configured to identify data access characteristics of the original data through a first data analysis process based on the initial data access representation, and generate an initial data heat state of the original data; an initial write decision module, configured to allocate a first target storage device to the original data according to the initial data heat state and a preset storage device performance correspondence, and store the original data in the first target storage device; an access behavior monitoring module, configured to monitor subsequent data access characteristics of the data stored in the first target storage device; a heat state determination module, configured to determine, based on the subsequent data access representation, through a second data analysis process, whether the data heat state of the stored data has changed compared to the initial data heat state; The storage migration execution module is used to allocate a second target storage device to the stored data and move the stored data from the first target storage device to the second target storage device when the data heat state of the stored data changes and the preset data movement conditions are met.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and an access representation-based data tiered storage program stored in the memory and capable of running on the processor. When the access representation-based data tiered storage program is executed by the processor, the steps of the access representation-based data tiered storage method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The storage medium stores a data tiered storage program based on access representation, which, when executed by a processor, implements the steps of the data tiered storage method based on access representation according to any one of claims 1 to 7.

Images