Data storage optimization method and system for solid state disk

Abstract

The invention relates to the technical field of data processing, and discloses a data storage optimization method and system for a solid state disk. The method comprises the steps of performing pattern recognition on small object metadata through multi-dimensional access feature extraction, performing five-level popularity level division based on a flash memory perception popularity evaluation algorithm, performing multi-level storage region division on an SSD, performing intelligent resource allocation according to popularity levels and storage regions, and realizing dynamic adjustment processing through adaptive load monitoring. The technical problem of how to carry out intelligent hot-cold separation and dynamic hierarchical storage on the small object metadata to reduce the SSD storage cost in a massive small object storage environment generated by a cloud native application is solved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data storage optimization method and system for a solid-state hard drive. Background Art

[0002] Existing object storage systems are widely used in cloud environments. Traditionally, these systems employ a hybrid storage architecture, storing object data on hard disk drives (HDDs) and object metadata on solid-state drives (SSDs). This design assumes that object metadata is much smaller than the object data. By placing frequently accessed metadata on high-performance SSDs, they achieve good I / O performance, while storing large-capacity object data on lower-cost HDDs to control overall storage costs. Existing SSD storage optimization methods primarily employ simple LRU caching algorithms and static wear leveling strategies, storing all metadata uniformly on SSDs and managing storage through basic hot and cold data separation.

[0003] However, with the prevalence of cloud-native applications, many applications have begun organizing data into massive amounts of small objects. The data size of these small objects is often close to or even smaller than the size of their corresponding metadata. This significantly increases storage costs due to traditional solutions that store all metadata on expensive SSDs. Existing SSD storage optimization methods have significant shortcomings: First, cost control is not precise enough to effectively handle scenarios where the metadata size of small objects approaches or exceeds the size of the data itself; second, the classification of hot and cold data is inaccurate. Existing methods are often based on simple access frequency statistics, making it difficult to accurately predict future data access patterns; third, there is a lack of specialized optimization for small object storage scenarios, and traditional optimization methods are ineffective when faced with massive amounts of small objects; and finally, dynamic adjustment capabilities are limited, making it impossible to quickly respond and adjust storage policies when workload patterns change.

[0004] Based on these limitations of existing technologies, further analysis revealed that in environments with massive small objects, deeper technical challenges need to be addressed: how to accurately identify the multi-dimensional access characteristics of small object metadata and establish a precise heat assessment model; how to intelligently manage partitions based on the physical characteristics of SSD flash blocks; how to implement fine-grained component-level tiered storage for small object metadata; and how to establish adaptive load monitoring and dynamic migration mechanisms to cope with real-time changes in access patterns. Solving these problems requires systematic technological innovation across multiple dimensions, including metadata access pattern analysis, SSD physical characteristic utilization, intelligent storage resource allocation, and dynamic load balancing. Summary of the Invention

[0005] This application provides a data storage optimization method and system for solid-state drives, which is used to solve the technical problem of how to intelligently separate hot and cold data and dynamically tier storage of small object metadata to reduce SSD storage costs in a massive small object storage environment generated by cloud-native applications.

[0006] In the first aspect, the present application provides a data storage optimization method for a solid-state hard drive, the data storage optimization method for a solid-state hard drive comprising: performing pattern recognition processing on access behavior data of small object metadata through multi-dimensional access feature extraction to obtain access frequency feature vectors, time locality features and application correlation features; performing data heat quantification processing through a flash memory-aware heat evaluation algorithm based on the access frequency feature vectors, time locality features and application correlation features to obtain a five-level data heat level division including super hot, hot, warm, cold and super cold; performing flash memory block characteristic analysis processing on the physical performance parameters of the SSD storage medium to obtain a multi-level storage area division based on performance differences and a flash memory block performance mapping relationship; performing intelligent storage resource allocation processing on small object metadata based on the five-level data heat level division and the multi-level storage area division to obtain an optimal matching scheme for metadata components and a storage cost optimization result based on the flash memory block performance mapping relationship; dynamically adjusting the data distribution status of each storage area in the optimal matching scheme of the metadata component through adaptive load monitoring to obtain balanced storage load distribution data based on the five-level data heat level division.

[0007] Optionally, the multi-dimensional access feature extraction is used to perform pattern recognition processing on the access behavior data of the small object metadata to obtain the access frequency feature vector, the time locality feature and the application correlation feature, including: The access timestamp, access operation type, and access source application identifier of small object metadata are monitored, collected, and processed in real time to obtain the original access behavior dataset; The original access behavior data set is segmented into time series according to the preset time window length to obtain access statistics and access frequency distribution matrix within multiple time windows; Performing quantitative analysis on the access patterns in each time window by frequency density calculation based on the access number distribution matrix to obtain the access frequency feature vector and access interval variation coefficient; The small object metadata from the same application source is subjected to correlation mining processing according to the access frequency feature vector and the access interval variation coefficient to obtain the time locality feature and the application correlation feature.

[0008] Optionally, the data heat quantification processing is performed using a flash memory-aware heat evaluation algorithm based on the access frequency feature vector, the time locality feature, and the application correlation feature to obtain a five-level data heat level classification including super hot, hot, warm, cold, and super cold, including: Performing logarithmic normalization on the access count data in the access frequency feature vector to obtain a frequency score and a corresponding frequency weight factor; Based on the time locality feature, a time decay score is calculated and processed on the time difference between the current time and the last access time by exponential decay calculation to obtain a time score and a time weight factor reflecting the freshness of the access time; Performing stability quantification processing on the small object metadata access patterns of the same application source according to the application correlation characteristics to obtain an application access stability score and an application weight factor; The frequency score, time score and application access stability score are weighted and summed according to the corresponding weight factors to obtain the comprehensive heat value of the small object metadata and the five-level data heat level classification based on the heat threshold interval, including super hot, hot, warm, cold and super cold.

[0009] Optionally, performing stability quantification processing on the small object metadata access patterns of the same application source according to the application correlation characteristics to obtain an application access stability score and an application weight factor includes: The variance of the access interval time series of the metadata of small objects from the same application source is calculated to obtain the access interval variance value and the access rhythm stability index. The access interval variance value is calculated by normalizing the coefficient of variation to perform quantitative analysis on the discrete degree of the access pattern, so as to obtain the access pattern coefficient of variation and a pattern stability benchmark value; Performing a correlation evaluation process on the access regularity of the small object metadata based on the access rhythm stability index and the pattern stability benchmark value to obtain an application access regularity score and an access prediction credibility value; The application access behavior is comprehensively evaluated and processed through stability weighted calculation according to the application access regularity score and the access prediction credibility value to obtain the application access stability score and the application weight factor based on the stability gradient.

[0010] Optionally, performing flash block characteristic analysis on the physical performance parameters of the SSD storage medium to obtain a multi-level storage area division based on performance differences and a flash block performance mapping relationship includes: Perform performance testing and collection of the read latency, write latency, and erase latency of each flash memory block in the SSD storage medium to obtain a flash memory block latency performance dataset and an IO response time benchmark value. Performing durability evaluation calculations on the flash memory block delay performance data set in combination with the current P / E cycle number and the maximum P / E cycle limit to obtain a flash memory block durability index and a remaining life expectancy value; Based on the IO response time benchmark value and the flash memory block durability index, each flash memory block is classified into a performance level by a comprehensive performance weight score to obtain flash memory block classification results of a high performance area, a standard performance area, an energy-saving area, and a spare area; The SSD storage space is subjected to regional boundary demarcation processing according to the flash memory block classification result, so as to obtain the multi-level storage area division based on performance difference and the flash memory block performance mapping relationship between each flash memory block and the storage area.

[0011] Optionally, performing intelligent storage resource allocation processing on small object metadata according to the five-level data heat level division and the multi-level storage area division to obtain an optimal matching solution for metadata components and a storage cost optimization result based on the flash block performance mapping relationship includes: Classify the metadata of small objects into metadata components according to basic attributes, extended attributes, index information and access control information, and obtain the data size and access feature identifier of each metadata component; Based on the five-level data heat level classification, heat level matching analysis processing is performed on each metadata component to obtain grouping results of super hot data components, hot data components, warm data components, cold data components and super cold data components; Performing storage area allocation processing on each data component by minimizing storage cost calculation according to the grouping results and the multi-level storage area division, thereby obtaining a high-performance area allocation plan, a standard performance area allocation plan, an energy-saving area allocation plan, and a spare area allocation plan; Each area allocation scheme is combined with the flash block performance mapping relationship to perform resource matching optimization processing, thereby obtaining the optimal matching scheme for metadata components based on the flash block performance mapping relationship and the storage cost optimization result including storage cost, access delay and space utilization.

[0012] Optionally, the dynamically adjusting the data distribution status of each storage area in the optimal matching solution of the metadata component through adaptive load monitoring to obtain balanced storage load distribution data based on the five-level data heat level division includes: Real-time monitoring, collection, and processing of data capacity occupancy, access frequency changes, and heat level migration in each storage area are performed to obtain storage area load status data and heat change trigger conditions; Comparing and analyzing the storage area load status data with a preset load balancing threshold to obtain an identifier of a load imbalance area and a list of data components that need to be migrated; Re-evaluate the heat level of the metadata of the small objects in the data component list based on the heat change trigger condition to obtain updated heat level allocation and data migration priority ranking; According to the updated heat level allocation and data migration priority ranking, the small object metadata is migrated across regions through a load redistribution algorithm to obtain balanced storage load distribution data based on the five-level data heat level division and optimized storage area utilization.

[0013] In a second aspect, the present application provides a data storage optimization system for a solid-state hard drive, the data storage optimization system for a solid-state hard drive comprising: The recognition module is used to perform pattern recognition processing on the access behavior data of small object metadata through multi-dimensional access feature extraction to obtain access frequency feature vectors, time locality features and application correlation features; a quantification module for performing data heat quantification processing using a flash memory-aware heat evaluation algorithm based on the access frequency feature vector, the temporal locality feature, and the application correlation feature, to obtain a five-level data heat classification including super hot, hot, warm, cold, and super cold; An analysis module is used to analyze the physical performance parameters of the SSD storage medium and process the flash block characteristics to obtain a multi-level storage area division based on performance differences and a flash block performance mapping relationship; An allocation module is configured to perform intelligent storage resource allocation processing on the small object metadata according to the five-level data heat level classification and the multi-level storage area classification, and obtain an optimal matching solution for metadata components and a storage cost optimization result based on the flash block performance mapping relationship; The adjustment module is used to dynamically adjust the data distribution status of each storage area in the optimal matching solution of the metadata component through adaptive load monitoring to obtain balanced storage load distribution data based on the five-level data heat level division.

[0014] In a third aspect, a data storage optimization device for a solid-state hard drive is provided, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the data storage optimization device for the solid-state hard drive executes the above-mentioned data storage optimization method for the solid-state hard drive.

[0015] In a fourth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, which, when executed on a computer, enables the computer to execute the above-mentioned data storage optimization method for a solid-state hard drive.

[0016] In the technical solution provided by this application, pattern recognition processing is performed on small object metadata through multi-dimensional access feature extraction, which can accurately capture key information such as the access timestamp, operation type and application source of small objects. Compared with the traditional single-dimensional analysis method based only on access frequency, multi-dimensional feature extraction significantly improves the accuracy of access pattern recognition, making subsequent heat evaluation more reliable. The flash memory-aware heat evaluation algorithm fully considers the physical characteristics of SSD flash storage, and weightedly fuses the access frequency feature vector, time locality feature and application correlation feature to form a dedicated heat quantification method for the characteristics of small object metadata. The algorithm effectively solves the data skew and time sensitivity problems in traditional heat evaluation through logarithmic normalization processing and exponential decay calculation. The generated five-level data heat level division provides an accurate data classification basis for subsequent tiered storage. The flash block characteristic analysis and processing of SSD storage media fully utilizes the differences in read / write performance and durability of flash blocks. By comprehensively evaluating the latency performance and P / E cycle status of each flash block, a multi-level storage area division based on performance differences is established. This physical characteristic-aware partitioning method can better leverage the differentiated advantages of SSD internal resources compared to traditional uniform allocation strategies, significantly improving storage resource utilization efficiency. Intelligent storage resource allocation processing manages small object metadata at the component level based on basic attributes, extended attributes, index information, and access control information, achieving more refined cost control than traditional overall storage. By matching metadata components of different heat levels to flash block areas with corresponding performance, it effectively reduces the waste of high-performance SSD storage space while ensuring access performance of hot data.

[0017] Adaptive load monitoring and dynamic adjustment processing establish an intelligent management mechanism that responds to changes in access patterns in real time. By continuously monitoring the capacity utilization, access frequency changes, and heat level migration of each storage area, it can promptly detect load imbalance problems and trigger corresponding data migration operations. This dynamic adjustment capability is more adaptable and flexible than traditional static storage allocation methods. Especially in cloud-native application scenarios, when the application access pattern suddenly changes, this solution can quickly re-evaluate the heat level of small object metadata and migrate data to the most suitable storage area through a load redistribution algorithm, thereby always maintaining the optimal performance-cost balance. The entire solution is specifically optimized for the specific scenario where the size of small object metadata approaches or exceeds the size of the data itself. Through multi-level technological innovation, it effectively addresses the shortcomings of traditional SSD storage optimization methods in cost control, precise classification, dynamic adaptation, etc., providing cloud storage services with a more economical and efficient small object metadata management solution. In actual applications, it can significantly reduce SSD storage costs while maintaining good access performance and system stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A schematic diagram of an embodiment of a data storage optimization method for a solid-state drive in an embodiment of the present application; Figure 2 A schematic diagram of an embodiment of a data storage optimization system for a solid-state hard disk in an embodiment of the present application; Figure 3 It is a schematic block diagram of the structure of a data storage optimization device for a solid state drive in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] Embodiments of the present application provide a data storage optimization method and system for a solid-state drive. The terms "first," "second," "third," "fourth," and the like (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this way are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products, or devices.

[0021] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In one embodiment of the present application, a method for optimizing data storage for a solid-state drive includes: Step S101: Perform pattern recognition processing on access behavior data of small object metadata through multi-dimensional access feature extraction to obtain access frequency feature vectors, time locality features, and application relevance features; Step S102: quantify the data heat using a flash memory-aware heat evaluation algorithm based on the access frequency feature vector, temporal locality feature, and application relevance feature, to obtain a five-level data heat classification including super hot, hot, warm, cold, and super cold. Step S103: Perform flash block characteristic analysis on the physical performance parameters of the SSD storage medium to obtain a multi-level storage area division based on performance differences and a flash block performance mapping relationship; Step S104: Intelligently allocate storage resources for small object metadata based on the five-level data heat level classification and multi-level storage area division, and obtain an optimal matching solution for metadata components and a storage cost optimization result based on the flash block performance mapping relationship; Step S105: Dynamically adjust the data distribution status of each storage area in the optimal matching solution of the metadata component through adaptive load monitoring to obtain balanced storage load distribution data based on the five-level data heat level division.

[0022] It is understandable that the execution subject of this application can be a data storage optimization system for a solid-state drive, or a terminal or a server, which is not limited here. The embodiment of this application is described by taking a server as the execution subject as an example.

[0023] Specifically, a small object metadata monitoring mechanism is established to collect raw data such as the access timestamp, access operation type (read, write, delete, modify), and access source application identifier for each small object in real time, forming a raw access behavior dataset. This dataset is then segmented into time series according to fixed time windows. For example, each time window is set to 1 hour, and access data within a 24-hour period is segmented into 24 time windows. The number of accesses within each time window is counted to generate an access count distribution matrix. Access patterns within each time window are quantitatively analyzed using frequency density calculations. The access density value for each time window is calculated as the number of accesses within the window divided by the time window length, forming an access frequency feature vector. The ratio of the standard deviation to the mean of the access interval is calculated to obtain the access interval coefficient of variation. Finally, correlation mining is performed on the metadata of small objects from the same application source to analyze the similarity of access behaviors within the same application in different time periods, thereby extracting temporal locality features and application correlation features.

[0024] Based on the obtained access frequency feature vector, temporal locality feature, and application relevance feature, data heat is quantified using a flash-aware heat assessment algorithm. This algorithm first logarithmically normalizes the access count data in the access frequency feature vector. A frequency score is calculated, equal to the logarithm of the access count plus 1, divided by the maximum logarithmic value, ensuring the score is between 0 and 1. A corresponding frequency weight factor is then assigned. Based on the temporal locality feature, a time decay score is calculated using an exponential decay function. This score is equal to a natural constant raised to the power of negative λ times the time difference, where λ is the decay coefficient and the time difference is the difference between the current time and the last access time, reflecting the freshness of the access time. A time weight factor is then assigned. For the application relevance feature, the variance of the access interval time series of metadata accesses to small objects from the same application source is calculated to obtain the access interval variance value. The coefficient of variation is then calculated and normalized to obtain the access pattern coefficient of variation and a pattern stability baseline value. Ultimately, the application access stability score and application weight factor are derived. The frequency score, time score, and application access stability score are weighted and summed according to the corresponding weight factors to obtain the comprehensive heat value of the small object metadata. It is then divided into five levels: super hot, hot, warm, cold, and super cold according to the preset heat threshold range.

[0025] The physical performance parameters of the SSD storage media are analyzed using flash block characteristics. By performing performance tests on each flash block in the SSD storage media, key performance indicators such as read latency, write latency, and erase latency are collected to form a flash block latency performance dataset. The average response time of each operation is recorded as the IO response time baseline. The flash block latency performance dataset is combined with the current P / E cycle count and the maximum P / E cycle limit to calculate the flash block endurance index, which is equal to the ratio of the current P / E cycle count to the maximum P / E cycle limit. The remaining life expectancy is predicted to be equal to the maximum P / E cycle limit minus the current P / E cycle count. By comprehensively considering the IO response time baseline and the flash block endurance index, a performance weighted composite score is calculated, and each flash block is classified into four performance levels: high-performance zone, standard-performance zone, energy-saving zone, and spare zone. Finally, based on the flash block classification results, regional boundaries are demarcated for the SSD storage space, and a performance mapping relationship between each flash block and storage zone is established.

[0026] Based on a five-level data heat classification and multi-tiered storage area division, small object metadata is intelligently allocated for storage resources. First, small object metadata is categorized into metadata components based on basic attributes (object size, creation time, modification time), extended attributes (user-defined tags, description information), index information (for fast retrieval of related data), and access control information (permission settings, security tags). The data size and access characteristics of each component are recorded. Based on the five-level data heat classification, each metadata component is matched to its corresponding heat level, resulting in groupings of super-hot data components, hot data components, warm data components, cold data components, and super-cold data components. Through storage cost minimization calculations, super-hot and hot data components are allocated to high-performance areas, warm data components to standard performance areas, cold data components to energy-saving areas, and super-cold data components to spare areas, forming an allocation plan for each area. Resource matching is optimized by combining flash block performance mapping to ensure that high-heat data is allocated to the flash blocks with the best performance. This results in an optimal metadata component matching plan and a storage cost optimization result that includes storage cost, access latency, and space utilization.

[0027] Adaptive load monitoring dynamically adjusts the data distribution status of each storage area in the optimal metadata component matching solution. The monitoring mechanism collects data capacity utilization, access frequency changes, and hotness level migration status of each storage area in real time to generate storage area load status data. When the capacity utilization of a monitored storage area exceeds the preset threshold of 85% or falls below 15%, load balancing adjustments are triggered. This storage area load status data is compared and analyzed with the preset load balancing threshold to identify areas with uneven loads and generate a list of data components requiring migration. Based on the hotness change trigger conditions, the hotness of small object metadata in the data component list is re-evaluated. When the access pattern of a small object changes significantly, its hotness level is updated, and data migration priority is determined based on urgency and data size. A load redistribution algorithm is used to perform cross-area data migration, migrating increasing hotness data from low-performance areas to high-performance areas and decreasing hotness data from high-performance areas to low-performance areas. This ultimately creates a balanced storage load distribution based on the five-level data hotness level.

[0028] In a specific embodiment, the process of executing step S101 may specifically include the following steps: The access timestamp, access operation type, and access source application identifier of small object metadata are monitored, collected, and processed in real time to obtain the original access behavior dataset; The original access behavior dataset is segmented into time series according to the preset time window length to obtain access statistics and access number distribution matrix within multiple time windows; Based on the access times distribution matrix, the access patterns in each time window are quantitatively analyzed by frequency density calculation to obtain the access frequency feature vector and access interval variation coefficient. According to the access frequency feature vector and access interval variation coefficient, the metadata of small objects from the same application source are mined for correlation, and the temporal locality feature and application correlation feature are obtained.

[0029] Specifically, when real-time monitoring, collection and processing are performed on the access timestamps, access operation types and access source application identifiers of small object metadata, an access monitoring hook function is deployed at the SSD storage controller level. When any application initiates an access request to the small object metadata, the monitoring hook immediately captures the access event and records the precise timestamp information. The timestamp adopts the Unix timestamp format with nanosecond precision. The access operation types are divided into four basic operations: read, write, delete and modify. The specific operation type is determined by analyzing the opcode of the IO instruction. The access source application identifier obtains a unique application identifier by parsing the process ID and application signature. The collected data is stored in a circular buffer in chronological order to form an original access behavior dataset. Each record contains four fields: object ID, timestamp, operation type and application identifier.

[0030] When the original access behavior dataset is segmented into time series according to the preset time window length, the time window length is first determined to be a fixed time interval, for example, set to 30 minutes. Then, starting from the start time of the original access behavior dataset, the continuous access records are divided into different time windows according to the time window length. Each time window contains the access records of all small object metadata in the time period. The number of visits to each small object in each time window is counted to form access statistics. Then, an access number distribution matrix is ​​constructed. The rows of the matrix represent different small objects, and the columns represent different time windows. The values ​​in the matrix represent the number of visits to the corresponding small object in the corresponding time window. In this way, the originally discrete access records are converted into structured matrix data.

[0031] When performing quantitative analysis on the access patterns in each time window through frequency density calculation based on the access count distribution matrix, frequency density calculation refers to dividing the number of accesses in each time window by the length of the time window to obtain the average access frequency per unit time. For each small object, its frequency density values ​​in all time windows are combined into a vector, which is the access frequency feature vector. This vector reflects the access activity of the small object in different time periods. At the same time, the access interval variation coefficient is calculated. The access interval refers to the time difference between two consecutive visits to the same small object. The variation coefficient is equal to the standard deviation of the access interval divided by the average value of the access interval. The smaller the variation coefficient, the more regular the access pattern, and the larger the variation coefficient, the more irregular the access pattern. By traversing each small object in the access count distribution matrix, the variation coefficient of its access interval sequence is calculated.

[0032] When performing correlation mining on metadata of small objects from the same application source based on access frequency feature vectors and access interval variation coefficients, small objects are first grouped according to application identifiers, and small objects from the same application are classified together. Then, the correlation between the access frequency feature vectors of each small object in the same application group is calculated, and the degree of correlation is quantified by calculating the Pearson correlation coefficient between the vectors. A correlation coefficient close to 1 indicates that the access patterns are highly similar, and a correlation coefficient close to 0 indicates that the access patterns are unrelated. The temporal locality feature is obtained by analyzing the access time distribution of small objects in the same application group, and calculating the concentration and dispersion of the access time of each small object. A high concentration indicates a strong locality of the access time. The application correlation feature is obtained by calculating the consistency of the access patterns of small objects in the same application group. A high consistency indicates that the application has a strong correlation in accessing these small objects.

[0033] In a specific embodiment, the process of executing step S102 may specifically include the following steps: Perform logarithmic normalization on the access count data in the access frequency feature vector to obtain the frequency score and the corresponding frequency weight factor; Based on the time locality feature, the time difference between the current time and the last access time is calculated through exponential decay calculation to obtain the time score and time weight factor reflecting the freshness of the access time; Based on the application correlation characteristics, the stability of the small object metadata access pattern of the same application source is quantified to obtain the application access stability score and application weight factor; The frequency score, time score, and application access stability score are weighted and summed according to the corresponding weight factors to obtain the comprehensive heat value of the small object metadata and the five-level data heat level classification based on the heat threshold interval, including super hot, hot, warm, cold, and super cold.

[0034] Specifically, when performing logarithmic normalization on the access count data in the access frequency feature vector, first add 1 to the access count of each element in the access frequency feature vector and then take the natural logarithm. This process is to prevent the situation where the logarithm is meaningless when the access count is 0, and at the same time compress the dynamic range of the data. Then, the maximum value of all logarithmic values ​​is used as the normalization benchmark, and each logarithmic value is divided by this maximum logarithmic value to obtain a frequency score between 0 and 1. The frequency weight factor is determined according to the overall distribution of the access count. Small object metadata with higher access counts is assigned a higher weight factor. The specific calculation method is to divide the access count of the small object by the sum of the access counts of all small objects, and then multiply it by the preset weight adjustment coefficient. The frequency score reflects the access activity of the small object metadata, and the frequency weight factor is used to balance small objects with different access intensities in subsequent weighted calculations.

[0035] When calculating the time decay score of the time difference between the current time and the last access time through exponential decay calculation based on the time locality feature, the time decay score is calculated using an exponential decay function. The basic form of this function is a negative exponential function with the natural constant e as the base. The decay coefficient in the exponential decay function determines the speed of time decay. The larger the decay coefficient, the stronger the time sensitivity and the faster the time score decreases. The calculation process first obtains the current system time and the last access time of the small object metadata, calculates the time difference between the two, and then multiplies the time difference by the preset decay coefficient as the exponent. The negative exponential power of the natural constant e is calculated to obtain a time score between 0 and 1. The closer the time score is to 1, the closer the last access time is to the current time. The time weight factor is allocated according to the distribution of the time score. Small object metadata with a high time score obtains a higher time weight factor. The calculation method is to divide the time score by the average of all small object time scores and then multiply it by the time weight adjustment coefficient.

[0036] When quantifying the stability of small object metadata access patterns from the same application source based on application correlation characteristics, stability quantification assesses the consistency of the same application's access behavior to its related small object metadata. First, the variance of the access interval time series of each small object within the same application group is calculated. A smaller variance indicates a more stable access interval. Then, the average of the access intervals is calculated and the coefficient of variation is obtained by dividing the variance by the square of the average. A smaller coefficient of variation indicates a more stable access pattern. The access rhythm stability index is then calculated using the inverse of the coefficient of variation. A higher stability index indicates a more regular access pattern. The pattern stability baseline is obtained by statistically analyzing the coefficient of variation of all application groups, typically taking the median as the baseline. The application access regularity score is calculated by comparing the stability index with the baseline value. A high score is assigned when the stability index exceeds the baseline, and a low score is assigned when it does not. The access prediction confidence value is calculated based on the consistency of historical access patterns. Higher consistency indicates higher prediction confidence. The application access stability score is calculated by weighted average, taking into account both the access regularity score and the prediction confidence value. The application weight factor is determined based on the gradient distribution of the stability score, with applications with higher stability scores receiving higher weight factors.

[0037] When the frequency score, time score and application access stability score are weighted and calculated according to the corresponding weight factors, the weighted summation is calculated using a linear weighted method. The three scores are multiplied by the corresponding weight factors, and then the sum is added to obtain the comprehensive heat value. The distribution of weight factors follows the constraint that the sum of frequency weight, time weight and application weight is equal to 1. The calculation formula for the comprehensive heat value is the frequency score multiplied by the frequency weight factor plus the time score multiplied by the time weight factor plus the application access stability score multiplied by the application weight factor. The resulting comprehensive heat value is between 0 and 1, the higher the heat value, the more active the small object metadata. The five-level data heat level division is achieved by setting the heat threshold interval. The super hot level corresponds to the interval with a heat value greater than or equal to 0.8, the hot level corresponds to the interval with a heat value between 0.6 and 0.8, the warm level corresponds to the interval with a heat value between 0.4 and 0.6, the cold level corresponds to the interval with a heat value between 0.2 and 0.4, and the super cold level corresponds to the interval with a heat value less than 0.2. By comparing the calculated comprehensive heat value with these threshold intervals, the heat level of each small object metadata is determined.

[0038] Take the thumbnail metadata management of a video processing application as an example. The application generates a large number of video thumbnails, each of which is about 20KB, and the corresponding metadata is about 16KB. A popular video thumbnail is accessed 120 times in a week. The logarithmic normalization process is to add 1 to the number of visits and then take the natural logarithm to get 4.8. Dividing it by the maximum logarithmic value of 5.2 gives a frequency score of 0.92. The frequency weight factor is calculated based on the proportion of the number of thumbnail visits to the total number of visits to get 0.35. In the time decay score calculation, it is found that the last visit of the thumbnail was 6 hours from the current time. The time difference of 6 hours multiplied by the decay coefficient of 0.1 is 0.6. The negative 0.6 power of e yields a time score of 0.55. The time weight factor, calculated based on the time score distribution, is 0.4. Application correlation analysis reveals that the variance of thumbnail access intervals for the same video processing application is 25, the average interval is 8 hours, the coefficient of variation is 0.39, and the stability index is 2.56. A comprehensive evaluation yields an application access stability score of 0.75, with an application weight factor of 0.25. The final weighted summation yields a comprehensive heat value of 0.92 times 0.35 plus 0.55 times 0.4 plus 0.75 times 0.25, equaling 0.73. Based on the heat threshold range, the thumbnail metadata is determined to be at the heat level.

[0039] In a specific embodiment, the process of performing stability quantification processing on the small object metadata access patterns from the same application source according to the application correlation characteristics may specifically include the following steps: The variance of the access interval time series of the metadata of small objects from the same application source is calculated to obtain the access interval variance value and the access rhythm stability index. The variance of the access interval is calculated by normalizing the coefficient of variation to quantify the discreteness of the access pattern, and the coefficient of variation of the access pattern and the reference value of the pattern stability are obtained. Based on the access rhythm stability index and pattern stability benchmark value, the access regularity of small object metadata is evaluated and processed to obtain the application access regularity score and access prediction credibility value; According to the application access regularity score and access prediction credibility value, the application access behavior is comprehensively evaluated through stability weighted calculation to obtain the application access stability score and the application weight factor based on the stability gradient.

[0040] Specifically, when performing variance calculation on the access interval time series of small object metadata from the same application source, it is first necessary to extract the access timestamps of all small object metadata under the same application identifier from the collected access behavior data, arrange these timestamps in chronological order to form a time series, and then calculate the time interval between two consecutive accesses to form an access interval time series. Each element in the access interval time series represents the time difference between two adjacent accesses. Then, variance calculation is performed on the access interval time series. The variance calculation process includes calculating the average value of the access interval, and then calculating the square of the difference between each access interval and the average value. The sum of all the difference squares is divided by the total number of access intervals minus 1 to obtain the access interval variance value. The access beat stability index is calculated by the inverse of the access interval variance value. The smaller the variance value, the more stable the access interval, and the higher the corresponding stability index. The stability index reflects the degree of regularity of the application's access to small object metadata.

[0041] When the access interval variance value is normalized by the coefficient of variation to quantitatively analyze the degree of dispersion of the access pattern, the coefficient of variation calculation requires taking the square root of the access interval variance value to obtain the standard deviation, and then dividing the standard deviation by the access interval average value to obtain the coefficient of variation. The coefficient of variation is a dimensionless relative variation measurement indicator that eliminates the impact of the absolute value differences in the access intervals between different applications, making the access pattern stability of different applications comparable. The smaller the coefficient of variation, the lower the degree of dispersion of the access pattern and the more regular the access behavior. The pattern stability baseline value is obtained by statistically analyzing the coefficient of variation data of all applications. The specific method is to collect the coefficient of variation data of all applications and calculate the median of these coefficients of variation as the pattern stability baseline value. The median can better reflect the central trend of the data and is less susceptible to extreme values ​​than the average value. The baseline value serves as a reference standard for judging whether the access pattern of an application is stable.

[0042] When performing a correlation evaluation on the access regularity of small object metadata based on the access rhythm stability index and the pattern stability benchmark value, the correlation evaluation determines the degree of access regularity by comparing the relationship between the access rhythm stability index and the pattern stability benchmark value. When the stability index is greater than the benchmark value, it indicates that the access pattern of the application is more regular than the average level, and a higher regularity score is given. When the stability index is less than the benchmark value, it indicates that the access pattern is relatively irregular, and a lower regularity score is given. The application access regularity score is calculated using a linear mapping method, mapping the ratio of the stability index to the benchmark value to a score range of 0 to 1. The access prediction credibility value is calculated based on the consistency between the access regularity score and the historical access pattern. The consistency degree is quantified by analyzing the similarity of access patterns in different time periods. The more consistent the access pattern, the higher the prediction credibility of future access behavior. The prediction credibility value is also normalized to the range of 0 to 1.

[0043] When comprehensively evaluating application access behavior through stability weighted calculation based on the application access regularity score and access prediction credibility value, the stability weighted calculation adopts a weighted average method, and the application access regularity score and access prediction credibility value are weighted and summed according to the preset weight ratio. The weight ratio is determined according to the actual needs of SSD storage optimization. In general, the regularity score weight is 0.6 and the credibility value weight is 0.4. The weighted summation result is the application access stability score, which comprehensively reflects the stability of the application's access behavior to small object metadata. The application weight factor calculation method based on the stability gradient is to sort the stability scores of all applications and divide them into different stability gradients according to the distribution of the stability scores. Applications with high stability scores are classified into the high stability gradient and assigned a higher weight factor. Applications with low stability scores are classified into the low stability gradient and assigned a lower weight factor. The distribution of weight factors follows the principle of positive correlation between stability gradient and weight factor.

[0044] In a specific embodiment, the process of executing step S103 may specifically include the following steps: Perform performance testing and collection of the read latency, write latency, and erase latency of each flash memory block in the SSD storage medium to obtain a flash memory block latency performance dataset and an IO response time benchmark value. The flash memory block latency performance data set is combined with the current P / E cycle number and the maximum P / E cycle limit to perform durability evaluation calculations to obtain the flash memory block durability index and remaining life expectancy; Based on the IO response time benchmark value and the flash memory block durability index, each flash memory block is classified into a performance level using a comprehensive performance weight score, and the flash memory block classification results are obtained as high-performance area, standard performance area, energy-saving area and spare area; Based on the flash block classification results, the SSD storage space is demarcated to obtain a multi-level storage area division based on performance differences and a flash block performance mapping relationship between each flash block and the storage area.

[0045] Specifically, when performing performance testing and collecting data for the read latency, write latency, and erase latency of each flash memory block in the SSD storage medium, it is first necessary to directly access each flash memory block through the underlying interface of the SSD controller and perform standardized performance testing operations. The read latency test is measured by sending a read instruction to each flash memory block and recording the time difference from the instruction issuance to the data return. The write latency test is obtained by writing a fixed-size data block to the flash memory block and measuring the time required for the write to complete. The erase latency test is measured by performing a flash memory block erase operation and recording the erase completion time. Each delay test is repeated multiple times and the average value is taken to ensure data accuracy. The read latency, write latency, and erase latency data of all flash memory blocks are aggregated to form a flash memory block latency performance dataset. The dataset is organized in a table format, with rows representing different flash memory blocks and columns representing different types of delay data. The IO response time benchmark value is determined by calculating the weighted average of various types of delays for all flash memory blocks. The weighting coefficient is set according to the frequency distribution of various operations in actual applications. The read operation has the highest frequency and therefore the largest weight, followed by the write operation, and the erase operation has the lowest frequency and the smallest weight.

[0046] When combining the flash memory block latency performance dataset with the current P / E cycle count and maximum P / E cycle limit for durability evaluation, the P / E cycle count refers to the cumulative number of program erase operations performed on the flash memory block. Each flash memory block has a maximum P / E cycle limit. Exceeding this limit significantly reduces the reliability of the flash memory block. The durability evaluation calculation first reads the current P / E cycle count of each flash memory block and compares it with the maximum P / E cycle limit of the flash memory block. The flash memory block durability index is calculated by subtracting the current P / E cycle count from the maximum P / E cycle limit and then dividing it by the maximum P / E cycle limit. A durability index closer to 1 indicates a newer flash memory block, while a value closer to 0 indicates that the flash memory block is nearing the end of its life. The remaining life expectancy is directly equal to the maximum P / E cycle limit minus the current P / E cycle count. This value indicates how many erase operations the flash memory block can still perform. The durability evaluation calculation also needs to consider the flash memory block's usage history and wear pattern. Future durability changes are predicted by analyzing the growth trend of the P / E cycle count.

[0047] Based on the IO response time baseline and the flash block endurance index, each flash block is classified into a performance level using a comprehensive performance weighted score. This score uses a multi-factor weighted scoring model that comprehensively considers two key factors: the flash block's response speed and lifespan. The response speed score is calculated by comparing the average IO response time of each flash block with the baseline value. Flash blocks with shorter response times receive higher speed scores. The endurance score directly uses the previously calculated endurance index. The comprehensive score is equal to the speed score multiplied by the speed weight coefficient plus the endurance score multiplied by the endurance weight coefficient. The weight coefficients are determined based on the SSD storage optimization strategy requirements. Speed ​​is weighted more heavily in high-performance application scenarios, while endurance is weighted more heavily in long-term storage scenarios. Classification thresholds are set based on the distribution of the comprehensive scores. The flash block with the highest score is classified into the high-performance zone, the block with an above-average score into the standard performance zone, the block with an above-average score into the energy-saving zone, and the block with the lowest score into the spare zone. The four performance zones are divided according to the principle that the score intervals do not overlap and fully cover all flash blocks.

[0048] When the SSD storage space is demarcated based on the flash block classification results, the regional boundary demarcation refers to allocating a continuous address range to each performance area in the logical address space of the SSD. The demarcation process first counts the number of flash blocks contained in each performance area, and then allocates address segments of corresponding sizes in the logical address space based on the number ratio. The high-performance area is allocated the optimal address segment, which is usually the area with a smaller starting address. The standard performance area is allocated the suboptimal address segment, and the energy-saving area and the spare area are allocated the remaining address segments in turn. After the multi-level storage area division is formed, a mapping relationship between each flash block and the storage area is established. The mapping relationship records the logical address range and performance area corresponding to each physical flash block. The flash block performance mapping relationship is stored in the form of a data structure, which contains key information such as the flash block physical address, logical address range, performance level, and comprehensive score. The mapping relationship supports bidirectional query, which can be used to search for the area to which it belongs through the flash block, and to search for the flash blocks contained in the area.

[0049] In a specific embodiment, the process of executing step S104 may specifically include the following steps: Classify the metadata of small objects into metadata components according to basic attributes, extended attributes, index information and access control information, and obtain the data size and access feature identifier of each metadata component; Based on the five-level data heat level classification, each metadata component is subjected to heat level matching analysis and processing to obtain the grouping results of super hot data components, hot data components, warm data components, cold data components and super cold data components; Based on the grouping results and multi-level storage area division, storage area allocation processing is performed on each data component through storage cost minimization calculation to obtain high-performance area allocation plan, standard performance area allocation plan, energy-saving area allocation plan and spare area allocation plan; The resource matching optimization processing is performed by combining each area allocation scheme with the flash block performance mapping relationship, and the optimal matching scheme of metadata components based on the flash block performance mapping relationship and the storage cost optimization results including storage cost, access latency and space utilization are obtained.

[0050] Specifically, when classifying small object metadata into metadata components based on basic attributes, extended attributes, index information, and access control information, metadata component classification refers to splitting the complete metadata of a small object into different logical components according to its function and purpose. The basic attribute component contains the core identification information of the object, such as object ID, file size, creation time, modification time, file type, and other required fields. This information must be read when the object is accessed. The extended attribute component contains optional fields such as user-defined tags, description information, version number, and notes. This information is mainly used for object management and retrieval. The index information component contains index data such as hash values, keyword indexes, and classification tags for quickly locating and retrieving objects. The access control information component contains security-related data such as permission settings, security levels, encryption parameters, and access policies. The classification process parses the structured storage format of small object metadata and classifies each field into the corresponding component according to predefined field mapping rules. At the same time, the data size of each metadata component is counted. The data size is obtained by calculating the sum of the bytes of the fields contained in each component. The access feature identification is determined by analyzing the frequency pattern of each component in historical access. Basic attributes have the highest access frequency, followed by extended attributes, and index information and access control information have relatively low access frequencies.

[0051] When performing heat level matching analysis on each metadata component based on the five-level data heat level division, the heat level matching analysis refers to applying the overall heat level of the small object calculated previously to its various metadata components, but the heat level allocation of different components follows the principle of differentiated access frequency. The basic attribute component has the highest access frequency, so its heat level is consistent with the overall heat level of the small object. The heat level of the extended attribute component is reduced by one level based on the overall heat level. For example, the extended attribute component of the small object with an overall super hot level is assigned to the hot level, the heat level of the index information component is reduced by two levels, and the heat level of the access control information component is reduced by two levels. Through this hierarchical matching mechanism, each metadata component is classified into the grouping results of super hot data components, hot data components, warm data components, cold data components and super cold data components. The grouping results are recorded in matrix form, with rows representing different small objects, columns representing different metadata component types, and matrix elements representing the heat levels of corresponding components.

[0052] When allocating storage areas to each data component through storage cost minimization calculation based on the grouping results and multi-level storage area division, the storage cost minimization calculation adopts an optimization strategy based on matching the heat level with the storage area performance. The core principle of this strategy is to allocate high-heat data components to high-performance storage areas and low-heat data components to low-performance storage areas. The specific allocation rules are that super-hot and hot data components are allocated to high-performance areas, warm data components are allocated to standard performance areas, cold data components are allocated to energy-saving areas, and ultra-cold data components are allocated to spare areas. The cost calculation considers two factors: storage cost and access cost. The storage cost is equal to the data size multiplied by the unit storage cost of the storage area, and the access cost is equal to the expected access frequency multiplied by the unit access cost of the storage area. The total cost is the sum of the storage cost and the access cost. The allocation plan with the lowest cost is found by enumerating all combinations and calculating the total cost. The high-performance area allocation plan records the list of data components allocated to the high-performance area and their storage locations. Other area allocation plans are recorded in the same way.

[0053] When combining each region allocation scheme with the flash block performance mapping for resource matching optimization, resource matching optimization involves further allocating data components to specific flash blocks based on the determined storage region allocation. The optimization goal is to maximize flash block utilization while meeting performance requirements. The matching process first determines the data components to be stored and their sizes based on each region allocation scheme. Then, the flash block performance mapping is queried to obtain a list of flash blocks in each storage region and their performance parameters. A best-fit algorithm is then used to allocate the data components to appropriate flash blocks. The best-fit algorithm prioritizes flash blocks with the highest space utilization that can still accommodate the target data components. The optimal matching scheme for the metadata component records the final storage location of each data component, including information such as the flash block number, starting address, and data size. The storage cost optimization results are evaluated by calculating the storage cost, average access latency, and space utilization of the actual allocation scheme. The storage cost is the sum of the size of each data component multiplied by the storage cost coefficient of the corresponding flash block. The access latency is the sum of the access frequency of each data component multiplied by the average access latency of the corresponding flash block. The space utilization is the used space divided by the total available space.

[0054] In a specific embodiment, the process of executing step S105 may specifically include the following steps: Real-time monitoring, collection, and processing of data capacity occupancy, access frequency changes, and heat level migration in each storage area are performed to obtain storage area load status data and heat change trigger conditions; Compare and analyze the storage area load status data with the preset load balancing threshold to obtain the load imbalance area identifier and the list of data components that need to be migrated; Based on the heat change trigger condition, the heat level of the small object metadata in the data component list is re-evaluated to obtain the updated heat level allocation and data migration priority ranking; According to the updated heat level allocation and data migration priority ranking, the small object metadata is migrated across regions through the load redistribution algorithm, and balanced storage load distribution data based on the five-level data heat level division and optimized storage area utilization are obtained.

[0055] Specifically, when real-time monitoring, collection and processing are performed on the data capacity occupancy, access frequency changes and heat level migration status of each storage area, real-time monitoring and collection refers to deploying a monitoring agent at the SSD controller level to continuously track and record the status changes of each storage area. Data capacity occupancy monitoring is calculated by regularly scanning the ratio of used space to total available space in each storage area. The monitoring cycle is set to perform statistical operations once every minute. When the occupancy rate exceeds the preset threshold, it is recorded as a capacity alarm event. Access frequency change monitoring is measured by counting the number of access requests received by each storage area per unit time. Access frequency change refers to the difference between the access frequency in the current time window and the historical average access frequency. When the difference exceeds the set range, it is marked as a frequency abnormality event. Heat level migration monitoring records the heat level changes of small object metadata. When the heat level of a small object changes from one level to another, a migration event record is generated. The storage area load status data includes key indicators such as the current capacity occupancy, access frequency, and number of active objects in each area. The heat change trigger condition is set to trigger the subsequent processing flow when the heat level changes across levels or the access frequency changes by more than a specific multiple of the baseline value.

[0056] When comparing and analyzing the storage area load status data with the preset load balancing threshold, the load balancing threshold refers to the ideal range limit of the load level of each storage area. The load balancing threshold of the high-performance area is set to a capacity occupancy rate not exceeding 90% of the total capacity and an access frequency not exceeding 80% of the design peak. The standard performance area, energy-saving area, and spare area are respectively set with corresponding threshold parameters. The comparative analysis is performed by comparing the actual monitored load status data with these preset thresholds item by item. When the capacity occupancy rate or access frequency of a storage area exceeds the corresponding threshold, the area is identified as a load imbalance area. The load imbalance area identification records the specific area number, the type of threshold parameter exceeded, and the degree of excess. The list of data components that need to be migrated is generated by analyzing the data distribution in the load imbalance area. Data components with relatively low access frequency or heat level that does not match the current storage area are preferentially selected as migration candidates. The data component list is sorted according to the urgency of migration, and the urgency is comprehensively evaluated based on the severity of the load exceeding the threshold and the access characteristics of the data component.

[0057] When re-evaluating the heat of small object metadata in the data component list based on the heat change trigger condition, heat re-evaluation refers to recalculating the comprehensive heat value of the small object metadata using the latest access behavior data. The re-evaluation process adopts the same flash-aware heat evaluation algorithm as the initial heat evaluation, but the input data is updated to the access statistics within the most recent time window. The access frequency feature vector, temporal locality feature, and application correlation feature are recalculated, and then the updated comprehensive heat value is obtained by weighted summation. The updated heat level assignment is determined based on the comparison result between the new comprehensive heat value and the heat threshold interval. The data migration priority ranking comprehensively considers three factors: the magnitude of the heat level change, the adaptability of the current storage location, and the migration cost. The smaller the heat level change, the higher the migration priority of the small object. The adaptability of the current storage location is evaluated by comparing the new heat level of the small object with the performance level of the storage area where it is currently located. The lower the adaptability, the more urgent the migration need. The migration cost considers the data transmission overhead and the space availability of the target area.

[0058] When performing cross-region migration of small object metadata through the load redistribution algorithm based on the updated heat level allocation and data migration priority ranking, the load redistribution algorithm adopts a phased migration strategy to perform data migration operations. The first phase processes the data component with the highest priority and migrates it from the current storage area to the target storage area that matches its new heat level. The migration process includes steps such as reading data from the source area, allocating storage space in the target area, writing data to the target location, updating the metadata mapping relationship, and releasing space in the source area. The second phase processes the data components with lower priority, and so on until all data components that need to be migrated are migrated. The migration operation is performed in the background to avoid affecting normal data access services. After each migration operation, the load status statistics of the storage area are updated. The balanced storage load distribution data based on the five-level data heat level division records the distribution of data of different heat levels in each storage area after the migration is completed. The optimized storage area utilization is evaluated by calculating the capacity utilization efficiency and access load balance of each area.

[0059] The above describes the data storage optimization method for solid state drives in the embodiment of the present application. The following describes the data storage optimization system for solid state drives in the embodiment of the present application. Figure 2 In one embodiment of the present application, a data storage optimization system for a solid-state hard disk includes: The recognition module is used to perform pattern recognition processing on the access behavior data of small object metadata through multi-dimensional access feature extraction to obtain access frequency feature vectors, time locality features and application correlation features; The quantification module is used to quantify data heat using a flash memory-aware heat assessment algorithm based on access frequency feature vectors, temporal locality features, and application relevance features, resulting in a five-level data heat classification system: super hot, hot, warm, cold, and super cold. An analysis module is used to analyze the physical performance parameters of the SSD storage medium and process the flash block characteristics to obtain a multi-level storage area division based on performance differences and a flash block performance mapping relationship; An allocation module is configured to perform intelligent storage resource allocation processing on the small object metadata according to the five-level data heat level classification and the multi-level storage area classification, and obtain an optimal matching solution for metadata components and a storage cost optimization result based on the flash block performance mapping relationship; The adjustment module is used to dynamically adjust the data distribution status of each storage area in the optimal matching solution of the metadata component through adaptive load monitoring to obtain balanced storage load distribution data based on the five-level data heat level division.

[0060] above Figure 2 The data storage optimization system for solid-state hard disks in an embodiment of the present invention is described in detail from the perspective of modular functional entities. The data storage optimization device for solid-state hard disks in an embodiment of the present invention is described in detail from the perspective of hardware processing.

[0061] Reference Figure 3 In an embodiment of the present invention, a data storage optimization device for a solid state hard disk is also provided. The data storage optimization device for a solid state hard disk can be a server, and its internal structure can be as follows: Figure 3 As shown. The data storage optimization device for solid-state hard disks includes a processor, a memory, a display screen, an input device, a network interface and a database connected via a system bus. Among them, the computer-designed processor is used to provide computing and control capabilities. The memory of the data storage optimization device for solid-state hard disks includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the data storage optimization device for solid-state hard disks is used to store the corresponding data in this embodiment. The network interface of the data storage optimization device for solid-state hard disks is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0062] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the data storage optimization device for a solid-state hard disk to which the solution of the present invention is applied.

[0063] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, cause the computer to execute the steps of the data storage optimization method for a solid-state hard drive.

[0064] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0065] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a data storage optimization device for a solid-state hard drive (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0066] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data storage optimization method for a solid state drive, characterized in that: The method comprises: Through multi-dimensional access feature extraction, the access behavior data of small object metadata is processed for pattern recognition to obtain access frequency feature vectors, time locality features and application correlation features; According to the access frequency feature vector, time locality feature and application correlation feature, data heat quantification is performed using a flash memory-aware heat evaluation algorithm to obtain a five-level data heat level classification including super hot, hot, warm, cold and super cold; The physical performance parameters of the SSD storage medium are analyzed and processed for flash block characteristics to obtain a multi-level storage area division based on performance differences and a flash block performance mapping relationship; Intelligently allocate storage resources for small object metadata based on the five-level data heat level division and the multi-level storage area division, and obtain an optimal matching solution for metadata components and a storage cost optimization result based on the flash block performance mapping relationship; The data distribution status of each storage area in the optimal matching solution of the metadata component is dynamically adjusted through adaptive load monitoring to obtain balanced storage load distribution data based on the five-level data heat level division.

2. The data storage optimization method for a solid state drive according to claim 1, wherein: The multi-dimensional access feature extraction is used to perform pattern recognition processing on the access behavior data of the small object metadata to obtain the access frequency feature vector, time locality feature and application correlation feature, including: The access timestamp, access operation type, and access source application identifier of small object metadata are monitored, collected, and processed in real time to obtain the original access behavior dataset; The original access behavior data set is segmented into time series according to the preset time window length to obtain access statistics and access frequency distribution matrix within multiple time windows; Performing quantitative analysis on the access patterns in each time window by frequency density calculation based on the access number distribution matrix to obtain the access frequency feature vector and access interval variation coefficient; The small object metadata from the same application source is subjected to correlation mining processing according to the access frequency feature vector and the access interval variation coefficient to obtain the time locality feature and the application correlation feature.

3. The data storage optimization method for a solid state drive according to claim 1, wherein: The data heat quantification processing is performed by using a flash memory-aware heat evaluation algorithm based on the access frequency feature vector, the time locality feature, and the application correlation feature to obtain a five-level data heat level classification including super hot, hot, warm, cold, and super cold, including: Performing logarithmic normalization on the access count data in the access frequency feature vector to obtain a frequency score and a corresponding frequency weight factor; Based on the time locality feature, a time decay score is calculated and processed on the time difference between the current time and the last access time by exponential decay calculation to obtain a time score and a time weight factor reflecting the freshness of the access time; Performing stability quantification processing on the small object metadata access patterns of the same application source according to the application correlation characteristics to obtain an application access stability score and an application weight factor; The frequency score, time score and application access stability score are weighted and summed according to the corresponding weight factors to obtain the comprehensive heat value of the small object metadata and the five-level data heat level classification based on the heat threshold interval, including super hot, hot, warm, cold and super cold.

4. The data storage optimization method for a solid state drive according to claim 3, wherein: The stability quantification processing of the small object metadata access pattern from the same application source according to the application correlation feature to obtain the application access stability score and the application weight factor includes: The variance of the access interval time series of the metadata of small objects from the same application source is calculated to obtain the access interval variance value and the access rhythm stability index. The access interval variance value is calculated by normalizing the coefficient of variation to perform quantitative analysis on the discrete degree of the access pattern, so as to obtain the access pattern coefficient of variation and a pattern stability benchmark value; Performing a correlation evaluation process on the access regularity of the small object metadata based on the access rhythm stability index and the pattern stability benchmark value to obtain an application access regularity score and an access prediction credibility value; The application access behavior is comprehensively evaluated and processed through stability weighted calculation according to the application access regularity score and the access prediction credibility value to obtain the application access stability score and the application weight factor based on the stability gradient.

5. The data storage optimization method for a solid state drive according to claim 1, wherein: The physical performance parameters of the SSD storage medium are subjected to flash block characteristic analysis to obtain a multi-level storage area division based on performance differences and a flash block performance mapping relationship, including: Perform performance testing and collection of the read latency, write latency, and erase latency of each flash memory block in the SSD storage medium to obtain a flash memory block latency performance dataset and an IO response time benchmark value. Performing durability evaluation calculations on the flash memory block delay performance data set in combination with the current P / E cycle number and the maximum P / E cycle limit to obtain a flash memory block durability index and a remaining life expectancy value; Based on the IO response time benchmark value and the flash memory block durability index, each flash memory block is classified into a performance level by a comprehensive performance weight score to obtain flash memory block classification results of a high performance area, a standard performance area, an energy-saving area, and a spare area; The SSD storage space is subjected to regional boundary demarcation processing according to the flash memory block classification result, so as to obtain the multi-level storage area division based on performance difference and the flash memory block performance mapping relationship between each flash memory block and the storage area.

6. The data storage optimization method for a solid state drive according to claim 1, wherein: The intelligent storage resource allocation processing of small object metadata according to the five-level data heat level division and the multi-level storage area division is performed to obtain the optimal matching solution of metadata components and the storage cost optimization result based on the flash block performance mapping relationship, including: Classify the metadata of small objects into metadata components according to basic attributes, extended attributes, index information and access control information, and obtain the data size and access feature identifier of each metadata component; Based on the five-level data heat level classification, heat level matching analysis processing is performed on each metadata component to obtain grouping results of super hot data components, hot data components, warm data components, cold data components and super cold data components; Performing storage area allocation processing on each data component by minimizing storage cost calculation according to the grouping results and the multi-level storage area division, thereby obtaining a high-performance area allocation plan, a standard performance area allocation plan, an energy-saving area allocation plan, and a spare area allocation plan; Each area allocation scheme is combined with the flash block performance mapping relationship to perform resource matching optimization processing, thereby obtaining the optimal matching scheme for metadata components based on the flash block performance mapping relationship and the storage cost optimization result including storage cost, access delay and space utilization.

7. The data storage optimization method for a solid state drive according to claim 1, wherein: The method of dynamically adjusting the data distribution status of each storage area in the optimal matching solution of the metadata component through adaptive load monitoring to obtain balanced storage load distribution data based on the five-level data heat level division includes: Real-time monitoring, collection, and processing of data capacity occupancy, access frequency changes, and heat level migration in each storage area are performed to obtain storage area load status data and heat change trigger conditions; Comparing and analyzing the storage area load status data with a preset load balancing threshold to obtain an identifier of a load imbalance area and a list of data components that need to be migrated; Re-evaluate the heat level of the metadata of the small objects in the data component list based on the heat change trigger condition to obtain updated heat level allocation and data migration priority ranking; According to the updated heat level allocation and data migration priority ranking, the small object metadata is migrated across regions through a load redistribution algorithm to obtain balanced storage load distribution data based on the five-level data heat level division and optimized storage area utilization.

8. A data storage optimization system for a solid state drive, characterized in that: For implementing the data storage optimization method for a solid-state hard disk according to any one of claims 1 to 7, the data storage optimization system for a solid-state hard disk comprises: The recognition module is used to perform pattern recognition processing on the access behavior data of small object metadata through multi-dimensional access feature extraction to obtain access frequency feature vectors, time locality features and application correlation features; a quantification module for performing data heat quantification processing using a flash memory-aware heat evaluation algorithm based on the access frequency feature vector, the temporal locality feature, and the application correlation feature, to obtain a five-level data heat classification including super hot, hot, warm, cold, and super cold; An analysis module is used to analyze the physical performance parameters of the SSD storage medium and process the flash block characteristics to obtain a multi-level storage area division based on performance differences and a flash block performance mapping relationship; An allocation module is configured to perform intelligent storage resource allocation processing on the small object metadata according to the five-level data heat level classification and the multi-level storage area classification, and obtain an optimal matching solution for metadata components and a storage cost optimization result based on the flash block performance mapping relationship; The adjustment module is used to dynamically adjust the data distribution status of each storage area in the optimal matching solution of the metadata component through adaptive load monitoring to obtain balanced storage load distribution data based on the five-level data heat level division.

9. A data storage optimization device for a solid state drive, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the method for optimizing data storage for a solid state drive according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the processor is caused to perform the data storage optimization method for a solid state drive according to any one of claims 1 to 7.

Images