Data storage optimization method and system for solid state drives

Abstract

The application relates to the technical field of data processing, and discloses a data storage optimization method and system for a solid state disk. The method comprises the following steps: performing pattern recognition on small object metadata through multi-dimensional access feature extraction, performing five-level heat level division based on a flash memory sensing heat degree evaluation algorithm, dividing the SSD into multi-level storage areas, intelligently allocating resources according to the heat levels and the storage areas, and dynamically adjusting the processing through adaptive load monitoring. The application solves the technical problem of how to intelligently separate hot and cold small object metadata and dynamically store the small object metadata in layers to reduce the SSD storage cost in a massive small object storage environment generated by cloud native application.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a data storage optimization method and system for a solid state disk. BACKGROUND

[0002] Existing object storage systems have been widely used in cloud environments. Traditionally, these systems use a hybrid storage architecture that stores object data on hard disk drives (HDD) and object metadata on solid state disks (SSD). This design is based on the assumption that the size of object metadata is much smaller than the size of object data. By placing frequently accessed metadata on high-performance SSDs, good I / O performance is achieved, and by storing large-capacity object data on low-cost HDDs, overall storage costs are controlled. Existing SSD storage optimization methods mainly use simple LRU cache algorithms and static wear leveling strategies to store all metadata on SSDs, and perform storage management through basic hot and cold data separation.

[0003] However, with the popularity of cloud-native applications, a large number of applications have begun to organize data into the form of a large number of small objects. The data size of these small objects is often close to or even smaller than the size of their corresponding metadata, which significantly increases the storage cost of the traditional scheme of storing all metadata on expensive SSDs. The SSD storage optimization methods in the prior art have significant shortcomings: first, the cost control is not fine enough, and it cannot effectively handle scenarios where small object metadata size is close to or exceeds the data itself; second, the hot and cold data classification is not accurate enough, and existing methods are often based on simple access frequency statistics, making it difficult to accurately predict future access patterns of data; third, there is a lack of specialized optimization for small object storage scenarios, and traditional optimization methods perform poorly when faced with a large number of small objects; and finally, the dynamic adjustment capability is limited, and when the workload pattern changes, the storage strategy cannot be quickly responded to and adjusted.

[0004] Based on these limitations of the prior art, further analysis finds that in the environment of storing a large number of small objects, more in-depth technical problems need to be solved: how to accurately identify the multi-dimensional access characteristics of small object metadata and establish an accurate hotness evaluation model, how to intelligently manage partitions according to the physical characteristics of SSD flash blocks, how to perform fine-grained component-level hierarchical storage of small object metadata, and how to establish an adaptive load monitoring and dynamic migration mechanism to respond to real-time changes in access patterns. The solution to these problems requires systematic technical innovation from multiple dimensions, including metadata access pattern analysis, SSD physical characteristics utilization, intelligent allocation of storage resources, and dynamic load balancing. SUMMARY

[0005] The application provides a data storage optimization method and system for a solid state disk, which is used to solve the technical problem of how to intelligently separate hot and cold small object metadata and dynamically store them in layers to reduce the storage cost of the SSD in a massive small object storage environment generated by cloud-native applications.

[0006] In a first aspect, the application provides a data storage optimization method for a solid state disk, which comprises: performing pattern recognition processing on the access behavior data of small object metadata through multi-dimensional access feature extraction to obtain an access frequency feature vector, a time locality feature, and an application correlation feature; performing data heat quantization processing through a flash memory-aware heat evaluation algorithm according to the access frequency feature vector, the time locality feature, and the application correlation feature to obtain a five-level data heat level division including super-hot, hot, warm, cold, and super-cold; performing flash block characteristic analysis processing on the physical performance parameters of the SSD storage medium to obtain a multi-level storage area division based on performance differences and a flash block performance mapping relationship; performing intelligent storage resource allocation processing on the small object metadata according to the five-level data heat level division and the multi-level storage area division to obtain a metadata component optimal matching scheme and a storage cost optimization result based on the flash block performance mapping relationship; and performing dynamic adjustment processing on the data distribution state of each storage area in the metadata component optimal matching scheme through adaptive load monitoring to obtain balanced storage load distribution data based on the five-level data heat level division.

[0007] Optionally, the pattern recognition processing on the access behavior data of small object metadata through multi-dimensional access feature extraction to obtain an access frequency feature vector, a time locality feature, and an application correlation feature comprises:

[0008] Real-time monitoring and collection processing are performed on the access timestamp, access operation type, and access source application identifier of the small object metadata to obtain an original access behavior data set;

[0009] Time series segmentation processing is performed on the original access behavior data set according to a preset time window length to obtain access statistical data and an access frequency distribution matrix in multiple time windows;

[0010] Quantitative analysis processing is performed on the access mode in each time window based on the access frequency distribution matrix through frequency density calculation to obtain the access frequency feature vector and an access interval coefficient of variation;

[0011] Correlation mining processing is performed on the small object metadata of the same application source according to the access frequency feature vector and the access interval coefficient of variation to obtain the time locality feature and the application correlation feature.

[0012] Optionally, the data hotness quantification processing is performed on the access frequency feature vector, the time locality feature, and the application correlation feature by a flash perception hotness evaluation algorithm to obtain a five-level data hotness grade division including super-hot, hot, warm, cold, and super-cold, which includes:

[0013] The access frequency data in the access frequency feature vector is subjected to logarithmic normalization processing to obtain a frequency score and a corresponding frequency weight factor;

[0014] The time difference between the current time and the last access time is subjected to time decay score calculation processing based on the time locality feature to obtain a time score reflecting the freshness of the access time and a time weight factor;

[0015] The application access stability score and the application weight factor are obtained by performing stability quantification processing on the access pattern of small object metadata of the same application source according to the application correlation feature;

[0016] The frequency score, the time score, and the application access stability score are subjected to weighted summation calculation processing according to the corresponding weight factors to obtain a comprehensive hotness value of the small object metadata and the five-level data hotness grade division including super-hot, hot, warm, cold, and super-cold based on the hotness threshold interval.

[0017] Optionally, the application access stability score and the application weight factor are obtained by performing stability quantification processing on the access pattern of small object metadata of the same application source according to the application correlation feature, which includes:

[0018] The access interval time series of small object metadata of the same application source is subjected to variance calculation processing to obtain an access interval variance value and an access beat stability index;

[0019] The access pattern dispersion degree is quantitatively analyzed by the access interval variance value through coefficient of variation normalization calculation to obtain a coefficient of variation of the access pattern and a pattern stability benchmark value;

[0020] The access regularity of small object metadata is subjected to correlation evaluation processing based on the access beat stability index and the pattern stability benchmark value to obtain an application access regularity score and an access prediction confidence value;

[0021] The application access behavior is comprehensively evaluated by stability weighted calculation according to the application access regularity score and the access prediction confidence value to obtain the application access stability score and the application weight factor based on the stability gradient.

[0022] Optionally, the flash block characteristic analysis processing of the physical performance parameters of the SSD storage medium obtains the multi-level storage area division based on performance differences and the flash block performance mapping relationship, including:

[0023] The performance test collection and processing are performed on the read delay, write delay, and erase delay of each flash block in the SSD storage medium to obtain a flash block delay performance dataset and an IO response time reference value;

[0024] The durability evaluation and calculation processing are performed on the flash block delay performance dataset in combination with the current P / E cycle number and the maximum P / E cycle limit to obtain a flash block durability index and a remaining life expectancy value;

[0025] The performance level classification processing is performed on each flash block based on the IO response time reference value and the flash block durability index through performance weight comprehensive scoring to obtain a flash block classification result of a high-performance area, a standard performance area, an energy-saving area, and a standby area;

[0026] The region boundary demarcation processing is performed on the SSD storage space according to the flash block classification result to obtain the multi-level storage area division based on performance differences and the flash block performance mapping relationship of each flash block and a storage area.

[0027] Optionally, the intelligent storage resource allocation processing of small object metadata is performed according to the five-level data heat level division and the multi-level storage area division to obtain a metadata component optimal matching scheme based on the flash block performance mapping relationship and a storage cost optimization result, including:

[0028] The metadata component classification processing is performed on small object metadata according to basic attributes, extended attributes, index information, and access control information to obtain data sizes and access feature identifiers of each metadata component;

[0029] The heat level matching analysis processing is performed on each metadata component based on the five-level data heat level division to obtain a grouping result of super-hot data components, hot data components, warm data components, cold data components, and super-cold data components;

[0030] The storage area allocation processing is performed on each data component according to the grouping result and the multi-level storage area division through storage cost minimization calculation to obtain a high-performance area allocation scheme, a standard performance area allocation scheme, an energy-saving area allocation scheme, and a standby area allocation scheme;

[0031] The resource matching optimization processing is performed on each area allocation scheme in combination with the flash block performance mapping relationship to obtain a metadata component optimal matching scheme based on the flash block performance mapping relationship and the storage cost optimization result including storage cost, access delay, and space utilization.

[0032] Optionally, the step of dynamically adjusting the data distribution status of each storage area in the optimal matching scheme of the metadata component through adaptive load monitoring to obtain balanced storage load distribution data based on the five-level data heat level division includes:

[0033] Real-time monitoring and data collection and processing are performed on the data capacity utilization, access frequency changes and popularity level migration of each storage area to obtain storage area load status data and popularity change trigger conditions.

[0034] The load status data of the storage area is compared and analyzed with the preset load balancing threshold to obtain the load imbalance area identifier and the list of data components that need to be migrated.

[0035] Based on the heat change triggering conditions, the heat re-evaluation process is performed on the metadata of small objects in the data component list to obtain the updated heat level allocation and data migration priority ranking.

[0036] Based on the updated heat level allocation and data migration priority sorting, the small object metadata is migrated across regions using a load redistribution algorithm to obtain balanced storage load distribution data based on the five-level data heat level division and optimized storage area utilization.

[0037] Secondly, this application provides a data storage optimization system for solid-state drives (SSDs), the data storage optimization system for SSDs comprising:

[0038] The identification module is used to perform pattern recognition processing on the access behavior data of small object metadata through multi-dimensional access feature extraction, and obtain access frequency feature vector, temporal locality feature and application relevance feature.

[0039] The quantization module is used to perform data heat quantification processing based on the access frequency feature vector, temporal locality feature and application correlation feature through the flash memory sensing heat evaluation algorithm to obtain a five-level data heat level classification including super hot, hot, warm, cold and super cold.

[0040] The analysis module is used to analyze the physical performance parameters of the SSD storage medium using flash block characteristics, and to obtain a multi-level storage area division based on performance differences and a flash block performance mapping relationship.

[0041] The allocation module is used to intelligently allocate storage resources to small object metadata based on the five-level data heat level division and the multi-level storage area division, so as to obtain the optimal matching scheme of metadata components and storage cost optimization results based on the flash block performance mapping relationship.

[0042] An adjusting module is configured to dynamically adjust the data distribution state of each storage area in the optimal matching scheme of the metadata component through adaptive load monitoring, and obtain balanced storage load distribution data based on the five-level data hotness level division.

[0043] In a third aspect, a data storage optimization device for a solid state disk is provided, comprising a memory and at least one processor, the memory storing instructions; the at least one processor invoking the instructions in the memory to cause the data storage optimization device for a solid state disk to perform the data storage optimization method for a solid state disk described above.

[0044] In a fourth aspect, a computer readable storage medium is provided, the computer readable storage medium storing instructions, when running on a computer, causing the computer to perform the data storage optimization method for a solid state disk described above.

[0045] In the technical scheme provided in the present application, the small object metadata is subjected to pattern recognition processing through multi-dimensional access feature extraction, which can accurately capture key information such as access timestamp, operation type and application source of the small object, and compared with the traditional single-dimensional analysis method based on access frequency, the multi-dimensional feature extraction significantly improves the accuracy of access pattern recognition, making the subsequent hotness evaluation more reliable. The flash-aware hotness evaluation algorithm fully considers the physical characteristics of SSD flash storage, and weights and fuses the access frequency feature vector, time locality feature and application correlation feature to form a special hotness quantification method for small object metadata characteristics. The algorithm effectively solves the data skew and time sensitivity problems in the traditional hotness evaluation through logarithmic normalization processing and exponential decay calculation, and the five-level data hotness level division generated provides an accurate data classification basis for subsequent hierarchical storage. The flash block characteristic analysis processing of the SSD storage medium fully utilizes the differences in read / write performance and durability of the flash block, and establishes a multi-level storage area division based on performance difference by comprehensively evaluating the delay performance and P / E cycle state of each flash block. Compared with the traditional uniform allocation strategy, this physical characteristic-aware partitioning method can better take advantage of the differentiation of internal resources of the SSD, significantly improving the utilization efficiency of storage resources. The intelligent storage resource allocation processing manages the small object metadata in a fine-grained component level according to the basic attributes, extended attributes, index information and access control information, realizes more fine-grained cost control than the traditional overall storage, and effectively reduces the waste of high-performance SSD storage space by matching metadata components of different hotness levels to corresponding performance flash block areas, while ensuring the access performance of hot data.

[0046] The adaptive load monitoring and dynamic adjustment process establishes an intelligent management mechanism that responds to access mode changes in real time. By continuously monitoring the capacity occupancy rate, access frequency changes and hotness level migration of each storage area, the load imbalance problem can be detected in time and the corresponding data migration operation is triggered. This dynamic adjustment capability has stronger adaptability and flexibility compared with the traditional static storage allocation method. Especially in the cloud native application scenario, when the access mode of the application changes suddenly, the scheme can quickly re-evaluate the hotness level of the small object metadata and migrate the data to the most suitable storage area through the load redistribution algorithm, so as to always maintain the optimal performance cost balance. The whole scheme is specially optimized for the specific scenario that the small object metadata size is close to or exceeds the data itself. Through multi-level technical innovation, it effectively solves the deficiencies of traditional SSD storage optimization methods in cost control, accurate classification, dynamic adaptation, etc., and provides a more economical and efficient small object metadata management solution for cloud storage services. In practical application, it can significantly reduce the SSD storage cost while maintaining good access performance and system stability. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained based on these drawings without creative labor.

[0048] Figure 1 An embodiment schematic diagram of the data storage optimization method for a solid state disk in the embodiments of the present application;

[0049] Figure 2 An embodiment schematic diagram of the data storage optimization system for a solid state disk in the embodiments of the present application;

[0050] Figure 3 An embodiment schematic diagram of the data storage optimization system for a solid state disk in the embodiments of the present application; DETAILED DESCRIPTION

[0051] The embodiment of the present application provides a data storage optimization method and system for a solid state disk. The terms "first", "second", "third", "fourth" and the like (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" or "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0052] For ease of understanding, the specific process of the embodiment of the present application is described below. Please refer to Figure 1 One embodiment of the data storage optimization method for a solid state disk in the embodiment of the present application comprises the following steps.

[0053] Step S101, mode recognition processing is performed on the access behavior data of the small object metadata by multi-dimensional access feature extraction, to obtain an access frequency feature vector, a time locality feature and an application correlation feature;

[0054] Step S102, data heat quantization processing is performed on the access frequency feature vector, the time locality feature and the application correlation feature by a flash memory perception heat evaluation algorithm, to obtain a five-level data heat level division including super-hot, hot, warm, cold and super-cold;

[0055] Step S103, flash memory block characteristic analysis processing is performed on the physical performance parameters of the SSD storage medium, to obtain a multi-level storage area division based on performance difference and a flash memory block performance mapping relationship;

[0056] Step S104, intelligent storage resource allocation processing is performed on the small object metadata according to the five-level data heat level division and the multi-level storage area division, to obtain a metadata component optimal matching scheme and a storage cost optimization result based on the flash memory block performance mapping relationship;

[0057] Step S105, dynamic adjustment processing is performed on the data distribution state of each storage area in the metadata component optimal matching scheme by adaptive load monitoring, to obtain balanced storage load distribution data based on the five-level data heat level division.

[0058] It can be understood that the execution subject of the present application can be a data storage optimization system for a solid state disk, and can also be a terminal or a server, and the specific place is not limited. The embodiment of the present application takes a server as an execution subject for example.

[0059] Specifically, a small object metadata monitoring mechanism is established to collect raw data such as access time stamp, access operation type (read, write, delete, modify) and access source application identifier of each small object in real time, forming a raw access behavior data set. Then the raw access behavior data set is time series segmented according to a fixed time window length, for example, each time window is set to 1 hour, and the access data within 24 hours is segmented into 24 time windows, and the access frequency in each time window is counted to generate an access frequency distribution matrix. The access mode in each time window is quantitatively analyzed by frequency density calculation, and the access density value of each time window is calculated as the access frequency in the window divided by the time window length, forming an access frequency feature vector. At the same time, the ratio of the standard deviation to the average value of the access interval is calculated to obtain the access interval coefficient of variation. Finally, the small object metadata of the same application source is associated and mined, the similarity of the access behavior of the same application in different time periods is analyzed, and the time locality feature and application correlation feature are extracted.

[0060] According to the obtained access frequency feature vector, time locality feature and application correlation feature, data hotness quantization processing is performed through a flash perception hotness evaluation algorithm. The algorithm first performs logarithmic normalization processing on the access frequency data in the access frequency feature vector, calculates the frequency score equal to the logarithm of the access frequency plus 1 divided by the maximum logarithm value, ensures that the score is between 0 and 1, and assigns the corresponding frequency weight factor. Based on the time locality feature, the time decay score is calculated through an exponential decay function, which is equal to the negative λ times the time difference power of the natural constant, where λ is the decay coefficient and the time difference is the difference between the current time and the last access time, reflecting the access time freshness, and assigning the time weight factor. For the application correlation feature, the access interval time series variance of the small object metadata of the same application source is calculated to obtain the access interval variance value, and the variation coefficient is normalized to obtain the access pattern variation coefficient and the pattern stability benchmark value, and finally the application access stability score and the application weight factor are obtained. The frequency score, time score and application access stability score are weighted and summed according to the corresponding weight factor to obtain the comprehensive hotness value of the small object metadata, and according to the preset hotness threshold interval, it is divided into five levels of super hot, hot, warm, cold and super cold.

[0061] The physical performance parameters of the SSD storage medium are analyzed and processed for flash block characteristics, the performance of each flash block in the SSD storage medium is tested, key performance indicators such as read delay, write delay and erase delay are collected, a flash block delay performance dataset is formed, and the average response time of each operation is recorded as an IO response time benchmark value. The flash block delay performance dataset is combined with the current P / E cycle number and the maximum P / E cycle limit, the flash block durability index is calculated as the ratio of the current P / E cycle number to the maximum P / E cycle limit, and the remaining life expectancy value is calculated as the maximum P / E cycle limit minus the current P / E cycle number. By comprehensively considering the IO response time benchmark value and the flash block durability index, the performance weight comprehensive score is calculated, each flash block is classified into four performance levels: high performance area, standard performance area, energy saving area and standby area, and finally the SSD storage space is divided according to the flash block classification results. The performance mapping relationship between each flash block and the storage area is established.

[0062] According to the five-level data heat level division and the multi-level storage area division, the small object metadata is intelligently stored and resource allocation processed. First, the small object metadata is classified according to basic attributes (object size, creation time, modification time), extended attributes (user-defined tags, description information), index information (quick retrieval of related data) and access control information (permission settings, security tags), and the data size and access feature identifier of each component are recorded. Based on the five-level data heat level division, each metadata component is matched to the corresponding heat level to form the grouping results of super-hot data components, hot data components, warm data components, cold data components and super-cold data components. Through storage cost minimization calculation, super-hot and hot data components are allocated to the high performance area, warm data components are allocated to the standard performance area, cold data components are allocated to the energy saving area, and super-cold data components are allocated to the standby area to form the allocation scheme of each area. The resource matching optimization is carried out in combination with the flash block performance mapping relationship to ensure that high-heat data is allocated to the flash block with the best performance, and the optimal matching scheme of the metadata component and the storage cost optimization result containing storage cost, access delay and space utilization are obtained.

[0063] The data distribution state of each storage area in the optimal matching scheme of the metadata component is dynamically adjusted by adaptive load monitoring. The monitoring mechanism collects the data capacity occupancy rate, access frequency change and hotness level migration of each storage area in real time to form storage area load state data. When the capacity occupancy rate of a certain storage area exceeds the preset threshold of 85% or is less than 15%, load balancing adjustment is triggered. The storage area load state data is compared and analyzed with the preset load balancing threshold to identify the unbalanced load area and generate a data component list that needs to be migrated. Based on the hotness change trigger condition, the small object metadata in the data component list is re-evaluated for hotness. When the access mode of the small object changes significantly, the hotness level allocation is updated, and the data migration priority is determined according to the urgency and data size. Through the load redistribution algorithm, cross-area data migration is performed to migrate data with rising hotness from low-performance areas to high-performance areas and data with declining hotness from high-performance areas to low-performance areas, finally forming balanced storage load distribution data based on five-level data hotness level division.

[0064] In a specific embodiment, the process of performing step S101 can specifically include the following steps:

[0065] The access time stamp, access operation type and access source application identifier of the small object metadata are monitored and collected in real time to obtain an original access behavior data set;

[0066] The original access behavior data set is time series segmented according to a preset time window length to obtain access statistical data and an access frequency distribution matrix in multiple time windows;

[0067] The access mode in each time window is quantitatively analyzed based on the access frequency distribution matrix through frequency density calculation to obtain an access frequency feature vector and an access interval coefficient of variation;

[0068] The small object metadata of the same application source are associated based on the access frequency feature vector and the access interval coefficient of variation to obtain time locality features and application correlation features.

[0069] Specifically, when monitoring and collecting the access timestamp, access operation type and access source application identifier of small object metadata in real time, a monitoring hook function is deployed at the SSD storage controller level. When any application initiates an access request for small object metadata, the monitoring hook immediately captures the access event and records the precise timestamp information. The timestamp is in the format of Unix timestamp with nanosecond precision. The access operation type is divided into four basic operations: read, write, delete and modify. The specific operation type is determined by analyzing the operation code of the IO instruction. The access source application identifier is obtained by analyzing the process ID and application signature to obtain a unique application identifier. The collected data is stored in a ring buffer in chronological order to form an original access behavior dataset. Each record contains four fields: object ID, timestamp, operation type and application identifier.

[0070] When the original access behavior dataset is divided into time series according to the preset time window length, the time window length is first determined as a fixed time interval, for example, 30 minutes. Then, starting from the start time of the original access behavior dataset, the continuous access records are divided into different time windows according to the time window length. Each time window contains all the access records of small object metadata in that time period. The access frequency of each small object in each time window is counted to form access statistical data. Then, an access frequency distribution matrix is constructed. The rows of the matrix represent different small objects, the columns represent different time windows, and the values in the matrix represent the access frequency of the corresponding small object in the corresponding time window. In this way, the originally discrete access records are converted into structured matrix data.

[0071] When the access frequency distribution matrix is quantitatively analyzed by frequency density calculation, the frequency density calculation means that the access frequency in each time window is divided by the length of the time window to obtain the average access frequency per unit time. For each small object, a vector is formed by the frequency density values in all time windows, which is the access frequency feature vector. This vector reflects the access activity level of the small object in different time periods. At the same time, the access interval coefficient of variation is calculated. The access interval refers to the time difference between two consecutive accesses of the same small object. The coefficient of variation is equal to the standard deviation of the access interval divided by the average value of the access interval. The smaller the coefficient of variation, the more regular the access pattern. The larger the coefficient of variation, the more irregular the access pattern. By traversing each small object in the access frequency distribution matrix, the coefficient of variation of the access interval sequence is calculated.

[0072] When performing relevance mining processing on small object metadata from the same application source according to the access frequency feature vector and the access interval coefficient of variation, first, the small objects are grouped according to the application identifier, and the small objects from the same application are classified together, then the correlation between the access frequency feature vectors of the small objects in the same application group is calculated, the correlation degree is quantified by calculating the Pearson correlation coefficient between the vectors, the correlation coefficient close to 1 indicates that the access mode is highly similar, and the correlation coefficient close to 0 indicates that the access mode is irrelevant, the time locality feature is obtained by analyzing the access time distribution of the small objects in the same application group, the concentration and dispersion of the access time of each small object are calculated, the high concentration indicates that the access time locality is strong, and the application relevance feature is obtained by calculating the consistency of the access mode of the small objects in the same application group, the high consistency indicates that the application has strong relevance to the access of these small objects.

[0073] In a specific embodiment, the process of performing step S102 can specifically include the following steps:

[0074] The access frequency data in the access frequency feature vector is logarithmically normalized to obtain a frequency score and a corresponding frequency weight factor;

[0075] Based on the time locality feature, the time difference between the current time and the last access time is calculated by exponential decay to obtain a time decay score, and a time score reflecting the freshness of the access time and a time weight factor are obtained;

[0076] According to the application relevance feature, the access mode of the small object metadata from the same application source is quantified to obtain an application access stability score and an application weight factor;

[0077] The frequency score, the time score and the application access stability score are weighted and summed according to the corresponding weight factors to obtain a comprehensive hotness value of the small object metadata and a five-level data hotness level division based on the hotness threshold interval, including super-hot, hot, warm, cold and super-cold.

[0078] Specifically, when the access frequency feature vector is logarithmically normalized, the access frequency of each element in the access frequency feature vector is first taken as the natural logarithm after being added by 1. This processing is to prevent the case of meaningless logarithm when the access frequency is 0, and to compress the dynamic range of the data. Then, the maximum value in all logarithmic values is taken as the normalization reference, and each logarithmic value is divided by the maximum logarithmic value to obtain a frequency score between 0 and 1. The frequency weight factor is determined according to the overall distribution of the access frequency. The higher the access frequency, the higher the weight factor of the small object metadata. The specific calculation method is to divide the access frequency of the small object by the sum of the access frequencies of all small objects, and then multiply by a preset weight adjustment coefficient. The frequency score reflects the access activity of the small object metadata, and the frequency weight factor is used to balance small objects with different access intensities in subsequent weighted calculation.

[0079] When the time decay score is calculated by exponential decay based on the time locality feature, the time decay score is calculated using an exponential decay function. The basic form of the function is a negative exponential function with the natural constant e as the base. The decay coefficient in the exponential decay function determines the speed of time decay. The larger the decay coefficient, the stronger the time sensitivity, and the faster the time score decreases. The calculation process first obtains the current system time and the last access time of the small object metadata, calculates the time difference between the two, and then multiplies the time difference by a preset decay coefficient as the index. The negative exponential power of the natural constant e is calculated to obtain a time score between 0 and 1. The closer the time score is to 1, the closer the last access time is to the current time. The time weight factor is assigned according to the distribution of the time score. Small object metadata with a high time score obtains a higher time weight factor. The calculation method is to divide the time score by the average of all small object time scores, and then multiply by a time weight adjustment coefficient.

[0080] When the stability quantification is performed on the small object metadata access patterns of the same application source according to the application correlation feature, the stability quantification refers to evaluating the consistency degree of the access behavior of the same application to its related small object metadata. First, the variance of the time interval sequence of each small object access in the same application group is calculated. The smaller the variance, the more stable the access interval. Then, the average value of the access interval is calculated. The variance is divided by the square of the average value to obtain the coefficient of variation. The smaller the coefficient of variation, the more stable the access pattern. Then, the reciprocal of the coefficient of variation is calculated to obtain the access beat stability index. The higher the stability index, the more regular the access pattern. The mode stability benchmark value is obtained by statistical analysis of the coefficients of variation of all application groups. Generally, the median is taken as the benchmark value. The application access regularity score is calculated according to the comparison result of the stability index and the benchmark value. When the stability index exceeds the benchmark value, a high score is given, and vice versa. The access prediction confidence value is calculated based on the consistency degree of the historical access pattern. The higher the consistency, the higher the prediction confidence. The application access stability score considers the access regularity score and the prediction confidence value. The application weight factor is determined according to the gradient distribution of the stability score. The application with a higher stability score obtains a higher weight factor.

[0081] When the frequency score, the time score and the application access stability score are weighted and summed according to the corresponding weight factors, the weighted sum calculation adopts a linear weighting method. The three scores are multiplied by the corresponding weight factors, and then added to obtain a comprehensive heat value. The distribution of the weight factors follows the constraint condition that the sum of the frequency weight, the time weight and the application weight is equal to 1. The calculation formula of the comprehensive heat value is the frequency score multiplied by the frequency weight factor plus the time score multiplied by the time weight factor plus the application access stability score multiplied by the application weight factor. The obtained comprehensive heat value is between 0 and 1. The higher the heat value, the more active the small object metadata. The five-level data heat level division is realized by setting heat threshold intervals. The super-hot level corresponds to the interval where the heat value is greater than or equal to 0.8. The hot level corresponds to the interval where the heat value is between 0.6 and 0.8. The warm level corresponds to the interval where the heat value is between 0.4 and 0.6. The cold level corresponds to the interval where the heat value is between 0.2 and 0.4. The super-cold level corresponds to the interval where the heat value is less than 0.2. By comparing the calculated comprehensive heat value with these threshold intervals, the heat level to which each small object metadata belongs is determined.

[0082] Taking thumbnail metadata management of a video processing application as an example, the application generates a large number of video thumbnails, each thumbnail is about 20 KB, and the corresponding metadata is about 16 KB. A certain popular video thumbnail is accessed 120 times in a week. After logarithmic normalization processing, the access frequency 120 plus 1 is taken to the natural logarithm to obtain 4.8. Divide by the maximum logarithmic value 5.2 to obtain a frequency score of 0.92. The frequency weight factor is calculated according to the proportion of the access frequency of the thumbnail in the total access frequency. The last access time of the thumbnail is 6 hours away from the current time, and the time difference 6 hours multiplied by the decay coefficient 0.1 obtains 0.6. Calculate the negative 0.6 power of e to obtain a time score of 0.55. The time weight factor is calculated according to the time score distribution to obtain 0.4. The application relevance analysis finds that the access interval variance of the thumbnails of the same video processing application is 25, the average interval is 8 hours, the coefficient of variation is 0.39, and the stability index is 2.56. The application access stability score is obtained by comprehensive evaluation to obtain 0.75. The application weight factor is 0.25. The final weighted sum calculation obtains a comprehensive heat value of 0.73. According to the heat threshold interval judgment, the thumbnail metadata belongs to the hot level.

[0083] In a specific embodiment, the execution step of performing stability quantization processing on the access mode of small object metadata of the same application source according to the application relevance characteristics can specifically include the following steps:

[0084] The access interval time series of the small object metadata of the same application source is subjected to variance calculation processing to obtain an access interval variance value and an access rhythm stability index.

[0085] The access interval variance value is subjected to variation coefficient normalization calculation to quantitatively analyze the dispersion degree of the access mode, to obtain a mode stability reference value and a mode variation coefficient.

[0086] Based on the access rhythm stability index and the mode stability reference value, the access regularity of the small object metadata is subjected to correlation evaluation processing to obtain an application access regularity score and an access prediction confidence value.

[0087] According to the application access regularity score and the access prediction confidence value, the application access behavior is subjected to comprehensive evaluation processing through stability weighting calculation to obtain an application access stability score and an application weight factor based on a stability gradient.

[0088] Specifically, when performing variance calculation processing on the access interval time sequence of small object metadata of the same application source, first, the access timestamps of all small object metadata under the same application identifier are extracted from the collected access behavior data, these timestamps are arranged in time order to form a time sequence, then the time interval between two consecutive accesses is calculated to form an access interval time sequence, each element in the access interval time sequence represents the time difference between two adjacent accesses, then the access interval time sequence is subjected to variance calculation, the variance calculation process includes calculating the average value of the access interval, then calculating the square of the difference between each access interval and the average value, adding all the square of the difference and dividing by the total number of access intervals minus 1 to obtain the access interval variance value, the access beat stability index is calculated by the reciprocal of the access interval variance value, the smaller the variance value, the more stable the access interval, and the higher the stability index, the stability index reflects the regularity of the application to the small object metadata access.

[0089] When the access interval variance value is normalized by the coefficient of variation to quantitatively analyze the dispersion degree of the access pattern, the coefficient of variation calculation needs to square the access interval variance value to obtain the standard deviation, then divide the standard deviation by the average value of the access interval to obtain the coefficient of variation, the coefficient of variation is a dimensionless relative variation measure index, which eliminates the influence of the absolute value difference of the access interval between different applications, making the access pattern stability of different applications comparable, the smaller the coefficient of variation, the lower the dispersion degree of the access pattern, the more regular the access behavior, the pattern stability benchmark value is obtained by statistical analysis of the coefficients of variation of all applications, the specific method is to collect the coefficients of variation of all applications, calculate the median of these coefficients of variation as the pattern stability benchmark value, the median is more reflective of the central tendency of the data than the average and is not easily affected by extreme values, the benchmark value is used as a reference standard for judging whether the access pattern of an application is stable.

[0090] When the access beat stability index and the pattern stability benchmark value are used to evaluate the regularity of the access to small object metadata, the correlation evaluation determines the regularity degree by comparing the relationship between the access beat stability index and the pattern stability benchmark value, when the stability index is greater than the benchmark value, it indicates that the access pattern of the application is more regular than the average level, and a higher regularity score is given, when the stability index is less than the benchmark value, it indicates that the access pattern is relatively irregular, and a lower regularity score is given, the application access regularity score is calculated using linear mapping, the ratio of the stability index to the benchmark value is mapped to the score interval of 0 to 1, the access prediction confidence value is calculated based on the access regularity score and the consistency degree of the historical access pattern, the consistency degree is quantified by analyzing the similarity of the access pattern in different time periods, the more consistent the access pattern, the higher the prediction confidence of future access behavior, the prediction confidence value is also normalized to the range of 0 to 1.

[0091] In the process of evaluating the application access behavior by the stability weighted calculation according to the application access regularity score and the access prediction confidence value, the stability weighted calculation adopts a weighted average method, the application access regularity score and the access prediction confidence value are weighted and summed according to a preset weight ratio, the weight ratio is determined according to the actual needs of SSD storage optimization, in general, the regularity score weight is 0.6, and the confidence value weight is 0.4, the weighted sum result obtains an application access stability score, which comprehensively reflects the stability degree of the application to the small object metadata access behavior, and the application weight factor calculation method based on the stability gradient is to sort the stability scores of all applications, and divide them into different stability gradients according to the distribution of the stability scores, the applications with high stability scores are classified into high stability gradients, and are allocated higher weight factors, the applications with low stability scores are classified into low stability gradients, and are allocated lower weight factors, and the allocation of the weight factors follows the principle that the stability gradient is positively correlated with the weight factor.

[0092] In a specific embodiment, the process of performing step S103 can specifically include the following steps:

[0093] The read delay, write delay and erase delay of each flash memory block in the SSD storage medium are tested and collected to obtain a flash memory block delay performance data set and an IO response time reference value;

[0094] The flash memory block delay performance data set is combined with the current P / E cycle number and the maximum P / E cycle limit to perform durability evaluation and calculation, to obtain a flash memory block durability index and a remaining life expectancy value;

[0095] Based on the IO response time reference value and the flash memory block durability index, the performance of each flash memory block is classified by a performance weight comprehensive score to obtain a flash memory block classification result of a high performance area, a standard performance area, an energy saving area and a standby area;

[0096] According to the flash memory block classification result, the SSD storage space is regionally demarcated to obtain a multi-level storage region division based on performance difference and a flash memory block performance mapping relationship between each flash memory block and the storage region.

[0097] Specifically, when collecting and processing the performance test of the read delay, write delay and erase delay of each flash block in the SSD storage medium, first, the SSD controller bottom interface is used to directly access each flash block, and the standardized performance test operation is executed. The read delay test is measured by sending a read instruction to each flash block and recording the time difference from the instruction issuance to the data return. The write delay test is obtained by writing a fixed size data block to the flash block and measuring the time required for the write completion. The erase delay test is measured by executing the flash block erase operation and recording the erase completion time. Each delay test is repeated multiple times to ensure data accuracy. The read delay, write delay and erase delay data of all flash blocks are summarized to form a flash block delay performance data set. The data set is organized in a table form, with rows representing different flash blocks and columns representing different types of delay data. The IO response time benchmark value is determined by calculating the weighted average of all flash block delay types. The weighting coefficient is set according to the frequency distribution of each type of operation in actual application. The read operation has the highest frequency and the largest weight, the write operation is next, and the erase operation has the lowest frequency and the smallest weight.

[0098] When the flash block delay performance data set is combined with the current P / E cycle number and the maximum P / E cycle limit for durability evaluation and calculation, the P / E cycle number refers to the cumulative number of program erase operations performed by the flash block. Each flash block has its maximum P / E cycle limit, and the reliability of the flash block will decrease significantly after exceeding the limit. The durability evaluation and calculation first reads the current P / E cycle number of each flash block, then compares it with the maximum P / E cycle limit of the flash block. The flash block durability index is calculated by subtracting the current P / E cycle number from the maximum P / E cycle limit and dividing by the maximum P / E cycle limit. The closer the durability index is to 1, the newer the flash block, and the closer it is to 0, the closer the flash block is to the end of life. The remaining life expectancy value is directly equal to the maximum P / E cycle limit minus the current P / E cycle number. This value represents how many erase operations the flash block can still perform. The durability evaluation and calculation also need to consider the usage history and wear pattern of the flash block. By analyzing the growth trend of the P / E cycle number, the future durability change can be predicted.

[0099] When the performance weight comprehensive score is used to classify and process the performance levels of each flash block based on the IO response time benchmark value and the flash block endurance index, the performance weight comprehensive score uses a multi-factor weighted score model. The model comprehensively considers two key factors of the response speed and the service life of the flash block. The response speed score is calculated by comparing the average IO response time of each flash block with the benchmark value. The shorter the response time of the flash block, the higher the speed score. The endurance score directly uses the endurance index calculated in the foregoing. The comprehensive score is equal to the speed score multiplied by the speed weight coefficient plus the endurance score multiplied by the endurance weight coefficient. The distribution of the weight coefficients is determined according to the strategy requirements of the SSD storage optimization. The speed weight is higher in a high-performance application scenario, and the endurance weight is higher in a long-term storage scenario. The classification threshold is set according to the distribution of the comprehensive score. The flash block with the highest score is classified into the high-performance area, the flash block with the middle-high score is classified into the standard-performance area, the flash block with the middle-low score is classified into the energy-saving area, and the flash block with the lowest score is classified into the standby area. The division of the four performance areas follows the principle that the score intervals do not overlap and completely cover all flash blocks.

[0100] When the region boundary demarcation is performed on the SSD storage space according to the flash block classification result, the region boundary demarcation refers to allocating a continuous address range for each performance area in the logical address space of the SSD. The demarcation process first counts the number of flash blocks included in each performance area, and then allocates an address segment of a corresponding size in the logical address space according to the number ratio. The high-performance area is allocated the optimal address segment, which is usually the area with a smaller starting address. The standard-performance area is allocated a suboptimal address segment. The energy-saving area and the standby area are sequentially allocated the remaining address segments. After the multi-level storage area is divided, the mapping relationship between the flash blocks and the storage areas is established. The mapping relationship records the logical address range and the performance area to which each physical flash block corresponds. The flash block performance mapping relationship is stored in the form of a data structure, which includes the physical address of the flash block, the logical address range, the performance level, the comprehensive score, and other key information. The mapping relationship supports bidirectional query, that is, the flash block can be used to find the corresponding area, and the area can be used to find the included flash block.

[0101] In an embodiment, the process of step S104 can specifically include the following steps:

[0102] The small object metadata is classified and processed into metadata components according to the basic attributes, the extended attributes, the index information, and the access control information, to obtain the data size and the access feature identifier of each metadata component.

[0103] Based on the five-level data heat level division, the heat level matching analysis is performed on each metadata component, to obtain the grouping results of the super-hot data component, the hot data component, the warm data component, the cold data component, and the super-cold data component.

[0104] According to the grouping result and the multi-level storage area division, a storage area allocation processing is performed on each data component through storage cost minimization calculation to obtain a high-performance area allocation scheme, a standard-performance area allocation scheme, an energy-saving area allocation scheme and a backup area allocation scheme.

[0105] The resource matching optimization processing is performed on each area allocation scheme combined with the flash block performance mapping relationship to obtain a metadata component optimal matching scheme based on the flash block performance mapping relationship and a storage cost optimization result including storage cost, access delay and space utilization.

[0106] Specifically, when the small object metadata is classified into metadata components according to basic attributes, extended attributes, index information and access control information, the metadata component classification refers to splitting the complete metadata of the small object into different logical components according to its functions and purposes. The basic attribute component includes the core identification information of the object such as object ID, file size, creation time, modification time, file type and other necessary fields. These information must be read during object access. The extended attribute component includes user-defined tags, description information, version number, notes and other optional fields. These information are mainly used for object management and retrieval. The index information component includes index data such as hash value, keyword index, classification label and other information used for fast positioning and retrieval of the object. The access control information component includes permission settings, security level, encryption parameters, access strategy and other security-related data. The classification processing process analyzes the structured storage format of the small object metadata, classifies each field into the corresponding component according to the pre-defined field mapping rule, and at the same time, counts the data size of each metadata component. The data size is obtained by calculating the total number of bytes of each component containing field. The access feature identifier is determined by analyzing the frequency pattern of each component in the historical access. The basic attribute has the highest access frequency, followed by the extended attribute, and the index information and access control information have relatively low access frequency.

[0107] When the metadata components are analyzed and processed by the heat level matching analysis based on the five-level data heat level classification, the heat level matching analysis refers to applying the overall heat level of the small object calculated in the foregoing to each metadata component of the small object. However, the heat level allocation of different components follows the differentiated access frequency principle. The heat level of the basic attribute component is consistent with the overall heat level of the small object because the basic attribute component has the highest access frequency. The heat level of the extended attribute component is reduced by one level based on the overall heat level. For example, the extended attribute component of a small object with an overall heat level of super-hot is allocated to a heat level of hot. The heat level of the index information component is reduced by two levels, and the heat level of the access control information component is reduced by two levels. Through the hierarchical matching mechanism, each metadata component is classified into the grouping results of the super-hot data component, the hot data component, the warm data component, the cold data component, and the super-cold data component. The grouping results are recorded in a matrix form. The rows represent different small objects, the columns represent different metadata component types, and the matrix elements represent the heat levels of the corresponding components.

[0108] When the storage area allocation processing of each data component is performed by the storage cost minimization calculation according to the grouping results and the multi-level storage area classification, the storage cost minimization calculation adopts an optimization strategy based on the matching of the heat level and the storage area performance. The core principle of the strategy is to allocate high-heat data components to high-performance storage areas and low-heat data components to low-performance storage areas. The specific allocation rules are as follows: super-hot and hot data components are allocated to high-performance areas, warm data components are allocated to standard-performance areas, cold data components are allocated to energy-saving areas, and super-cold data components are allocated to standby areas. The cost calculation considers two factors: the storage cost and the access cost. The storage cost is equal to the data size multiplied by the unit storage cost of the storage area, and the access cost is equal to the expected access frequency multiplied by the unit access cost of the storage area. The total cost is the sum of the storage cost and the access cost. The cost-minimized allocation scheme is found by enumerating all combinations and calculating the total cost. The high-performance area allocation scheme records the list of data components allocated to the high-performance area and their storage locations, and the allocation schemes of other areas are recorded in the same way.

[0109] When the resource matching optimization is performed on the basis of the storage area allocation scheme and the flash block performance mapping relationship, the resource matching optimization refers to further allocating the data components to specific flash blocks on the basis of the determined storage area allocation, and the optimization target is to maximize the utilization efficiency of the flash blocks on the premise of meeting the performance requirements. The matching process first determines the data components to be stored and their sizes according to the storage area allocation scheme, then queries the flash block performance mapping relationship to obtain the flash block list contained in each storage area and the performance parameters thereof, and then adopts the best fit algorithm to allocate the data components to appropriate flash blocks. The best fit algorithm preferentially selects the flash block with the highest space utilization rate and still capable of accommodating the target data component. The metadata component optimal matching scheme records the final storage location of each data component, including the flash block number, the starting address, the data size and the like. The storage cost optimization result is evaluated by calculating the storage cost, the average access delay and the space utilization rate of the actual allocation scheme. The storage cost is equal to the sum of the sizes of the data components multiplied by the storage cost coefficients of the corresponding flash blocks. The access delay is equal to the sum of the access frequencies of the data components multiplied by the average access delays of the corresponding flash blocks. The space utilization rate is equal to the used space divided by the total available space.

[0110] In a specific embodiment, the process of performing step S105 can specifically include the following steps:

[0111] The data capacity occupancy rate, access frequency change and hotness level migration in each storage area are monitored, collected and processed in real time to obtain storage area load state data and hotness change trigger conditions.

[0112] The storage area load state data is compared and analyzed with the preset load balancing threshold to obtain a load imbalance area identifier and a data component list to be migrated.

[0113] Based on the hotness change trigger condition, the hotness of the small object metadata in the data component list is re-evaluated to obtain an updated hotness level allocation and a data migration priority order.

[0114] According to the updated hotness level allocation and the data migration priority order, the small object metadata is migrated across the areas by a load redistribution algorithm to obtain balanced storage load distribution data based on the five-level data hotness level division and an optimized storage area utilization rate.

[0115] Specifically, when monitoring and collecting data capacity occupancy, access frequency change and hotness level migration in each storage area in real time, real-time monitoring and collection refers to continuously tracking and recording the state changes of each storage area by deploying a monitoring agent program at the SSD controller level, data capacity occupancy monitoring is calculated by periodically scanning the ratio of used space to total available space in each storage area, the monitoring period is set to perform statistical operations once every minute, and when the occupancy rate exceeds the preset threshold, it is recorded as a capacity alarm event, access frequency change monitoring is measured by counting the number of access requests received by each storage area in a unit of time, access frequency change refers to the difference between the current time window access frequency and the historical average access frequency, and when the difference exceeds the set range, it is marked as a frequency anomaly event, and hotness level migration monitoring records the change of the hotness level of small object metadata, and generates a migration event record when the hotness level of a small object changes from one level to another, the storage area load state data includes the current capacity occupancy, access frequency, active object quantity and other key indicators of each area, and the hotness change trigger condition is set to trigger the subsequent processing flow when the hotness level changes across levels or the access frequency change exceeds a certain multiple of the reference value.

[0116] When comparing and analyzing the storage area load state data with the preset load balancing threshold, the load balancing threshold refers to the ideal range limit of the load level of each storage area, the load balancing threshold of the high-performance area is set to not more than 90% of the total capacity and not more than 80% of the design peak value of the access frequency, the standard performance area, the energy-saving area and the standby area are set with corresponding threshold parameters, the comparison and analysis processing is executed by comparing the actual monitored load state data with these preset threshold values item by item, when the capacity occupancy or access frequency of a certain storage area exceeds the corresponding threshold value, the area is identified as a load imbalance area, the load imbalance area identification records the specific area number, the threshold parameter type and the exceeding degree, the data component list that needs to be migrated is generated by analyzing the data distribution in the load imbalance area, the data component with relatively low access frequency or hotness level that does not match the current storage area is preferentially selected as the migration candidate, and the data component list is sorted according to the migration urgency, and the urgency is comprehensively evaluated according to the severity of the load exceeding the threshold and the access characteristics of the data component.

[0117] When the hotness re-evaluation of the small object metadata in the data component list is triggered based on the hotness change triggering condition, the hotness re-evaluation refers to the re-computation of the comprehensive hotness value of the small object metadata by using the latest access behavior data, the re-evaluation process adopts the same flash-aware hotness evaluation algorithm as the initial hotness evaluation, but the input data is updated to the access statistics in the latest time window, the access frequency feature vector, the time locality feature and the application correlation feature are re-computed, and then the updated comprehensive hotness value is obtained by weighted summation, the updated hotness level assignment is determined according to the comparison result of the new comprehensive hotness value and the hotness threshold interval, the data migration priority ranking comprehensively considers three factors of the hotness level change amplitude, the adaptation degree of the current storage location and the migration cost, the small object with the greater hotness level change amplitude obtains the higher migration priority, the adaptation degree of the current storage location is evaluated by comparing the new hotness level of the small object and the performance level of the current storage area where the small object is located, the lower adaptation degree indicates the more urgent migration demand, and the migration cost considers the data transmission overhead and the space availability of the target area.

[0118] When the cross-area migration of the small object metadata is performed by the load redistribution algorithm according to the updated hotness level assignment and the data migration priority ranking, the load redistribution algorithm adopts a staged migration strategy to perform the data migration operation, the first stage processes the data component with the highest priority, and migrates the data component from the current storage area to the target storage area matching the new hotness level of the data component, the migration process includes the steps of reading data from the source area, allocating storage space in the target area, writing data to the target location, updating the metadata mapping relationship, and releasing the source area space, the second stage processes the data component with the second priority, and so on until the migration of all data components to be migrated is completed, the migration operation is performed in a background mode to avoid affecting the normal data access service, the load state statistical information of the storage area is updated after each migration operation is completed, the distribution of data with different hotness levels in each storage area after the migration of the balanced storage load distribution data record based on the five-level data hotness level division is completed, and the utilization rate of the optimized storage area is evaluated by calculating the capacity utilization efficiency and the access load balancing degree of each area.

[0119] The data storage optimization method for a solid state disk in the embodiments of the present application is described above, and the data storage optimization system for a solid state disk in the embodiments of the present application is described below. Please refer to Figure 2 The data storage optimization system for a solid state disk in the embodiments of the present application includes one embodiment:

[0120] The identification module is configured to perform pattern recognition processing on the access behavior data of the small object metadata by multi-dimensional access feature extraction, to obtain the access frequency feature vector, the time locality feature and the application correlation feature.

[0121] The quantification module is configured to perform data hotness quantification processing on the access frequency feature vector, the time locality feature and the application correlation feature by using a flash memory-aware hotness evaluation algorithm to obtain five-level data hotness grade division including super-hot, hot, warm, cold and super-cold.

[0122] The analysis module is configured to perform flash block characteristic analysis processing on the physical performance parameters of the SSD storage medium to obtain a multi-level storage area division based on performance difference and a flash block performance mapping relationship.

[0123] The allocation module is configured to perform intelligent storage resource allocation processing on the small object metadata according to the five-level data hotness grade division and the multi-level storage area division to obtain a metadata component optimal matching scheme and a storage cost optimization result based on the flash block performance mapping relationship.

[0124] The adjustment module is configured to perform dynamic adjustment processing on the data distribution state of each storage area in the metadata component optimal matching scheme by using adaptive load monitoring to obtain balanced storage load distribution data based on the five-level data hotness grade division.

[0125] The above Figure 2 The data storage optimization system for a solid state disk in the embodiment of the present application is described in detail from the perspective of a modular functional entity, and the data storage optimization device for a solid state disk in the embodiment of the present application is described in detail from the perspective of hardware processing.

[0126] Referring to Figure 3 , the embodiment of the present application further provides a data storage optimization device for a solid state disk. The data storage optimization device for a solid state disk can be a server, and the internal structure thereof can be as shown in Figure 3 The data storage optimization device for a solid state disk includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. The processor of the computer is configured to provide computing and control capabilities. The memory of the data storage optimization device for a solid state disk includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the data storage optimization device for a solid state disk is configured to store corresponding data in the embodiment. The network interface of the data storage optimization device for a solid state disk is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement the above method.

[0127] Those skilled in the art can understand Figure 3The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the data storage optimization device for a solid state disk to which the scheme of the present application is applied.

[0128] The present application also provides a computer readable storage medium, which can be a non-volatile computer readable storage medium, and can also be a volatile computer readable storage medium, and the computer readable storage medium stores instructions, and when the instructions are run on a computer, the computer executes the steps of the data storage optimization method for a solid state disk.

[0129] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, system and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0130] The integrated unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical scheme of the present application or the whole or part of the technical scheme that essentially contributes to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a data storage optimization device for a solid state disk (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0131] The above embodiments are only used to illustrate the technical scheme of the present application, rather than limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical scheme recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical scheme deviate from the spirit and scope of the technical scheme of each embodiment of the present application.

Claims

1. A data storage optimization method for a solid state drive, characterized by, The method comprises: The access behavior data of the small object metadata is processed by multi-dimensional access feature extraction for pattern recognition, to obtain access frequency feature vector, time locality feature and application correlation feature; According to the access frequency feature vector, time locality feature and application correlation feature, data hotness quantization processing is performed through flash perception hotness evaluation algorithm to obtain five-level data hotness level division including super-hot, hot, warm, cold and super-cold, including: the access frequency data in the access frequency feature vector is processed by logarithmic normalization to obtain frequency score and corresponding frequency weight factor; based on the time locality feature, time decay score calculation processing is performed on the time difference between the current time and the last access time through exponential decay calculation to obtain time score reflecting access time freshness and time weight factor; according to the application correlation feature, the access mode of small object metadata with the same application source is quantitatively processed to obtain application access stability score and application weight factor; the frequency score, time score and application access stability score are weighted and summed according to the corresponding weight factor to obtain the comprehensive hotness value of the small object metadata and the five-level data hotness level division including super-hot, hot, warm, cold and super-cold based on the hotness threshold interval; The physical performance parameters of the SSD storage medium are analyzed and processed for flash block characteristics to obtain multi-level storage area division based on performance difference and flash block performance mapping relationship; According to the five-level data hotness level division and the multi-level storage area division, the small object metadata is intelligently stored and the resource allocation processing is performed to obtain the metadata component optimal matching scheme based on the flash block performance mapping relationship and the storage cost optimization result; Through adaptive load monitoring, the data distribution state of each storage area in the metadata component optimal matching scheme is dynamically adjusted to obtain balanced storage load distribution data based on the five-level data hotness level division.

2. The data storage optimization method for solid state hard drives according to claim 1, characterized in that, The access behavior data of the small object metadata is processed by multi-dimensional access feature extraction for pattern recognition, to obtain access frequency feature vector, time locality feature and application correlation feature, including: The access timestamp, access operation type and access source application identifier of the small object metadata are monitored and collected in real time to obtain the original access behavior data set; The original access behavior data set is processed by time series segmentation according to the preset time window length to obtain access statistical data and access frequency distribution matrix in multiple time windows; Based on the access frequency distribution matrix, the access mode in each time window is quantitatively analyzed by frequency density calculation to obtain the access frequency feature vector and the access interval coefficient of variation; According to the access frequency feature vector and the access interval coefficient of variation, the small object metadata with the same application source is correlated to obtain the time locality feature and the application correlation feature.

3. The data storage optimization method for solid state hard drives of claim 1, wherein, The access mode of small object metadata with the same application source is quantitatively processed according to the application correlation feature to obtain application access stability score and application weight factor, including: The variance calculation process is performed on the access interval time sequence of the small object metadata from the same application source to obtain an access interval variance value and an access beat stability index; The access interval variance value is normalized by a coefficient of variation to quantitatively analyze the discrete degree of the access mode, and an access mode coefficient of variation and a mode stability benchmark value are obtained; Based on the access beat stability index and the mode stability benchmark value, the access regularity of the small object metadata is evaluated to obtain an application access regularity score and an access prediction confidence value; Based on the application access regularity score and the access prediction confidence value, the application access behavior is comprehensively evaluated by stability weighting calculation to obtain the application access stability score and the application weight factor based on the stability gradient.

4. The data storage optimization method for solid state hard drives of claim 1, wherein, The physical performance parameters of the SSD storage medium are analyzed to obtain a multi-level storage area division based on performance differences and a flash memory block performance mapping relationship, including: The read delay, write delay, and erase delay of each flash memory block in the SSD storage medium are tested and collected to obtain a flash memory block delay performance dataset and an IO response time benchmark value; The flash memory block delay performance dataset is combined with the current P / E cycle number and the maximum P / E cycle limit to perform durability evaluation calculation to obtain a flash memory block durability index and a remaining life expectancy value; Based on the IO response time benchmark value and the flash memory block durability index, the performance of each flash memory block is classified by performance weight comprehensive scoring to obtain a flash memory block classification result of a high-performance area, a standard-performance area, an energy-saving area, and a standby area; Based on the flash memory block classification result, the boundaries of the SSD storage space are determined to obtain the multi-level storage area division based on performance differences and the flash memory block performance mapping relationship between each flash memory block and the storage area.

5. The data storage optimization method for solid state hard drives of claim 1, wherein, The small object metadata is intelligently stored by resource allocation based on the five-level data heat level division and the multi-level storage area division to obtain a metadata component optimal matching scheme based on the flash memory block performance mapping relationship and a storage cost optimization result, including: The small object metadata is classified into metadata components according to basic attributes, extended attributes, index information, and access control information to obtain data sizes and access feature identifiers of the metadata components; Based on the five-level data heat level division, the heat level matching analysis of each metadata component is performed to obtain a grouping result of super-hot data components, hot data components, warm data components, cold data components, and super-cold data components; Based on the grouping result and the multi-level storage area division, the storage area allocation of each data component is performed by storage cost minimization calculation to obtain a high-performance area allocation scheme, a standard-performance area allocation scheme, an energy-saving area allocation scheme, and a standby area allocation scheme; Each area allocation scheme is combined with the flash memory block performance mapping relationship to perform resource matching optimization to obtain a metadata component optimal matching scheme based on the flash memory block performance mapping relationship and a storage cost optimization result including storage cost, access delay, and space utilization.

6. The data storage optimization method for solid state hard drives of claim 1, wherein, The dynamic adjustment processing of the data distribution state of each storage area in the metadata component optimal matching scheme through adaptive load monitoring obtains balanced storage load distribution data based on the five-level data heat level division, comprising: Real-time monitoring, collection and processing of the data capacity occupancy rate, access frequency change and heat level migration situation in each storage area obtain storage area load state data and heat change trigger conditions; Comparative analysis processing of the storage area load state data and the preset load balancing threshold obtains load imbalance area identification and a data component list that needs to be migrated; Heat re-evaluation processing of the small object metadata in the data component list based on the heat change trigger conditions obtains updated heat level allocation and data migration priority ranking; Cross-area migration processing of the small object metadata through a load redistribution algorithm according to the updated heat level allocation and data migration priority ranking obtains balanced storage load distribution data based on the five-level data heat level division and optimized storage area utilization.

7. A data storage optimization system for a solid state drive, comprising: The data storage optimization system for a solid state disk comprises: An identification module for mode identification processing of access behavior data of small object metadata through multi-dimensional access feature extraction obtains access frequency features, time locality features and application correlation features; A quantization module for data heat quantization processing through a flash memory-aware heat evaluation algorithm according to the access frequency features, time locality features and application correlation features obtains a five-level data heat level division including super-hot, hot, warm, cold and super-cold; An analysis module for flash block characteristic analysis processing of physical performance parameters of an SSD storage medium obtains a multi-level storage area division based on performance differences and a flash block performance mapping relationship; An allocation module for intelligent storage resource allocation processing of small object metadata according to the five-level data heat level division and the multi-level storage area division obtains a metadata component optimal matching scheme and a storage cost optimization result based on the flash block performance mapping relationship; An adjustment module for dynamic adjustment processing of the data distribution state of each storage area in the metadata component optimal matching scheme through adaptive load monitoring obtains balanced storage load distribution data based on the five-level data heat level division.

8. A data storage optimization device for a solid state drive, characterized by, The computer program, when executed by the processor, causes the processor to perform the data storage optimization method for a solid state disk.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, causes the processor to perform the data storage optimization method for a solid state disk.

Images