Self-adaptive storage capacity adjusting method and system for industrial-grade solid state disk

Abstract

The invention relates to the technical field of data storage and management, and discloses a self-adaptive storage capacity adjusting method and system for an industrial-grade solid state disk. The method and the system can intelligently analyze and predict future storage requirements by collecting key performance indexes and user data access modes of the solid state disk in real time, remarkably improve the efficiency of storage management, and can automatically identify behavior modes of storage use and formulate corresponding storage strategies through a machine learning algorithm, thereby improving the efficiency of storage management. According to the self-adaptive method, hot data is ensured to be quickly accessed, cold data is reasonably migrated, so that the utilization rate of storage space is optimized, in addition, the storage performance is further improved and the risk of data loss or damage is reduced by adopting a garbage collection algorithm and a TRIM command, and the self-adaptive method not only improves the response speed and the reliability of storage equipment, but also reduces the maintenance cost, and is suitable for popularization and application. And a more efficient and flexible storage solution is provided for enterprises.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage and management, and in particular to a method and system for adaptively adjusting the storage capacity of an industrial-grade solid-state hard disk. Background Art

[0002] With the rapid development of information technology and the surge in data volumes, the demand for storage devices continues to increase. Solid-state drives (SSDs) are becoming the preferred storage medium in data centers and industrial applications due to their high read and write speeds, low latency, and high durability. Compared to traditional mechanical hard drives, SSDs offer faster data access speeds and higher input and output operations per second (IOPS), making them excellent at processing large amounts of data. However, SSD storage resources are limited, making efficient and intelligent management and optimization of their storage capacity to meet ever-changing user needs a key research topic.

[0003] Traditional methods lack intelligent decision-making support when migrating and cleaning data. The data migration and cleanup process is usually time-consuming and complex, relying on manual intervention or simple rules, and is prone to errors or delays. This not only affects storage efficiency but may also lead to data loss or corruption. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention provides an adaptive storage capacity adjustment method and system for industrial-grade solid-state hard drives. By real-time collection of key performance indicators and user data access patterns of solid-state hard drives, it can intelligently analyze and predict future storage needs, significantly improving the efficiency of storage management. Compared with traditional methods, this method effectively solves the problem of lack of intelligent decision-making support in data migration and cleaning processes. Moreover, through machine learning algorithms, it can automatically identify storage usage behavior patterns and formulate corresponding storage strategies to ensure fast access to hot data while reasonably migrating cold data, thereby optimizing the utilization of storage space. In addition, the garbage collection algorithm and TRIM command are used to further improve storage performance and reduce the risk of data loss or damage. This adaptive method not only improves the response speed and reliability of storage devices, but also reduces maintenance costs, providing enterprises with more efficient and flexible storage solutions.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for adaptively adjusting storage capacity of an industrial-grade solid-state drive, comprising the following steps: S1, real-time collection of SSD read and write rates, storage usage, I / O load, temperature, and health status data, as well as user write patterns and data access frequency data; S2 uses machine learning algorithms to analyze the data collected in S1, identify behavioral patterns in storage usage, and use time series analysis to predict future storage needs; S3. Based on predicted future storage needs, a storage strategy is developed to ensure fast access to hot data and migration of cold data to slower-speed areas. A storage area partitioning scheme is created, and the storage ratio of hot and cold data is calculated to ensure sufficient space in the hot data storage area. S4. Perform data move, copy, and delete operations based on the established storage policy, and use garbage collection algorithms and TRIM commands to optimize storage space utilization. S5. After the storage capacity is adjusted, continue to monitor the performance of the solid-state drive and calculate the utilization and performance indicators of the adjusted storage to evaluate the effect of the capacity adjustment.

[0006] Preferably, the identification of the behavior pattern of storage usage is achieved by applying the following formula: ; In the formula, Indicates the behavior pattern of storage usage, Indicates the average amount of data written / read for a single storage block. Indicates the maximum amount of data written / read, Indicates the access frequency to the storage block, Indicates the highest access frequency, represents the average temperature of the storage block, Indicates the maximum temperature of the storage block, express Load index, Indicates the maximum load.

[0007] Preferably, the formula used to predict future storage requirements using time series analysis is as follows: ; In the formula, Indicates a time point The storage demand forecast value, represents the smoothing coefficient, Indicates at a point in time The actual storage requirements, Indicates at the previous time point The predicted value of .

[0008] Preferably, the formula for calculating the storage ratio of hot data and cold data is as follows: ; In the formula, Indicates the storage ratio of hot data, Indicates the The amount of hot data access per storage block, Indicates the The total number of accesses to a storage block is the sum of the accesses to hot data and cold data.

[0009] Preferably, the storage area division scheme determines the storage space size by the following formula: ; ; In the formula, Indicates the size of the hot data storage area. Indicates the size of the cold data storage area. Indicates the current utilization of the storage device. Indicates the storage ratio of hot data, Indicates the total capacity of the storage device.

[0010] Preferably, the garbage collection algorithm of the application is as follows: ; In the formula, Indicates the total amount of space freed by garbage collection operations. Indicates the blocks that are recycled as free, Indicates the total number of storage blocks.

[0011] Preferably, the TRIM command is as follows: ; in, Waiting The logical block address, Represents a set of label spaces To be erasable, this The command tells the hard disk controller to move the logical block Marked as free, no longer contains valid data, allowing the space to be freed up later in actual erasure.

[0012] Preferably, the formula for the adjusted storage utilization is as follows: ; In the formula, Indicates the utilization of the adjusted storage, Indicates the used space before adjustment. Indicates the amount of space deleted in this operation. Indicates the amount of space copied to other areas. Indicates the amount of space that has been moved to other areas. Indicates the total capacity of the storage device.

[0013] Preferably, the performance index is achieved by measuring the average read and write speed, and the calculation formula is as follows: ; In the formula, Indicates the average read and write speed change ratio, Indicates the average read and write speed measured after storage adjustment. Indicates the average read and write speed before adjustment.

[0014] An adaptive storage capacity adjustment system for industrial-grade solid-state hard drives, comprising a data acquisition unit, a behavior pattern analysis unit, a storage strategy formulation unit, a data operation execution unit, and a performance monitoring unit; The data acquisition unit is used to collect real-time data on the read and write rates, storage usage, I / O load, temperature, and health status of the solid-state drive, as well as user write patterns and data access frequency data; The behavior pattern analysis unit uses a machine learning algorithm to analyze the data collected in S1, identifies the behavior pattern of storage usage, and uses time series analysis to predict future storage needs; The storage strategy formulation unit formulates a storage strategy based on predicted future storage needs to ensure that hot data is quickly accessed and cold data is migrated to a low-speed area. It also creates a storage area partitioning scheme and calculates the storage ratio of hot data to cold data to ensure that the storage area for hot data has sufficient space. The data operation execution unit performs data move, copy and delete operations according to the formulated storage strategy, and optimizes the utilization of storage space by using garbage collection algorithm and TRIM command; After the storage capacity is adjusted, the performance monitoring unit continues to monitor the performance of the solid state drive and calculates the utilization rate and performance index of the adjusted storage to evaluate the effect of the capacity adjustment.

[0015] Compared with the prior art, the present invention provides a method and system for adaptively adjusting the storage capacity of an industrial-grade solid-state drive, which has the following beneficial effects: By collecting key performance indicators and user data access patterns of solid-state drives in real time, the present invention can intelligently analyze and predict future storage needs, significantly improving the efficiency of storage management. Compared with traditional methods, this method effectively solves the problem of lack of intelligent decision-making support in data migration and cleaning processes. Moreover, through machine learning algorithms, it can automatically identify behavioral patterns of storage usage and formulate corresponding storage strategies to ensure fast access to hot data while reasonably migrating cold data, thereby optimizing the utilization of storage space. In addition, the garbage collection algorithm and TRIM command are used to further improve storage performance and reduce the risk of data loss or damage. This adaptive method not only improves the response speed and reliability of storage devices, but also reduces maintenance costs, providing enterprises with more efficient and flexible storage solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Schematic diagram of the steps of the method of the present invention; Figure 2 Schematic diagram of the system flow of the present invention. DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0018] Traditional methods lack intelligent decision support when migrating and cleaning data. The data migration and cleaning process is usually time-consuming and complicated, relying on manual intervention or simple rules, and prone to errors or delays. This not only affects storage efficiency but may also lead to data loss or corruption. To this end, we propose an adaptive storage capacity adjustment method for industrial-grade solid-state drives. Figure 1 , the method comprises the following steps: S1, real-time collection of SSD read and write rates, storage usage, I / O load, temperature, and health status data, as well as user write patterns and data access frequency data; To achieve comprehensive monitoring and analysis of the solid-state drive (SSD) status, the system will adopt a variety of high-precision real-time data acquisition technologies. First, it will use high-performance monitoring chips and sensors to accurately obtain the SSD's read and write rates from the hardware level, including the amount of data read and written per second (for example, in MB / s or IOPS). Low-latency, high-frequency data acquisition will be achieved through hardware counters or DMA (direct memory access). Second, embedded circuits and software drivers will continuously monitor the storage usage of storage areas, and count the used and free space in real time. Combined with SMART (Self-Monitoring, Analysis and Reporting) Data from the SSD storage system (SSD) is analyzed using IO performance monitoring tools (such as IO monitoring managers or dedicated interfaces) to obtain dynamic parameters of storage utilization. I / O load is detected using IO performance monitoring tools (such as IO monitoring managers or dedicated interfaces). The system accurately records the number of I / O requests, request type (read / write), response time, and queuing delay at each point in time to assess the workload level of the storage device. For temperature monitoring, the system deploys micro-temperature sensors (such as thermistors or MEMS temperature sensors) in close proximity to the SSD controller or storage chip for 24-hour continuous monitoring. These sensors reflect changes in device temperature rise in real time, effectively providing early warning of thermal management issues. In addition, by integrating the SSD health status monitoring module, SMART parameters (such as disk health, write cycles, and number of bad blocks) are used to continuously track the overall physical status of the SSD to ensure the security of the storage system. Access log and IO scheduling analysis technology are used to analyze user write patterns and data access frequency. By collecting information about the operating system's IO request queue, file system access logs, and storage layer write / read request statistics, the time distribution, hot spots, access depth, and frequency of storage behavior are analyzed to further identify the storage preferences of hot and cold data. This information is combined with software-level analysis algorithms through an efficient collection framework (such as NDI or a dedicated hardware monitoring interface) to form a complete real-time monitoring system, providing a solid data foundation for subsequent storage management and optimization. S2 uses machine learning algorithms to analyze the data collected in S1, identify behavioral patterns in storage usage, and use time series analysis to predict future storage needs; When using machine learning algorithms to analyze the multi-dimensional data collected in step S1, the data will first be feature extracted and standardized, including statistical information on the maximum and average values ​​of the storage device. For example, using the formula: ; Each parameter here is normalized to make different indicators comparable. Specifically: Indicates the average amount of accessed data during the monitoring period (such as the average amount of data written / read per second), and It is the maximum value of the metric during the monitoring period and is used to measure the peak value of storage behavior and thus identify extreme load conditions. refers to the frequency of access (i.e., the number of requests) within a specific time period, while Represents the historical maximum access frequency, which helps to identify access hotspots; and The average and maximum temperatures of the storage device during the monitoring period are used to determine the device's thermal load and potential heat dissipation risks. and They represent the I / O load (such as the number of I / O requests per second) and its historical maximum value, reflecting the working pressure of the device; By normalizing these metrics, machine learning models (such as random forests, support vector machines, or deep neural networks) can learn typical storage behavior patterns, identifying behavioral characteristics such as "hotspot storage areas," "consistently high-load areas," and "idle areas," thereby understanding the dynamic utilization of storage space. This identification process is the foundation for intelligent storage management and provides accurate input for subsequent storage demand forecasting. In predicting future storage demand, time series analysis algorithms (such as exponential smoothing or ARIMA models) are used to use known demand data. , through the formula Continuously adjust sensitivity to the latest data; Combining the aforementioned feature analysis and time series prediction, we can predict storage capacity trends and identify potential capacity bottlenecks in advance, enabling proactive storage resource scheduling. For example, before demand exceeds current capacity, the system can migrate cold data, expand hot storage areas, or optimize storage policies to avoid performance degradation. This technical framework, which leverages machine learning to identify behavioral patterns and predict future demand, significantly enhances the intelligence of storage systems, enabling them to adapt to changing access loads and storage pressures, achieving efficient, stable, and scalable storage operations. S3. Based on predicted future storage needs, a storage strategy is developed to ensure fast access to hot data and migration of cold data to slower-speed areas. A storage area partitioning scheme is created, and the storage ratio of hot and cold data is calculated to ensure sufficient space in the hot data storage area. To efficiently manage storage resources, the system will formulate dynamic storage strategies based on the prediction results of future storage needs. It will rationally divide hot data and cold data into different storage areas, thereby ensuring that hot data can be quickly accessed and improving overall performance. In specific implementation, the system first uses a prediction model to predict the storage capacity demand within a certain period of time in the future. For example, a time series analysis model is used to obtain the future storage growth value. Then, based on the current utilization rate and total storage capacity of the storage device, the storage space is rationally divided. The calculation formula for the storage ratio of hot data is: ; in, Indicates the The amount of hot data access or write per storage block, is the total number of visits to the corresponding block, The number of storage blocks during the monitoring period. This ratio reflects the proportion of hot data in the storage and provides a basis for storage area division. Next, calculate the size of the hot data storage area based on this ratio: ; here, Indicates the current utilization of the storage device (for example, 0.75 means 75% of the space is occupied), ensuring that the existing space usage is taken into account when allocating space. It is calculated based on the proportion of hot data on the basis of the above, in this way, sufficient space is provided for hot data to ensure its fast access; At the same time, cold data can be migrated to a slower, larger storage area with a size of: ; This partitioning scheme makes storage resource utilization more rational and centralized, ensuring that hot data areas have sufficient space to handle future access peaks and avoiding performance bottlenecks caused by insufficient space. By combining a "predictive" mechanism for future storage needs with a space partitioning formula, storage strategies can be dynamically adjusted while reducing the frequency of hot and cold data migration, improving overall system efficiency and stability. This technical approach, leveraging mathematical models for prediction and space management, provides strong support for intelligent and flexible storage management. S4. Perform data move, copy, and delete operations based on the established storage policy, and use garbage collection algorithms and TRIM commands to optimize storage space utilization. When executing data management operations based on established storage policies, the system uses efficient algorithms and hardware instructions to ensure that data migration, replication, and deletion are efficient without impacting overall system performance. Data migration and replication are typically implemented through dedicated storage controllers or high-speed channels to minimize write latency and bandwidth usage, while batch processing techniques are used to balance the load. For deletion operations, the system invokes a garbage collection (GC) algorithm, combined with a defragmentation strategy, to properly reclaim deleted or unreachable storage blocks, ensuring storage space continuity and improved utilization. The effectiveness of garbage collection can be quantified using the following formula: ; in, Indicates the A memory block (block or page) that is deleted or marked as free, and is the total number of reclaimed blocks, which helps quantify the space recovery efficiency and reflects the actual effect of space cleaning. At the same time, to further optimize the utilization of storage space, the SSD controller or storage management layer will use the TRIM command to notify the hardware of the blocks that are no longer used in the logical space (that is, logical blocks marked as unused or deleted) to actually erase them. The formula is: ; Here Represents the set of logical block addresses to be TRIMed. After sending the TRIM command, the SSD maps these logical blocks to physical erase blocks and performs the erase operation asynchronously in the background, reducing write wait time. The combination of garbage collection and TRIM commands not only maximizes the recovery of free space and reduces the write amplification effect, but also extends the service life of the storage device and improves the overall storage space utilization efficiency. This complete set of space management technologies ensures the continuous and efficient operation of the storage system under high load, providing a solid hardware and algorithm foundation for intelligent storage management and maintenance. S5. After the storage capacity is adjusted, continue to monitor the performance of the solid-state drive and calculate the utilization and performance indicators of the adjusted storage to evaluate the effect of the capacity adjustment. After the storage capacity is adjusted, to ensure continued efficient system operation, the system will continuously monitor the performance indicators and storage space utilization of the SSDs. Using high-precision performance monitoring tools and data collection interfaces, the system collects real-time data such as read and write speeds (such as MB / s or IOPS), storage utilization, I / O latency, and response time. The adjusted storage utilization is calculated using the following formula: ; This formula can dynamically reflect the remaining available space in the storage system and help evaluate the actual effect of space adjustment. To measure changes in storage performance, the ratio of speed indicators is used to evaluate the performance changes before and after adjustment: ; This ratio is used to measure the improvement (e.g., a ratio greater than 1) or degradation (a ratio less than 1) of storage performance, helping technicians determine whether adjustments have had a positive impact on system performance. By combining real-time data monitoring, utilization calculation, and performance indicator evaluation, the system can dynamically adjust subsequent storage policies to ensure the rational use of storage capacity and continuous optimization of system performance, thereby achieving efficient management of storage resources and long-term stable operation.

[0019] See also Figure 2 , an adaptive storage capacity adjustment system for industrial-grade solid-state drives, comprising a data acquisition unit, a behavior pattern analysis unit, a storage strategy formulation unit, a data operation execution unit, and a performance monitoring unit; The data acquisition unit is used to collect real-time data on the read and write rates, storage usage, I / O load, temperature, and health status of the solid-state drive, as well as user write patterns and data access frequency data; The behavioral pattern analysis unit uses machine learning algorithms to analyze the data collected in S1, identify behavioral patterns of storage usage, and use time series analysis to predict future storage needs; The storage strategy formulation unit formulates a storage strategy based on predicted future storage needs, ensuring fast access to hot data and migrating cold data to slower storage areas. It also creates a storage area partitioning scheme and calculates the storage ratio of hot and cold data to ensure sufficient space in the hot data storage area. The data operation execution unit performs data movement, copying, and deletion operations according to the established storage strategy, and uses garbage collection algorithms and TRIM commands to optimize storage space utilization; After the storage capacity is adjusted, the performance monitoring unit continues to monitor the performance of the solid-state drive and calculates the utilization and performance indicators of the adjusted storage to evaluate the effect of the capacity adjustment.

[0020] The comprehensive application of the above-mentioned methods and systems effectively solves the problem of lack of intelligent decision-making support in the data migration and cleanup process. Moreover, through machine learning algorithms, it is possible to automatically identify storage usage behavior patterns and formulate corresponding storage strategies to ensure fast access to hot data while rationally migrating cold data, thereby optimizing storage space utilization. In addition, the use of garbage collection algorithms and TRIM commands further improves storage performance and reduces the risk of data loss or corruption. This adaptive approach not only improves the response speed and reliability of storage devices, but also reduces maintenance costs, providing enterprises with more efficient and flexible storage solutions.

[0021] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for adaptively adjusting storage capacity of an industrial-grade solid-state drive, characterized in that: The following steps are involved: S1, real-time collection of SSD read and write rates, storage usage, I / O load, temperature, and health status data, as well as user write patterns and data access frequency data; S2 uses machine learning algorithms to analyze the data collected in S1, identify behavioral patterns in storage usage, and use time series analysis to predict future storage needs; S3. Based on predicted future storage needs, a storage strategy is developed to ensure fast access to hot data and migration of cold data to slower-speed areas. A storage area partitioning scheme is created, and the storage ratio of hot and cold data is calculated to ensure sufficient space in the hot data storage area. S4. Perform data move, copy, and delete operations based on the established storage policy, and use garbage collection algorithms and TRIM commands to optimize storage space utilization. S5. After the storage capacity is adjusted, continue to monitor the performance of the solid-state drive and calculate the utilization and performance indicators of the adjusted storage to evaluate the effect of the capacity adjustment.

2. The method for adaptively adjusting storage capacity of an industrial-grade solid-state drive according to claim 1, wherein: The identification of the behavior pattern of storage usage is achieved by applying the following formula: ； In the formula, Indicates the behavior pattern of storage usage, Indicates the average amount of data written / read for a single storage block. Indicates the maximum amount of data written / read, Indicates the access frequency to the storage block, Indicates the highest access frequency, represents the average temperature of the storage block, Indicates the maximum temperature of the storage block, express Load index, Indicates the maximum load.

3. The method for adaptively adjusting storage capacity of an industrial-grade solid-state drive according to claim 2, wherein: The formula used to predict future storage requirements using time series analysis is as follows: ； In the formula, Indicates a time point The storage demand forecast value, represents the smoothing coefficient, Indicates at a point in time The actual storage requirements, Indicates at the previous time point The predicted value of .

4. The method for adaptively adjusting storage capacity of an industrial-grade solid-state drive according to claim 3, wherein: The formula for calculating the storage ratio of hot data and cold data is as follows: ； In the formula, Indicates the storage ratio of hot data, Indicates the The amount of hot data access per storage block, Indicates the The total number of accesses to a storage block is the sum of the accesses to hot data and cold data.

5. The method for adaptively adjusting storage capacity of an industrial-grade solid-state drive according to claim 4, wherein: The storage area division scheme determines the storage space size by the following formula: ； ； In the formula, Indicates the size of the hot data storage area. Indicates the size of the cold data storage area. Indicates the current utilization of the storage device. Indicates the storage ratio of hot data, Indicates the total capacity of the storage device.

6. The method for adaptively adjusting storage capacity of an industrial-grade solid-state drive according to claim 5, wherein: The garbage collection algorithm of the application is as follows: ； In the formula, Indicates the total amount of space freed by garbage collection operations. Indicates the blocks that are recycled as free, Indicates the total number of storage blocks.

7. The method for adaptively adjusting storage capacity of an industrial-grade solid-state drive according to claim 6, wherein: The TRIM command is as follows: ； in, Indicates the logical block address to be waited for, Represents a set of label spaces To be erasable, this The command tells the hard disk controller to move the logical block Marked as free, no longer contains valid data, allowing the space to be freed up later in actual erasure.

8. The method for adaptively adjusting storage capacity of an industrial-grade solid-state drive according to claim 7, wherein: The formula for the adjusted storage utilization is as follows: ； In the formula, Indicates the utilization of the adjusted storage, Indicates the used space before adjustment. Indicates the amount of space deleted in this operation. Indicates the amount of space copied to other areas. Indicates the amount of space that has been moved to other areas. Indicates the total capacity of the storage device.

9. The method for adaptively adjusting storage capacity of an industrial-grade solid-state drive according to claim 8, wherein: The performance indicators are achieved by measuring the average read and write speeds, and the calculation formula is as follows: ； In the formula, Indicates the average read and write speed change ratio, Indicates the average read and write speed measured after storage adjustment. Indicates the average read and write speed before adjustment.

10. An adaptive storage capacity adjustment system for an industrial-grade solid-state hard drive, characterized by: It includes data collection unit, behavior pattern analysis unit, storage strategy formulation unit, data operation execution unit and performance monitoring unit; The data acquisition unit is used to collect real-time data on the read and write rates, storage usage, I / O load, temperature, and health status of the solid-state drive, as well as user write patterns and data access frequency data; The behavior pattern analysis unit uses a machine learning algorithm to analyze the data collected in S1, identifies the behavior pattern of storage usage, and uses time series analysis to predict future storage needs; The storage strategy formulation unit formulates a storage strategy based on predicted future storage needs to ensure that hot data is quickly accessed and cold data is migrated to a low-speed area. It also creates a storage area partitioning scheme and calculates the storage ratio of hot data to cold data to ensure that the storage area for hot data has sufficient space. The data operation execution unit performs data move, copy and delete operations according to the formulated storage strategy, and optimizes the utilization of storage space by using garbage collection algorithm and TRIM command; After the storage capacity is adjusted, the performance monitoring unit continues to monitor the performance of the solid state drive and calculates the utilization rate and performance index of the adjusted storage to evaluate the effect of the capacity adjustment.

Images