Multi-level cache management method, system and device for solid-state storage device and medium

Abstract

The invention relates to the technical field of computer storage, in particular to a multi-level cache management method, system and equipment for solid-state storage equipment and a medium. The method comprises the following steps that multiple layers of caches are configured in the solid-state storage device and comprise a high-speed cache layer, a medium-speed cache layer and a low-speed cache layer, and each layer of cache has different access speeds and capacities and is used for storing data blocks with different priorities and access frequencies; dynamically scheduling the data blocks to a proper cache layer through a cache allocation algorithm according to the real-time access frequency, size, life cycle and importance of the data blocks; establishing a comprehensive data block priority evaluation model, and calculating a priority score of each data block; according to the multi-level cache management method, by configuring the multi-layer cache, the priority evaluation model and the multi-path scheduling mechanism, dynamic scheduling and load balancing of the data blocks among different cache layers are achieved, and the overall efficiency of data access is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer storage technology, and in particular to a multi-level cache management method, system, device and medium for solid-state storage devices. Background Art

[0002] In solid-state storage devices (SSDs), cache management is a key technology to improve data access speed and device performance. Traditionally, single-layer cache management methods are mostly used, but single-layer cache management methods are prone to bottlenecks when processing large amounts of parallel data access, and cannot fully utilize the high-speed performance of storage devices. It can no longer meet the current complex and diverse application requirements and data access patterns. Summary of the invention

[0003] The present invention provides a multi-level cache management method, system, device and medium for a solid-state storage device to solve the technical problem of speed performance bottleneck faced by single-level cache management of a solid-state storage device.

[0004] The technical solution of the present invention to solve the above technical problems is as follows: In one aspect, a multi-level cache management method for a solid-state storage device is provided, the method comprising the following steps: A multi-layer cache is configured in the solid-state storage device, including a high-speed cache layer, a medium-speed cache layer, and a low-speed cache layer. Each layer of cache has a different access speed and capacity and is used to store data blocks with different priorities and access frequencies. Dynamically schedule data blocks to the appropriate cache layer through the cache allocation algorithm based on the real-time access frequency, size, life cycle, and importance of the data blocks; Establish a comprehensive data block priority evaluation model and calculate the priority score of each data block; A multi-path scheduling mechanism is used to dynamically allocate data blocks to different cache layers according to priority scores, optimizing error distribution to balance the load between cache layers; Monitor the load status of each cache layer in real time, and dynamically adjust the capacity and data distribution of the cache layer through the load balancing function; Implement data consistency protection mechanisms to check and correct data blocks to ensure data integrity during migration between cache layers.

[0005] In another aspect, a multi-level cache management system for a solid-state storage device is provided, the system comprising: Multi-layer cache, including high-speed cache layer, medium-speed cache layer and low-speed cache layer; Cache scheduling module, used to perform dynamic scheduling of data blocks to appropriate cache layers; The load monitoring module is used to monitor the load status of each cache layer in real time and dynamically adjust the capacity and data distribution of the cache layer through the load balancing function; The data migration module is used to perform the data migration process between cache layers; A data check and error correction module, used to implement check and error correction when migrating data between cache layers; The parallel processing module supports multi-threaded parallel read and write operations to improve data access efficiency.

[0006] In another aspect, a multi-level cache management device for a solid-state storage device includes: A multi-level cache management system for solid-state storage devices as above; A storage device for storing data scheduled by the cache management system; and The processor is used to dynamically schedule data blocks to appropriate cache layers, execute data migration processes between cache layers, implement checksum and error correction when migrating data between cache layers, and perform multi-threaded parallel read and write operations.

[0007] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the multi-level cache management method of a solid-state storage device as described above can be implemented.

[0008] The beneficial effects of the present invention are: The multi-level cache management method of the present invention realizes dynamic scheduling and load balancing of data blocks between different cache layers by configuring multi-level cache, priority evaluation model and multi-path scheduling mechanism, thereby improving the overall efficiency of data access. The multi-level cache management method manages data blocks in a refined manner according to their real-time access frequency, size, life cycle and importance, effectively improving the cache hit rate of data blocks with high access frequency.

[0009] Furthermore, by real-time monitoring of the load status of each cache layer and adopting a dynamic adjustment mechanism, load balance between cache layers is ensured, thus avoiding performance bottlenecks caused by uneven load.

[0010] In addition, the multi-level cache management method of the present invention also adopts a data verification and error correction mechanism to ensure the integrity of data during migration between cache layers, effectively reducing data loss and inconsistency problems caused by improper data error correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A flowchart of a multi-level cache management method according to an embodiment of the present invention; Figure 2 A multi-level cache management simulation diagram in one embodiment of the present invention; Figure 3 A schematic diagram of a multi-path scheduling mechanism in one embodiment of the present invention; Figure 4A load balancing grid diagram based on a cache layer in one embodiment of the present invention; Figure 5 A topological diagram of data distribution and cache optimization in one embodiment of the present invention; Figure 6 This is a diagram of the architecture of a multi-level cache management device in one embodiment of the present invention. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] The term "comprise" and any variation thereof in the specification and claims of the present application are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. In addition, the use of "and / or" in the specification and claims to indicate at least one of the connected objects, such as A and / or B, means including A alone, B alone, and A and B.

[0014] In the embodiments of the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0015] The present invention provides the following preferred embodiments: Embodiment 1 In order to solve the problem that the single-layer cache management method is prone to bottlenecks when processing a large amount of parallel data access and cannot fully utilize the high-speed performance of the storage device, this embodiment proposes a multi-level cache management method for solid-state storage devices, such as Figure 1 As shown, the method specifically includes the following steps: S101. Configure multiple layers of cache in a solid-state storage device, including a high-speed cache layer, a medium-speed cache layer, and a low-speed cache layer. Each layer of cache has a different access speed and capacity and is used to store data blocks with different priorities and access frequencies.

[0016] S102: Dynamically dispatch data blocks to appropriate cache layers through a cache allocation algorithm according to the real-time access frequency, size, life cycle, and importance of the data blocks.

[0017] S103: Establish a comprehensive data block priority evaluation model to calculate the priority score of each data block.

[0018] S104: Use a multi-path scheduling mechanism to dynamically allocate data blocks to different cache layers according to priority scores, and optimize error distribution to balance the load between cache layers.

[0019] S105: Monitor the load status of each cache layer in real time, and dynamically adjust the capacity and data distribution of the cache layer through a load balancing function.

[0020] S106: Implement a data consistency protection mechanism to verify and correct data blocks to ensure the integrity of data during migration between cache layers.

[0021] Specifically, this embodiment configures multiple layers of cache in a solid-state storage device, including a high-speed cache layer, a medium-speed cache layer, and a low-speed cache layer. Each layer of cache has different access speeds and capacities, in order to adapt to the storage requirements of data blocks with different priorities and access frequencies. Furthermore, the specific access speed, capacity, and data block access strategy of each layer of cache can be configured according to actual business needs. It is understandable that the high-speed cache layer can use the latest storage media, such as DDR4 or DDR5 memory, to ensure the highest access speed and lower latency; the medium-speed cache layer can use a performance-balanced NAND flash memory; and the low-speed cache layer can use a relatively low-priced large-capacity storage medium, such as QLC NAND flash memory. This hierarchical design enables high-frequency access data to be stored in the fast cache closest to the processor, while low-frequency access data is stored on slower but more economical storage media.

[0022] like Figure 2 As shown, this embodiment dynamically schedules data blocks to appropriate cache layers through a cache allocation algorithm based on the real-time access frequency, size, life cycle, and importance of the data blocks. In this process, the algorithm will comprehensively consider the above-mentioned characteristics of each data block for priority sorting. It should be understood that this dynamic scheduling is based on a comprehensive evaluation of the data blocks, and a weighted algorithm can be used to calculate the priority score of the data block. In practical applications, variants of algorithms such as LRU and LFU can be used, and weights can be adjusted in combination with real-time access frequency to meet the needs of different business scenarios.

[0023] Furthermore, in order to establish a comprehensive data block priority evaluation model, this embodiment introduces a multi-parameter weighted scoring mechanism. It is understandable that this evaluation model can flexibly and dynamically adjust different types of data block attributes to better match the actual workload.

[0024] Furthermore, if Figure 3 As shown, when implementing dynamic scheduling, this embodiment adopts a multi-path scheduling mechanism to dynamically allocate each data block to different cache layers according to its priority score, thereby optimizing the error distribution and ensuring load balance between cache layers. This multi-path scheduling mechanism not only improves the efficiency of data access, but also avoids the overload of a single cache layer. It should be understood that the optimization of error distribution can achieve the optimal balance of data access and storage in different cache layers by real-time monitoring of the load of each cache layer and combining it with a dynamic priority adjustment strategy.

[0025] Furthermore, if Figure 4 As shown, the load status of each cache layer is monitored in real time. Through the load monitoring module, the system can obtain the current load status of each cache layer in real time, and dynamically adjust the capacity and data distribution of the cache layer through the load balancing function, such as Figure 5 As shown in the figure, it is understandable that the design of the load balancing function should fully consider the access speed differences between cache layers and the changes in data access patterns to ensure the overall smooth operation and efficient response of the system.

[0026] Furthermore, in order to ensure the integrity of data during migration between cache layers, this embodiment implements a data consistency protection mechanism. By performing verification and error correction when migrating between data blocks, data loss and inconsistency problems caused by data errors can be effectively reduced. In practical applications, the verification mechanism can use methods such as CRC (cyclic redundancy check), and the error correction mechanism can use technologies such as Hamming Codes to automatically correct data errors when they are detected.

[0027] The benefit of this embodiment is that, through the multi-level cache management method, dynamic scheduling and load balancing of data blocks between different cache layers are realized, which significantly improves the overall efficiency of data access. At the same time, through the priority evaluation model and multi-path scheduling mechanism, refined management of high-priority data blocks is achieved, thereby improving the response speed and overall performance of the system. In addition, through the data consistency protection mechanism, the stability and consistency of data in a multi-layer cache environment are enhanced.

[0028] Embodiment 2 In order to solve the bottleneck problem of the single-layer cache management method of solid-state storage devices when processing a large amount of parallel data access, this embodiment further optimizes the implementation method, especially describes in detail the optimization method for calculating the priority score and the error distribution.

[0029] Based on the multi-level cache management method of this embodiment, the priority score P(d i ) is a comprehensive consideration of the access frequency of data blocks A(d i ), size S(d i )、Life cycle L(d i ) and importance I(d i ) are determined by four key parameters. Specifically, the priority score is calculated as follows: Among them, A(di) is the access frequency of the data block, S(d i ) is the size of data block di, L(d i ) is the life cycle of the data block, I(d i ) is the importance of the data block, and α, β, γ, and θ are weighting coefficients used to correct calculation deviations. It should be understood that this weighted scoring mechanism allows the weights of different features to be adjusted according to actual needs to adapt to different application scenarios. For example, for data access scenarios that require fast response, the weight α of access frequency can be set relatively high, while in some scenarios involving large data block transmission, the weight β of data block size may occupy a more important position.

[0030] Furthermore, this embodiment also optimizes the error distribution between cache layers and realizes load balancing between cache layers through a dynamic distribution optimization mechanism. The core of this mechanism is to calculate the priority score P(d i ,c), and use the following formula to measure and optimize the error distribution: Among them, P(d i ,c) represents data block d i The priority score in cache layer c, P avg is the average of the priority scores of all cache layers, C is the number of cache layers, and N is the total number of data blocks. It can be understood that the goal of optimizing this error distribution is to make the data block priority scores of each cache layer as close to the average as possible, so as to achieve a load balancing effect. The optimization method of error distribution can be achieved by dynamically adjusting the distribution state of data blocks in each cache layer, for example, by gradually migrating high-priority data blocks to cache layers with faster access speeds, and scheduling low-priority data blocks to cache layers with relatively slower access speeds but larger capacity.

[0031] It should be understood that in practical applications, the calculation process of error distribution optimization needs to continuously monitor the real-time load status of each cache layer and dynamically adjust the data block allocation strategy according to the changes in the load status. This can effectively avoid the system performance bottleneck problem caused by uneven load. By combining the above-mentioned priority score calculation and error distribution optimization mechanism, this embodiment can achieve more refined and dynamic data block scheduling between different cache layers, thereby significantly improving the overall performance and response speed of the system.

[0032] The benefit of this embodiment is that, through the refined calculation of priority scores and the dynamic optimization of error distribution, efficient scheduling and load balancing of data blocks in a multi-layer cache environment are achieved. Not only does it improve the overall efficiency of data access, allowing the system to better adapt to complex and diverse application scenarios, but it also maximizes the access speed and capacity of each layer of cache through dynamic priority adjustment strategies. Through this embodiment, the performance bottleneck of the traditional single-layer cache management method when processing a large amount of parallel data access can be effectively solved, providing a reliable guarantee for the stable operation of solid-state storage devices in a high-load environment.

[0033] Embodiment 3 In order to solve the performance bottleneck problem existing in the traditional single-layer cache management method, this embodiment further optimizes the implementation method of load balancing and data verification and error correction, thereby improving the overall performance of the multi-layer cache management system in the solid-state storage device.

[0034] First, this embodiment describes in detail the calculation method of the load balancing function, which is used to dynamically adjust the data distribution state between cache layers to ensure load balancing of caches at each layer. The calculation formula of the load balancing function is as follows: Where U(c) is the utilization of cache layer c, U total is the total utilization rate.

[0035] It should be understood that by calculating the sum of squares of the deviations between the utilization of each cache layer and the total utilization, this embodiment can accurately evaluate the load balancing status of each cache layer. Furthermore, when the utilization of a cache layer is significantly higher than that of other layers, the function will adjust the migration strategy of data blocks to migrate some low-priority data from the high-load layer to the low-load layer, thereby reducing the load difference between the layers and ensuring the stable and efficient operation of the system.

[0036] Furthermore, to ensure the integrity and consistency of data during migration between cache layers, this embodiment introduces a data verification and error correction mechanism. Specifically, each data migration generates and verifies the hash value H(d i ) and cyclic redundancy check code CRC(di ), which is calculated as: Among them, H(d i ) is the data block d i The hash value, CRC(d i ) is the data block d i ⊕ represents a bitwise XOR operation. It should be understood that the dual verification mechanism of hash value and cyclic redundancy check code can accurately detect and correct errors generated during data migration. For example, the hash value calculation can use advanced hash algorithms such as SHA-256, while the cyclic redundancy check can use standard algorithms such as CRC-32, thereby providing reliable error detection and correction capabilities during data block migration.

[0037] In actual applications, when a data block is migrated from one cache layer to another, the system will first calculate and store the hash value and CRC checksum of the data block, and then recheck the data block after the migration is completed. If data inconsistency is detected during the verification process, the system will re-read the data or restore it from a backup to ensure data consistency and integrity. This process not only ensures data accuracy, but also effectively prevents data corruption caused by hardware failure or transmission noise, thereby improving system reliability.

[0038] Through these optimizations, this embodiment effectively solves the technical challenges of multi-layer cache management system in data migration and load balancing, and provides good guarantee for efficient management and stable operation of data in solid-state storage devices. The load balancing function not only improves the performance of the system under high load environment, but also realizes the optimal configuration of each layer of cache by dynamically adjusting the utilization rate of cache layer. The data verification and error correction mechanism provides double guarantee for data integrity, ensuring the consistency and integrity of data during multiple migrations.

[0039] The benefit of this embodiment is that by describing in detail the implementation methods of load balancing and data verification and error correction, the performance and reliability of the multi-layer cache management system are fully optimized. Through the optimization of this embodiment, efficient scheduling and load balancing of data blocks between different cache layers are achieved, and the consistency and integrity of data are ensured during data migration. This not only improves the overall performance of the system, but also enhances the system's ability to operate stably in complex application environments, providing a reliable solution for the application of solid-state storage devices in high-load environments.

[0040] Embodiment 4 In order to address the shortcomings of traditional single-layer cache management in processing real-time data access, this embodiment further refines the calculation steps of the data block priority score, and ensures that the calculation of the data block priority score is accurate and dynamically adaptable through the application of a multi-parameter model.

[0041] Specifically, the access frequency A(d i ) performs exponential smoothing weighted calculation to reflect the recent access frequency in real time. The formula for exponential smoothing weighted calculation is as follows: Among them, λ is the smoothing coefficient, A past (d i ) is the past visit frequency, A current (d i ) is the current cycle access frequency. It should be understood that the smoothing coefficient λ controls the weight distribution of historical access frequency and current access frequency. The larger λ is, the greater the impact of historical access frequency; conversely, the greater the impact of current access frequency. It can be understood that in this way, the system can dynamically reflect recent data access changes and adjust the cache scheduling strategy in real time to improve cache hit rate and access efficiency.

[0042] Furthermore, in this embodiment, the life cycle L(d i ) and predict future access needs based on historical access patterns. The life cycle is calculated as follows: Among them, T create (d i ) is the creation time of the data block, T last (d i ) is the last access time, and μ is the weight coefficient. It should be understood that this calculation method can predict possible future access needs by monitoring the time interval from the creation of a data block to the last access, combined with historical access records. It not only helps the system consider the activity of data blocks when scheduling data, but also adopts a longer-term storage strategy for data blocks with a longer life cycle, avoiding frequent data migration and resource waste.

[0043] Furthermore, this embodiment combines the user-set policy and historical access records to determine the importance of the data block I(d i ) for scoring, the scoring formula is as follows: Among them, w k is the weight coefficient, f k (d i) is the function value of the kth importance indicator, and K is the number of importance indicators. It can be understood that this scoring mechanism can flexibly adjust the weights of importance indicators according to different application scenarios and user needs, thereby ensuring that more important data is processed first during cache scheduling. For example, users can define some key indicators according to the specific needs of the application, such as the sensitivity, usage frequency, and business relevance of data blocks, and adjust the weights of these indicators w k , the system can evaluate and schedule data priorities more intelligently.

[0044] The benefit of this embodiment is that by introducing exponential smoothing weighted calculation, life cycle monitoring and importance scoring mechanism, the calculation of data block priority score is refined. In practical applications, the multi-parameter model not only enhances the system's sensitivity to real-time access frequency, enabling the scheduling strategy to respond more promptly to frequently changing access patterns, but also ensures that high-priority data blocks can obtain better storage locations through life cycle monitoring and importance scoring, thereby improving the access performance of the overall system. Through this multi-parameter model, this embodiment achieves efficient data scheduling between cache layers and optimizes the overall performance of solid-state storage devices.

[0045] Embodiment 5 In order to solve the data integrity problem during data migration in multi-level cache management, this embodiment further optimizes the data verification and error correction method to ensure that data corruption can be efficiently identified and corrected during data migration, thereby improving the overall reliability and data security of the system.

[0046] Specifically, this embodiment introduces a detailed verification code damaged area recording mechanism for each data migration operation. During the data migration process, the system will record the verification code of each data block and identify the damaged area. When the data verification code is damaged, the system will use the shift error correction method to repair it. The calculation formula is as follows: Among them, d i represents the damaged data bit, n is the length of the damaged data bit, and m is the number of shift bits. It should be understood that through this shift error correction mechanism, the system can flexibly adjust the offset position of the data bit, thereby repairing the errors caused by the partial bit damage of the data block. In practical applications, the selection of the number of shift bits m will be dynamically adjusted based on the distribution characteristics of the damaged area and the specific damage situation of the check code.

[0047] Furthermore, in order to ensure the integrity and consistency of the modified data, this embodiment performs a comprehensive recheck on the data blocks after data migration. The recheck is completed by calculating and verifying the modified cyclic redundancy check code CRC(i), and its calculation formula is as follows: Among them, S correct (i) is the corrected data, and CRC(i) is the corrected cyclic redundancy check code. It can be understood that through the recheck process, the system not only verifies the integrity of the data after correction, but also detects new possible damaged areas, so as to further perform refined correction operations. This ensures that after each data migration, all data blocks have high availability and data consistency, avoiding system instability and data loss caused by data damage.

[0048] In addition, this embodiment adopts a distributed processing mechanism during the data migration and verification process, and improves the efficiency of data verification and error correction through the collaborative work between distributed storage nodes. After receiving the data block migration request, each storage node will independently perform preliminary verification and damage records, and feed back the results to the master control node. The master control node will comprehensively review the feedback results of each storage node and adjust the shift correction strategy and re-verification process. It should be understood that this distributed coordination working mode not only improves the verification and correction efficiency of the system, but also further improves the fault tolerance of the system through multi-node collaboration.

[0049] The benefit of this embodiment is that by describing in detail the specific implementation method of data verification and error correction, the system can efficiently identify and correct data corruption during data migration to ensure data integrity and consistency. The combination of shift error correction and recheck mechanism enables data to have higher reliability in each cache layer after migration, avoiding system failure and data loss caused by data corruption.

[0050] Embodiment 6 In order to solve the delay problem caused by data migration between cache layers, this embodiment further optimizes the migration delay optimization mechanism to ensure that data migration is carried out efficiently and with low latency, thereby improving the overall system performance.

[0051] Specifically, this embodiment introduces a migration delay calculation method in the data migration process, and calculates the migration delay of each data block. The calculation formula is as follows: Among them, S(d i ) is the size of the data block di, B is the bandwidth between cache layers, ε is the migration delay coefficient, and N is the number of cache layers. It should be understood that by calculating the migration delay of the data block, this embodiment can accurately evaluate the delay time of different data blocks during the migration process, thereby optimizing the migration order according to the delay time. At the same time, the introduction of the delay coefficient ε and the number of cache layers N further refines the dynamic evaluation of the migration delay, ensuring that the calculation results can truly reflect the delay situation in the actual migration.

[0052] Furthermore, according to the calculation result of the migration delay, this embodiment preferentially schedules high-priority data blocks for migration. Specifically, when the system detects that data block migration is required, it first determines the priority of the data block according to the aforementioned priority scoring mechanism, and then calculates the migration delay for the high-priority data block. It should be understood that preferential scheduling of high-priority data blocks can ensure that in multi-layer cache management, important data can be migrated to the appropriate cache layer in a timely manner, reducing access delays and improving data response speed.

[0053] The benefit of this embodiment is that by introducing a migration delay optimization mechanism, the system can accurately evaluate and optimize the migration delay during data migration, ensuring that high-priority data blocks can be migrated in a timely manner. At the same time, through dynamic bandwidth allocation and a prediction mechanism based on historical data, the system resource utilization efficiency and overall performance are further improved. Through the optimization of this embodiment, efficient and low-latency data migration in multi-layer cache management is achieved, providing technical support for the efficient operation of solid-state storage devices in complex application environments.

[0054] Embodiment 7 In order to solve the problems of dynamic scheduling of data blocks, load balancing, data migration, verification and error correction, and efficient read and write operations in a multi-level cache management system, this embodiment further proposes the characteristics and interaction methods of each functional module to form an efficient and reliable multi-level cache management system.

[0055] The multi-layer cache in this embodiment not only includes a high-speed cache layer, a medium-speed cache layer and a low-speed cache layer, but each cache layer assumes different important responsibilities according to its performance characteristics. The high-speed cache layer is mainly responsible for storing frequently accessed data. It has the highest read and write speed and is suitable for data blocks that need to be accessed frequently. The medium-speed cache layer is used to store data with a general access frequency, and serves as a data buffer between the high-speed cache layer and the low-speed cache layer. The low-speed cache layer specifically stores the data with the lowest access frequency to ensure the maximization of storage capacity. It should be understood that this hierarchical storage mechanism not only optimizes the efficiency and access speed of data storage, but also improves the overall performance of the system.

[0056] Furthermore, the cache scheduling module in this embodiment is responsible for dynamically scheduling data blocks to the appropriate cache layer according to the real-time demand for data access. The scheduling module determines that each data block is stored in the most appropriate cache layer by deeply analyzing the data access frequency and the characteristics of the data itself. The core of the scheduling strategy is to improve the hit rate of the cache layer, thereby reducing the overall access delay of the system. It can be understood that this scheduling mechanism not only improves the utilization efficiency of a single-layer cache, but also realizes the collaborative work between multiple layers of cache through dynamic migration of data.

[0057] Furthermore, in order to solve the load balancing problem in the multi-layer cache system, the present embodiment introduces a load monitoring module. This module monitors the load status of each cache layer in real time, including the usage of cache capacity and data access frequency. Combined with the load balancing function, the system can dynamically adjust the capacity and data distribution of each cache layer to avoid overload or vacancy of a certain cache layer. The specific implementation of the load balancing function can be adjusted according to the real-time data of the system, for example, through hash functions, minimum load priority principles, etc. It should be understood that the key to the dynamic adjustment of the load monitoring module between cache layers is to maintain the stable operation of each cache layer, thereby improving the overall efficiency of the system.

[0058] Furthermore, in order to ensure the integrity and accuracy of data when migrating between data cache layers, a data migration module and a data verification and error correction module are designed in this embodiment. The data migration module is responsible for the specific data block migration task, combined with the migration delay optimization mechanism to minimize the delay during the migration process. At the same time, before and after the migration process starts, the data verification and error correction module will verify the data block and implement necessary error correction. By using a cyclic redundancy check (CRC) and a shift error correction algorithm, it is ensured that the data remains intact and reliable after migration. It can be understood that this dual protection mechanism not only reduces the risk of data corruption during the migration process, but also improves the fault tolerance of the system.

[0059] Furthermore, the parallel processing module is designed to support multi-threaded parallel read and write operations and improve the efficiency of data access. When the system performs large-scale data operations, it can distribute a large number of read and write requests through multi-threaded processing to avoid the performance bottleneck caused by single-threaded operations. Parallel processing not only improves the execution speed of a single operation, but also enhances the concurrent access capability of the system, which can significantly improve the user experience, especially in high-concurrency scenarios.

[0060] Through this embodiment, the multi-level cache management system can achieve efficient scheduling, load balancing, data migration and error checking of data blocks through the organic combination of various functional modules, further improving the overall performance and reliability of the system. The logical connection between each module is close to ensure data integrity and access efficiency during system operation.

[0061] Embodiment 8 In order to solve the efficiency and accuracy problems in the data allocation and migration process in the multi-level cache system, this embodiment further refines the management mechanism of the cache scheduling module, adopts multiple priority evaluation algorithms and data balancing mechanisms to achieve refined management of data allocation at each cache layer.

[0062] Specifically, the cache scheduling module in this embodiment optimizes the data allocation and migration strategy between cache layers by real-time calculation of data block priorities. Specifically, when evaluating the priority of data blocks, the system comprehensively considers multiple factors such as access frequency, data block size, life cycle and importance. It should be understood that by introducing multi-dimensional evaluation criteria, the system can more accurately identify which data should be allocated to the cache layer first, thereby reducing the access delay of key data.

[0063] Furthermore, the priority evaluation algorithm of this embodiment is not limited to a single evaluation criterion, but ensures the comprehensiveness and accuracy of the evaluation results through the combination of multiple algorithms. For example, when evaluating the access frequency of a data block, the system will use a sliding window algorithm based on the access history records to capture the short-term and long-term access patterns of the data block. In terms of data block life cycle evaluation, the system will comprehensively consider factors such as the storage length of the data since it was first written to the cache and the most recent access time to ensure that data with a short life cycle but frequent access is given priority. It should be understood that this multi-dimensional priority evaluation mechanism can help the system dynamically adjust the data allocation of the cache layer in a complex and changeable storage environment to adapt to different application scenarios.

[0064] In order to further improve the data distribution balance between cache layers, this embodiment also introduces a data balancing mechanism. This mechanism monitors the capacity utilization and data distribution of each cache layer in real time through the load monitoring module, and dynamically adjusts the migration strategy of data blocks between cache layers in combination with the priority evaluation results. It can be understood that the data balancing mechanism not only ensures that high-priority data blocks can be quickly migrated to the cache layer, but also avoids load imbalance between cache layers, preventing a certain layer of cache from being overloaded or vacant and affecting the overall performance of the system.

[0065] During the implementation process, this embodiment adopts a hierarchical and progressive cache scheduling strategy. Specifically, when the cache layer load is close to saturation, the system will use a priority evaluation algorithm to determine which data blocks should be migrated to the medium-speed cache layer or the low-speed cache layer, while ensuring that the data blocks still in the cache layer are the most frequently accessed and highest priority data. This dynamic migration strategy based on priority evaluation and data balancing mechanism ensures that data blocks of different priorities can be stored in the most suitable cache layer, minimizes unnecessary data movement between cache layers, and improves the storage efficiency of the overall system.

[0066] In addition, this embodiment also considers the consistency and integrity of data blocks during the migration process. In order to ensure that no data errors occur during the migration process, the data verification and error correction module will verify the data blocks before and after the data migration, and use algorithms such as cyclic redundancy check (CRC) to implement error correction to ensure the integrity of the data after migration between different cache layers. It can be understood that this verification and error correction mechanism provides higher reliability for the system, ensuring that in a complex storage management environment, the data migration process will not introduce errors or cause data loss.

[0067] The benefit of this embodiment is that by introducing a priority evaluation algorithm and a data balancing mechanism, the system can dynamically schedule and finely manage the data allocation and migration of each layer of cache according to the multi-dimensional characteristics of the data block. Combined with real-time load monitoring and the verification and error correction functions during the migration process, the system not only improves the utilization rate of cache resources, but also improves the response speed of data access and the overall reliability of the system. These optimization measures lay a solid foundation for the efficient operation of solid-state storage devices in high-concurrency and complex application environments.

[0068] Embodiment 9 In order to solve the problems of efficient scheduling, migration, verification and error correction of storage data faced by solid-state storage devices in multi-layer cache systems, this embodiment proposes a multi-level cache management device for solid-state storage devices, which combines the optimized collaboration of storage devices and processors to achieve efficient data management and processing.

[0069] The multi-level cache management device in this embodiment is closely integrated with the multi-level cache management system through the storage device to ensure the effective storage and management of data in the system. Specifically, the storage device is used to store data scheduled by the cache management system, which includes data blocks dynamically allocated and migrated in the high-speed cache layer, the medium-speed cache layer and the low-speed cache layer. The storage device adopts high-performance non-volatile memory technology to ensure that data access requests can be quickly responded to during the multi-layer cache scheduling process. At the same time, the storage device cooperates with each layer of cache in the cache system to achieve efficient migration of data blocks between different cache layers, ensuring that frequently accessed data is preferentially stored in the high-speed cache layer, while infrequently accessed data is reasonably stored in the medium-speed or low-speed cache layer according to priority and life cycle. It should be understood that the design goal of the storage device is to optimize the utilization of storage space through a multi-layer cache mechanism to maximize the overall performance of the system.

[0070] Furthermore, the processor in this embodiment is not only responsible for executing the data scheduling operation in the cache management system during the operation of the system, but also responsible for managing the migration process of data between cache layers. Specifically, the processor dynamically schedules the data blocks to the most suitable cache layer by analyzing the priority of the data blocks in real time. This process is based on the results of multiple priority evaluation algorithms and data balancing mechanisms to ensure that data blocks of different priorities can be scheduled to the corresponding cache layer according to their access frequency, size, life cycle and other characteristics. The multi-threaded architecture of the processor supports parallel read and write operations, and can maintain efficient processing performance in scenarios where multiple data blocks are migrated or scheduled at the same time. This multi-threaded parallel processing capability improves the processing efficiency of the system in high-concurrency scenarios.

[0071] Furthermore, the processor is also responsible for performing verification and error correction operations when performing data migration between cache layers. In the process of migrating data blocks from one cache layer to another, the system introduces a cyclic redundancy check (CRC) and error correction mechanism. The processor verifies the migrating data blocks to ensure that the data is not damaged or lost during the migration process. It can be understood that this verification function of the processor not only ensures the consistency and integrity of the system data, but also improves the reliability of the system, especially in the face of large-scale data migration, it can effectively reduce the risk of data corruption.

[0072] The benefit of this embodiment is that by organically combining the processor, storage device and multi-level cache management system, the system can efficiently schedule and manage data blocks, realize multi-threaded parallel operation, and ensure the integrity and accuracy of data during the migration process. The processor's efficient scheduling algorithm, multi-dimensional priority evaluation mechanism and real-time data verification and error correction function enable the multi-level cache management device to maintain excellent performance in high-concurrency, large data access scenarios.

[0073] Embodiment 10 In order to solve the problem of low data storage and transmission efficiency in the multi-level cache management process of existing solid-state storage devices, this embodiment further optimizes the data scheduling and transmission mechanism in the cache management method, aiming to improve the overall system performance and data processing efficiency. By storing the corresponding computer program on a computer-readable storage medium, the multi-level cache management method of the solid-state storage device can be implemented. In turn, data can be efficiently managed and dynamically adjusted between the high-speed cache, the medium-speed cache and the low-speed cache.

[0074] In this embodiment, the cache uses a storage medium based on NVMe technology, which is selected to store frequently accessed hot data because of its high bandwidth and low latency. This configuration ensures that the processor can quickly access the required data, thereby improving the response speed and processing efficiency of the system. It should be understood that the cache is mainly for data that is frequently accessed in a short period of time to reduce the load on the processor and reduce waiting time. This method optimizes the I / O performance of the system.

[0075] Furthermore, the medium-speed cache uses SATA interface technology. Although its data transmission rate is lower than NVMe, its capacity is larger and it is suitable for storing data with low access frequency. In order to effectively schedule data between the high-speed cache and the medium-speed cache, this embodiment introduces a dynamic scheduling algorithm based on data access frequency and heat. When the access frequency of the data drops to a certain threshold, the cache management system will automatically migrate the data from the high-speed cache to the medium-speed cache. This process ensures that the resources of the high-speed cache area can be used efficiently, and the medium-speed cache area maintains the smooth operation of the system by storing cooled data. It can be understood that the scheduling algorithm is based on real-time data access analysis to ensure that the most commonly used data can always be accessed in the cache with the lowest latency.

[0076] Furthermore, the low-speed cache layer uses QLC NAND storage technology, which is mainly used to store cold data with the lowest access frequency. QLC NAND is an ideal choice for storing large amounts of cold data due to its high storage density and low cost. Furthermore, in order to improve the storage efficiency and data integrity of the low-speed cache layer, this embodiment adopts an optimization method of block storage combined with ECC technology. Through this method, the system can efficiently manage large data blocks and perform automatic error correction during data storage to ensure data integrity and stability. It can be understood that this structural design not only improves the storage density of the low-speed cache layer, but also improves the reliability of data storage through the error correction mechanism.

[0077] Through this embodiment, the multi-level cache management method realizes efficient data storage and dynamic scheduling of solid-state storage devices, significantly improving the overall performance and data processing capabilities of storage devices. The reasonable layout and dynamic management strategy of high-speed cache, medium-speed cache and low-speed cache not only optimizes the speed of data access, but also enhances the security of data storage.

[0078] The above embodiments further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-level cache management method for a solid-state storage device, characterized in that: The method comprises the following steps: A multi-layer cache is configured in the solid-state storage device, including a high-speed cache layer, a medium-speed cache layer and a low-speed cache layer, each layer of cache having a different access speed and capacity for storing data blocks of different priorities and access frequencies; Dynamically dispatching the data block to an appropriate cache layer through a cache allocation algorithm according to the real-time access frequency, size, life cycle and importance of the data block; Establishing a comprehensive data block priority evaluation model to calculate the priority score of each of the data blocks; A multi-path scheduling mechanism is used to dynamically allocate data blocks to different cache layers according to the priority scores, and error distribution is optimized to balance the load between cache layers; Monitor the load status of each cache layer in real time, and dynamically adjust the capacity and data distribution of the cache layer through the load balancing function; A data consistency protection mechanism is implemented to verify and correct the data blocks to ensure the integrity of the data during migration between cache layers.

2. The multi-level cache management method of a solid-state storage device according to claim 1, characterized in that: The priority score is calculated as follows: , where A(di) is the access frequency of the data block, S(d i ) is the size of data block di, L(d i ) is the life cycle of the data block, I(d i ) is the importance of the data block, α, β, γ and θ are weighting coefficients used to correct calculation deviations; The calculation method of the optimized error distribution is: , where P(d i ,c) represents data block d i The priority score in cache layer c, P avg is the average priority score of all cache layers, C is the number of cache layers, and N is the total number of data blocks.

3. The multi-level cache management method of a solid-state storage device according to claim 2, characterized in that: The calculation formula of the load balancing function is: , where U(c) is the utilization of cache layer c, U total is the total utilization rate; The calculation formula for checking and correcting the data block is: , where H(d i ) is the data block d i The hash value, CRC(d i ) is the data block d i The cyclic redundancy check code of , ⊕ represents the bitwise exclusive OR operation.

4. The multi-level cache management method for a solid-state storage device according to claim 2, characterized in that: The step of calculating the priority score of each data block comprises: The access frequency A(d i ) performs exponential smoothing weighted calculation to reflect the recent access frequency in real time. The formula is as follows: , where λ is the smoothing coefficient, A past (d i ) is the past visit frequency, A current (d i ) is the current cycle access frequency; The life cycle L(d i ) to monitor and predict future access needs based on historical access patterns. The formula is as follows: , where T create (d i ) is the creation time of the data block, T last (d i ) is the last access time, μ is the weight coefficient; Combined with the user-set policy and historical access records, the importance of the data block I(d i ) is scored, and the formula is as follows: , where w k is the weight coefficient, f k (d i ) is the function value of the kth importance index, and K is the number of importance indexes.

5. The multi-level cache management method for a solid-state storage device according to claim 3, characterized in that: The steps of checking and correcting the data block are: For each data migration, the checksum damaged area is recorded and shift error correction is performed. The calculation formula of the shift error correction is as follows: , where d i represents the damaged data bit, n is the length of the damaged data bit, and m is the number of shift bits; The data migration is re-inspected for complete data, and the re-inspection is performed in the following manner: , where S correct (i) is the corrected data, and CRC(i) is the corrected cyclic redundancy check code.

6. The multi-level cache management method of a solid-state storage device according to claim 5, characterized in that: The method further includes adopting a migration delay optimization mechanism for data migration between cache layers to reduce the delay generated during the migration process, and the steps of the migration delay optimization mechanism include: For each of the data blocks, the migration delay is calculated as follows: , where S(d i ) is the size of the data block di, B is the bandwidth between cache layers, ε is the migration delay coefficient, and N is the number of cache layers; Based on the calculation result, the data blocks with high priority are preferentially scheduled for migration.

7. A multi-level cache management system for a solid-state storage device, characterized in that: The system comprises: Multi-layer cache, including high-speed cache layer, medium-speed cache layer and low-speed cache layer; Cache scheduling module, used to perform dynamic scheduling of data blocks to appropriate cache layers; The load monitoring module is used to monitor the load status of each cache layer in real time and dynamically adjust the capacity and data distribution of the cache layer through the load balancing function; The data migration module is used to perform the data migration process between cache layers; A data check and error correction module, used to implement check and error correction when migrating data between cache layers; The parallel processing module supports multi-threaded parallel read and write operations to improve data access efficiency.

8. The multi-level cache management system of a solid-state storage device according to claim 7, characterized in that: The cache scheduling module uses multiple priority evaluation algorithms and data balancing mechanisms to finely manage data allocation and migration at each cache layer, calculate data block priority in real time, and comprehensively evaluate factors such as access frequency, data block size, life cycle and importance.

9. A multi-level cache management device for a solid-state storage device, characterized in that: The device comprises: A multi-level cache management system for a solid-state storage device as claimed in any one of claims 7 or 8; A storage device, used to store data scheduled by the cache management system; and The processor is used to dynamically schedule data blocks to appropriate cache layers, execute data migration processes between cache layers, implement checksum and error correction when migrating data between cache layers, and perform multi-threaded parallel read and write operations.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the multi-level cache management method for a solid-state storage device according to any one of claims 1 to 6 can be implemented.

Images