Method, device, storage medium and electronic device for writing data into storage system

Abstract

The embodiments of the present application provide a method, device, computer-readable storage medium and electronic device for writing data to a storage system, the method comprising: obtaining historical data writing conditions of multiple logical address segments of the storage system; determining the writing heat of the multiple logical address segments respectively according to the historical data writing conditions of the multiple logical address segments; and performing heat grading on the data to be written according to the writing heat of the multiple logical address segments to determine the heat level of each data to be written. The data to be written with the same heat level are written into the same physical address segment of the storage system. The data to be written with the same heat level are written into the same physical address segment, so that the data is written according to the classification of heat level. Since the heat of the data in the physical address segment is relatively consistent, when the data is recovered, since the data of different heat levels have been classified, the invalid data can be recovered more effectively, the amount of data migration is reduced, and the recovery efficiency is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and more specifically, to a method and device for writing data into a storage system, a computer-readable storage medium, and an electronic device. Background Art

[0002] With the development of high-end server performance, flash memory technology has emerged, such as all-flash storage systems composed of all-flash hard drives, which have the advantages of stronger performance and lower latency. In all-flash storage systems, data writing and recycling management are crucial to maintaining system performance and extending device life.

[0003] In the related art, an all-flash storage system is divided into a plurality of physical address segments. When the storage space in the physical address segment is used to a certain extent, the garbage data in the physical address segment is recycled, thereby ensuring the utilization rate of the storage space.

[0004] However, the physical address segment in the related art has low efficiency when performing data recovery, which affects the performance of the storage system. Summary of the invention

[0005] The embodiments of the present application provide a method, device, computer-readable storage medium and electronic device for writing data to a storage system, which can write data according to heat classification, so as to improve the efficiency of data recovery.

[0006] According to an embodiment of the present application, a method for writing data to a storage system is provided, the method comprising: obtaining historical data writing conditions of multiple logical address segments of the storage system; determining the writing heat of the multiple logical address segments respectively according to the historical data writing conditions of the multiple logical address segments; performing heat grading on the data to be written according to the writing heat of the multiple logical address segments to determine the heat level of each data to be written, wherein each data to be written corresponds to a logical address segment; and writing the data to be written with the same heat level into the same physical address segment of the storage system.

[0007] In an exemplary embodiment, based on the historical data writing status of multiple logical address segments, the amount of written data, the number of write requests, and the location distribution of written data for the multiple logical address segments within a preset time length are determined respectively; based on the amount of written data, the number of write requests, the location distribution of written data for the multiple logical address segments within the preset time length, and the corresponding weights, the write heat of the multiple logical address segments is determined.

[0008] In an exemplary embodiment, based on the historical data writing status of multiple logical address segments, the amount of data written and the number of times data are written to the multiple logical address segments within a preset time length are counted to respectively determine the amount of data written and the number of times data are written to the multiple logical address segments within the preset time length; the median values ​​of the logical addresses of the multiple logical address segments are respectively determined; and based on the average values ​​of the logical addresses of the write requests for the multiple logical address segments within the preset time length and the median values ​​of the logical addresses, the location distribution of the write data of the multiple logical address segments within the preset time length is respectively determined.

[0009] In an exemplary embodiment, the method further includes: determining the historical life cycle of data in multiple physical address segments of the storage system, wherein the life cycle is the length of time from when data is written to the physical address segment to when data is migrated out of the physical address segment; and determining a preset length of time based on the historical life cycle of data in multiple physical address segments.

[0010] In an exemplary embodiment, after determining the write heat of multiple logical address segments based on the amount of written data, the number of write requests, the location distribution of written data, and the corresponding weights of the multiple logical address segments within a preset time period, the method also includes: recording the write heat of the logical address segments within multiple consecutive preset time periods; assigning corresponding weights to the multiple consecutive preset time periods according to the time intervals between the multiple consecutive preset time periods and the current moment; determining the current write heat of the logical address segment according to the write heat of the logical address segment within the multiple consecutive preset time periods and the weights corresponding to the multiple consecutive preset time periods.

[0011] In an exemplary embodiment, data to be written are heat graded according to the write heat of multiple logical address segments to determine the heat level of each data to be written, including: determining the association between the data to be written and the multiple logical address segments; in a preset heat grade table, querying and obtaining the heat level corresponding to the data to be written according to the association, wherein the heat grade table is used to indicate the correspondence between the write heat of the logical address segment and the heat level; and establishing an index information table based on the heat level of each data to be written, the index information table is used to indicate the heat level of each data to be written.

[0012] In an exemplary embodiment, data to be written with the same heat level are written into the same physical address segment of a storage system, including: determining the amount of data to be written at each heat level; determining the writing order of the data to be written at each heat level according to the amount of data to be written at each heat level; and writing the data to be written with the same heat level into the same physical address segment of the storage system in sequence according to the writing order.

[0013] In an exemplary embodiment, data to be written with the same heat level are written into the same physical address segment of the storage system in sequence according to the write order, and also includes: according to the write order, write requests for data to be written with the same heat level are aggregated in the same data management block in sequence; when the write requests aggregated in the data management block reach a preset granularity, the write requests aggregated in the data management block are executed to flush the data to be written with the same heat level to the same physical address segment of the storage system in sequence.

[0014] In an exemplary embodiment, before aggregating write requests for data to be written with the same heat level in the same data management block in sequence according to the write order, the method also includes: allocating corresponding data management blocks for data to be written of each heat level according to the heat level of the data to be written, and identifying the heat level of the data management block; and identifying the order of the data management blocks according to the write order.

[0015] In an exemplary embodiment, write requests for data to be written with the same heat level are aggregated in the same data management block, and also include: determining the write mode of the data to be written with the same heat level, wherein the write mode includes sequential writing and random writing; and aggregating write requests for data to be written with the same heat level and the same write mode in the same data management block.

[0016] In an exemplary embodiment, the method also includes: when the remaining space in the physical address segment is less than a preset threshold, respectively determining the proportion of invalid data in multiple data management blocks of the physical address segment and the existence duration of the invalid data in the multiple data management blocks; and determining the order of recovering invalid data from the multiple data management blocks based on the proportion of invalid data in the multiple data management blocks and the existence duration of the invalid data in the multiple data management blocks.

[0017] Another aspect of the present application provides a device for writing data to a storage system, the device comprising: a write status acquisition module, used to acquire the historical data writing status of multiple logical address segments of the storage system; a write heat determination module, used to determine the write heat of multiple logical address segments respectively according to the historical data writing status of the multiple logical address segments; a heat level determination module, used to perform heat grading on the data to be written according to the write heat of the multiple logical address segments, and determine the heat level of each data to be written, wherein each data to be written corresponds to a logical address segment; a write execution module, used to write the data to be written with the same heat level into the same physical address segment of the storage system.

[0018] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0019] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0020] According to another embodiment of the present application, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0021] The data writing method of the storage system of the present application first collects the historical data writing situation, so as to analyze which logical address segments are frequently accessed and which segments are less used, thereby providing a data basis for the subsequent heat evaluation and ensuring the accuracy and effectiveness of the subsequent steps. Then, based on the historical data writing situation, the writing heat of the logical address segment can be determined, which is convenient for the subsequent data classification, so as to more efficiently perform data aggregation and space management. Then, the data to be written is classified according to the heat. Through the heat classification, the data in the cache is pre-classified, which is convenient for the system to quickly select and aggregate data according to the data heat and write it into the physical address segment, thereby reducing the delay of data processing and improving the efficiency of data writing. At the same time, it also helps to optimize the use of storage space, reduce the overhead of garbage collection, and improve the performance of the storage system. Finally, the data to be written of the same heat level is written into the same physical address segment, and the classified writing of the data according to the heat level is realized. Since the heat of the data in the physical address segment is relatively consistent, when the data is recovered, since the data of different heats have been classified, the invalid data can be recovered more effectively, the amount of data migration is reduced, and the recovery efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 It is a hardware structure block diagram of a server device of a method for writing data to a storage system according to an embodiment of the present application;

[0024] Figure 2 is a flow chart of a method for writing data to a storage system according to an embodiment of the present application;

[0025] Figure 3 This is a second flowchart of a method for writing data to a storage system according to an embodiment of the present application;

[0026] Figure 4 This is a third flowchart of a method for writing data to a storage system according to an embodiment of the present application;

[0027] Figure 5 This is a fourth flowchart of a method for writing data to a storage system according to an embodiment of the present application;

[0028] Figure 6 is a fifth flowchart of a method for writing data to a storage system according to an embodiment of the present application;

[0029] Figure 7 is a sixth flowchart of a method for writing data to a storage system according to an embodiment of the present application;

[0030] Figure 8 is a seventh flowchart of a method for writing data to a storage system according to an embodiment of the present application;

[0031] Fig. 9 is a flowchart of a method for writing data to a storage system according to an embodiment of the present application;

[0032] Fig.10 is a ninth flowchart of a method for writing data to a storage system according to an embodiment of the present application;

[0033] Fig.11 is a tenth flowchart of a method for writing data to a storage system according to an embodiment of the present application;

[0034] Fig.12 is a flowchart of a method for writing data to a storage system according to an embodiment of the present application;

[0035] Fig.13 is a structural block diagram of a device for writing data into a storage system according to an embodiment of the present application;

[0036] Fig.14 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0039] The method embodiment of the data writing method of the storage system provided in the embodiment of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a server device of a method for writing data to a storage system according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0040] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method of writing data to the storage system in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0041] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0042] As described in the background technology, the physical address segment in the related technology is inefficient when performing data recovery, which affects the performance of the storage system. The inventors have found that this is because the related technology has some limitations in data writing and space management, resulting in low efficiency in data recovery in the physical address segment, which in turn affects the performance of the entire storage system. Specifically, the existing all-flash storage system does not distinguish between the writing frequency and distribution characteristics of data when writing data, that is, hot and cold data (data with high or low access frequency) and random or sequential write data are mixed and written in the same data block. This non-classified writing strategy will lead to low recovery efficiency, that is, cold data and hot data are mixed and written in the same data block, resulting in the need to process active data and inactive data at the same time during garbage collection, which increases the complexity and overhead of data migration. Due to the random distribution of data in the data block, it may be necessary to frequently migrate valid data during garbage collection to clear the space where invalid data is located, which not only consumes a lot of system resources, but also may introduce additional delays, affecting the overall response time of the storage system. The write amplification effect will also increase because randomly written data needs to be read and rewritten to new data blocks during garbage collection, which increases the number of disk writes, resulting in a write amplification effect that affects the life of the disk and the long-term performance of the storage system.

[0043] In view of the above problems, a method for writing data to a storage system is provided in this embodiment. Figure 2 As shown, the method includes the following steps S200-230:

[0044] Step S200, obtaining historical data writing status of multiple logical address segments of the storage system.

[0045] Specifically, the storage system maintains various data structures and logs, recording historical read and write operations, including information such as the written logical address, data volume, and timestamp. In order to more accurately assess the popularity of data, these historical data writes can be collected and analyzed. Specifically, the logical volume address space can be divided into multiple logical address segments with a certain continuous sector as the granularity, and all write records (including the number of write requests, the amount of written data, and the write location distribution) of each segment within a period of time can be collected according to a certain statistical period.

[0046] Exemplarily, the historical data writing situation can be obtained by regularly querying the metadata log inside the storage system or using a special monitoring tool. The writing situation of each segment includes but is not limited to the number of writes, the total amount of writes, the distribution of write time, and the distribution of write locations. For example, the system can be set to perform heat statistics every 24 hours, and the number of writes and the amount of written data for each segment within the statistical interval.

[0047] Step S210 , determining the write heat of the plurality of logical address segments respectively according to the historical data writing status of the plurality of logical address segments.

[0048] Specifically, write heat is an indicator that quantifies the access activity of data. Based on the analysis of historical write data, a heat value can be assigned to each logical address segment. The high or low heat value directly reflects the access frequency and importance of the data in the segment.

[0049] Exemplarily, the system uses a weighted calculation formula to determine the write heat according to the collected write times and data volume, as well as the distribution of write locations. For example, the heat calculation formula can be: Heat = Write times weight * Write times + Write data volume weight * Write data volume + Write location distribution weight * Write location distribution consistency. The consistency of the write location distribution here can be measured by calculating the offset of the address distribution, that is, the closeness of the average value of the address distribution to the middle address of the segment.

[0050] Step S220 , performing heat grading on the data to be written according to the write heats of the plurality of logical address segments to determine a heat level of each data to be written.

[0051] Each data to be written corresponds to a logical address segment. The logical address space of the logical volume of the storage system is divided into multiple continuous logical address segments. These logical address segments can be regarded as sub-areas in the logical address space, and their size and position are pre-defined according to the configuration and requirements of the storage system. When data is written to the logical volume, it will be assigned to a specific logical address segment according to its own logical address. In other words, the logical address of each data to be written determines the target address of its write operation, which also determines which logical address segment the data belongs to. This process is usually completed automatically by the file system or storage management layer of the storage system without direct user intervention, so each data to be written will have a corresponding logical address segment. The storage system can collect all write records (including the number of write requests, the amount of written data, and the distribution of write locations) of each logical address segment within a period of time according to a certain statistical period. By analyzing these write records, the system can identify which logical address segments are write hotspots, that is, they are hot.

[0052] Specifically, the buffer pool plays the role of intermediate storage in the storage system, and is used to temporarily store the host's write request for subsequent data aggregation and write optimization. The data to be written in the buffer pool is graded according to the popularity of the logical address segment to which the data belongs.

[0053] For example, a heat index can be established for the data in the cache, and the data can be classified and stored in cache queues or linked lists of different heat levels according to the heat of the logical address segment corresponding to the data. For example, a multi-level cache structure is used, in which each heat level corresponds to a cache queue, hot data will be stored in a high-priority queue, and cold data will be stored in a low-priority queue.

[0054] Step S230, writing the to-be-written data with the same heat level into the same physical address segment of the storage system.

[0055] Specifically, the physical address segment is a virtualization unit used to manage physical storage space in the storage system, which is composed of multiple data management blocks. This step aims to write data with the same popularity in the cache to the same data management block to achieve aggregate writing of data at the physical level.

[0056] Specifically, in storage systems, especially all-flash storage systems, there is a connection between logical address segments and physical address segments (Solid State Disk, SSD) at the data management and physical storage level. A logical address segment refers to a sub-area of ​​the address space in a logical volume, while a physical address segment is a physical storage device (such as an SSD). The relationship between them is as follows: A logical address segment is a logical unit used by a storage system to aggregate data. When a write request arrives, the system aggregates data from logical address segments of the same temperature based on the temperature of the data, and then flushes the aggregated data to the physical space on the physical address segment. This aggregate write strategy helps reduce the write amplification effect of SSDs, because aggregate writes can increase the amount of data written each time, thereby reducing the number of erases and writes to the SSD.

[0057] Specifically, writing the data to be written with the same heat level into the same physical address segment of the storage system can improve the efficiency of subsequent data recovery. This is because heat classification storage stores data in different data management blocks according to its access frequency (i.e., heat), which means that cold data (data with low access frequency) and hot data (data with frequent access) are physically isolated. During the garbage collection process, the data management blocks where cold data are located often contain more invalid data (i.e., data that has been logically deleted but not physically cleared). Therefore, giving priority to recycling the data management blocks of cold data can significantly reduce the amount of valid data that needs to be migrated, because the proportion of invalid data in the data management blocks of hot data is low, and frequent physical migration of data is not required. Secondly, since different data heats mean different access modes and life cycles, storage by heat classification makes garbage collection operations more targeted. For example, the proportion of invalid data in the data management blocks of cold data is high, and space can be released more efficiently during recycling, while the data management blocks of hot data may need to reserve more space due to their high access frequency to reduce frequent recycling operations. This strategy avoids unnecessary migration of valid hot data in mixed data and improves recycling efficiency.

[0058] In this embodiment, firstly, by collecting the historical data writing situation, it is convenient to analyze which logical address segments are frequently accessed and which segments are less used, thereby providing a data basis for the subsequent heat evaluation and ensuring the accuracy and effectiveness of the subsequent steps. Then, based on the historical data writing situation, the writing heat of the logical address segment can be determined, which is convenient for the subsequent data classification, so as to more efficiently perform data aggregation and space management. Then, the data to be written is classified according to the heat. Through the heat classification, the data in the cache is pre-classified, which is convenient for the system to quickly select and aggregate data according to the data heat and write it into the physical address segment, thereby reducing the delay of data processing and improving the efficiency of data writing. At the same time, it also helps to optimize the use of storage space, reduce the overhead of garbage collection, and improve the performance of the storage system. Finally, the data to be written of the same heat level is written into the same physical address segment, realizing the classified writing of data according to the heat level. Since the heat of the data in the physical address segment is relatively consistent, when the data is recovered, since the data of different heats have been classified, it is possible to more effectively recover invalid data, reduce the amount of data migration, and improve the recovery efficiency.

[0059] In one embodiment, Figure 3 As shown, step S210, according to the historical data writing status of the multiple logical address segments, respectively determines the writing heat of the multiple logical address segments. It includes: steps S300-S310:

[0060] Step S300, according to the historical data writing status of the multiple logical address segments, respectively determine the amount of written data, the number of write requests, and the location distribution of the written data in the multiple logical address segments within a preset time period.

[0061] Specifically, the data writing activities of each logical address segment within a specific time range are quantified for subsequent analysis and decision making. The amount of written data, the number of write requests, and the distribution of written data locations are important indicators for evaluating data heat.

[0062] Exemplarily, the system periodically monitors and records the write activity of each logical address segment, including the amount of data written, the total number of write operations, and the relative position distribution of the written data. For example, each segment can be further subdivided, and the write activities of different sub-segments can be counted to evaluate the distribution uniformity of the written data. For the position distribution of the written data, it can be quantified as the degree of deviation of the write position from the midpoint position of the segment.

[0063] Step S310 , determining the write heat of the plurality of logical address segments according to the amount of written data, the number of write requests, the location distribution of the written data, and the corresponding weights of the plurality of logical address segments within a preset time period.

[0064] Specifically, write heat is a comprehensive indicator used to evaluate the activity of data. The write heat of each logical address segment is calculated based on the previously collected data and the pre-set weight.

[0065] Exemplarily, the system uses a formula to calculate write heat, taking into account the amount of written data, the number of write requests, and the consistency of location distribution, and assigning different weights to these indicators. For example, the heat calculation formula can be: heat = α * amount of written data + β * number of write requests + γ * location distribution consistency, where α, β, and γ are preset weight coefficients, and their values ​​can be adjusted according to the specific needs and optimization goals of the storage system.

[0066] In this embodiment, by collecting and analyzing the historical data writing within a preset time, the system can accurately identify which segments are active and which are inactive, providing data support for realizing heat-based data management, helping to reduce invalid data migration, reduce write amplification effect, and improve the overall performance of the storage system. By introducing a weight mechanism to determine the write heat, the system can more accurately reflect the true activity of the data. This comprehensive evaluation method takes into account the amount of written data, the write frequency, and the uniformity of data distribution, which helps to achieve finer-grained control in data management. Using data heat as the basis for space management decisions can significantly improve the utilization efficiency of physical address segments and reduce the number and complexity of garbage collection operations, thereby optimizing the overall performance of the storage system, extending the service life of the SSD, reducing the write amplification effect, and providing users with more stable and efficient data storage services.

[0067] In one embodiment, Figure 4 As shown, step S300, according to the historical data writing status of multiple logical address segments, respectively determines the amount of written data, the number of write requests, and the location distribution of the written data in multiple logical address segments within a preset time. It includes: steps S400-S420:

[0068] Step S400, based on the historical data writing status of multiple logical address segments, count the amount of written data and the number of write requests for multiple logical address segments within a preset time period, so as to respectively determine the amount of written data and the number of write requests for multiple logical address segments within the preset time period.

[0069] Specifically, this is the first step in data heat analysis. By statistics on historical data writing, we can obtain the write activity profile of each logical address segment within the preset period, including the total capacity of written data and the total number of requests, providing basic data for subsequent heat calculations.

[0070] Exemplarily, within a preset time interval (such as one day, one week, etc.), the metadata log is queried to count the amount of written data and the number of writes for each logical address segment. This usually involves scanning the log file, extracting information related to the write operation, and summarizing it by logical address segment. For example, a database query or specialized data analysis software can be used to process this data and generate a statistical report.

[0071] Step S410, respectively determining the middle values ​​of the logical addresses of the plurality of logical address segments.

[0072] Specifically, the middle value of the logical address is a key reference point for evaluating the distribution of data write locations. It represents the center of each logical address segment and helps analyze whether the written data is concentrated in the first half, second half, or evenly distributed in the segment. The ratio of the average value of the logical address of each write request in the logical address segment (segment) is calculated to the middle value of the logical address of the logical address segment. If it is equal to 1, it indicates that it is very even; if it is less than 1, it indicates that the write density is biased towards the first half of the segment; if it is greater than 1, it indicates that the write density is biased towards the second half of the segment.

[0073] Exemplarily, for each logical address segment, the system calculates the middle value according to its logical address range. For example, if the logical address range of a segment is from LBA1 to LBA1000, then the middle value of the logical address is (LBA1 + LBA1000) / 2. For asymmetric or discontinuous logical address segments, the middle value can be determined in a corresponding manner.

[0074] Step S420 , determining the position distribution of write data in the plurality of logical address segments within the preset time length according to the average values ​​of the logical addresses of the write requests in the plurality of logical address segments within the preset time length and the median values ​​of the logical addresses.

[0075] Specifically, by comparing the average value of the write request logical address with the median value of the logical address of the segment, it can be determined whether the data is evenly distributed or tends to be at one end of the segment.

[0076] Exemplarily, the system calculates the average value of the write request logical address within a preset time length for each logical address segment, and then compares this average value with the middle value of the logical address of the segment. The result of the comparison can be quantified as a distribution index, which reflects the uniformity of the distribution of the data write location. For example, if the average value is exactly equal to the middle value, then the distribution index is 1, indicating that the data is evenly distributed; if the average value is less than the middle value, the distribution index is less than 1, indicating that the data tends to be in the first half of the segment; conversely, if the average value is greater than the middle value, the distribution index is greater than 1, indicating that the data tends to be in the second half of the segment.

[0077] In this embodiment, by regularly counting the amount of written data and the number of writes within a preset time, the system can capture the dynamic changes in the data access mode, adjust the data management strategy in time, and avoid performance bottlenecks or resource waste caused by sudden changes in data heat. At the same time, these statistical results also provide an important reference for the subsequent write location distribution analysis. Determining the middle value of the logical address provides an accurate reference point for the subsequent analysis of the location distribution of the written data, so that the system can quantitatively evaluate the distribution of the written data, which helps to achieve more refined data management. For example, by identifying the phenomenon that the data is concentrated in a certain part of the segment, the physical layout of the data can be optimized, the delay of read and write operations can be reduced, and the read and write performance of the storage system can be improved. By determining the location distribution of the written data, the system can further refine the analysis of the data heat and identify potential trends in the data access mode. This helps to adopt more effective strategies when writing data, such as giving priority to writing data to segments with more uniform distribution, reducing the addressing time of the write operation, reducing the write amplification effect, optimizing the garbage collection process, and ultimately significantly improving the efficiency, performance and reliability of the all-flash storage system and extending the service life of the disk.

[0078] In one embodiment, Figure 5 As shown, the method for writing data into the storage system further includes: steps S500-S510:

[0079] Step S500, determining the historical life cycle of data in multiple physical address segments of the storage system.

[0080] The life cycle is the time from when data is written into the physical address segment to when data is migrated out of the physical address segment.

[0081] Specifically, the historical life cycle of stored data reflects the residence time of the data in the physical address segment, which is an important basis for evaluating data popularity and predicting future data behavior. The longer the life cycle, the more stable the data is or the less frequently accessed it is; conversely, a short life cycle indicates that the data may be more active or will be updated soon.

[0082] For example, the historical life cycle of the data in each physical address segment can be counted by tracking the time interval between each data management block (or other space unit of the physical address segment) being allocated for write operation and its data being garbage collected (or reallocated, deleted). All operations related to data writing and deletion can be recorded, including timestamp, data size, write location and other information.

[0083] Step S510, determining a preset duration according to the historical life cycle of data in the plurality of physical address segments.

[0084] Specifically, the preset duration is used to periodically count and analyze the write heat of data. It should match the average life cycle of data in the physical address segment to ensure that the periodicity of heat statistics is consistent with the behavior pattern of the data, avoiding inaccurate data heat assessment caused by too short or too long statistical period.

[0085] Exemplarily, historical life cycle data is analyzed to identify the trend of the average residence time of data in the physical address segment. The preset duration can be set to be close to or equal to the average life cycle, or to a time range that covers most data life cycles. For example, if the analysis results show that the average data life cycle is one week, the preset duration can be set to one week to ensure that the heat statistics period matches the data life cycle.

[0086] In this embodiment, the historical life cycle of the statistical data helps determine the cycle of data update. Setting the preset time length to a value that is compatible with the data life cycle can improve the accuracy and effectiveness of the heat statistics and be more in line with the actual situation.

[0087] In one embodiment, Figure 6 As shown, in step S310, after determining the write heat of the plurality of logical address segments according to the amount of written data, the number of write requests, the location distribution of the written data, and the corresponding weights of the plurality of logical address segments within a preset time period, the method for writing data in the storage system further includes steps S600-S620:

[0088] Step S600 , recording the write heat of the logical address segment within a plurality of consecutive preset time lengths.

[0089] Specifically, during the life cycle of a storage system, the popularity of data (i.e., access frequency and write activity) may change over time. Therefore, recording write popularity over multiple consecutive preset time periods can more comprehensively reflect the dynamic changes in data popularity and provide richer information for subsequent decision-making.

[0090] Exemplarily, a database table or log may be used to record the write heat of each logical address segment within each preset duration. This generally involves calculating the write heat of each logical address segment at the end of each cycle according to the heat statistics method described above, and storing the result in the data structure. For example, a heat history list may be created for each logical address segment, and each element in the list corresponds to a heat value of a preset duration.

[0091] Step S610: assigning corresponding weights to the plurality of consecutive preset time periods according to the time intervals between the plurality of consecutive preset time periods and the current moment.

[0092] Specifically, since data popularity changes over time, earlier popularity values ​​may no longer accurately reflect current data access patterns. Therefore, assigning weights to popularity values ​​of different preset durations can ensure that popularity calculations focus more on recent data behaviors.

[0093] Exemplarily, the system calculates a weight for each preset duration of heat value, and the weight is inversely proportional to the time interval, that is, the heat value closer to the current cycle has a greater weight. For example, a decreasing weight coefficient sequence can be used, such as 1, 1 / 2, 1 / 4, 1 / 8, ..., corresponding to the most recent, the second most recent, the third most recent cycle, etc. In this way, the heat value of the latest cycle will have the greatest impact, while the heat value of the earlier cycle will gradually reduce its contribution.

[0094] Step S620 , determining the current write heat of the logical address segment according to the write heat of the logical address segment within a plurality of consecutive preset time lengths and the weights corresponding to the plurality of consecutive preset time lengths.

[0095] Specifically, a comprehensive indicator reflecting the current data heat is calculated, which takes into account the change in data heat over multiple consecutive periods and the importance of the heat value in each period.

[0096] Exemplarily, the system multiplies the write heat of the logical address segment within multiple consecutive preset time lengths by the weight corresponding to each cycle, then adds the results, and finally divides by the sum of the weights to obtain a weighted average, i.e., the current write heat. For example, if the heat of logical address segment A in the last three cycles is 10, 8, and 6, respectively, and the corresponding weights are 1, 1 / 2, and 1 / 4, then the current write heat calculation formula is: (10*1 + 8*1 / 2 + 6*1 / 4) / (1 + 1 / 2 + 1 / 4) = (10 + 4 + 1.5) / 1.75 = 15.5 / 1.75 = 8.85.

[0097] In this embodiment, recording historical heat data enables the system to track data heat trends and better predict future heat changes. This is crucial for optimizing data management and space allocation strategies, avoiding performance problems caused by sudden changes in heat, ensuring that the allocation of system resources matches the real-time needs of data, and improving the response speed and overall performance of the storage system. By introducing a weight mechanism for heat values ​​of different periods, the system can more accurately reflect the current heat of the data and reduce the negative impact of historical heat data on current decisions. This method of dynamically adjusting weights can ensure that the storage system always optimizes the management of physical address segments based on the most relevant data heat information, improves the efficiency of data writing and garbage collection, and reduces the write amplification effect of the disk, thereby increasing the life of the device. By calculating the weighted average heat, the system can accurately evaluate the current heat of the data based on historical and current data behavior. This evaluation method not only takes into account the recent activity of the data, but also takes into account the long-term trend, which helps to achieve a more accurate and stable data management strategy, optimize the resource allocation of the physical address segment, reduce the frequency and complexity of garbage collection operations, and ultimately improve the performance and efficiency of the storage system, extend the service life of the disk, and reduce operating costs.

[0098] In one embodiment, Figure 7 As shown, step S220, heat grading is performed on the data to be written according to the writing heat of multiple logical address segments to determine the heat level of each data to be written. It includes: steps S700-S720:

[0099] Step S700, determining the association between the data to be written and a plurality of logical address segments.

[0100] Specifically, the data will be temporarily saved before being written to the back-end physical address segment. In order to achieve data writing optimization based on heat, it is necessary to determine which logical address segment or segments each piece of data to be written is associated with, because the heat of the data is calculated based on the logical address segment.

[0101] Exemplarily, by parsing the logical address of the data to be written, it is mapped to the corresponding logical address segment. The storage system's logical address to physical address mapping table can be queried to determine the segment to which the data belongs. For example, if the logical address of the data to be written is LBA 12345, and the system's logical address segment division rule is that every 1000 LBAs are a segment, then the system will associate the data with the logical address segment of LBA 12000-12999.

[0102] Step S710, in a preset heat classification table, query and obtain the heat level corresponding to the data to be written according to the association situation.

[0103] The heat classification table is used to indicate the correspondence between the write heat and heat level of the logical address segment.

[0104] Specifically, the heat classification table is a mapping table that maps the write heat of the logical address segment to different heat levels, which facilitates the system to perform data classification operations. After determining the association between the data and the logical address segment, the system needs to query the heat level of the data to determine the subsequent data writing strategy.

[0105] Exemplarily, the heat classification table lists all the counted logical address segments and their corresponding write heats. After determining the associated segment of the data to be written, the system queries the write heat of the segment in the heat classification table and maps the heat to a specific heat level according to the preset heat classification rules.

[0106] Step S720: establishing an index information table based on the heat level of each data to be written, where the index information table is used to indicate the heat level of each data to be written.

[0107] Specifically, the index information table is a data structure used to quickly find and retrieve the heat level information of the data to be written in the cache pool, so as to optimize the subsequent data processing and writing operations.

[0108] Exemplarily, the system creates an index information table for each heat level, which records all the data to be written at the corresponding heat level. When the heat level of the data to be written is determined, the system will add the index information of the data (such as the data's identifier, size, logical address, etc.) to the corresponding heat level index information table.

[0109] In this embodiment, by clarifying the association between the data to be written and the logical address segment, the system can quickly find the level to which its heat belongs based on the logical address information of the data, laying the foundation for subsequent data classification and space optimization management. This improves the speed and efficiency of data writing processing and ensures the accuracy of data management and space allocation decisions. The heat classification table provides the system with a tool for quickly querying and classifying data heat levels, so that the system can efficiently classify data into different heat levels based on the heat of the logical address segment to which the data belongs. This classification is the core of the write processing and space management strategy of the all-flash storage system, which helps to reduce the frequency of garbage collection operations, reduce the write amplification effect, and optimize the physical layout of data, thereby improving the performance of the all-flash storage system and the life of the disk. The establishment of the index information table realizes the rapid association between the data heat level and the data information, so that the system can quickly select the appropriate write strategy according to the heat level of the data when making data writing decisions, such as writing hot data to a high-priority data management block and writing cold data to a low-priority data management block. This greatly improves the flexibility and efficiency of data writing and management, reduces processing delays, improves the performance of all-flash physical address segments and data accessibility, while also reducing maintenance costs and extending the life of the device.

[0110] In one embodiment, Figure 8 As shown, step S230 writes the to-be-written data with the same heat level into the same physical address segment of the storage system. It includes: steps S800-S820:

[0111] Step S800, determining the amount of data to be written at each heat level.

[0112] Specifically, the data to be written will be temporarily stored in the cache pool until the system writes it to the backend physical address segment. In order to optimize the write operation, it is necessary to know how much data there is at different heat levels, which helps to determine the data aggregation write strategy.

[0113] Exemplarily, by scanning the data in the cache pool, the data of each heat level is identified and the total amount of data is counted. This usually involves maintaining a data structure, such as a hash table or list, to track the cumulative amount of data at different heat levels. For example, the system creates lists for hot data and cold data respectively, and updates the amount of data in these lists in real time.

[0114] Step S810, determining the writing order of the data to be written at each heat level according to the amount of the data to be written at each heat level.

[0115] Specifically, the order in which data is written determines its layout in the physical address segment and the efficiency of garbage collection. By sorting based on data volume, the system can manage storage space more efficiently, reduce data migration, and improve write performance.

[0116] Exemplarily, based on the above statistical data volume, the data to be written at each heat level is sorted. Data with a larger data volume is selected for writing first, because this can reduce the number of write operations and improve write efficiency. Sorting can be implemented using various algorithms, such as quick sort, heap sort, etc. The specific selection depends on the data volume and system performance requirements.

[0117] Step S820, writing the to-be-written data with the same heat level into the same physical address segment of the storage system in sequence according to the writing order.

[0118] Specifically, the process of writing data from the cache pool to the physical address segment not only follows the data heat level, but also considers the data volume and writing order to achieve a more optimized data layout.

[0119] Exemplarily, data is written to the physical address segment in the determined writing order. For data of a certain heat level, the system will find or create a dedicated data management block, and then aggregate the data and write it to the data management block until the available space of the data management block is insufficient to accommodate more data. When all the data of a heat level is written, the system will start processing the data of the next heat level.

[0120] In this embodiment, by counting the amount of data at each heat level, the system can determine the priority and strategy of aggregate writing according to the activity of the data. After determining the write order, the system can better manage the data write process, reduce cache pressure, and increase write speed. In addition, by aggregating data into appropriate data management blocks, future garbage collection operations can be reduced, write amplification can be reduced, and hot data can be written quickly. Writing data with the same heat level to the same data management block in the same physical address segment can significantly improve the efficiency of garbage collection because they have similar life cycles and access patterns, reducing the need for data migration during garbage collection.

[0121] In one embodiment, Fig. 9 As shown, step S820, according to the writing order, write the to-be-written data with the same heat level into the same physical address segment of the storage system in sequence. It includes: steps S900-S910:

[0122] Step S900 , according to the writing order, sequentially aggregate write requests for data to be written with the same heat level into the same data management block.

[0123] Specifically, the data management block (i.e., Extent) is the basic unit for data aggregation writing and garbage collection in the all-flash storage system. Aggregating write requests for data to be written with the same heat level in the same data management block can ensure that data with the same access mode is physically stored in adjacent spaces, thereby improving the read and write performance and management efficiency of the storage system.

[0124] Exemplarily, the storage system starts processing the data in the cache pool according to the determined write order. The system checks whether there is a data management block that has been allocated but not fully written. If so, and the heat level of the data management block is consistent with the data to be aggregated, the system adds the data write request to the write queue of the data management block. If there is no suitable data management block, the system allocates a new data management block and sets its heat level to the heat level of the current data to be aggregated, and then adds the data write request to the write queue of the new data management block. This process continues until data of all heat levels are appropriately aggregated.

[0125] Step S910, when the write requests aggregated in the data management block reach a preset granularity, execute the write requests aggregated in the data management block to sequentially flush the to-be-written data with the same heat level to the same physical address segment of the storage system.

[0126] Specifically, the preset granularity is a trigger condition for the write operation of the data management block, which aims to reduce the frequency of write operations, improve write efficiency, and reduce the impact on the back-end storage device by aggregating write operations.

[0127] For example, the system sets a write granularity threshold, such as 128MB or the size of a disk stripe. When the accumulated data volume of write requests in a data management block of a certain heat level reaches or exceeds this threshold, the system will perform a write operation on the data management block, flush all aggregated write requests to the backend physical address segment at one time, and write them to the corresponding disk group. After the write operation is completed, the status of the data management block will be updated to reflect its latest usage and data layout.

[0128] In this embodiment, by aggregating write requests of the same heat level in the same data management block, the system can significantly reduce the addressing operations when writing data and improve the writing speed. In addition, this aggregation method is also convenient for the operation of the garbage collection mechanism, because the data in the aggregated data management block has a similar life cycle, and cold data can be cleared more efficiently during garbage collection, reducing data migration and improving recycling efficiency. Writing operations that reach a preset granularity can significantly reduce the number of write operations, reduce wear on storage devices, and reduce write amplification. At the same time, by flushing data of the same heat level at one time, the storage system can allocate and manage storage resources more effectively, reduce data fragmentation, improve read and write performance, and reduce the complexity and frequency of garbage collection, thereby reducing the overall maintenance cost of the storage system, extending the service life of the equipment, and improving the stability and efficiency of the system.

[0129] In one embodiment, Fig.10 As shown, in step S900, before the write requests of the to-be-written data with the same heat level are aggregated in the same data management block in sequence according to the write order, the method further includes: steps S1000-S1010:

[0130] Step S1000 , allocating corresponding data management blocks to the data to be written of each heat level according to the heat level of the data to be written, and identifying the heat level of the data management blocks.

[0131] Specifically, data management blocks are allocated based on the heat level of the data, which helps optimize the space usage and garbage collection efficiency of the physical address segment. The heat level of the data management block is identified to identify and track the access mode and life cycle of the data in subsequent management operations.

[0132] Exemplarily, the storage system allocates one or more data management blocks (i.e., Extents) to the data of the heat level according to the heat level information of the data to be written. When a data management block is allocated, the system will add a heat level identifier to its metadata. This identifier is used to record the heat level of the data in the data management block as a basis for subsequent data management and space optimization. For example, for data with a high heat level, the system may allocate a series of adjacent data management blocks, while for data with a low heat level, it may be allocated more dispersedly to adapt to its access pattern.

[0133] Step S1010, identifying the order of the data management blocks according to the writing order.

[0134] Specifically, the sequential identification of the data management block is to ensure that the data can be written to the back-end storage device in a preset order, which is crucial for optimizing read and write performance and garbage collection strategies.

[0135] Exemplarily, after the data management blocks are allocated and identified with heat levels, a sequence identifier is assigned to the data management blocks according to the determined write order. This is usually a numerical value, and the smaller the numerical value, the higher the write priority, or it is identified according to the order of timestamps to ensure that the data can be written to the physical address segment in order according to the heat level and data size. For example, data management blocks with high heat levels will be assigned smaller sequence identifiers, and data management blocks with low heat levels will be assigned larger sequence identifiers.

[0136] In this embodiment, by keeping the heat level of the data management block consistent with the heat level of the data to be written, the system can realize dynamic stratification of data, physically isolate hot data and cold data, and avoid efficiency loss caused by mixed writing. This not only improves data access speed and reduces write amplification, but also simplifies the garbage collection process and reduces the migration of invalid data, thereby increasing the life of the disk, reducing maintenance costs, and enhancing the overall performance and stability of the storage system. The introduction of sequential identification enables the system to accurately control the writing order of data and optimize the physical layout of data on the storage device. This orderly writing strategy also helps to optimize garbage collection, because the system can prioritize the recovery of inefficient data management blocks according to the sequential identification of the data management blocks, reduce the complexity of garbage collection, improve recovery efficiency, further reduce write amplification, and extend the service life of the disk, thereby improving the performance and resource utilization of the storage system as a whole.

[0137] In one embodiment, Fig.11 As shown, step S900, according to the writing order, sequentially aggregates the write requests of the data to be written with the same heat level in the same data management block. It includes: steps S1100-S1110:

[0138] Step S1100, determining the writing mode of the data to be written with the same heat level.

[0139] The write mode includes sequential write and random write.

[0140] Specifically, the system needs to identify the write mode of the data to be written in order to further optimize the data writing and space management. Different write modes have different effects on the performance and garbage collection strategy of the storage system. Therefore, classifying data according to the write mode can make the write operation more efficient.

[0141] Exemplarily, the system determines whether the write mode is sequential write or random write by analyzing the logical address of the data to be written. Sequential write usually refers to data being written according to continuous logical addresses, while random write has no obvious address continuity. For example, by checking the logical address sequence involved in the write request, if the address continuity is above a certain threshold, the system identifies it as sequential write; otherwise, if it is below the threshold, it is identified as random write. This identification can be achieved through a simple algorithm, such as calculating the average difference between addresses and comparing it with a preset sequential threshold.

[0142] Step S1110 , aggregating write requests for data to be written with the same heat level and the same write mode into the same data management block.

[0143] Specifically, in an all-flash storage system, aggregating data with the same heat level and write mode in the same data management block (i.e., Extent) helps to further optimize data layout and space management and improve system performance.

[0144] Exemplarily, after the system determines the heat level and write mode of the data to be written, it will search for a data management block that matches these conditions. If such a data management block exists, the system will aggregate these write requests into the data management block; if there is no matching data management block, the system will create a new data management block and aggregate the write requests into it. The newly created data management block will be marked with its heat level and write mode to ensure that subsequent data requests are aggregated according to the same rules.

[0145] In this embodiment, by distinguishing the write mode of the data, the system can adopt the most effective aggregation strategy for different types of writes. For example, data written sequentially can be aggregated in continuous data management blocks, which helps to reduce the addressing time of write operations and improve write speed. Aggregating data with the same heat level and write mode in the same data management block can significantly reduce the addressing operations when writing data, improve write efficiency, and reduce write amplification. For sequentially written data, this aggregation method also helps to improve read performance because the data is more physically continuous, the seek time is reduced, and the access speed is improved. In addition, this strategy simplifies the garbage collection process because the data in the same data management block has similar life cycles and access patterns. Cold data can be cleared more efficiently during garbage collection, reducing data migration, thereby reducing the overall maintenance cost of the storage system and improving the stability and resource utilization of the system.

[0146] In one embodiment, when executing a write request aggregated in a data management block, the storage system identifies the spatial continuity of the logical addresses between the data management block corresponding to the write request and the data management block of the last executed write request.

[0147] Specifically, by determining whether the newly received write request is continuous with the previous write request in terms of logical address, it is possible to determine whether the data is written in sequence. Logical address continuity is a key indicator for determining whether data is written sequentially. Sequential writing usually occurs in scenarios such as batch data upload and continuous data writing in the file system, while random writing occurs in scenarios such as random access to small files and database updates.

[0148] Exemplarily, when a new write request arrives at the cache layer, the system checks whether its logical address is immediately after the logical address of the previous write request. If so, the request is recorded as a sequential write request; if not, it is recorded as a random write request. Thus, by recording the logical address of the previous write request, the continuity of the new write request can be quickly determined.

[0149] Secondly, with the help of the storage system's recognition of the continuity of data writing, the continuity of the dirty data cached in the cache can be determined.

[0150] Specifically, in the cache, data that has been written to the cache but has not yet been flushed to the physical address segment is called dirty data. Identifying the continuity of dirty data can more effectively flush the continuous dirty data as a whole, reduce the number of flush operations, and improve storage efficiency.

[0151] Exemplarily, a dirty data record table is maintained in the cache layer, which contains the logical address and status information (such as data volume, timestamp, etc.) of each dirty data. The dirty data record table is checked regularly to identify and mark continuous dirty data, which may come from multiple different write requests. Continuous dirty data are aggregated together to form dirty data blocks. When the data volume of the dirty data block reaches the preset data volume, the dirty data block is flushed all at once.

[0152] In this embodiment, by identifying the spatial continuity of the logical addresses between the data management blocks, the storage system can determine whether the write request is continuous, and then determine whether the data is continuous. Therefore, when processing dirty data in the cache, the storage system can identify the continuity of the dirty data, and can more effectively flush the continuous dirty data as a whole, reduce the number of flush operations, improve storage efficiency, reduce the fragmentation of data, and improve the flush efficiency. In addition, sequentially written data is more easily aggregated, thereby reducing flush operations and write amplification, and extending the service life of the SSD. At the same time, this ability to identify continuity helps to more efficiently process continuous data in the subsequent garbage collection process, reduce data migration and erase operations, and further improve the overall performance and reliability of the storage system.

[0153] In one embodiment, Fig.12As shown, the method for writing data into the storage system further includes: steps S1200-S1210:

[0154] Step S1200 , when the remaining space in the physical address segment is less than a preset threshold, respectively determining the proportion of invalid data in multiple data management blocks in the physical address segment and the existence duration of the invalid data in the multiple data management blocks.

[0155] Specifically, when the remaining space of the physical address segment approaches a preset threshold, the system needs to start the garbage collection mechanism to free up space. The proportion and existence time of invalid data are important indicators for evaluating the recycling priority of the data management block (DMB).

[0156] Exemplarily, the system checks the data in each data management block periodically or when the remaining space in the physical address segment is below a preset threshold. Through metadata records, the system can determine which data in each data management block is invalid, that is, data that has been overwritten by new data but has not yet been physically deleted. The system calculates the proportion of invalid data in each data management block and the length of time since these invalid data were marked as invalid. This usually involves analyzing the metadata of the data management block, possibly using a bitmap or other data structure to represent the status of each data page (valid or invalid), and recording the marking time of each invalid data page.

[0157] Step S1210 , determining an order for reclaiming invalid data from the multiple data management blocks according to the proportion of invalid data in the multiple data management blocks and the existence duration of the invalid data in the multiple data management blocks.

[0158] Specifically, the system determines the priority order of garbage collection based on the proportion and existence duration of invalid data in each data management block. This is to most effectively recover the space of the physical address segment under limited resources.

[0159] Exemplarily, a recycling priority is assigned to each data management block based on the calculated invalid data ratio and existence time. The calculation of priority can be a composite function, such as priority = invalid data ratio * weight of existence time. This ensures that data management blocks with a high proportion of invalid data and a long existence time are recycled first. The system maintains a recycling priority list and arranges all data management blocks to be recycled in order of priority. In summary, data management blocks with high inefficiency are given priority for recycling. For data management blocks with the same inefficiency, data management blocks with longer duration are given priority for recycling based on the length of time since the data management block was allocated to the present, because data management blocks with shorter durations indicate more activity, and retaining them for a longer time may further increase the inefficiency.

[0160] In this embodiment, by regularly checking and evaluating the proportion and existence time of invalid data, data management blocks that need to be garbage collected first can be identified in time to avoid the performance of physical address segments being affected by insufficient space. By giving priority to recycling data management blocks with a high proportion of invalid data and a long existence time, the system can release space more efficiently while maintaining the high-speed response and data access performance of the storage system. Determining the recycling order based on the proportion and existence time of invalid data can ensure the optimization of space release in the physical address segment. Giving priority to recycling data management blocks with a high proportion of invalid data and a long existence time not only reduces the overall data migration amount of the garbage collection operation, but also because the data in these data management blocks is more likely to be no longer needed, the recycling efficiency is higher and the impact on system performance is smaller. This strategy helps to maintain the efficient operation of the storage system, while extending the service life of the disk, reducing maintenance costs, and improving the reliability of the system.

[0161] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0162] In the present embodiment, a device for writing data to a storage system is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the modules described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0163] Fig.13 1 is a structural block diagram of a device for writing data into a storage system according to an embodiment of the present application, the device comprising:

[0164] The write status acquisition module 1301 is used to acquire the historical data write status of multiple logical address segments of the storage system.

[0165] The write heat determination module 1302 is used to determine the write heat of multiple logical address segments respectively according to the historical data writing conditions of the multiple logical address segments.

[0166] The heat level determination module 1303 is used to perform heat grading on the data to be written according to the write heat of multiple logical address segments, and determine the heat level of each data to be written, wherein each data to be written corresponds to a logical address segment.

[0167] The write execution module 1304 is used to write the to-be-written data with the same heat level into the same physical address segment of the storage system.

[0168] The above device first collects historical data writing conditions, so that it is easy to analyze which logical address segments are frequently accessed and which segments are less used, thereby providing a data basis for subsequent heat evaluation and ensuring the accuracy and effectiveness of subsequent steps. Then, based on the historical data writing conditions, the writing heat of the logical address segment can be determined, which is convenient for subsequent data classification, so as to more efficiently perform data aggregation and space management. Then, the data to be written is classified according to the heat. Through the heat classification, the data in the cache is pre-classified, which is convenient for the system to quickly select and aggregate data according to the data heat and write it into the physical address segment, thereby reducing the delay of data processing and improving the efficiency of data writing. At the same time, it also helps to optimize the use of storage space, reduce the overhead of garbage collection, and improve the performance of the storage system. Finally, the data to be written with the same heat level is written into the same physical address segment, realizing the classified writing of data according to the heat level. Since the heat of the data in the physical address segment is relatively consistent, when the data is recovered, since the data of different heats have been classified, it can be more effectively recovered. Invalid data, the amount of data migration is reduced and the recovery efficiency is improved.

[0169] In an exemplary embodiment, the write heat determination module 1302 includes:

[0170] The write status determination unit is used to determine the amount of written data, the number of write requests, and the location distribution of the written data in the multiple logical address segments within a preset time period according to the historical data writing status of the multiple logical address segments.

[0171] The write heat determination unit is used to determine the write heat of multiple logical address segments according to the amount of written data, the number of write requests, the location distribution of the written data, and the corresponding weights of the multiple logical address segments within a preset time period.

[0172] In an exemplary embodiment, the write situation determination unit is further used to count the amount of written data and the number of write requests of the multiple logical address segments within the preset time length according to the historical data writing situation of the multiple logical address segments, so as to respectively determine the amount of written data and the number of write requests of the multiple logical address segments within the preset time length. The logical address median values ​​of the multiple logical address segments are respectively determined. According to the logical address average values ​​of the write requests of the multiple logical address segments within the preset time length and the logical address median value, the location distribution of the written data of the multiple logical address segments within the preset time length is respectively determined.

[0173] In an exemplary embodiment, the above device further includes:

[0174] The life cycle determination module is used to determine the historical life cycle of data in multiple physical address segments of the storage system, wherein the life cycle is the length of time from when data is written into the physical address segment to when data is migrated out of the physical address segment.

[0175] The preset duration determination module is used to determine the preset duration according to the historical life cycle of the data in the multiple physical address segments.

[0176] In an exemplary embodiment, the above device further includes:

[0177] The write heat recording module is used to record the write heat of the logical address segment within a plurality of consecutive preset time lengths.

[0178] The weight determination module is used to assign corresponding weights to multiple consecutive preset time lengths according to the time intervals between the multiple consecutive preset time lengths and the current moment.

[0179] The write heat determination module is used to determine the current write heat of the logical address segment according to the write heat of the logical address segment within multiple continuous preset time lengths and the weights corresponding to the multiple continuous preset time lengths.

[0180] In an exemplary embodiment, the heat level determination module 1303 includes:

[0181] The association determination unit is used to determine the association between the data to be written and the multiple logical address segments.

[0182] The heat level query unit is used to query the heat level corresponding to the data to be written in a preset heat level classification table according to the associated situation, wherein the heat level classification table is used to indicate the corresponding relationship between the write heat and the heat level of the logical address segment.

[0183] The information table determination unit is used to establish an index information table based on the heat level of each data to be written, and the index information table is used to indicate the heat level of each data to be written.

[0184] In an exemplary embodiment, the write execution module 1304 includes:

[0185] The data amount determination unit is used to determine the data amount of the data to be written at each heat level.

[0186] The write order determination unit is used to determine the write order of the data to be written at each heat level according to the data amount of the data to be written at each heat level.

[0187] The writing unit is used to write the to-be-written data with the same heat level into the same physical address segment of the storage system in sequence according to the writing order.

[0188] In an exemplary embodiment, the writing unit is further used to aggregate write requests for data to be written with the same heat level in the same data management block in sequence according to the writing order. When the aggregated write requests in the data management block reach a preset granularity, the aggregated write requests in the data management block are executed to sequentially flush the data to be written with the same heat level to the same physical address segment of the storage system.

[0189] In an exemplary embodiment, the above device further includes:

[0190] The heat level identification module is used to allocate corresponding data management blocks to the data to be written at each heat level according to the heat level of the data to be written, and to identify the heat level of the data management block.

[0191] The sequence identification module is used to identify the sequence of data management blocks according to the writing sequence.

[0192] In an exemplary embodiment, the writing unit is further used to determine a writing mode of the data to be written with the same heat level, wherein the writing mode includes sequential writing and random writing. The writing requests of the data to be written with the same heat level and the same writing mode are aggregated in the same data management block.

[0193] In an exemplary embodiment, the above device further includes:

[0194] The invalid data determination module is used to determine the proportion of invalid data in multiple data management blocks of the physical address segment and the existence time of the invalid data in multiple data management blocks when the remaining space in the physical address segment is less than a preset threshold.

[0195] The invalid data recovery order determination module is used to determine the order of recovering invalid data from multiple data management blocks according to the proportion of invalid data in the multiple data management blocks and the existence time of the invalid data in the multiple data management blocks.

[0196] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to this: the above modules are all located in the same processor. Alternatively, the above modules are located in different processors in any combination.

[0197] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0198] Optionally, in this embodiment, the computer program may be configured to perform the following steps by means of the computer program:

[0199] S1, obtaining historical data writing status of multiple logical address segments of a storage system.

[0200] S2, determining the writing heat of the multiple logical address segments respectively according to the historical data writing status of the multiple logical address segments.

[0201] S3, performing heat grading on the data to be written according to the writing heat of the plurality of logical address segments to determine the heat level of each data to be written, wherein each data to be written corresponds to a logical address segment.

[0202] S4, writing the to-be-written data with the same heat level into the same physical address segment of the storage system.

[0203] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0204] The embodiment of the present application also provides an electronic device, such as Fig.14 As shown, the electronic device includes a memory 1402 and a processor 1404. The memory 1402 stores a computer program, and the processor 1404 is configured to execute the steps in any of the above method embodiments through the computer program.

[0205] Optionally, in this embodiment, the processor 1404 may be configured to perform the following steps through a computer program:

[0206] S1, obtaining historical data writing status of multiple logical address segments of a storage system.

[0207] S2, determining the writing heat of the multiple logical address segments respectively according to the historical data writing status of the multiple logical address segments.

[0208] S3, performing heat grading on the data to be written according to the writing heat of the plurality of logical address segments to determine the heat level of each data to be written, wherein each data to be written corresponds to a logical address segment.

[0209] S4, writing the to-be-written data with the same heat level into the same physical address segment of the storage system.

[0210] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0211] Alternatively, a person skilled in the art may understand that: Fig.14 The structure shown is for illustration only. Fig.14 The structure of the electronic device is not limited. For example, the electronic device may also include Fig.14 More or fewer components (such as network interfaces, etc.) as shown in, or with Fig.14 Different configurations are shown.

[0212] Among them, the memory 1402 can be used to store software programs and modules, such as the program instructions / modules corresponding to the method of writing data to the storage system and the device of writing data to the storage system in the embodiment of the present application. The processor 1404 executes various functional applications and data processing by running the software programs and modules stored in the memory 1402, that is, realizing the method of writing data to the storage system mentioned above. The memory 1402 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1402 may further include a memory remotely arranged relative to the processor 1404, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 1402 can be specifically, but not limited to, used to store information such as system configuration files. As an example, such as Fig.14 As shown, the memory 1402 may include but is not limited to the modules in the device for writing data into the storage system. In addition, it may also include but is not limited to other module units in the device for writing data into the storage system, which will not be described in detail in this example.

[0213] Optionally, the transmission device 1406 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 1406 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 1406 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0214] In addition, the electronic device further includes: a display 1408; and a connection bus 1414, which is used to connect various module components in the electronic device.

[0215] In other embodiments, the electronic device may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. The nodes may form a peer-to-peer (P2P) network, and any form of computing device, such as a server, terminal, or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0216] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0217] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0218] The embodiments of the present application also provide a computer program, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any one of the above method embodiments.

[0219] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0220] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for writing data to a storage system, characterized in that: The method comprises: Acquire historical data writing status of multiple logical address segments of the storage system; According to the historical data writing conditions of the multiple logical address segments, respectively determine the amount of written data, the number of write requests, and the location distribution of the written data in the multiple logical address segments within a preset time length; Determine the write heat of the multiple logical address segments according to the amount of written data, the number of write requests, the location distribution of the written data, and the corresponding weights of the multiple logical address segments within a preset time period; Performing heat grading on the data to be written according to the write heat of the plurality of logical address segments to determine the heat level of each of the data to be written, wherein each of the data to be written corresponds to one of the logical address segments; The to-be-written data with the same heat level are written into the same physical address segment of the storage system.

2. The method for writing data into a storage system according to claim 1, characterized in that: The step of determining the amount of written data, the number of write requests, and the location distribution of the written data in the plurality of logical address segments within a preset time period according to the historical data writing conditions of the plurality of logical address segments comprises: According to the historical data writing conditions of the multiple logical address segments, the amount of written data and the number of write requests of the multiple logical address segments within the preset time length are counted to respectively determine the amount of written data and the number of write requests of the multiple logical address segments within the preset time length; respectively determining the middle values ​​of the logical addresses of the plurality of logical address segments; According to the average values ​​of the logical addresses of the write requests of the multiple logical address segments within the preset time length and the intermediate value of the logical addresses, the position distribution of the write data of the multiple logical address segments within the preset time length is determined respectively.

3. The method for writing data into a storage system according to claim 2, characterized in that: The method further comprises: Determine the historical life cycle of data in the plurality of physical address segments of the storage system, wherein the life cycle is the length of time from when data is written into the physical address segment to when data is migrated out of the physical address segment; The preset time length is determined according to the historical life cycle of the data in the multiple physical address segments.

4. The method for writing data into a storage system according to claim 1, characterized in that: After determining the write heat of the plurality of logical address segments according to the amount of written data, the number of write requests, the location distribution of the written data, and the corresponding weights of the plurality of logical address segments within a preset time period, the method further includes: Recording the writing heat of the logical address segment within a plurality of consecutive preset time lengths; According to the time intervals between the multiple consecutive preset time lengths and the current moment, respectively assigning corresponding weights to the multiple consecutive preset time lengths; The current write heat of the logical address segment is determined according to the write heat of the logical address segment within a plurality of consecutive preset time lengths and the weights respectively corresponding to the plurality of consecutive preset time lengths.

5. The method for writing data into a storage system according to any one of claims 1 to 4, characterized in that: The step of performing heat grading of the data to be written according to the write heat of the plurality of logical address segments and determining the heat level of each of the data to be written comprises: Determining the association between the data to be written and the multiple logical address segments; In a preset heat classification table, the heat level corresponding to the to-be-written data is obtained by querying according to the association situation, wherein the heat classification table is used to indicate the corresponding relationship between the write heat and the heat level of the logical address segment; An index information table is established based on the heat level of each of the data to be written, and the index information table is used to indicate the heat level of each of the data to be written.

6. The method for writing data into a storage system according to any one of claims 1 to 4, characterized in that: Writing the to-be-written data with the same heat level into the same physical address segment of the storage system includes: Determine the amount of data to be written at each heat level; Determine the writing order of the data to be written at each heat level according to the amount of data to be written at each heat level; According to the writing order, the to-be-written data with the same heat level are sequentially written into the same physical address segment of the storage system.

7. The method for writing data into a storage system according to claim 6, characterized in that: The step of sequentially writing the to-be-written data with the same heat level into the same physical address segment of the storage system according to the writing order further includes: According to the writing order, sequentially aggregate write requests for data to be written with the same heat level in the same data management block; When the write requests aggregated in the data management block reach a preset granularity, the write requests aggregated in the data management block are executed to sequentially flush the to-be-written data with the same heat level to the same physical address segment of the storage system.

8. The method for writing data into a storage system according to claim 7, characterized in that: Before aggregating write requests for data to be written with the same heat level in the same data management block in sequence according to the write order, the method further includes: Allocating corresponding data management blocks to the data to be written at each heat level according to the heat level of the data to be written, and identifying the heat level of the data management blocks; According to the writing order, the order of the data management blocks is identified.

9. The method for writing data into a storage system according to claim 7, characterized in that: The step of aggregating write requests for data to be written with the same heat level in the same data management block also includes: Determine a writing mode of data to be written with the same heat level, wherein the writing mode includes sequential writing and random writing; Write requests for data to be written with the same heat level and the same write mode are aggregated in the same data management block.

10. The method for writing data into a storage system according to any one of claims 1 to 4, characterized in that: The method further comprises: When the remaining space in the physical address segment is less than a preset threshold, respectively determining the proportion of invalid data in a plurality of data management blocks in the physical address segment and the existence duration of the invalid data in the plurality of data management blocks; The order of recovering invalid data from the multiple data management blocks is determined according to the proportion of invalid data in the multiple data management blocks and the existence time of the invalid data in the multiple data management blocks.

11. A device for writing data into a storage system, characterized in that: The device comprises: A writing status acquisition module, used to acquire historical data writing status of multiple logical address segments of the storage system; a write heat determination module, configured to determine the amount of written data, the number of write requests, and the location distribution of the written data in the multiple logical address segments within a preset time period according to the historical data writing conditions of the multiple logical address segments; and determine the write heat of the multiple logical address segments according to the amount of written data, the number of write requests, the location distribution of the written data in the preset time period, and the corresponding weights respectively; A heat level determination module, used for performing heat grading on the data to be written according to the write heat of the plurality of logical address segments, and determining the heat level of each of the data to be written, wherein each of the data to be written corresponds to one of the logical address segments; The write execution module is used to write the to-be-written data with the same heat level into the same physical address segment of the storage system.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 10 when executed by a processor.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 10 are implemented.

14. A computer program product comprising a computer program, characterized in that The computer program is executed by the processor Carry out the steps of the method described in any one of claims 1 to 10.

Images