A method, device, computer storage medium and terminal for realizing data processing

By distinguishing the writing methods and recording physical block information, and giving priority to random write garbage collection, the problem of low garbage collection efficiency in the FTL algorithm is solved and the service life of the solid-state hard disk is extended.

CN113608702BActive Publication Date: 2024-11-08HEFEI DATANG STORAGE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110949965.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-18
Publication Date
2024-11-08
Estimated Expiration
2041-08-18

AI Technical Summary

Technical Problem

When existing FTL algorithms write sequential data and random data, they lead to low garbage collection efficiency, affecting the write amplification efficiency and service life of solid-state drives.

Method used

By determining the data writing method to write to the namespace, sequential data and random data are written to different physical blocks, and the writing method is recorded. The randomly written physical blocks are garbage collected first, and then the sequentially written physical blocks are written to Physical blocks are garbage collected.

Benefits of technology

It improves garbage collection efficiency, extends the working time of the solid-state drive, and improves write amplification efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113608702B_ABST
    Figure CN113608702B_ABST
Patent Text Reader

Abstract

This article discloses a method, device, computer storage medium and terminal for implementing data processing, including: determining the writing method of data written into the namespace; writing the data into different physical blocks according to the different writing methods determined, and recording the physical block information of the data written when the data is written using different writing methods; and performing garbage collection on the physical blocks where the data is written using different writing methods according to the recorded physical block information. After distinguishing the writing methods of the data written into the namespace, the embodiment of the present invention records the physical block information written by different writing methods, and performs garbage collection according to the recorded physical block information, thereby providing a data basis for improving the garbage collection efficiency and extending the working time of the solid-state hard disk; further, garbage collection is first performed on the randomly written physical blocks, and then on the sequentially written physical blocks, thereby extending the working time of the solid-state hard disk by improving the garbage collection efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This article relates to but is not limited to testing technology, and in particular to a method, device, computer storage medium and terminal for implementing data processing. Background Art

[0002] The non-volatile memory host controller interface specification (NVMe) protocol of the high-speed serial computer expansion bus standard (PCIe) solid-state drive (SSD) supports the namespace (NAMESPACE) function. Each NAMESPACE is an independent block device, which appears as an independent hard disk device on the host side. The current flash translation layer (FTL) algorithm will write the write command issued by the host into the same physical block. For example, data written to NAMESPACE1 and NAMESPACE2 will be written to the same physical block; data written to different NAMESPACEs may have different characteristics, for example, random data is written to NAMESPACE1, and sequential data is written to NAMESPACE2; according to the current FTL algorithm, sequential data and random data will be written to the same physical block. Summary of the invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The embodiments of the present invention provide a method, device, computer storage medium and terminal for implementing data processing, which can improve the garbage collection efficiency and extend the working time of a solid-state hard disk.

[0005] An embodiment of the present invention provides a method for implementing data processing, including:

[0006] Determines how data written to the namespace is written;

[0007] Writing the data to different physical blocks according to different determined writing modes, and recording the physical block information to which the data is written when the data is written using different writing modes;

[0008] According to the recorded physical block information, garbage collection is performed on the physical blocks in which data is written using different writing methods.

[0009] On the other hand, an embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method for realizing data processing is implemented.

[0010] In another aspect, an embodiment of the present invention further provides a terminal, comprising: a memory and a processor, wherein the memory stores a computer program; wherein:

[0011] The processor is configured to execute the computer program in the memory;

[0012] When the computer program is executed by the processor, the method for data processing as described above is implemented.

[0013] In another aspect, an embodiment of the present invention further provides a device for implementing data processing, comprising: a writing mode determination unit, a recording unit and a garbage collection unit; wherein:

[0014] The determining writing mode unit is configured to: determine a writing mode for data written into the name space;

[0015] The recording unit is configured to: write data to different physical blocks according to different determined writing modes, and record information of the physical blocks to which the data is written when the data is written in different writing modes;

[0016] The garbage collection unit is configured to perform garbage collection on physical blocks in which data is written in different writing modes according to recorded physical block information.

[0017] The technical solution of the present application includes: determining the writing method of the data written into the namespace; writing the data into different physical blocks according to the different writing methods determined, and recording the physical block information of the data written when the data is written using different writing methods; and performing garbage collection on the physical blocks where the data is written using different writing methods according to the recorded physical block information. After distinguishing the writing methods of the data written into the namespace, the embodiment of the present invention records the physical block information written by different writing methods, and performs garbage collection according to the recorded physical block information, thereby providing a data basis for improving the garbage collection efficiency and extending the working time of the solid-state hard disk; further, garbage collection is first performed on the randomly written physical blocks, and then on the sequentially written physical blocks, thereby extending the working time of the solid-state hard disk by improving the garbage collection efficiency.

[0018] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0020] Figure 1 A flowchart of a method for implementing data processing according to an embodiment of the present invention;

[0021] Figure 2 A structural block diagram of a device for implementing data processing according to an embodiment of the present invention;

[0022] Figure 3 The figure is a flowchart of an FTL algorithm according to an example of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention more clear, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other arbitrarily without conflict.

[0024] The steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions. Also, although a logical sequence is shown in the flowchart, in some cases, the steps shown or described can be performed in a sequence different from that shown here.

[0025] The inventor of the present application has analyzed and found that after sequential data and random data are written to the same physical block, when garbage collection (GC) is performed, the valid data in the physical block needs to be written back. The GC processing efficiency is low, which affects the write amplification (WA) efficiency of the SSD and affects the service life of the SSD.

[0026] Figure 1 Flow chart of a method for implementing data processing in an embodiment of the present invention, such as Figure 1 As shown, including:

[0027] Step 101, determining a writing method for data written into a namespace (NAMESPACE);

[0028] In an exemplary embodiment, the writing method in the embodiment of the present invention may include: sequential writing and random writing.

[0029] Step 102: writing the data into different physical blocks according to different determined writing modes, and recording the physical block information into which the data is written when the data is written using different writing modes;

[0030] When the embodiment of the present invention determines that the writing mode is different, the physical blocks to which the data is written are different, that is, data with different writing modes will not be written to the same physical block; for example, it is determined that data 1 is written sequentially, data 2 is written randomly, data 3 is written sequentially, and data 4 is written randomly. Then, according to the method of the embodiment of the present invention, if data 1 is written to physical block 1, data 2 will be written to other physical blocks except physical block 1, such as physical block 2. The writing mode of data 3 is sequential writing. When physical block 1 is not full, data 3 will be written to physical block 1. Similarly, data 4 will be written to physical block 2.

[0031] Step 103: According to the recorded physical block information, garbage collection (GC) is performed on the physical blocks into which data is written using different writing modes.

[0032] After distinguishing the writing methods of the data written into the namespace, the embodiment of the present invention records the physical block information written by different writing methods, and performs garbage collection according to the recorded physical block information, thereby providing a data basis for improving the garbage collection efficiency and extending the working time of the solid-state hard disk; further, garbage collection is first performed on the randomly written physical blocks, and then garbage collection is performed on the sequentially written physical blocks, thereby extending the working time of the solid-state hard disk by improving the garbage collection efficiency.

[0033] In an exemplary embodiment, the embodiment of the present invention determines the writing mode of data written into the namespace, including:

[0034] The writing method is determined according to the data writing command.

[0035] In an exemplary embodiment, the write command of the embodiment of the present invention includes: a command sent from a host that executes data writing.

[0036] In an exemplary embodiment, the embodiment of the present invention determines a writing mode according to a data writing command, including:

[0037] The data writing mode is determined according to the first logical block address (LBA) in the current write command of the host and the second LBA in the previous write command.

[0038] In an exemplary embodiment, the embodiment of the present invention determines a data writing method, including:

[0039] Subtracting the second write end address in the second LBA from the first write start address in the first LBA to obtain a first difference;

[0040] Subtracting the second write start address in the second LBA from the first write end address in the first LBA to obtain a second difference;

[0041] When the absolute value of the first difference or the absolute value of the second difference is less than a preset standard threshold, determining that the data writing method is sequential writing;

[0042] When the absolute value of the first difference and the absolute value of the second difference are both greater than or equal to a preset standard threshold, it is determined that the data writing method is random writing.

[0043] In an exemplary embodiment, the embodiment of the present invention writes data to different physical blocks according to different determined writing modes, including:

[0044] Determining a physical block for data writing according to the determined writing mode;

[0045] Determine whether the physical block used for data writing is full;

[0046] When it is determined that the physical block for data writing is full, a new physical block for data writing is obtained from the idle queue, and data writing is performed;

[0047] When it is determined that the physical block used for data writing is not full, the data is directly written into the physical block used for data writing.

[0048] In an exemplary embodiment, the writing mode of the embodiment of the present invention includes sequential writing and random writing, and the physical block information of data written when data is written using different writing modes is recorded, including:

[0049] Establishing a first queue and a second queue;

[0050] The physical block information of the data written when the data is written sequentially is recorded in the first queue, and the physical block information of the data written when the data is written randomly is recorded in the second queue.

[0051] In an exemplary embodiment, the embodiment of the present invention performs garbage collection on physical blocks in which data is written in different writing modes, including:

[0052] When it is determined that the second queue contains physical blocks that have not been garbage collected, garbage collection is performed on the physical blocks that have not been garbage collected and are recorded in the second queue;

[0053] When it is determined that garbage collection has been completed for all physical blocks recorded in the second queue, garbage collection is performed on the physical blocks recorded in the first queue that have not been garbage collected.

[0054] The embodiment of the present invention first performs garbage collection on randomly written physical blocks, and then performs garbage collection on sequentially written physical blocks, thereby extending the working time of the solid state drive by improving the garbage collection efficiency.

[0055] The above method for implementing data processing in the embodiment of the present invention may be loaded into a flash translation layer (FTL) algorithm for implementation, that is, the above method for implementing data processing may be used to adjust the FTL algorithm.

[0056] An embodiment of the present invention further provides a computer storage medium, in which a computer program is stored. When the computer program is executed by a processor, the above-mentioned method for realizing data processing is implemented.

[0057] The embodiment of the present invention further provides a terminal, comprising: a memory and a processor, wherein a computer program is stored in the memory; wherein:

[0058] The processor is configured to execute the computer program in the memory;

[0059] When the computer program is executed by a processor, the method for implementing data processing as described above is implemented.

[0060] Figure 2 FIG. 1 is a structural block diagram of a device for implementing data processing according to an embodiment of the present invention, such as Figure 2 As shown, it includes: a recording unit and a garbage collection unit; wherein,

[0061] The recording unit is configured as: a writing mode determination unit, a recording unit and a garbage collection unit; wherein,

[0062] The determining writing mode unit is configured to: determine a writing mode for data written into the name space;

[0063] The recording unit is configured to: write data to different physical blocks according to different determined writing modes, and record information of the physical blocks to which the data is written when the data is written in different writing modes;

[0064] The garbage collection unit is configured to perform garbage collection on physical blocks in which data is written in different writing modes according to recorded physical block information.

[0065] After distinguishing the writing methods of the data written into the namespace, the embodiment of the present invention records the physical block information written by different writing methods, and performs garbage collection according to the recorded physical block information, thereby providing a data basis for improving the garbage collection efficiency and extending the working time of the solid-state hard disk; further, garbage collection is first performed on the randomly written physical blocks, and then garbage collection is performed on the sequentially written physical blocks, thereby extending the working time of the solid-state hard disk by improving the garbage collection efficiency.

[0066] In an exemplary embodiment, the device of the embodiment of the present invention determines that the writing mode unit is set to:

[0067] The writing method is determined according to the data writing command.

[0068] In an exemplary embodiment, the device of the embodiment of the present invention determines that the writing mode unit is set to:

[0069] The data writing mode is determined according to the first logical block address LBA in the current write command and the second LBA in the previous write command.

[0070] In an exemplary embodiment, the writing mode determining unit of the apparatus of the embodiment of the present invention is configured to determine the writing mode according to the writing command of the data, including:

[0071] Subtracting the second write end address in the second LBA from the first write start address in the first LBA to obtain a first difference;

[0072] Subtracting the second write start address in the second LBA from the first write end address in the first LBA to obtain a second difference;

[0073] When the absolute value of the first difference or the absolute value of the second difference is less than a preset standard threshold, determining that the data writing method is sequential writing;

[0074] When the absolute value of the first difference and the absolute value of the second difference are both greater than or equal to a preset standard threshold, it is determined that the data writing method is random writing.

[0075] In an exemplary embodiment, the writing unit of the embodiment of the present invention is configured as follows:

[0076] Determining a physical block for data writing according to the determined writing mode;

[0077] Determine whether the physical block used for data writing is full;

[0078] When it is determined that the physical block for data writing is full, a new physical block for data writing is obtained from the idle queue, and data writing is performed;

[0079] When it is determined that the physical block used for data writing is not full, the data is directly written into the physical block used for data writing.

[0080] In an exemplary embodiment, after the embodiment of the present invention determines the writing method, the physical block used for data writing can be determined based on writing the data to different physical blocks according to the different determined writing methods. Generally, after distinguishing the physical blocks used to write different writing methods, the data can be written in the physical blocks corresponding to each writing method.

[0081] In an exemplary embodiment, the recording unit of the embodiment of the present invention is configured as follows:

[0082] Establishing a first queue and a second queue;

[0083] The physical block information of the data written when the data is written sequentially is recorded in the first queue, and the physical block information of the data written when the data is written randomly is recorded in the second queue.

[0084] In an exemplary embodiment, the garbage collection unit of the embodiment of the present invention is configured as follows:

[0085] When it is determined that there are physical blocks that have not been garbage collected recorded in the second queue, garbage collection is performed on the physical blocks that have not been garbage collected recorded in the second queue; when it is determined that all physical blocks recorded in the second queue have completed garbage collection, garbage collection is performed on the physical blocks that have not been garbage collected recorded in the first queue.

[0086] The following briefly describes the embodiments of the present invention through application examples. The application examples are only used to illustrate the embodiments of the present invention and are not used to limit the protection scope of the present invention.

[0087] Application Examples

[0088] This application example uses the FTL algorithm that carries the method for implementing data processing as the execution subject to illustrate the embodiment of the present invention. This application example implements the FTL algorithm of multiple NAMESPACEs; Figure 3 The flowchart of the FTL algorithm of the present invention is shown as follows: Figure 3 As shown, including:

[0089] Step 301, establishing a first queue and a second queue; wherein the first queue is used to record and manage the physical block information of the physical blocks to which data is written when sequential writing is adopted, and the second queue is used to record and manage the physical block information of the physical blocks to which data is written when random writing is adopted; in this application example, the first queue is defined as queue0, and the second queue is defined as queue1;

[0090] Step 302: upon receiving a write command from the host, read a first logical block address (LBA) in the current write command and a second LBA in the previous write command;

[0091] Step 303: Determine a data writing method according to the read first LBA and second LBA;

[0092] In an exemplary embodiment, the present application example determines a data writing method, including: obtaining a first difference by subtracting a first write start address in a first LBA from a second write end address in a second LBA; obtaining a second difference by subtracting a first write end address in the first LBA from a second write start address in a second LBA; when an absolute value of the first difference or an absolute value of the second difference is less than a preset standard threshold, determining that the data writing method is sequential writing; when both the absolute value of the first difference and the absolute value of the second difference are greater than or equal to the preset standard threshold, determining that the data writing method is random writing; in an exemplary embodiment, the preset standard threshold of the present application example can be determined by a person skilled in the art through analysis based on the characteristics of NAMESPACE data;

[0093] Step 304: write the data to different physical blocks according to the different writing modes determined, and record the physical block information of the data when writing the data in different writing modes through the first queue and the second queue;

[0094] In an exemplary embodiment, the present application example writes data into a physical block corresponding to a writing method, including: determining a physical block for data writing according to a determined writing method; determining whether the physical block for data writing is full; when it is determined that the physical block for data writing is full, obtaining a new physical block for data writing from an idle queue and executing data writing; when it is determined that the physical block for data writing is not full, directly writing the data into the physical block for data writing.

[0095] Step 305: perform garbage collection of physical blocks according to the first queue and the second queue;

[0096] In an exemplary embodiment, this application example performs garbage collection of physical blocks according to the first queue and the second queue, including: when performing garbage collection processing, when it is determined that there are physical blocks that have not been garbage collected recorded in the second queue, garbage collection is performed on the physical blocks that have not been garbage collected recorded in the second queue; when it is determined that garbage collection has been completed for all physical blocks recorded in the second queue, garbage collection is performed on the physical blocks that have not been garbage collected recorded in the first queue.

[0097] This application example distinguishes data streams and writes them into different physical blocks, giving priority to obtaining physical blocks that need to be garbage collected from a queue with random characteristics. Since the sequentially written data stream can reduce garbage collection or even eliminate the need for garbage collection, WAI is greatly reduced and the service life of the SSD is extended.

[0098] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

Claims

1. A method for implementing data processing, comprising: Determines how data written to the namespace is written; Writing the data to different physical blocks according to different determined writing modes, and recording the physical block information to which the data is written when the data is written using different writing modes; According to the recorded physical block information, garbage collection is performed on the physical blocks where data is written using different writing methods; Among them, the writing method includes sequential writing and random writing, and the recording of physical block information of data written when data is written using different writing methods includes: establishing a first queue and a second queue; recording the physical block information of data written when data is written using sequential writing in the first queue, and recording the physical block information of data written when data is written using random writing in the second queue; the garbage collection of physical blocks where data is written using different writing methods includes: when it is determined that there are physical blocks recorded in the second queue that have not been garbage collected, garbage collection is performed on the physical blocks recorded in the second queue that have not been garbage collected; when it is determined that the physical blocks recorded in the second queue have all completed garbage collection, garbage collection is performed on the physical blocks recorded in the first queue that have not been garbage collected.

2. The method according to claim 1, characterized in that The step of determining a writing method of data written into the namespace includes: The writing mode is determined according to a writing command of the data.

3. The method according to claim 2, characterized in that The determining the writing mode according to the writing command of the data includes: The data writing mode is determined according to the first logic block address LBA in the current write command and the second logic block address LBA in the previous write command.

4. The method according to claim 3, characterized in that The method of determining the data writing method includes: Subtracting a second write end address in the second logical block address LBA from a first write start address in the first logical block address LBA to obtain a first difference; Subtracting a second write start address in the second logical block address LBA from a first write end address in the first logical block address LBA to obtain a second difference; When the absolute value of the first difference or the absolute value of the second difference is less than a preset standard threshold, determining that the data writing mode is sequential writing; When the absolute value of the first difference and the absolute value of the second difference are both greater than or equal to the preset standard threshold, it is determined that the data writing method is random writing.

5. The method according to any one of claims 2 to 4, characterized in that: The step of writing data to different physical blocks according to different determined writing modes includes: Determining a physical block for data writing according to the determined writing mode; Determining whether the physical block for data writing is full; When it is determined that the physical block for data writing is full, a new physical block for data writing is obtained from the idle queue, and data writing is performed; When it is determined that the physical block for data writing is not full, the data is directly written into the physical block for data writing.

6. A computer storage medium, wherein a computer program is stored in the computer storage medium, and when the computer program is executed by a processor, the method for realizing data processing according to any one of claims 1 to 5 is implemented.

7. A terminal, comprising: A memory and a processor, wherein the memory stores a computer program; wherein, The processor is configured to execute the computer program in the memory; When the computer program is executed by the processor, the method for implementing data processing according to any one of claims 1 to 5 is implemented.

8. A device for implementing data processing, comprising: Determine a write mode unit, a recording unit and a garbage collection unit; wherein, The determining writing mode unit is configured to: determine a writing mode for data written into the name space; The recording unit is configured to: write data to different physical blocks according to different determined writing modes, and record information of the physical blocks to which the data is written when the data is written in different writing modes; The garbage collection unit is configured to: perform garbage collection on physical blocks to which data is written in different writing modes according to recorded physical block information; Among them, the writing method includes sequential writing and random writing, and the recording of physical block information of data written when data is written using different writing methods includes: establishing a first queue and a second queue; recording the physical block information of data written when data is written using sequential writing in the first queue, and recording the physical block information of data written when data is written using random writing in the second queue; the garbage collection of physical blocks where data is written using different writing methods includes: when it is determined that there are physical blocks recorded in the second queue that have not been garbage collected, garbage collection is performed on the physical blocks recorded in the second queue that have not been garbage collected; when it is determined that the physical blocks recorded in the second queue have all completed garbage collection, garbage collection is performed on the physical blocks recorded in the first queue that have not been garbage collected.