An SSD storage fragmentation optimization management method, electronic device and storage medium

Abstract

The present invention discloses an SSD storage fragmentation optimization management method, an electronic device and a storage medium, relating to the technical field of data storage. The method includes: obtaining the logical address and physical address corresponding to the file data in the solid state drive; determining, according to the logical address and the physical address, a plurality of logical fragments corresponding to the file data in the logical space and a plurality of physical fragments in the physical space; determining, based on the fragmentation information of the plurality of logical fragments and the plurality of physical fragments, the fragments with a fragment size smaller than a set size threshold as critical fragments; sending a fragmentation reorganization instruction to the storage end, so that the storage end transfers the data corresponding to the critical fragments to an independent segment space, and the independent segment space is a preset part of the storage space in the solid state drive. The present invention can reduce the data volume of fragmentation reorganization, reduce the I / O cost of the host end, avoid the interference between the fragmentation reorganization process and other write operations, improve the fragmentation reorganization efficiency, and optimize the storage performance.

Description

Technical Field

[0001] This application relates to the technical field of data storage, and in particular, to a method for optimizing and managing SSD storage fragmentation, an electronic device, and a storage medium. Background Art

[0002] Facing the increasingly rich application types and usage scenarios, traditional hard disk drives (HDDs) can no longer meet the performance requirements of various applications for data storage. Solid State Drives (SSDs) with flash memory as the storage medium are gradually becoming the mainstream storage devices due to their advantages such as high performance, low power consumption, and large capacity.

[0003] When an SSD is in use, it needs to perform frequent data read and write operations, and a large amount of storage fragmentation will be generated during this process, that is, data storage fragmentation occurs. Data storage fragmentation leads to multiple addressings and reads when reading files, reducing the read efficiency and thus affecting the read and write performance of the SSD.

[0004] Therefore, there is an urgent need for a solution for optimizing and managing data storage fragmentation for solid state drives. Summary of the Invention

[0005] Aiming at the above technical problems and deficiencies, the purpose of the present invention is to provide a method for optimizing and managing SSD storage fragmentation, an electronic device, and a storage medium, which can reduce the amount of data for defragmentation, reduce the I / O cost of the host side, avoid interference between the defragmentation process and other write operations, improve the defragmentation efficiency, and optimize the storage performance.

[0006] To achieve the above purpose, in a first aspect, the present invention provides a method for optimizing and managing SSD storage fragmentation, which is applied to the host side of an electronic device. The electronic device further includes a storage side, and the storage side includes a solid state drive. The method includes: obtaining the logical address and physical address corresponding to the file data in the solid state drive, where the logical address is the address of the file data in the logical space of the file system, and the physical address is the address of the file data in the physical space of the solid state drive; determining, according to the logical address and the physical address, multiple logical fragments corresponding to the file data in the logical space, and multiple physical fragments corresponding to the file data in the physical space; based on the fragment information of the multiple logical fragments and the multiple physical fragments, determining fragments with a fragment size smaller than a set size threshold as key fragments; sending a defragmentation instruction to the storage side so that the storage side transfers the data corresponding to the key fragments to an independent segment space, and the independent segment space is a preset part of the storage space in the solid state drive.

[0007] By adopting the above embodiments, by obtaining the logical address and physical address of file data in the solid-state drive, the distribution of file data on the storage medium can be accurately located, providing a solid data foundation for subsequent fragment identification and sorting. This not only helps reduce data access latency but also improves the utilization rate of storage space. And based on the logical address and physical address information, this technology can accurately identify logical fragments and physical fragments, and then determine the key fragments. By preferentially processing the key fragments, this technology can significantly reduce the impact of fragments on storage performance, improving the system's response speed and stability. Finally, by sending a defragmentation instruction to the storage end, the storage end transfers the data corresponding to the key fragments to the independent segment space, thus realizing automatic defragmentation. In this way, the work of data migration is offloaded to the storage end, and tasks such as fragment analysis are left on the host side. This asynchronous method of completing the defragmentation work can reduce the impact of data sorting on the read and write performance of foreground applications. The entire process requires no manual intervention, greatly simplifying the complexity of fragment management and improving the maintenance efficiency of the storage system. The present invention application realizes the efficient identification and automatic sorting of SSD storage fragments, significantly improving the performance and efficiency of the storage system and providing a more stable and efficient storage fragment optimization solution.

[0008] In some embodiments, the file data includes I / O request access data; after the step of determining, according to the logical address and the physical address, that the file data corresponds to multiple logical fragments in the logical space and multiple physical fragments in the physical space, the method further includes: obtaining I / O information of the I / O request access data, where the I / O information includes a file index and / or an in-file offset, and the I / O information is used as fragment metadata of the logical fragment and the physical fragment; saving the I / O information to a doubly linked list.

[0009] By adopting the above embodiments, by obtaining the I / O information of the I / O request access data, including the file index and / or the in-file offset, which are used as fragment metadata of the logical fragment and the physical fragment, accurate identification and tracking of the fragments are realized. Subsequently, saving the I / O information to a doubly linked list not only ensures the integrity and orderliness of the information but also improves the efficiency and flexibility of fragment management, providing strong data support for subsequent fragment sorting and optimization operations.

[0010] In some embodiments, after the step of determining, according to the logical address and the physical address, that the file data corresponds to multiple logical fragments in the logical space and multiple physical fragments in the physical space, the method further includes: sorting the logical fragments and the physical fragments respectively by a red-black tree according to the fragment information to obtain a logical fragment red-black tree and a physical fragment red-black tree; maintaining the fragment information of the logical fragment or the physical fragment respectively based on the logical fragment red-black tree or the physical fragment red-black tree.

[0011] Using the above embodiments, by utilizing the fragmentation information and adopting the efficient data structure of a red - black tree to sort the logical fragments and physical fragments, a logical - fragment red - black tree and a physical - fragment red - black tree are obtained. This sorting method ensures the rapid retrieval and orderly management of the fragmentation information. Meanwhile, based on the logical - fragment red - black tree or the physical - fragment red - black tree, the fragmentation information of the logical fragments or physical fragments can be respectively maintained, realizing the real - time tracking and dynamic update of the fragment status, providing strong support for fragment management and optimization.

[0012] In some embodiments, after the step of determining, based on the fragmentation information of the plurality of logical fragments and the plurality of physical fragments, that a fragment with a size smaller than a set size threshold is a critical fragment, the method further includes: allocating a new logical address and a new physical address for the critical fragment; updating the fragment metadata of the critical fragment.

[0013] Using the above embodiments, for the identified critical fragments, new logical addresses and physical addresses are allocated to ensure the optimization of their storage locations and the improvement of performance. Meanwhile, to ensure the accuracy and consistency of the fragmentation information, the fragment metadata of the critical fragments is updated in a timely manner, including their new address information and other related attributes, thereby providing a solid foundation for the fragment management and optimization of the system. This technical effect not only improves the read - write performance of the storage device but also enhances the efficiency and reliability of data management.

[0014] In some embodiments, before the step of obtaining the logical address and the physical address corresponding to the file data in the solid - state drive, the method further includes: obtaining the load characteristics of the file system in real - time; determining whether to perform a defragmentation task according to the load characteristics.

[0015] Using the above embodiments, by obtaining the load characteristics of the file system in real - time, the operating status and performance bottlenecks of the system can be comprehensively understood. According to these load characteristics, the host side can intelligently determine whether to perform a defragmentation task to effectively reduce the impact of fragmentation on system performance without affecting the normal operation of the business. This technical effect not only improves the overall performance of the file system but also reduces the additional overhead caused by defragmentation, achieving a balance between performance optimization and resource utilization.

[0016] In some embodiments, the fragmentation information includes the fragment size and the number of fragments. Before the step of determining, based on the fragmentation information of the plurality of logical fragments and the plurality of physical fragments, that a fragment with a size smaller than a set size threshold is a critical fragment, the method further includes: determining the total capacity size of the solid - state drive; determining the size threshold of the critical fragment according to the total capacity size and a preset percentage threshold.

[0017] By adopting the above embodiments, by accurately determining the total capacity of the solid-state drive and combining with a preset percentage threshold, the size threshold of critical fragments can be scientifically and reasonably determined. This technical effect provides an accurate basis for subsequent fragment management and optimization work, ensuring that critical fragments can be accurately identified and processed, thereby effectively improving the performance and stability of the solid-state drive and meeting the user's demand for the efficient operation of the storage system.

[0018] In some embodiments, before the step of sending a defragmentation instruction to the storage end, it further includes: determining the degree of fragmentation according to the number of critical fragments, the number of logical fragments, and the number of physical fragments; when the degree of fragmentation is greater than a preset degree threshold, it is determined that the read and write performance of the electronic device is affected.

[0019] By adopting the above embodiments, through comprehensive consideration of the number of critical fragments, the number of logical fragments, and the number of physical fragments, the degree of fragmentation of the storage device can be accurately determined. When this degree of fragmentation exceeds the preset threshold, the host can quickly identify and determine that the read and write performance of the electronic device has been significantly affected. This technical effect provides an important basis for taking defragmentation measures in a timely manner, which helps to ensure the stable operation and high performance of the electronic device.

[0020] In a second aspect, an embodiment of the present application provides an electronic device, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions, when the above instructions run on an electronic device, enabling the above electronic device to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer program product including instructions, when the above computer program product runs on an electronic device, enabling the above electronic device to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0023] It can be understood that the electronic device provided in the second aspect, the storage medium provided in the third aspect, and the computer program product provided in the fourth aspect are all used to execute the method provided in the embodiments of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, and will not be elaborated here.

[0024] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0025] 1. By obtaining the logical address and physical address of file data in a solid-state drive, the distribution of file data on the storage medium can be accurately located, providing a solid data foundation for subsequent fragment identification and sorting. This not only helps reduce data access latency but also improves storage space utilization. Moreover, based on the logical address and physical address information, this technology can accurately identify logical fragments and physical fragments, and then determine the critical fragments. By preferentially processing the critical fragments, this technology can significantly reduce the impact of fragments on storage performance, improve the system's response speed and stability. Finally, by sending a defragmentation instruction to the storage end, the storage end transfers the data corresponding to the critical fragments to the independent segment space, thus realizing automatic defragmentation. In this way, the work of data migration is offloaded to the storage end, and tasks such as fragment analysis are left on the host side. This asynchronous way of completing the defragmentation work can reduce the impact of data sorting on the read and write performance of foreground applications. The entire process requires no manual intervention, greatly simplifies the complexity of fragment management, and improves the maintenance efficiency of the storage system. The present invention application realizes the efficient identification and automatic sorting of SSD storage fragments, significantly improves the performance and efficiency of the storage system, and provides a more stable and efficient storage fragment optimization solution.

[0026] 2. By obtaining the I / O information of the I / O request accessing data, including file index and / or in-file offset, these information are used as fragment metadata of logical fragments and physical fragments, thereby realizing the accurate identification and tracking of fragments. Subsequently, these I / O information are saved in a doubly linked list, which not only ensures the integrity and orderliness of the information but also improves the efficiency and flexibility of fragment management, providing strong data support for subsequent fragment sorting and optimization operations.

[0027] 3. Using the fragment information, the red-black tree, an efficient data structure, is adopted to sort logical fragments and physical fragments, thereby obtaining the logical fragment red-black tree and the physical fragment red-black tree. This sorting method ensures the quick retrieval and orderly management of fragment information. At the same time, based on the logical fragment red-black tree or the physical fragment red-black tree, the fragment information of logical fragments or physical fragments can be respectively maintained, realizing the real-time tracking and dynamic update of fragment status, providing strong support for fragment management and optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. In the drawings:

[0029] Figure 1 is a flowchart of a method for optimizing and managing SSD storage fragmentation in an embodiment of the present application;

[0030] Figure 2 is a schematic diagram of another method for optimizing and managing SSD storage fragmentation in an embodiment of the present application;

[0031] Figure 3 is a schematic diagram of an architecture of an electronic device in an embodiment of the present application. Detailed implementation manners

[0032] The terms used in the following embodiments of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention, the singular forms "a", "an", "above-mentioned", "the", and "this" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present invention refers to any or all possible combinations including one or more of the listed items.

[0033] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise stated, the meaning of "a plurality" is two or more.

[0034] Solid State Drives (SSDs) are increasingly widely used in electronic devices. SSDs use flash memory as the storage medium and are trending towards becoming the mainstream storage device due to their high performance, low power consumption, large capacity, etc.

[0035] Due to the physical characteristics limitations of flash memory, SSDs can only access and store data in a write-after-erase manner. To abstract SSDs into general block devices, related technologies have designed a Flash Translation Layer (FTL) based on the log-structured idea, which converts the physical addresses of flash memory into logical addresses in the file system that users can directly access through address mapping, and also completes functions such as garbage collection and wear leveling.

[0036] During the read and write processes of a solid-state drive (SSD), a large amount of storage fragmentation is generated, which leads to data storage fragmentation. Data storage fragmentation causes multiple addressings and reads when reading files, reducing the read efficiency and thus affecting the read and write performance of the SSD.

[0037] In the embodiments of the present invention, the storage fragmentation of data mainly includes logical fragmentation and physical fragmentation.

[0038] Logical fragmentation refers to the fragmentation phenomenon caused by the discontinuous storage of file data in the logical space of the file system. In the file system, each file is usually divided into multiple logical blocks, and these logical blocks are continuously numbered in the logical space. However, with operations such as file creation, modification, and deletion, the free space in the file system becomes discontinuous, resulting in the inability of new files or the expansion of existing files to be continuously stored in new logical blocks. Therefore, the data of the file is split into multiple discontinuous parts in the logical space, and these discontinuous parts constitute logical fragmentation.

[0039] Among them, the file system is a mechanism used by a computer operating system to manage and organize files and directories on a storage device. In the file system, data is organized in the form of files and directories. A file is the logical storage unit of data, and a directory is a structure used to organize and manage files. The file system is responsible for storing, retrieving, updating, and deleting files and directories, and provides a file access interface for applications and users. Common file systems include FAT, NTFS, ext4, etc. The file system is crucial for the operating system and applications. It provides the function of data persistence, ensuring that data will not be lost after the power is turned off. In addition, the file system is also responsible for managing the storage space allocation of files, ensuring that data can be correctly read after being written.

[0040] Physical fragmentation refers to the fragmentation phenomenon caused by the discontinuous physical storage locations of file data in the physical space of the storage device. In a physical storage medium (such as a solid-state drive SSD), file data is stored as a series of physical blocks. With operations such as data writing, deletion, and update, the free space in the physical blocks becomes scattered and discontinuous. When new data needs to be written, it may be allocated to discontinuous physical blocks, resulting in the discontinuous storage of file data in the physical space. This physical discontinuity is physical fragmentation. Physical fragmentation also has a negative impact on the performance and reliability of the storage device. It may lead to a reduction in the efficiency of disk read and write operations because the system needs to access multiple scattered physical locations to read or write file data.

[0041] Therefore, the embodiments of the present invention provide a technical solution for optimizing and managing SSD storage fragmentation. By obtaining the logical addresses and physical addresses of file data in the solid-state drive, this solution can accurately locate the distribution of file data on the storage medium, providing a solid data foundation for subsequent fragmentation identification and sorting. This not only helps reduce data access latency but also improves the utilization rate of storage space. Moreover, based on the logical address and physical address information, this technology can accurately identify logical fragmentation and physical fragmentation, and then determine the critical fragmentation. By preferentially processing the critical fragmentation, this technology can significantly reduce the impact of fragmentation on storage performance, improving the system's response speed and stability. Finally, by sending a defragmentation instruction to the storage end, the storage end transfers the data corresponding to the critical fragmentation to an independent segment space, thereby achieving automatic defragmentation. In this way, the work of data migration is offloaded to the storage end, and tasks such as fragmentation analysis are left on the host side. This asynchronous method of completing the defragmentation work can reduce the impact of data sorting on the read and write performance of foreground applications. The entire process requires no manual intervention, greatly simplifying the complexity of fragmentation management and improving the maintenance efficiency of the storage system. The present invention application realizes the efficient identification and automatic sorting of SSD storage fragmentation, significantly improving the performance and efficiency of the storage system and providing a more stable and efficient storage fragmentation optimization solution.

[0042] The technical solution of this embodiment adopts an SSD storage fragmentation optimization management method, which is applied to the host side of an electronic device. The electronic device further includes a storage side, and the storage side includes a solid-state drive. The electronic device in the embodiments of this application has a storage function and can store data.

[0043] Among them, the host side refers to computing units such as the central processing unit (CPU) and memory of a computer system, and is also referred to as the host or main computer. The host side is responsible for executing application programs, performing data processing and other computing tasks, and managing access to storage devices. The storage side refers to a storage device connected to the host side, such as a hard disk, SSD, etc. The storage side is responsible for storing data and responding to read and write requests from the host side.

[0044] The technical solution of the embodiment of the present application can be applied to various scenarios where storage space needs to be fragmented, including but not limited to enterprise storage, distributed storage, cloud storage, etc. The present application does not specifically limit the type of electronic device. In some embodiments, the electronic device in the embodiment of the present application can be a mobile phone, a wearable device (such as a smart bracelet), a tablet computer, a laptop, a handheld computer, a computer, an ultra-mobile personal computer (UMPC), a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) / virtual reality (VR) device and other portable devices. It can also be an in-vehicle device, a projector and other devices, and can also be a cloud storage device, etc.

[0045] To achieve the above effects, a method for optimizing and managing SSD storage fragmentation provided in this embodiment is as Figure 1 shown, and includes the following steps:

[0046] Step 101, obtain the logical address and physical address corresponding to the file data in the solid-state drive.

[0047] Among them, the logical address is the address of the file data in the logical space of the file system, and the physical address is the address of the file data in the physical space of the solid-state drive.

[0048] Specifically, when the host obtains the logical address and physical address corresponding to the file data in the solid-state drive, it will first communicate with the solid-state drive through the file system interface. The file system interface is a bridge for interaction between the host and the storage end, and it provides access and control functions for the storage end.

[0049] In the file system, each file is assigned a unique logical address, which is numbered in the logical space of the file system and is used to identify the position of the file in the logical structure. Through the file system interface, the host can query the start address and length information of the file in the logical space, so as to determine the logical address range of the file data.

[0050] At the same time, the solid-state drive has its own address mapping mechanism inside, which maps the logical address to the physical address. The physical address is the specific position of the file data in the physical space of the solid-state drive, and it identifies the block or page where the data is actually stored. When the host needs to read or write file data, it will send an access request with the logical address to the solid-state drive. After receiving the request, the solid-state drive will convert the logical address into the corresponding physical address according to its internal address mapping table and locate the actual data storage location.

[0051] Therefore, through the interaction between the host and the solid-state drive via the file system interface and the address mapping mechanism inside the solid-state drive, the host can accurately obtain the address information of file data in the logical space and the physical space, providing necessary support for subsequent file operations.

[0052] Step 102: Based on the logical address and the physical address, determine multiple logical fragments corresponding to the file data in the logical space and multiple physical fragments in the physical space.

[0053] Specifically, after the host obtains the logical address and physical address of the file data in the solid-state drive, it will further analyze this address information to determine the fragmentation distribution of the file data in the logical space and the physical space.

[0054] First, the host will locate the start and end positions of the file in the logical space of the file system according to the logical address. Since the file may be stored discontinuously in the logical space due to multiple operations such as modification, deletion, and re-writing, there will be multiple logical fragments. The host will traverse the logical address range to identify all discontinuous logical blocks, which are the logical fragments of the file.

[0055] Next, the host will locate the actual storage location of the file data in the physical space of the solid-state drive according to the physical address. Due to the allocation and release operations of the physical storage space, the file data may also be split into multiple discontinuous blocks in the physical space, forming physical fragments. The host will use the interface or command provided by the solid-state drive to obtain the physical block information corresponding to the file data and identify all the physical fragments.

[0056] Through this process, the host can accurately determine the fragmentation distribution of the file data in the logical space and the physical space. This fragmentation information is crucial for subsequent fragmentation reorganization and optimization, which can help the host manage the storage space more effectively and improve the read / write performance and stability of the system.

[0057] Step 103: Based on the fragmentation information of the multiple logical fragments and the multiple physical fragments, determine the fragments with a fragment size smaller than the set size threshold as critical fragments.

[0058] Specifically, after the host obtains the fragmentation information of the multiple logical fragments and physical fragments of the file data, it will further analyze and process these fragments to determine which fragments are critical fragments.

[0059] First, the host will set a fragment size threshold, which is determined by comprehensively considering factors such as the performance requirements of the storage system, the characteristics of the file system, and the characteristics of the solid-state drive. This threshold represents the minimum size of the fragments that the host believes have a greater impact on the system performance.

[0060] Next, the host will traverse all logical and physical fragments and check the size of each fragment. For fragments smaller than the set size threshold, the host will mark them as critical fragments. This is because these smaller fragments may cause more addressing and switching overhead during read and write operations, thus increasing system latency and reducing performance.

[0061] In addition, the host can also consider other factors to determine critical fragments, such as the degree of fragmentation continuity, the access frequency of fragments, etc. Considering these factors comprehensively, the host can more accurately identify the critical fragments that have the greatest impact on system performance.

[0062] The following is an example. Suppose the host manages a large file in a solid-state drive. Due to multiple modification and write operations, this file has multiple fragments in both logical and physical spaces. To optimize storage performance, the host decides to analyze and process these fragments.

[0063] First, the host sets a fragment size threshold, such as 512KB. This threshold is determined based on the performance requirements of the storage system and the characteristics of the solid-state drive, meaning that fragments smaller than this size may have a greater impact on system performance.

[0064] Next, the host obtains all fragment information of this file in logical and physical spaces. Suppose there are three logical fragments A, B, and C with sizes of 1MB, 256KB, and 50KB respectively; and three physical fragments D, E, and F with sizes of 768KB, 400KB, and 80KB respectively.

[0065] Then, the host starts to traverse these fragments and checks whether their sizes are smaller than the set size threshold. In this example, the sizes of logical fragment C and physical fragment F are both smaller than the 512KB threshold.

[0066] Based on this check result, the host determines logical fragment C and physical fragment F as critical fragments. This is because their sizes are relatively small, which may cause more disk addressing and switching operations, thus increasing system latency and reducing read and write performance.

[0067] Through this process, the host can, based on the fragment size threshold and other relevant factors, determine those small fragments that have a greater impact on performance as critical fragments, providing a basis for subsequent fragmentation and optimization.

[0068] Step 104: Send a defragmentation instruction to the storage end so that the storage end transfers the data corresponding to the critical fragments to an independent segment space, which is a preset part of the storage space in the solid-state drive.

[0069] Specifically, after the host determines the critical fragments, it will initiate a defragmentation process to optimize storage performance. A crucial step is to send a defragmentation instruction to the storage side.

[0070] First, based on the critical fragment information obtained from the previous analysis, the host generates corresponding defragmentation instructions. These instructions contain the identifiers, locations of the critical fragments to be defragmented, and information about the target transfer space.

[0071] Next, the host sends these defragmentation instructions to the storage side through a storage interface or a specific communication protocol. After receiving the instructions, the storage side will parse and execute them.

[0072] During the execution process, the storage side will locate the specific physical locations of the critical fragments and read the data corresponding to these fragments. Then, the storage side will transfer the read data to a preset independent segment space. This independent segment space is a part of the storage space reserved in the solid-state drive, specifically used to store the defragmented data. This can avoid the interlacing of the defragmentation process and ordinary user write operations, and can also avoid secondary fragmentation introduced by concurrent write operations.

[0073] After the transfer is completed, the storage side will update the relevant metadata and data structures to ensure the consistency of the file system and the integrity of the data. At the same time, the storage side will also send a confirmation message to the host, informing that the defragmentation operation has been completed.

[0074] Through this process, the host successfully guides the storage side to transfer the data corresponding to the critical fragments to the independent segment space, realizing the automatic defragmentation and optimization of the fragments, and improving the performance and stability of the storage system.

[0075] In some embodiments, the file system extends the ZNS (Zoned Namespace) command interface to further optimize the data management and performance of the storage side (flash device). In this extension, a new defragmentation command defrag is added. The core idea of this command is to offload the work of data migration to the flash device, while leaving relatively complex tasks such as fragment analysis, address space allocation, and fragment metadata update to the host side for processing.

[0076] Specifically, when the host identifies critical fragments and decides to perform defragmentation, it will first perform fragment analysis. In this step, the host will carefully check the distribution of the fragments, determine which fragments need to be defragmented, and calculate the required space size after defragmentation. Next, the host will perform address space allocation to reserve sufficient space for the data blocks to be migrated. At the same time, to ensure the consistency and integrity of the data, the host will also update the relevant metadata.

[0077] After completing these preparations, the host sends a defrag command to the storage side through the ZNS command interface. This command contains the identifiers, source addresses, target addresses, and other relevant information of the data blocks to be migrated. After receiving this command, the flash device is responsible for the actual data migration work. It reads data from the source address according to the instructions in the command and writes it to the target address. This process is asynchronous, that is, while the flash device is performing data migration, the host can continue to process other tasks without waiting for the migration to complete.

[0078] This asynchronous defragmentation method brings significant advantages. First, it reduces the impact of data defragmentation on the read and write performance of foreground applications. Since the data migration work is offloaded to the flash device, the host does not need to spend a lot of time and computing resources on this operation, so it can process other tasks more efficiently. Second, since flash devices usually have faster data processing speeds, the data migration speed will also be faster, further improving the defragmentation efficiency.

[0079] Generally speaking, the system's extension of the ZNS command interface and the addition of the defrag command is an effective optimization measure. By offloading the data migration work to the flash device and adopting an asynchronous processing method, it reduces the impact of defragmentation on the performance of foreground applications and improves the performance and stability of the entire system.

[0080] The method of this embodiment obtains the logical addresses and physical addresses of file data in the SSD through the host, and identifies logical fragments and physical fragments. Filters out key fragments smaller than the set threshold to reduce the amount of data for defragmentation. Offloads the data migration work to the device side to complete, reducing the I / O cost of the host. Avoids interference between the defragmentation process and other write operations, improving the defragmentation efficiency. Achieves effective management of SSD storage fragments and reduces the impact of defragmentation on system performance.

[0081] In some embodiments, the file data includes I / O request access data, that is, the data accessed through I / O requests. The host mainly conducts data interaction through the I / O interface, so the file data mainly consists of I / O request access data.

[0082] After step 102, it further includes: First, obtain the I / O information of the I / O request access data. The I / O information includes a file index and / or an offset within the file. The I / O information is used as the fragment metadata of the logical fragment and the physical fragment; then save the I / O information to a doubly linked list.

[0083] Specifically, the host side captures the detailed information of the data accessed in the I / O request, including the file index and / or the offset within the file. The file index is the unique identifier that specifies the file being accessed in the file system, while the offset within the file is an offset of the position of the data relative to the start position of the file, which can indicate the specific position within the file. These pieces of information together constitute the information of the I / O request, which not only describe the context of data access but also can serve as the fragmentation metadata for both logical and physical fragments.

[0084] Next, the host side saves the obtained I / O information into a doubly linked list. A doubly linked list is a data structure that allows traversing data elements in two directions. In the context of fragmentation management, using a doubly linked list can facilitate tracking and managing the metadata information of logical and physical fragments.

[0085] By saving the I / O information into the doubly linked list, the host side can establish the association between the fragments and the I / O requests. This association is very important for subsequent operations such as defragmentation, performance optimization, and fault recovery. For example, during defragmentation, the host side can, based on the information in the doubly linked list, give priority to processing those fragments that are frequently accessed to improve the overall performance of the system. Also, if data loss or corruption occurs, the host side can use the I / O information in the doubly linked list to restore the integrity of the data.

[0086] Therefore, obtaining the metadata of the I / O request and saving it into the doubly linked list is an essential step in fragmentation management, which provides the host side with rich information to manage and optimize the fragments in the storage system.

[0087] In some embodiments, after step 102, it further includes: according to the fragmentation information, respectively sorting the logical fragment and the physical fragment through a red - black tree to obtain a logical - fragment red - black tree and a physical - fragment red - black tree; based on the logical - fragment red - black tree or the physical - fragment red - black tree, respectively maintaining the fragmentation information of the logical fragment or the physical fragment.

[0088] Among them, a red - black tree is a self - balancing binary search tree data structure that can maintain a relatively balanced tree height during insertion, deletion, and search operations, thus ensuring that the time complexity of these operations is approximately logarithmic. This is very beneficial for the host side that needs to frequently access and update the fragmentation information.

[0089] By sorting the logical fragment and the physical fragment through the red - black tree, the host side can obtain a logical - fragment red - black tree and a physical - fragment red - black tree. These two trees sort the logical fragment and the physical fragment respectively according to specific sorting rules (such as fragment size, access frequency, etc.). This sorting enables the host side to retrieve and locate specific fragments more efficiently, providing convenience for subsequent defragmentation and optimization.

[0090] Based on the logical fragmentation red - black tree or the physical fragmentation red - black tree, the host can respectively maintain the fragmentation information of logical fragments or physical fragments. This means that when the fragmentation information changes (such as fragments being sorted, merged, or new fragments being generated), the host will correspondingly update the node information in the red - black tree to maintain the accuracy and consistency of the fragmentation information.

[0091] By maintaining the logical fragmentation red - black tree and the physical fragmentation red - black tree, the host can achieve fast access and efficient management of the fragmentation information. This helps the host quickly locate key fragments when needed and take corresponding measures for optimization, thereby improving the performance and stability of the storage system.

[0092] In some embodiments, after step 103, it further includes: allocating new logical addresses and physical addresses for the key fragment; updating the fragment metadata of the key fragment.

[0093] Specifically, the host will allocate new logical addresses and physical addresses for each key fragment. This process involves finding suitable positions for the key fragments in the logical space and the physical space. The host will determine the optimal address allocation strategy based on the current state of the storage system, the access frequency of the fragments, their sizes, and other relevant factors. Among them, the new logical addresses and physical addresses correspond to the independent segment space. By allocating new addresses for the key fragments, the addressing overhead during data access can be reduced, and the read - write performance of the system can be improved.

[0094] Next, the host will update the fragment metadata of the key fragment. The fragment metadata contains various information about the fragment, such as the size, location, access frequency, etc. After allocating the new logical addresses and physical addresses, the host needs to update these new address information into the fragment metadata to ensure that the system can accurately locate and manage these key fragments. At the same time, the host can also update other metadata related to the fragment as needed, such as the status of the fragment, the file to which it belongs, etc.

[0095] By allocating new addresses for the key fragments and updating their metadata, the host can effectively manage and optimize the fragments in the storage system. This can not only improve the read - write performance of the system but also reduce the impact of fragments on the system stability and reliability. At the same time, this processing method also provides convenience for subsequent fragmentation sorting and optimization operations, enabling the host to more efficiently maintain and manage the data in the storage system.

[0096] In some embodiments, before step 101, it further includes: obtaining the load characteristics of the file system in real - time; determining whether to perform a fragmentation sorting task according to the load characteristics.

[0097] Among them, the load characteristics refer to the performance parameters and status information exhibited by the file system during operation, including but not limited to CPU usage rate, memory occupancy rate, disk I / O read / write rate, concurrent access volume, etc. These characteristics can reflect the current workload of the file system and potential performance bottlenecks.

[0098] After obtaining the load characteristics, the host will determine whether to perform a defragmentation task based on these characteristics. Although the defragmentation task helps optimize storage performance, it also consumes certain system resources during execution, such as CPU time, memory space, and disk I / O bandwidth. Therefore, if the current load of the file system is already high, performing the defragmentation task may further exacerbate the system burden and lead to performance degradation.

[0099] Based on this consideration, the host will set one or more thresholds to determine whether the load characteristics have reached the conditions for triggering the defragmentation task. For example, if the CPU usage rate or memory occupancy rate exceeds a certain threshold, the host may decide not to perform the defragmentation task temporarily to avoid further exacerbating the system burden. On the contrary, if the system load is low and the disk I / O read / write rate is slow, this may indicate that the storage performance is affected by fragmentation. In this case, the host may decide to perform the defragmentation task to optimize the performance.

[0100] In this way, the host can dynamically determine whether to perform the defragmentation task according to the real-time load characteristics of the file system. This helps balance the relationship between system load and performance optimization, ensuring effective defragmentation and management without affecting the normal operation of the system.

[0101] In some embodiments, the fragmentation information includes the fragmentation size and the number of fragments. Before step 103, it further includes: determining the total capacity size of the solid-state drive; determining the size threshold of critical fragments according to the total capacity size and a preset percentage threshold.

[0102] Specifically, the host will determine the total capacity size of the solid-state drive. This can usually be done by querying the specification information of the storage device or directly reading the capacity attribute of the device. Understanding the total capacity size is the basis for setting the size threshold because it helps the host formulate a suitable fragmentation management strategy according to the actual capabilities of the storage device.

[0103] Next, the host will determine the size threshold of critical fragments based on the total capacity and a preset percentage threshold. The percentage threshold is a preset parameter that represents the proportion of the total SSD capacity that the host considers should be regarded as critical fragments. This percentage threshold can be adjusted according to the actual application scenario and performance requirements. For example, if the system has high requirements for storage performance, a lower percentage threshold may need to be set to identify and process critical fragments earlier. Critical fragments are fragments that have a greater impact on the storage performance.

[0104] By multiplying the total capacity by the percentage threshold, the host can calculate the size threshold of critical fragments. This threshold will be used as the basis for subsequent judgment of whether a fragment is critical. When the size of a fragment is less than this threshold, the host will regard it as a critical fragment and perform corresponding processing.

[0105] In this way, the host can dynamically set the size threshold of critical fragments according to the actual capacity and performance requirements of the SSD. This helps to ensure the pertinence and effectiveness of the fragment management strategy, while avoiding overprocessing of non-critical fragments, thus improving the overall performance and stability of the storage system.

[0106] In some embodiments, before step 104, it further includes: determining the degree of fragmentation according to the number of critical fragments, the number of logical fragments, and the number of physical fragments; when the degree of fragmentation is greater than a preset degree threshold, determining that the read / write performance of the electronic device is affected.

[0107] Specifically, the host will calculate the degree of fragmentation based on the determined number of critical fragments, the number of logical fragments, and the number of physical fragments. The degree of fragmentation is a relative indicator used to measure the severity of fragments in the storage. The specific calculation method may vary depending on the system, but usually factors such as the number, size, and distribution of fragments will be considered.

[0108] After calculating the degree of fragmentation, the host will compare it with a preset degree threshold. The degree threshold is a preset standard for judging whether the degree of fragmentation has reached the level that requires defragmentation. This threshold can be set according to the system's performance requirements, the characteristics of the storage device, and the actual application scenario.

[0109] If the degree of fragmentation is greater than the preset degree threshold, the host will determine that the read / write performance of the electronic device has been affected. This means that the fragment problem in the storage device has become serious enough to have a negative impact on the system performance, and defragmentation is required to optimize the performance.

[0110] After determining that the fragmentation level affects performance, the host will perform subsequent steps, such as sending a defragmentation instruction to the storage end to trigger the defragmentation task. This step aims to improve the read and write performance of the storage device by reducing fragmentation and optimizing the data storage structure, thereby ensuring the stable and reliable overall performance of the electronic device.

[0111] Through this series of steps and judgments, the host can accurately evaluate the fragmentation level of the storage device and trigger the defragmentation task at the appropriate time to maintain the read and write performance of the electronic device and avoid unnecessary performance overhead.

[0112] In some embodiments, such as Figure 2 shown, the SSD storage fragmentation optimization management method can also be specifically as follows:

[0113] (1) Fragment identification and quantification

[0114] Similar to logical fragments, physical fragments also have a greater impact on the read performance of the flash device (storage end). The host makes full use of the characteristics of the SSD to analyze the logical and physical addresses corresponding to the data accessed by the I / O request at the file system layer, calculate the fragment size and number of fragments it is divided into in the logical space, and the number of available chips in the physical space, to complete the identification and quantification of logical and physical fragments.

[0115] In Figure 2 , the fragmentation is identified and quantified through I / O characteristics. The ino, start, end, and count in the I / O characteristics represent the following:

[0116] ino: File index, used to identify different files.

[0117] start: Offset within the file, indicating the starting position of the accessed data in the file.

[0118] end: Offset within the file, indicating the ending position of the accessed data in the file.

[0119] count: Count, indicating the number of times of this I / O access.

[0120] Therefore, ino, start, end, and count respectively represent the file index, offset within the file, and access count, used to identify and count the access characteristics of different I / O requests. These characteristics are used by the system to analyze the logical and physical addresses corresponding to the data accessed by the I / O request, and then identify and quantify the fragments.

[0121] (2) Fragment metadata management

[0122] Taking the data accessed by I / O requests as the object for defragmentation requires maintaining a large amount of fragmentation metadata. The system uses the file index and the offset within the file as the unique identifiers of fragmentation information, saves I / O information with a doubly linked list, and uses a red-black tree to sort logical and physical fragments respectively, reducing the metadata access cost during the process of selecting defragmentation objects.

[0123] (3)Optimization of Defragmentation Object Selection and Target Segment Space Management

[0124] Fragments of different degrees have significant differences in their impact on the system. After the system completes the process of fragment identification and quantification, it periodically triggers defragmentation tasks, filters out the key fragments that affect performance using a pre-set percentage threshold, and reduces data migration during the defragmentation process. To avoid the interleaving of the defragmentation process and ordinary user write operations, the system writes fragmented data to a separately allocated segment space, avoiding secondary fragmentation introduced by concurrent write operations.

[0125] (4)Data Migration Offloading Strategy

[0126] Traditional defragmentation tools involve a large amount of data transfer between the CPU, memory, and device side, occupying I / O bandwidth resources and affecting the read and write performance of foreground applications. The system extends the ZNS command interface, adds a defragmentation command defrag, offloads the work of data migration to the flash device to complete, and leaves tasks such as fragment analysis, address space allocation, and metadata update on the host side. Completing the defragmentation work asynchronously can reduce the impact of data defragmentation on the read and write performance of foreground applications.

[0127] In Figure 2 , the solid-state drive includes multiple storage chips, such as four storage chips: chip0, chip1, chip2, and chip3. Each storage chip includes multiple storage fragments. In the figure, dark gray, light gray, and white squares represent different storage fragments, and the white squares represent key fragments. Seg6 is an ordinary storage segment space, and Seg7 is a separately allocated segment space.

[0128] Through the above embodiments, first monitor the I / O activities of applications at the file system layer, analyze and quantify the fragmentation degree of the data accessed by these I / O operations, then filter out the fragments that affect performance and use them as defragmentation objects to improve the efficiency of defragmentation. During defragmentation, write the data to a separate log segment to avoid mixing with other data and causing secondary fragmentation. Finally, extend the ZNS command interface and hand over the work of data copying to the device side to avoid data transfer between the host side and the device side, reduce the I / O cost of the host side, and improve the chip parallelism of the flash device.

[0129] The following describes the electronic device in the embodiments of the present invention application from the perspective of hardware processing. Please refer toFigure 3 , which is a schematic structural diagram of an entity device of the electronic device in the embodiment of the present application.

[0130] It should be noted that Figure 3 the structure of the electronic device shown is only an example and should not bring any limitations to the functions and usage scope of the embodiments of the present invention.

[0131] As Figure 3 shown, the electronic device includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage section 408 into the random access memory (RAM) 403, such as executing the method described in the above embodiments. In the RAM 403, various programs and data required for system operation are also stored. The CPU 401, ROM 402, and RAM 403 are connected to each other via a bus 404. The input / output (I / O) interface 405 is also connected to the bus 404.

[0132] The following components are connected to the I / O interface 405: an input section 406 including an audio input device, a button switch, etc.; an output section 407 including a liquid crystal display (LCD), an audio output device, an indicator light, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. The drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed so that the computer program read from it can be installed into the storage section 408 as needed.

[0133] Specifically, according to the embodiments of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments of the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication section 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, various functions defined in the present invention are executed.

[0134] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. Among them, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings.

[0136] Specifically, the electronic device of this embodiment includes a processor and a memory. The memory is coupled to one or more processors, and the memory is used to store computer program code. The computer program code includes computer instructions, and one or more processors call the computer instructions to enable the electronic device to execute the method provided in the above embodiment.

[0137] On the other hand, the present invention also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or it may exist separately without being assembled into the electronic device. The above storage medium carries one or more computer programs, and when the one or more computer programs are executed by a processor of the electronic device, the electronic device is enabled to implement the method provided in the above embodiment.

[0138] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present application.

[0139] As used in the foregoing embodiments, depending on the context, the term "when" may be construed to mean "if" or "after" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "upon determining" or "if (the stated condition or event) is detected" may be construed to mean "if determined" or "in response to determining" or "when (the stated condition or event) is detected" or "in response to detecting (the stated condition or event)".

[0140] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the foregoing embodiments can be implemented, and such processes can be completed by relevant hardware instructed by a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the foregoing method embodiments. The foregoing storage media include: various media such as ROM or random access memory RAM, magnetic disks, or optical discs that can store program codes.

Claims

1. A method for optimizing and managing SSD storage fragmentation, characterized in that: The method is applied to a host end of an electronic device, wherein the electronic device further comprises a storage end, wherein the storage end comprises a solid state hard disk, and the method comprises: Obtaining a logical address and a physical address corresponding to the file data in the solid state drive, wherein the logical address is an address of the file data in a logical space of the file system, and the physical address is an address of the file data in a physical space of the solid state drive; Determine, according to the logical address and the physical address, a plurality of logical fragments in the logical space and a plurality of physical fragments in the physical space corresponding to the file data; Determining the total capacity of the solid state drive; Determine a size threshold of a key fragment according to the total capacity and a preset percentage threshold; Based on fragment information of the plurality of logical fragments and the plurality of physical fragments, fragments having fragment sizes smaller than a set size threshold are determined as key fragments, wherein the fragment information includes fragment sizes and fragment numbers; A defragmentation instruction is sent to the storage end so that the storage end transfers the data corresponding to the key fragment to an independent segment space, where the independent segment space is a preset part of the storage space in the solid state drive.

2. The method according to claim 1, characterized in that: The file data includes I / O request access data; after the step of determining, according to the logical address and the physical address, that the file data corresponds to a plurality of logical fragments in the logical space and a plurality of physical fragments in the physical space, the method further includes: Acquire I / O information of the I / O request to access data, the I / O information including a file index and / or an offset within a file, and the I / O information is used as fragment metadata of the logical fragment and the physical fragment; The I / O information is saved in a bidirectional linked list.

3. The method according to claim 1 or 2, characterized in that: After the step of determining, according to the logical address and the physical address, that the file data corresponds to a plurality of logical fragments in the logical space and a plurality of physical fragments in the physical space, the method further includes: According to the fragment information, the logical fragments and the physical fragments are sorted respectively by a red-black tree to obtain a logical fragment red-black tree and a physical fragment red-black tree; Based on the logical fragment red-black tree or the physical fragment red-black tree, fragment information of the logical fragment or the physical fragment is maintained respectively.

4. The method according to claim 1, characterized in that After the step of determining fragments whose fragment sizes are smaller than a set size threshold as key fragments based on the fragment information of the plurality of logical fragments and the plurality of physical fragments, the method further includes: Allocating new logical addresses and physical addresses to the key fragments; The shard metadata of the key shard is updated.

5. The method according to claim 1, characterized in that Before the step of obtaining the logical address and the physical address corresponding to the file data in the solid state hard disk, the method further includes: Acquiring the load characteristics of the file system in real time; According to the load characteristics, it is determined whether to execute a defragmentation task.

6. The method according to claim 1, characterized in that Before the step of sending the defragmentation instruction to the storage end, the method further includes: Determining a degree of fragmentation according to the number of the key fragments, the number of the logical fragments, and the number of physical fragments; When the fragmentation degree is greater than a preset degree threshold, it is determined that the reading and writing performance of the electronic device is affected.

7. An electronic device, characterized in that: include: one or more processors and memory; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 6.

9. A computer program product, characterized in that When the computer program product is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 6.

Images