Memory recovery method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a memory recovery method and device, electronic equipment and a storage medium, and relates to the technical field of data storage. Under the condition that both the file system and the storage device support the perceptual garbage collection function, the device state of the storage device is obtained in response to the starting of the perceptual garbage collection function; according to the equipment state, determining whether an execution condition for sensing a garbage collection function is met or not; and if the execution condition of the garbage collection sensing function is met, the file system is linked to perform memory collection on the storage device, so that the file system can know the data blocks moved by the storage device, the file system is prevented from moving the data blocks moved by the storage device again, and the efficiency of the file system is improved. In addition, invalid blocks in the file system and the storage device can be synchronized, the situation that invalid data blocks in the file system are moved when the storage device executes memory recovery is avoided, and therefore the service life expenditure of the storage device and the write-in amplification value of the storage device are reduced.

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and more specifically, to a memory recovery method, device, electronic device and storage medium. Background Art

[0002] In Android, the file system is used to organize and store computer data, such as documents, pictures, videos, applications, and other data that are stored in some way on the computer's hard drive or other storage media. The file system allows users to easily find, read, and write files by associating data with metadata (such as file name, date, time, etc.).

[0003] However, file systems are not completely reliable and sometimes generate garbage or redundant data. The generation of garbage is the result of the interaction of multiple factors. For example, garbage data may include, but is not limited to, data generated by application execution errors, temporary files generated by browsers, residual files after software uninstallation, redundant files restored by the system, and data generated based on user behavior. Although the industry has developed a wide variety of file systems and storage devices based on different business requirements for file data storage, the garbage collection technology of the currently developed file systems and storage devices still has certain defects, such as large lifespan overhead of storage devices and high write amplification values ​​of storage devices. Summary of the invention

[0004] The embodiments of the present application provide a memory recovery method, device, electronic device and storage medium to improve the above-mentioned problems.

[0005] In a first aspect, an embodiment of the present application provides a memory recovery method, the method comprising: in a case where both a file system and a storage device support a perceptual garbage collection function, in response to the perceptual garbage collection function being enabled, obtaining a device status of the storage device; determining whether an execution condition of the perceptual garbage collection function is met based on the device status; if the execution condition of the perceptual garbage collection function is met, linking the file system to perform memory recovery on the storage device.

[0006] In a second aspect, an embodiment of the present application provides a memory recovery device, which includes: a response module, which is used to obtain the device status of the storage device in response to the activation of the perceptual garbage collection function when both the file system and the storage device support the perceptual garbage collection function; a decision module, which is used to determine whether the execution conditions of the perceptual garbage collection function are met based on the device status; and a linkage module, which is used to link the file system to perform memory recovery on the storage device if the execution conditions of the perceptual garbage collection function are met.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes: a memory and a processor. An application program is stored in the memory, and the application program is used to execute the method provided by the embodiment of the present application when called by the processor.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which program code is stored. The program code is used to cause the processor to execute the method provided by the embodiment of the present application when called by the processor.

[0009] In the memory recycling method, device, electronic device and storage medium provided by the embodiments of the present application, both the file system and the storage device support the awareness garbage collection function, so that the file system and the storage device can perceive each other's states. When it is determined according to the device state of the storage device that the execution condition for the awareness garbage collection function is met, the file system is linked to perform memory recycling on the storage device. Thus, the file system dominates the memory recycling of the storage device, which can not only enable the file system to know the data blocks that have been moved by the storage device, avoid the file system from moving the data blocks that have been moved by the storage device again, but also synchronize the invalid blocks in the file system and the storage device, avoid the storage device from moving the invalid data blocks in the file system during memory recycling, and thus can reduce the lifetime overhead and write amplification factor of the storage device. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, rather than all embodiments. Based on the embodiments of the present application, all other embodiments and drawings obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0011] Figure 1 is a structural block diagram of an awareness garbage collection system provided by an embodiment of the present application;

[0012] Figure 2 is provided by an embodiment of the present application Figure 1 detailed structural block diagram of the awareness garbage collection system shown;

[0013] Figure 3 is a flowchart of a memory recycling method provided by an embodiment of the present application;

[0014] Figure 4 is a flowchart of issuing an erasure instruction provided by an exemplary embodiment of the present application;

[0015] Figure 5 is a flowchart of determining a plurality of target sections provided by an exemplary embodiment of the present application;

[0016] Figure 6 It is a flowchart for determining multiple target sections provided by another exemplary embodiment of the present application;

[0017] Figure 7 It is a flowchart of a memory recycling method provided by an exemplary embodiment of the present application;

[0018] Figure 8 It is a flowchart of a memory recycling method provided by another embodiment of the present application;

[0019] Figure 9 It is a flowchart of an initialization stage provided by an exemplary embodiment of the present application;

[0020] Figure 10 It is a flowchart of an initialization stage provided by another exemplary embodiment of the present application;

[0021] Figure 11 It is a flowchart of an initialization stage provided by yet another exemplary embodiment of the present application;

[0022] Figure 12 It is a flowchart of an initialization stage provided by still another exemplary embodiment of the present application;

[0023] Figure 13 It is a flowchart of an initialization stage provided by still another exemplary embodiment of the present application;

[0024] Figure 14 It is a structural block diagram of a memory recycling device provided by an embodiment of the present application;

[0025] Figure 15 It is a structural block diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0026] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0027] Explanation of related concepts involved in memory recycling: The memory recycling process involves three types of data blocks, namely, used blocks, valid blocks, and invalid blocks. Among them, a used block is a data block that has been written with useful data, a valid block is a data block to which new data can be written, and an invalid block is obtained by converting a used block. An invalid block is a data block whose data can be erased, and the invalid data can also be called garbage data. In memory recycling, a data block is the smallest logical unit of memory recycling, that is, the smallest granularity unit of data storage. Usually, what is read and written from the storage medium (such as a disk) of a storage device is a data block. Several adjacent data blocks form an extent. A group of extents forms a segment.

[0028] The embodiments of this application mainly relate to a file system and a storage device with the ability of memory recycling (also known as garbage collection), such as F2FS (Flash Friendly File System) and Universal Flash Storage (UFS) devices. Both F2FS and UFS devices perform memory recycling internally to organize fragmented space, thereby meeting the design of the file system and the usage limitations of the flash media.

[0029] F2FS is a file system designed for flash storage devices, with characteristics such as efficient writing, memory recycling, and scalability. In F2FS, memory recycling is an important mechanism for maintaining the health and performance of the file system. The memory recycling process of F2FS is mainly achieved by scanning the metadata of the file system, marking and recycling invalid blocks. Specifically, when a user deletes a file or directory, F2FS will mark the corresponding inode node as "free" and mark the data blocks occupied by the inode node as "free". When F2FS performs memory recycling by itself, the data blocks marked with "free" as invalid blocks will be recycled, thereby improving the space utilization rate and performance of F2FS.

[0030] UFS devices are widely used in mobile devices and other storage systems. The storage technology of UFS devices has higher reliability and stability compared to file systems, with higher data transfer speeds and higher storage capacities. The memory recycling process of UFS devices mainly involves two parts: logical block marking and data block erasure. Specifically, when a user deletes a file or directory, the UFS device marks the corresponding logical block as "deleted" and marks the data block occupied by the logical block as "reusable". When the UFS device performs memory recycling, it traverses the entire storage medium to find the data blocks (invalid blocks) marked as "reusable" in the storage medium, and uses the TRIM command to erase the data blocks marked as "reusable" from the storage medium to recycle the data blocks marked as "reusable", thereby improving the stability and service life of the storage device. Among them, the TRIM command is used to notify the storage device by the operating system which data blocks are no longer considered for use and can be internally erased.

[0031] In the process of long-term research on memory recycling, the inventors found that: currently, file systems and storage devices are independent of each other, and are in a black box state relative to each other, unable to perceive each other's operating status, logic, and goals. This leads to the following problems when the file system and storage device independently perform memory recycling:

[0032] (1) In a short period of time, one or more data blocks are moved by the storage device and then moved again by the file system. This not only causes fragmentation of the flash blocks after the storage device is organized, but also causes the data to be written twice in total, resulting in the Write amplification (WAF) problem of the storage device and an increase in lifespan overhead.

[0033] (2) The file system generates invalid blocks due to data deletion or off-site data updates. At this time, the file system needs to synchronize this information to the storage device to let the storage device clean up the invalid data. Otherwise, when the storage device performs memory recycling, it may move the invalid data in the file system, resulting in invalid movement, causing an increase in the lifespan overhead and write amplification value of the storage device.

[0034] To solve the above problems where the file system and the storage device cannot perceive each other's states, the present application proposes a memory recycling method, apparatus, electronic device, and storage medium. By enabling the file system and the storage device to perceive each other's states, when the execution conditions for the perceived garbage collection function are met, the file system is linked to perform memory recycling on the storage device, thereby allowing the file system to dominate the memory recycling of the storage device. This can not only enable the file system to know the data blocks that have been migrated by the storage device and avoid the file system from migrating the data blocks that have already been migrated by the storage device again, but also synchronize the invalid blocks in the file system and the storage device, preventing the storage device from migrating the invalid data blocks in the file system during memory recycling, thus reducing the lifetime overhead and write amplification factor of the storage device.

[0035] See Figure 1 , Figure 1 FIG. is a structural block diagram of a perceived garbage collection system provided by an embodiment of the present application. The software part of the perceived garbage collection (Universal Flash Storage, abbreviated as Garbage Collection) system involved in the present application includes: a user-mode module and a kernel module. The user-mode module is located on the Android platform. The user-mode module includes one or more applications, such as WeChat and ByteDance. As Figure 1 shown, the user-mode module includes Applications 1-4. The Android platform provides a Volume Daemon (abbreviated as Vold). Vold is the control center of the external storage system in the Android platform and is a background process for managing and controlling external storage devices in the Android platform. The kernel module includes a file system, a block layer, and a driver for the storage device. The file system includes a garbage collection module. The hardware part of the perceived garbage collection system includes: a storage device, on which a Flash Translation Layer (abbreviated as FLT) firmware is stored. The FLT firmware includes a garbage collection module, and the garbage collection module deletes memory blocks by the flash driver. For example, the driver of the storage system can be a driver that meets the SCSI (Small Computer System Interface) standard, denoted as the SCSI-UFS driver, and the driver of the flash in the storage device can be denoted as the UFS driver. SCSI refers to an independent processor standard for the system-level interface between a computer and its peripheral devices (hard disks, floppy drives, optical drives, printers, scanners, etc.).

[0036] See Figure 2 , Figure 2 is provided by an embodiment of the present application Figure 1The detailed structural block diagram of the perception garbage collection system shown. The user-mode module includes a maintenance management module. The kernel module includes the sysfs file system and a file system with garbage collection function (such as F2FS). The file system with garbage collection function includes a statistical maintenance module and a garbage collection module. The block layer includes an input / output processing module. The driver of the storage device includes a command processing module. The storage device includes a garbage collection module. For example, the UFS hardware device includes the FLT garbage collection module. Among them,

[0037] The maintenance management module is at least used to perform the following three functions: (1) Dynamically enable the perception garbage collection function and adjust the perception garbage collection parameters; (2) Obtain the statistical information in the kernel module and upload it regularly; (3) Identify whether the system state is idle and update the state of the garbage collection module of the file system with garbage collection function (such as F2FS).

[0038] The statistical maintenance module is used to count the data related to the perception garbage collection function and provide interfaces related to the perception garbage collection function. For example, the interface for switching the perception garbage collection function and the parameter setting interface related to the perception garbage collection function. Exemplarily, the path of the switch interface for the perception garbage collection function of the storage device is: / sys / fs / f2fs / <device> / aware_gc_enable. The method to enable the awareness-based garbage collection function is: echo 1 > / sys / fs / f2fs / <device> / aware_gc_enable. The method to turn off the awareness recycling function is: echo 0> / sys / fs / f2fs / <device> / aware_gc_enable.

[0039] For example, the path of the data migration statistics interface is: / sys / fs / f2fs / <device> / aware_gc_status. The method to display the execution status of aware garbage collection is: cat sys / fs / f2fs / <device> / aware_gc_status. Among them, the perceived garbage collection execution status may include but is not limited to the following: the total number of times of perceived garbage collection execution "cnt", the number of times "skip_cnt" when the background operation (BKOPS) status is not satisfied, the number of times "intr_cnt" interrupted during the execution, the number of segments "segs" moved (counting nodes and data separately), the number of data blocks "blks" moved (counting nodes and data separately), the longest execution time "max_time" of perceived garbage collection, the shortest execution time "min_time" of perceived garbage collection, the average execution time "avg_time" of perceived garbage collection, the maximum time overhead "lca_max_time" for a single acquisition of the logical address list (Logical Cluster address List, referred to as the LCA list), the minimum time overhead "lca_min_time" for a single acquisition of the logical address list, the average time overhead "lca_avg_time" for a single acquisition of the logical address list (Logical Cluster address List, referred to as the LCA list), the total number of times "lca_cnt" of acquiring the logical address list, and threshold / parameter control (read / write).

[0040] Exemplarily, the parameter setting interface path may include but is not limited to the following paths: the interface path for the selection algorithm aging time (mtime) threshold: / sys / fs / f2fs / <device> / aware_gc_mtime_threshold. Interface path for selecting the number of algorithm segments: / sys / fs / f2fs / <device> / aware_gc_victim_segs. Interface path for obtaining the buffer size of the logical address list: / sys / fs / f2fs / <device> / aware_gc_lca_list_size, where the size of the logical address list can be preset. For example, the valid range of the size of the logical address list is [4K, the maximum value allowed by the storage device]. When it exceeds the maximum value allowed by the storage device, it is configured according to the maximum value. Interface path for the data block threshold to sense whether to perform a synchronization point (Check Point, abbreviated as CP) after garbage collection migration: / sys / fs / f2fs / <device> / aware_gc_blks, where the maximum threshold of the number of data blocks sensed for garbage collection migration can be preset. When the number of data blocks sensed for garbage collection migration exceeds the maximum threshold, the acquisition of the logical address list will no longer continue. The valid range of the number of migrated data blocks is [the minimum value, the maximum value allowed for the logical address list by the storage device]. When the number of migrated data blocks exceeds the maximum value, it is set according to the maximum value. When the number of migrated data blocks is less than the minimum value, it is set according to the minimum value.

[0041] The garbage collection module of a file system with garbage collection function (such as F2FS) has at least the following functions: (1) Query whether the storage device (such as UFS) supports the aware garbage collection function; (2) Query the device status of the storage device, and the device status may include but is not limited to the background operation status, the total number of valid blocks, and the available WriteBooster Buffer Size; (3) Switch the aware garbage collection function of the storage device (such as UFS); (4) Query the logical address list to be recycled, that is, obtain the logical address list of the data that the storage device (such as UFS) is about to perform garbage collection on, and select the migrated data by the method of selecting the target section described below according to the data distribution in the logical address list. (5) According to the number of valid blocks in the storage device (such as UFS) and the actual number of valid blocks in the file system, obtain the virtual occupation situation (invalid data on the storage device side) of the invalid data in the file system with garbage collection function (such as F2FS) in the storage device (such as UFS). When the number of invalid data exceeds the quantity threshold, execute the synchronization point and the UMMAP command (hereinafter referred to as the erase command) to clean up the invalid data. Among them, the function of the UMMAP command is similar to the above-mentioned TRIM command, and is used to notify the storage device by the operating system which data blocks are no longer considered in use and can be internally erased.

[0042] To support the aware garbage collection function, the storage device needs to add the following attributes: (1) wDeviceValidDataSize, which represents the current valid data size of the storage device; (2) wExceptionEventStatus, which represents that the current used blocks of the storage device are within the aware garbage collection window. The relevant configurable parameters of the storage device are shown in Table 1:

[0043] Table 1

[0044]

[0045] It should be noted that the memory recycling method in the embodiments of the present application can be applied to the above-mentioned awareness garbage collection system, file system with garbage collection function, memory recycling device or electronic device in the following text. In all the method embodiments in the following text, the file system refers to a file system with garbage collection function, such as F2FS, and all the storage devices referred to in the method embodiments refer to storage devices with garbage collection function, such as UFS.

[0046] See Figure 3 , Figure 3 is a flowchart of the memory recycling method provided by an embodiment of the present application. The memory recycling method may include steps S110 to S130.

[0047] Step S110: When both the file system and the storage device support the awareness garbage collection function, in response to the activation of the awareness garbage collection function, obtain the device state of the storage device.

[0048] The awareness garbage collection function refers to the function of jointly recycling memory by the file system and the storage device. In the awareness garbage collection function, the file system takes the lead in recycling memory from the storage device.

[0049] In some embodiments, the awareness garbage collection function can be activated in response to the file system meeting a specified condition. Among them, the specified condition can be pre-customized by the developer. As an example, the specified condition can be to activate the awareness garbage collection function at a specified time (such as 12 o'clock at night), then when the file system detects that the time is the specified time, it will activate the awareness garbage collection function. As another example, the specified condition can be that the file system is in a specified state (such as the charging state), then the awareness garbage collection function will be activated when the file system is in the specified state.

[0050] In some other embodiments, the execution frequency of the perception garbage collection function can be determined according to the overall load of the input / output and the proportion of the input / output foreground occupancy, and the perception garbage collection function can be enabled according to the execution frequency. Among them, the overall load of the input / output and the proportion of the input / output foreground occupancy can be detected and output by the input / output processing module of the block layer. The overall load of the input / output can include three cases: high, medium, and low, and the proportion of the input / output foreground occupancy can include three cases: high, medium, and low. Exemplarily, the mapping relationship between the execution frequency of the perception garbage collection function and the overall load of the input / output and the proportion of the input / output foreground occupancy can be preset (see Table 2), and the execution frequency corresponding to the current overall load of the input / output and the proportion of the input / output foreground occupancy can be determined according to this mapping relationship. For example, referring to the mapping relationship shown in Table 2, assuming that the current proportion of the input / output foreground occupancy is high and the overall load of the input / output is low, the frequency corresponding to the low frequency is used as the execution frequency of the perception garbage collection function. Among them, the specific values of the frequencies corresponding to the low frequency, high frequency, and medium frequency can be set according to actual requirements.

[0051] Table 2

[0052]

[0053]

[0054] After the perception garbage collection function is enabled, the file system can send a device status acquisition request to the storage device, and in response to the device status acquisition request, the storage device can send its current device status to the file system.

[0055] Step S120: Determine whether the execution condition of the perception garbage collection function is satisfied according to the device status.

[0056] In the embodiments of the present application, the device status of the storage device may include, but is not limited to, the background operation status of the storage device, the number of valid blocks, and the available write enhancement buffer size. Among them, the background operation status may include a first state, a second state "0x1 (BGGC)", and a third state "0x2 / 0x3 (FGGC)".

[0057] Exemplarily, the first state can be represented as 0x0 (idle). The first state indicates that there is relatively sufficient available space in the storage device. In this case, it is necessary to check the sub-state to determine whether the execution conditions for the awareness garbage collection function are met. That is to say, the first state can include sub-states, and the sub-states can include a first sub-state (e.g., represented as 1) and a second sub-state (e.g., represented as 0). If the sub-state is the first sub-state "1", it means that the storage device hopes that the file system will cooperate in memory recycling. At this time, it can be determined that the execution conditions for the awareness garbage collection function are met, and step S130 can be continued. If the sub-state is the second sub-state "0", the file system does not need to cooperate with the storage device in memory recycling. At this time, it can be determined that the execution conditions for the awareness garbage collection function are not met, and both the storage device and the file system perform memory recycling according to their original garbage collection processes. Among them, the available space refers to the space that has been partitioned but has no stored data.

[0058] The second state can be represented as 0x1 (BGGC), where BGGC is the abbreviation of Background Garbage Collection. The second state indicates that there are some data blocks in the storage device that need to be recycled in memory, but there is still relatively sufficient available space in the storage device, and the background operation of the storage device is about to perform the background memory recycling operation. In this case, the file system can initiate the awareness recycling function to cooperate with the storage device in memory recycling. At this time, it can be determined that the execution conditions for the awareness garbage collection function are met.

[0059] The third state can be represented as 0x2 / 0x3 (FGGC), where FGGC is the abbreviation of Frontend Garbage Collection. The third state indicates that there are more data blocks in the storage device that need to be recycled in memory, and the available space in the storage device is less. The storage device needs to actively perform memory recycling. In this case, it is not necessary for the file system to cooperate with the storage device in memory recycling. At this time, it can be determined that the execution conditions for the awareness garbage collection function are not met, and both the storage device and the file system perform memory recycling according to their original garbage collection processes.

[0060] Based on the analysis of the first state to the third state, if the device state of the storage device is the first state and the sub-state is the first sub-state, it indicates that the available space in the storage device is relatively sufficient and it hopes to link with the file system. At this time, it can be determined that the storage device has a need to link with the file system. If the device state of the storage device is the second state, it indicates that the available space in the storage device is relatively sufficient and the background memory recycling operation is about to be initiated. At this time, it can be determined that the storage device has a need to link with the file system. Except for the above two cases, in other cases, it is determined that the storage device has no need to link with the file system.

[0061] If the device status indicates a need to link to the file system, the number of valid blocks in the storage device and the number of valid blocks in the file system can be obtained, and the virtual occupation status of the system data in the file system in the storage device can be calculated based on the number of valid blocks in the storage device and the number of valid blocks in the file system. The virtual occupation status indicates whether there is invalid data in the storage device, and the virtual occupation status can be represented by the number of invalid blocks, or by the ratio of the number of valid blocks in the storage device to the number of valid blocks in the file system.

[0062] In some embodiments, after obtaining the number of valid blocks in the storage device and the number of valid blocks in the file system, the number of invalid blocks in the storage device can be determined based on the number of valid blocks in the storage device and the number of valid blocks in the file system; if the number of invalid blocks in the storage device is greater than the quantity threshold, it is determined that the execution condition for the awareness garbage collection function is satisfied; if the number of invalid blocks in the storage device is less than or equal to the quantity threshold, it is determined that the execution condition for the awareness garbage collection function is not satisfied. Among them, the quantity threshold can be predefined according to actual requirements.

[0063] In other embodiments, after obtaining the number of valid blocks in the storage device and the number of valid blocks in the file system, the ratio of the number of valid blocks in the storage device to the number of valid blocks in the file system can be determined based on the number of valid blocks in the storage device and the number of valid blocks in the file system; if the ratio is less than the ratio threshold, it is determined that the execution condition for the awareness garbage collection function is satisfied; if the number of invalid blocks in the storage device is greater than or equal to the ratio threshold, it is determined that the execution condition for the awareness garbage collection function is not satisfied. Among them, the ratio threshold can be predefined according to actual requirements.

[0064] Step S130: If it is determined that the execution condition for the awareness garbage collection function is satisfied, link the file system to perform memory recovery on the storage device.

[0065] See Figure 4 , in response to determining that the execution condition for the awareness garbage collection function is satisfied, the file system can send an erase command to the storage device. The erase command includes the invalid blocks that need to be recycled, so that the storage device deletes the invalid blocks that need to be recycled according to the erase command and feeds back the execution result. The erase command is used to notify the storage device by the operating system which data blocks are no longer considered in use and can be internally erased. Exemplarily, the erase command is the UNMAP command. After receiving the erase command, the driver of the storage device will parse the erase command into corresponding operations and send them to the controller of the flash device in the storage device. The flash device deletes the invalid blocks that need to be recycled specified in the erase command from the file system and marks them as unused status to obtain the execution result. The storage device sends the execution result to the file system.

[0066] In some embodiments, in response to receiving an execution result, the file system closes the background operation of the storage device and sends a request to the storage device to obtain a logical address list. After receiving the request to obtain the logical address list, the storage device feeds back the logical addresses corresponding to the data to be recycled to the file system in the form of a logical address list. In response to receiving the logical address list sent by the storage device, the file system determines multiple target segments from the logical address list; and sends the multiple target segments (which can be referred to as source segments) to the storage device, so that the storage device moves the data blocks located in the multiple target intervals and in the logical address list. Among them, the logical address list includes the logical addresses (Logical Cluster Address, abbreviated as LCA) and physical addresses (Physical Cluster Address, abbreviated as) of the data that the storage device needs to recycle. By way of example, the storage device can use the write / xcopy method to move data. The "xcopy" of extended copy is an extension of "copy". The "xcopy" of extended copy can copy files and directory trees, that is, copy the specified directory together with the file and directory structure, but cannot copy system files.

[0067] In some embodiments, refer to Figure 5 , the implementation manner of determining multiple target segments from the logical address list is as follows: the addresses in the logical address list can be sorted in ascending order, and the sorted logical address list is divided into multiple segments, and each segment includes multiple data blocks; the first N segments with the most data blocks and the number of read / write modifications less than the number threshold are determined as target segments, where N is greater than zero and less than the maximum number of target segments set by the storage device. In the embodiments of the present application, a segment with the number of read / write modifications greater than or equal to the number threshold can be referred to as a hot segment, and a segment with the number of read / write modifications less than the number threshold can be referred to as a cold segment, and the number threshold can be predefined according to actual needs.

[0068] In other embodiments, refer to Figure 6 , after dividing the logical address list into multiple segments, the implementation manner of determining multiple target segments from the logical address list is as follows: obtain the aging time of each segment, and determine the first N segments with the longest aging time (which can also be understood as the longest storage time of the data) as target segments, where N is greater than zero and less than the maximum number of target segments set by the storage device.

[0069] By way of example, for ease of understanding, an example is provided here to illustrate steps S110 to S130. Refer to Figure 7 , the file system determines whether the garbage collection awareness function is enabled; if the garbage collection awareness function is enabled, the device status of the storage device is queried; if the garbage collection awareness function is not enabled, the file system is not linked, and the file system and the storage device use their respective original memory recovery processes for memory recovery.

[0070] Based on the device status, it is determined whether the garbage collection awareness execution condition is met; if the garbage collection awareness execution condition is met, it is determined whether the storage device has invalid data according to the number of valid blocks in the storage device and the file system; if the garbage collection awareness execution condition is not met, the file system is not linked, and the file system and the storage device use their respective original memory recovery processes for memory recovery.

[0071] Based on the number of valid blocks in the storage device and the file system, it is determined whether the storage device has invalid data; if the storage device has invalid data, a synchronization point is executed and an erase command is issued, and the background operation is closed. The role of the synchronization point is to insert a waiting time and then perform the next operation; if the storage device has no invalid data, the background operation of the storage device is closed.

[0072] After closing the background operation, the logical address list is obtained. If the logical address list is successfully obtained, N target sections are determined according to the logical address list; if the logical address list is not obtained, a synchronization point is executed, the synchronization point is submitted, and the background operation is started. It should be noted that the storage device allows the file system to obtain the logical address list multiple times during a single background operation switch. However, after the storage device returns all the logical address lists to the file system, if it still receives a request from the file system to obtain the logical address list, an error is returned.

[0073] After determining the N target sections, it is necessary to determine again whether the garbage collection awareness function is currently enabled and whether the garbage collection awareness execution condition is met. If the garbage collection awareness function is currently enabled and the garbage collection awareness execution condition is met, the data to be recycled is moved using the write / xcopy method. Otherwise, if the garbage collection awareness function is not currently enabled or the garbage collection awareness execution condition is not met, the garbage collection awareness is interrupted.

[0074] After moving the data, it is determined whether the N target sections have been moved. If the N target sections have been moved, it is determined whether the amount of data moved this time exceeds the preset threshold; if the N target sections have not been moved, the garbage collection awareness function is closed.

[0075] It is determined whether the amount of data moved this time exceeds the preset threshold. If the amount of data moved this time exceeds the preset threshold, the garbage collection awareness function is closed; if the amount of data moved this time does not exceed the preset threshold, wait for the move input / output to complete.

[0076] After the input / output transfer is completed, it is determined whether to execute a synchronization point. If a synchronization point is executed, the synchronization point is submitted, and the current awareness garbage collection is completed, and background operations are started. If the synchronization point is not executed, the current awareness garbage collection process ends.

[0077] Based on steps S110 to S130, the file system and the storage device can mutually perceive each other's states. When the execution conditions for the awareness garbage collection function are met, the file system takes the lead in the memory recovery of the storage device. This can not only enable the file system to know the data blocks that have been transferred by the storage device, avoiding the file system from transferring the data blocks that have already been transferred by the storage device again, but also synchronize the invalid blocks in the file system and the storage device, preventing the storage device from transferring the invalid data blocks in the file system during memory recovery, thereby reducing the life overhead and write amplification factor of the storage device.

[0078] See Figure 8 , Figure 8 is a flowchart of a memory recovery method provided by another embodiment of the present application. The memory recovery method may include steps S210 to S230.

[0079] Step S210: Send a query instruction to the storage device so that the storage device responds to the query instruction and returns a query result.

[0080] See Figure 9 , the driver of the storage device (such as the SCSI-UFS driver) sends a query request to the storage device (such as UFS) to query whether the storage device supports the awareness garbage collection function. When the driver of the storage device receives the query result of "the storage device supports the awareness garbage collection function", it marks that the request queue of the storage device supports the awareness garbage collection function. When the driver of the storage device receives the query result of "the storage device does not support the awareness garbage collection function", it marks that the request queue of the storage device does not support the awareness garbage collection function, or does not perform any processing.

[0081] The file system sends a query request to the driver of the storage device to query whether the request queue of the storage device supports the awareness garbage collection function. If the driver of the storage device has marked that the request queue of the storage device supports the awareness garbage collection function, the driver of the storage device sends the query result of "the request queue of the storage device supports the awareness garbage collection function" to the file system. If the driver of the storage device has marked that the request queue of the storage device does not support the awareness garbage collection function, or the driver of the storage device has no mark on whether the request queue of the storage device supports the awareness garbage collection function, the driver of the storage device sends the query result of "the request queue of the storage device does not support the awareness garbage collection function" to the file system.

[0082] See Figure 10 , the implementation method for querying whether a storage device supports the awareness garbage collection function is as follows: In response to receiving a query request sent by the driver of the storage device, the storage device detects whether the manufacturer of the storage device supports the awareness garbage collection function based on whether the manufacturer identifier of the storage device is a manufacturer identifier that supports the awareness garbage collection function. If the manufacturer identifier of the storage device is not a manufacturer identifier that supports the awareness garbage collection function, it is determined that the manufacturer of the storage device does not support the awareness garbage collection function, and then the query result of "the storage device does not support the awareness garbage collection function" is returned. If the manufacturer identifier of the storage device is a manufacturer identifier that supports the awareness garbage collection function, it is determined that the manufacturer of the storage device supports the awareness garbage collection function, and then the device description of the storage device itself is read to check whether there is a "Get Logical Address List (GetLCA List)" command in the device description. If the "Get Logical Address List" command exists in the device description, the storage device returns the query result of "the storage device supports the awareness garbage collection function" to the driver of the storage device. If the "Get Logical Address List" command does not exist in the device description, the storage device returns the query result of "the storage device does not support the awareness garbage collection function" to the driver of the storage device.

[0083] It should be noted that referring to Figure 11 , when an input / output request of the file system is sent to the driver of the storage device (such as the SCSI-UFS driver), before executing the command of the driver of the storage device (such as the SCSI command), if it is found that the current request is an awareness garbage collection request, it is necessary to set the information for receiving the return from the storage device, that is, set the direct memory access (Direct Memory Access, abbreviated as DMA) direction to the data reading direction from the device "FROM_DEVICE", and "FROM_DEVICE" means reading data from the device to a certain memory area. Then, according to the received awareness garbage collection request, the awareness garbage collection request is converted into a command of the driver of the storage device (such as the SCSI command) through configuration, and a mapping command buffer is established to store the mapping relationship between the awareness garbage collection request and the command of the driver of the storage device. If it is not an awareness garbage collection request, a failure will be returned.

[0084] Step S220: Receive the query result.

[0085] The file system receives the query result of "the request queue of the storage device supports the awareness garbage collection function" or the query result of "the request queue of the storage device does not support the awareness garbage collection function".

[0086] Step S230: If the query result indicates that the request queue of the storage device supports the awareness garbage collection function, determine that the storage device supports the awareness garbage collection function.

[0087] Referring to Figure 12 After determining that the storage device supports the awareness garbage collection function, the block layer device marks whether it supports the awareness garbage collection function and obtains the logical address list in the request queue for the block layer device query result. And a new operation type is added to the request: background operation status query and obtaining the logical address list. In the initialization stage, parameters related to the awareness garbage collection function are obtained according to the device description of the storage device (such as input / output parameters "bio parameters" and request parameters), the parameters related to the awareness garbage collection function of the storage device are initialized, and the parameter information related to the awareness garbage collection function is displayed through the interface of the memory-based file system (such as the sysfs file system). When the block layer device receives the awareness garbage collection request submitted by the upper layer, it checks whether the device supports the awareness garbage collection function and confirms whether the buffer size in the request parameters exceeds the maximum value allowed by the storage device. If the storage device does not support the awareness garbage collection function or the request parameters do not meet the requirements, a failure will be returned.

[0088] After the file system receives the feedback result, the file system determines whether it needs to be led by the file system to perform awareness garbage collection with the storage device according to its current own state and the state of the storage device. Exemplarily, the idle and non-idle states can be identified according to the system state, and the state of the file system garbage collection module is updated through the interface (for example, enabling / disabling the awareness garbage collection function). Refer to Figure 13 It can be judged whether the file system supports the awareness garbage collection function according to the mount option. After the file system is mounted, the awareness garbage collection function is enabled by parsing the mount parameters. If the mount parameters enable the awareness garbage collection function, it is determined that the file system supports the awareness garbage collection function. If the mount parameters cannot enable the awareness garbage collection function, it is determined that the file system does not support the awareness garbage collection function. After the awareness garbage collection function is enabled, it is checked whether the underlying device (such as the storage device) supports the awareness garbage collection function. If it supports the awareness garbage collection function, the awareness garbage collection function is enabled (for the specific enabling trigger conditions, refer to step S110 above). If it does not support the awareness garbage collection function, a failure is returned. Among them, file system mounting refers to connecting a file system to a specified directory of the operating system so that applications in the operating system can access the files in the file system.

[0089] Based on steps S210 to S230, the file system and the storage device can be initialized to ensure that both the file system and the storage device support the awareness garbage collection function.

[0090] Step S240: When both the file system and the storage device support the awareness garbage collection function, in response to the enabling of the awareness garbage collection function, obtain the device state of the storage device.

[0091] Step S250: Determine whether the execution conditions for the perception garbage collection function are met according to the device status.

[0092] Step S260: If the execution conditions for the perception garbage collection function are met, link the file system to perform memory recycling on the storage device.

[0093] Step S270: If the execution conditions for the perception garbage collection function are not met, do not link the file system to perform memory recycling on the storage device, so that the file system and the storage device perform memory recycling according to their original memory recycling methods respectively.

[0094] Compared with Steps S110 to S130, Steps S210 to S270 also have the following additional technical effects: Through the initialization stage, it is confirmed that both the file system and the storage device support the perception garbage collection function, thus ensuring that the perception garbage collection function can be executed.

[0095] It should be noted that in the above perception garbage collection process, the storage device triggers perception garbage collection when the following conditions are met: (1) The background operation is closed; (2) There is no other perception garbage collection currently in progress; (3) The number of used blocks triggers the perception garbage collection window: Perception garbage collection will only perform perception garbage collection when the background operation status is 0x00 and 0x01, which means on the storage device side that the storage device has not initiated perception garbage collection or is performing background garbage collection. The perception garbage collection window is configured according to the waterline of the background garbage collection, the write enhancement size configured by the host, and the value that can expand the above block calculation window.

[0096] The storage device's perception garbage collection supports reporting data item by item for multi-stream data in the set shunt order. For example, the shunt data is pin->hot->cold->undefine, where pin data is the data initiated by the host pin, hot data is the hot data frequently read and modified by the host, cold data is the cold data with fewer read and write modifications by the host, and undefine data is other data.

[0097] See Figure 14 , Figure 14 is the structural block diagram of the memory recycling device provided by an embodiment of the present application. The memory recycling device can be applied to a perception garbage collection system or a file system or an electronic device. The memory recycling device 100 includes a response module 110, a decision module 120, and a linkage module 130.

[0098] The response module 110 is used to obtain the device state of the storage device in response to the activation of the awareness garbage collection function when both the file system and the storage device support the awareness garbage collection function. In some embodiments, the response module 110 is further used to detect whether the file system and the storage device support the awareness garbage collection function, and activate the awareness garbage collection function. For the specific working process of the response module 110, please refer to step S110 and steps S210 to S240.

[0099] The decision-making module 120 is used to determine whether the execution conditions of the awareness garbage collection function are met according to the device state. For the specific working process of the decision-making module 120, please refer to step S120 and step S250.

[0100] The linkage module 130 is used to, if the execution conditions of the awareness garbage collection function are met, link the file system to perform memory recycling on the storage device; if the execution conditions of the awareness garbage collection function are not met, not link the file system to perform memory recycling on the storage device, so that the storage device performs memory recycling by itself. For the specific working process of the decision-making module 120, please refer to step S130, step S260, and step S270.

[0101] Those skilled in the art can clearly understand that the above devices provided in the embodiments of the present application can implement the methods provided in the embodiments of the present application. For the specific working processes of the above-described devices and modules, reference can be made to the corresponding processes of the methods in the embodiments of the present application, which will not be elaborated here.

[0102] In the embodiments provided by the present application, the coupling, direct coupling, or communication connection between the modules shown or discussed with each other can be indirect coupling or communication coupling through some interfaces, devices, or modules, and can be in electrical, mechanical, or other forms. The embodiments of the present application do not make specific limitations on this.

[0103] In addition, in the embodiments of the present application, each functional module can be integrated into a processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0104] See Figure 15 , Figure 15 is a structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device 200 may include a memory 210 and a processor 220. An application program is stored in the memory 210, and the application program is configured to execute the method provided in the embodiment of the present application when called by the processor 220.

[0105] The processor 220 may include one or more processing cores. The processor 220 is connected to various parts within the entire electronic device 200 through various interfaces and circuits, and is used to run or execute instructions, programs, code sets, or instruction sets stored in the memory 210, and to call and run or execute data stored in the memory 210, so as to perform various functions of the electronic device 200 and process data.

[0106] The processor 220 may be implemented in at least one of the hardware forms of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 220 may integrate one or a combination of several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the displayed content; the modem is used to process wireless communications. It can be understood that the above modem may not be integrated into the processor 220 and may be implemented separately through a communication chip.

[0107] The memory 210 may include random access memory (RAM) and may also include read-only memory (ROM). The memory 210 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 210 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for implementing at least one function, instructions for implementing the above method embodiments, etc. The data storage area may store data created during the use of the electronic device 200.

[0108] The embodiment of the present application also provides a computer-readable storage medium, on which program code is stored, and the program code is configured to execute the method provided by the embodiment of the present application when called by a processor.

[0109] A computer-readable storage medium may be an electronic memory such as a flash memory, an Electrically-Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a hard disk, or a ROM.

[0110] In some embodiments, the computer-readable storage medium includes a Non-Transitory Computer-Readable Storage Medium (Non-TCRSM). The computer-readable storage medium has a storage space for program code that executes any of the method steps in the above-described method. This program code can be read out from or written into one or more computer program products. The program code can be compressed in a suitable form.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.< / device> < / device> < / device> < / device> < / device> < / device> < / device> < / device> < / device>

Claims

1. A memory recycling method, characterized in that, include: In a case where both the file system and the storage device support the aware garbage collection function, in response to the aware garbage collection function being enabled, obtaining a device status of the storage device; Determining whether an execution condition of a garbage collection function is satisfied according to the device state; If the execution condition of the garbage collection function is met, the file system is linked to reclaim memory of the storage device.

2. The method according to claim 1, characterized in that, The linking the file system to reclaim memory from the storage device includes: An erase command is sent to the storage device, the erase command including the invalid blocks to be reclaimed, so that the storage device deletes the invalid blocks to be reclaimed according to the erase command and feeds back the execution result.

3. The method according to claim 2, wherein The linking the file system to reclaim memory from the storage device also includes: In response to receiving the execution result, shutting down the background operation of the storage device, and sending a request for obtaining a logical address list to the storage device; In response to receiving the logical address list sent by the storage device, determining a plurality of target segments from the logical address list; Multiple target sections are sent to the storage device so that the storage device moves data blocks in the multiple target sections and in the logical address list.

4. The method according to claim 3, wherein The step of determining a plurality of target segments from the logical address list comprises: Dividing the logical address list into a plurality of sections, each section including a plurality of data blocks; Determine the first N segments that contain the most data blocks and whose read and write modification times are less than a threshold as target segments, where N is greater than zero and less than the maximum number of target segments set by the storage device; or The first N segments with the longest aging time are determined as target segments.

5. The method according to any one of claims 1-4, characterized in that, The determining, according to the device state, whether an execution condition of a garbage collection function is satisfied includes: If the device status indicates that there is a need to link with the file system, obtaining the number of valid blocks in the storage device and the number of valid blocks in the file system; Determine the number of invalid blocks in the storage device according to the number of valid blocks in the storage device and the number of valid blocks in the file system; If the number of invalid blocks in the storage device is greater than the number threshold, it is determined that the execution condition of the aware garbage collection function is met.

6. The method according to any one of claims 1 to 4, characterized in that Before acquiring the device status of the storage device in response to sensing that the garbage collection function is enabled, the method further includes: In response to the file system satisfying a specified condition, enabling a perceptual garbage collection function; or Determine the execution frequency of the perceptual garbage collection function based on the overall input and output load and the proportion of the input and output foreground, and enable the perceptual garbage collection function according to the execution frequency.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Sending a query instruction to the storage device, so that the storage device feeds back a query result in response to the query instruction; receiving the query result; If the query result indicates that the request queue of the storage device supports the aware garbage collection function, it is determined that the storage device supports the aware garbage collection function.

8. A memory recycling device, characterized in that, include: A response module, for obtaining a device status of the storage device in response to the activation of the awareness garbage collection function when both the file system and the storage device support the awareness garbage collection function; A decision module, used to determine whether an execution condition of a garbage collection function is satisfied according to the device state; The linkage module is used to, if the execution conditions for the perceived garbage collection function are met, link the file system to perform memory recovery on the storage device.

9. An electronic device, characterized in that, It includes: A memory and a processor, wherein an application program is stored in the memory, and the application program is used to execute the method according to any one of claims 1-7 when called by the processor.

10. A computer-readable storage medium, characterized in that, Program code is stored on the computer-readable storage medium, and the program code is used to execute the method according to any one of claims 1-7 when called by the processor.

Images