Data processing method and device in memory, storage medium and electronic device

Abstract

The embodiment of the present application provides a data processing method and device in a memory, a storage medium and an electronic device. The method includes: obtaining a set writable space corresponding to the current storage set in the target memory; when the set writable space is less than or equal to the target threshold, obtaining a first storage state parameter and a second storage state parameter of each of the multiple storage units, wherein the first storage state parameter is used to indicate the number of historical erase and write operations of the storage unit, and the second storage state parameter is used to indicate the amount of valid data stored in the storage unit; determining the operation order corresponding to each of the multiple storage units according to the first storage state parameter and the second storage state parameter, and based on the operation order, sequentially performing a cache operation on the valid data stored in each of the multiple storage units; performing a data erase operation on the storage unit that has performed a data cache operation.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and more specifically, to a method and device for processing data in a memory, a storage medium, and an electronic device. Background Art

[0002] As the amount of data grows, the capacity and performance of data storage systems are also growing rapidly. In particular, enterprises currently need large-scale data centers to support the digitalization of their businesses and enterprise data storage. The huge demand for data storage has put forward higher requirements for the performance and reliability of storage devices. SSDs are widely used in the construction of large data centers because they have huge advantages in performance, reliability, and power consumption.

[0003] However, in the existing SSD technology, when writing data to the SSD, a large amount of additional data will be written in addition to the data to be written. This process is called write amplification. The generation of write amplification will seriously reduce the read and write performance and service life of the SSD. In other words, in the related art, when writing data to the hard disk, a large amount of write amplification will occur.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a data processing method and device in a memory, a storage medium, and an electronic device, so as to at least solve the technical problem of excessive write amplification when writing to a hard disk in the related art.

[0006] According to one embodiment of the present application, a method for processing data in a memory is provided, comprising: obtaining a set writable space corresponding to a current storage set in a target memory, wherein the current storage set includes a plurality of storage units, the set writable space is the sum of the writable spaces of the plurality of storage units, the writable space is the data storage space not occupied by data objects, and the data objects include valid data objects and invalid data objects; when the set writable space is less than or equal to a target threshold, obtaining a first storage state parameter and a second storage state parameter of each of the plurality of storage units, wherein the first storage state parameter is used to indicate the number of historical erase and write operations of the storage unit, and the second storage state parameter is used to indicate the amount of valid data stored in the storage unit; determining an operation order corresponding to each of the plurality of storage units according to the first storage state parameter and the second storage state parameter, and based on the operation order, sequentially performing a cache operation on the valid data stored in each of the plurality of storage units; performing a data erase operation on the storage unit on which the data cache operation has been performed, wherein the data erase operation is used to configure the data storage space in the storage unit as a writable space.

[0007] According to another aspect of an embodiment of the present application, a data processing device of a memory is also provided, including: a first acquisition unit, used to acquire a set writable space corresponding to a current storage set in a target memory, wherein the current storage set includes multiple storage units, the set writable space is the sum of the writable spaces of the multiple storage units, the writable space is the data storage space not occupied by data objects, and the data objects include valid data objects and invalid data objects; a second acquisition unit, used to acquire a first storage state parameter and a second storage state parameter of each of the multiple storage units when the set writable space is less than or equal to a target threshold, wherein the first storage state parameter is used to indicate the number of historical erase and write operations of the storage unit, and the second storage state parameter is used to indicate the amount of valid data stored in the storage unit; a determination unit, used to determine an operation order corresponding to each of the multiple storage units according to the first storage state parameter and the second storage state parameter, and based on the operation order, sequentially perform a cache operation on the valid data stored in each of the multiple storage units; an erasing unit, used to perform a data erasing operation on the storage unit that has performed a data cache operation, wherein the data erasing operation is used to configure the data storage space in the storage unit as a writable space.

[0008] Optionally, the above-mentioned determination unit includes: a first acquisition module, used to obtain a first weight coefficient matching the first storage state parameter, and a second weight coefficient matching the second storage state parameter; a second acquisition module, used to sequentially acquire the first storage state parameter and the second storage state parameter of each of the multiple storage units, and determine the priority indexes that match each of the multiple storage units according to the weighted summation result between the first weight coefficient, the second weight coefficient, the first storage state parameter and the second storage state parameter; a determination module, used to determine the operation order corresponding to each of the multiple storage units based on the priority indexes that match each of the multiple storage units.

[0009] Optionally, the above-mentioned second acquisition module is also used to: obtain a third weight coefficient, and obtain the third storage state parameter of each of the multiple storage units in turn; based on the third weight coefficient, the third storage state parameter of each of the multiple storage units and the weighted summation result, determine the priority index matching each of the multiple storage units respectively, wherein the third storage state parameter is used to indicate the data amount of invalid data stored in the storage unit, and the third weight coefficient is a weight coefficient matching the third storage state parameter; obtain a fourth weight coefficient, and obtain the fourth storage state parameter of each of the multiple storage units in turn; based on the fourth weight coefficient, the fourth storage state parameter of each of the multiple storage units and the weighted summation result, determine the priority index matching each of the multiple storage units respectively, wherein the fourth storage state parameter is used to indicate the ratio between the first data amount of valid data and the second data amount of invalid data stored in the storage unit, and the fourth weight coefficient is a weight coefficient matching the fourth storage state parameter.

[0010] Optionally, the first acquisition module is further used to: determine an erase balance index matching the current storage set according to respective first state parameters of a plurality of storage units in the current storage set; and acquire a first weight coefficient and a second weight coefficient matching the erase balance index.

[0011] Optionally, the above-mentioned first acquisition module is also used to: obtain a first weight coefficient and a second weight coefficient matching the current index interval according to the current index interval corresponding to the erasure balance index, wherein the preset multiple index intervals correspond to a group of first weight coefficients and second weight coefficients respectively, and the multiple index intervals include the current index interval; determine the product of the erasure correlation coefficient and the erasure balance index as the first weight coefficient, wherein the erasure correlation coefficient is used to indicate the positive correlation between the first weight coefficient and the erasure balance index; obtain the preset second weight coefficient.

[0012] Optionally, the above-mentioned erasing unit is also used to: when the set writable space is greater than or equal to a reference threshold, suspend the sequential execution of caching operations on valid data stored in each of the multiple storage units; when the set writable space is greater than or equal to a reference threshold, suspend the execution of data erasing operations on the storage units that have executed data caching operations; wherein the reference threshold is greater than or equal to the target threshold.

[0013] Optionally, the above-mentioned determination unit is also used to: determine the current storage unit to be processed from multiple storage units based on the operation order; when a target storage unit matching the current storage unit is determined from multiple storage units, cache the valid data stored in the current storage unit to the data cache area.

[0014] Optionally, the above-mentioned determination unit is also used to: write valid data in the data cache area into the target storage unit.

[0015] Optionally, the above-mentioned determination unit is also used to: obtain a data bitmap matching the current storage unit, wherein the data bitmap is used to determine the storage position corresponding to the valid data stored in the current storage unit; obtain a position sorting result of at least one storage position determined according to the data bitmap; and according to the position sorting result, cache the valid data stored in at least one storage position in the current storage unit in sequence to the data cache area.

[0016] Optionally, the above-mentioned determination unit is also used to: obtain the handle type of the current operation handle used to operate the current storage unit, wherein the handle type includes an initial independent type and a continuous independent type, the initial independent type is used to indicate that the current operation handle is allowed to operate on the reference storage unit, and the continuous independent type is used to indicate that the current operation handle is not allowed to operate on the reference storage unit, the reference storage unit is a storage unit operated according to the reference operation handle, and the reference operation handle is an operation handle different from the current operation handle; according to the handle type of the current operation handle, determine the target storage unit from multiple storage units included in the first storage subset and the second storage subset, wherein the current storage set includes the first storage subset and the second storage subset, the first storage unit in the first storage subset does not store data objects, the second storage unit in the second storage subset has stored data objects, and the writable space corresponding to the second storage unit is greater than 0.

[0017] Optionally, the above-mentioned determination unit is also used to: when the handle type of the current operation handle is the initial independent type and the second storage subset includes at least one second storage unit corresponding to the initial independent type, determine the second storage unit with the lowest first storage state parameter as the target storage unit from the at least one second storage unit; when the handle type of the current operation handle is the initial independent type and the second storage subset does not include the second storage unit corresponding to the initial independent type, determine the first storage unit with the lowest first storage state parameter as the target storage unit from the at least one first storage unit included in the first storage subset; when the handle type of the current operation handle is the persistent independent type and the second storage subset includes at least one second storage unit associated with the current operation handle, determine the second storage unit with the highest second storage state parameter as the target storage unit from the at least one second storage unit; when the handle type of the current operation handle is the persistent independent type and the second storage subset does not include the second storage unit associated with the current operation handle, determine the first storage unit with the lowest first storage state parameter as the target storage unit from the at least one first storage unit included in the first storage subset.

[0018] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the data processing method in the above-mentioned memory when it is run.

[0019] According to another aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data processing method in the above memory.

[0020] According to another aspect of an embodiment of the present application, there is further provided an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the data processing method in the memory through the computer program.

[0021] Through the present application, the writable space of the memory can be obtained first. When the writable space is less than or equal to a preset threshold, it is necessary to perform garbage collection on the storage units in the memory to clear invalid data and increase the writable space of the memory. At this time, the effective data volume and historical erase and write times of the storage units are obtained, and the storage units are sorted according to the effective data volume and the historical erase and write times to ensure that the storage units with small effective data volume and historical erase and write times can be garbage collected first, and starting from the storage units at the front of the sorting, the operation of moving the effective data to the cache is performed, and then the storage units that have performed the data caching operation are erased to release the storage space so that the value of the writable space is higher than or equal to the preset threshold. Since in this embodiment, data transfer is preferentially performed on the storage units with small effective data volume and small historical erase and write times in the memory, and then, while taking into account the wear leveling of the memory, less effective data is rewritten from the cache area to the memory, which means smaller write amplification and less memory wear, thereby solving the technical problems of excessive write amplification and excessive wear of storage devices in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 It is a hardware structure block diagram of a server device of a method for processing data in a memory according to an embodiment of the present application;

[0024] Figure 2is a flow chart of a data processing method in a memory according to an embodiment of the present application;

[0025] Figure 3 is a schematic diagram of a data processing method in a memory according to an embodiment of the present application;

[0026] Figure 4 is a schematic diagram of another data processing method in a memory according to an embodiment of the present application;

[0027] Figure 5 is a schematic diagram of another data processing method in a memory according to an embodiment of the present application;

[0028] Figure 6 is a schematic diagram of another data processing method in a memory according to an embodiment of the present application;

[0029] Figure 7 is a schematic diagram of another data processing method in a memory according to an embodiment of the present application;

[0030] Figure 8 is a schematic diagram of another data processing method in a memory according to an embodiment of the present application;

[0031] Fig. 9 is a flow chart of another method for processing data in a memory according to an embodiment of the present application;

[0032] Fig.10 is a structural schematic diagram of a data processing device in a memory according to an embodiment of the present application;

[0033] Fig.11 It is a structural schematic diagram of a data processing electronic device in a memory according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0036] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a server device of a data processing method in a memory in an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1Only one is shown) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0037] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the data processing method of the memory in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0038] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0039] As an optional implementation, Figure 2 As shown, the data processing method in the above memory includes:

[0040] S202, obtaining a set writable space corresponding to a current storage set in a target storage, wherein the current storage set includes a plurality of storage units, the set writable space is the sum of the writable spaces of the plurality of storage units, the writable space is the data storage space not occupied by data objects, and the data objects include valid data objects and invalid data objects;

[0041] S204, when the set writable space is less than or equal to the target threshold, obtaining a first storage state parameter and a second storage state parameter of each of the plurality of storage units, wherein the first storage state parameter is used to indicate the number of historical erase and write operations of the storage unit, and the second storage state parameter is used to indicate the amount of valid data stored in the storage unit;

[0042] S206, determining the operation order corresponding to each of the plurality of storage units according to the first storage state parameter and the second storage state parameter, and performing cache operations on the valid data stored in each of the plurality of storage units in sequence based on the operation order;

[0043] S208, performing a data erasing operation on the storage unit that has performed the data caching operation, wherein the data erasing operation is used to configure the data storage space in the storage unit as a writable space.

[0044] In an optional embodiment, it should be noted that in the above step S202, the target memory can be a solid-state hard disk with any structure, and the storage unit is a different data storage area for storing data in the memory. In step S206, the above first storage state parameter can be the number of historical erase and write operations of a storage unit in the memory, and the second storage parameter is the amount of valid data of a storage unit in the memory.

[0045] The following Figure 3 The data processing method in the above memory is described by taking the solid state hard disk 304 with FDP (Flexible Data Placement) structure as an example. Figure 3 As shown, when the host 301 needs to access the solid state drive 304, the host 301 accesses the solid state drive 304 through the host interface 306 and the specified address 302. After receiving the instruction sent by the host 301 to access the specified address 302, the host interface 306 sends the instruction to the flash conversion layer 308. The flash conversion layer 308 then determines which function needs to be executed based on the content of the instruction. When the instruction sent by the host 301 requires the solid state drive 304 to obtain its own writable space, the flash conversion layer 308 accesses the flash memory 312 in the solid state drive through the flash memory interface 310, and determines the space in the flash memory 312 that is not occupied by data as the writable space. The above data includes valid data and invalid data. The specific method for determining the writable space can be: obtain the space of all storage units in the flash memory 312 that is not occupied by data, and determine the space of each storage unit that is not occupied by data as the writable space of the target memory.

[0046] In use Figure 3When the solid state drive 304 of the FDP structure shown in the figure performs data storage, the data is stored in different storage units according to different workloads, such as Figure 4 As shown in FIG. 1 , in a hard disk without an FDP structure, data with different workloads and invalid data are mixed in different storage units, while in a hard disk with an FDP structure, each storage unit stores data with different workloads and invalid data in different data storage units in addition to storing the check bit XOR. Figure 4 As shown, invalid data is centrally stored in storage unit 401, and workload B and workload C are centrally stored in other storage units respectively. When the host needs to write data with workload A to the memory, in the hard disk using the FDP structure, garbage collection is performed on storage unit 401, and the invalid data in storage unit 401 is cleared. The storage unit 401 after the invalid data is cleared is used as storage unit 402 that can be used to store data, and then the data of workload A is stored in storage unit 402.

[0047] It should be noted that the type of the above workload can be the magnitude of the number of times the data is accessed. For example, the data is divided into three access levels, and the three access levels correspond to workloads A, B, and C respectively, where workload A represents data that is frequently accessed, workload C represents data that is occasionally accessed, and workload B represents data with an access level between A and C.

[0048] Optionally, storing data of different loads in different storage units can be as follows: Figure 5 As shown, Figure 5 As shown, there is a handle module in the hard disk, which is responsible for writing data with different workloads into one or more storage units in a centralized manner.

[0049] For example, handle 0 is responsible for writing the data set with workload A into multiple storage units in storage sets 0-3, such as Figure 8 As shown, after receiving the write command for data with workload A sent by the host, the hard disk determines the storage unit to be written through handle 0. Each handle is associated with a storage unit in each storage set, such as Figure 8 The handle 0 in is associated with storage sets 0-2, and the data with workload A is written to different storage sets through handle 0.

[0050] It should be noted that the storage unit in the above step S202 refers to Figure 6 The storage area of ​​the flash memory shown is composed of blocks with the same physical location on every 32 wafers.

[0051] In the above step S202, each storage unit in the storage set has four states: idle, open, closed, and ready to be erased, and is located at Figure 7 In the free pool, open pool, closed pool and to-be-erased pool shown, it should be noted that the free pool, open pool, closed pool and to-be-erased pool here are not physical entities with fixed storage addresses, but can be a collection for recording storage units. For example, the free pool can be a collection of addresses, which records the addresses of all storage units in an idle state. The above-mentioned idle state refers to a state in which no data is written. Similarly, the address of a storage unit that is in a state of starting to write data but not fully written is recorded in the open pool, the address of a storage unit in a state of fully written data is recorded in the closed pool, and the address of a storage unit waiting to be erased is recorded in the to-be-erased pool.

[0052] Optionally, in the above step S202, the number of storage units in the free pool and the unwritten data storage space in the open pool in the storage set may be obtained, and the size of the writable space of the storage set may be further determined.

[0053] It should be noted that in the memory, the number of times each memory cell can be erased and written is limited. If some memory cells are erased frequently, they may reach the life limit faster than other cells and their performance will also decline, such as slower write speed or increased read error rate. By giving priority to erasing cells with fewer historical erase times, the erase operation can be distributed more evenly, thereby extending the overall life of the memory and maintaining the performance of the memory.

[0054] Then, step S204 can be executed to compare the writable space obtained in step S202 with the preset target threshold. When the writable space is smaller than the target threshold, the garbage collection mechanism of the hard disk data is started, all storage units in the flash memory that need garbage collection are determined, and the historical number of erase and write operations and the amount of valid data of each storage unit that needs garbage collection are obtained.

[0055] The above-mentioned garbage collection operation is generally to move the valid data of the current storage unit to other storage units, and to erase the data in the current storage unit. It should be noted that in the memory, the number of erase and write times of each storage unit is limited. If some storage units are frequently erased, they may reach the life limit faster than other units, and their performance will also decline, such as slower write speed or increased read error rate. By giving priority to erasing units with fewer historical erase times, the erase operation can be distributed more evenly, thereby extending the overall life of the memory and maintaining the performance of the memory. Therefore, further, in order to reduce write amplification while taking into account the wear leveling of the memory, it is necessary to determine the storage unit to perform garbage collection first based on the amount of valid data and the historical erase and write times of the storage unit.

[0056] Then, step S206 is performed to determine the order in which the storage units perform garbage collection based on the historical erase and write operation counts and valid data amounts of the multiple storage units that need to perform garbage collection obtained in step S204. For example, the order can be determined according to the following mathematical model:

[0057]

[0058] The above V is the ranking value. The smaller the ranking value, the higher the ranking result. A is the first weight coefficient, and B is the second weight coefficient. is the historical erase operation number, It is the amount of valid data. It should be noted that the above is only an optional mathematical model for determining the sorting value, and the present application does not limit it here. Afterwards, starting from the storage unit determined to be at the front in the order, the operation of writing valid data to the cache is executed in sequence.

[0059] Optionally, the above method for determining the order in which storage units perform garbage collection may also be: first determine a first sorting result based on the historical number of erase and write operations, and then sort according to the amount of valid data. Specifically: first determine the number of historical erase and write operations of the storage unit, and then sort the storage units from small to large according to the size of the historical number of erase and write operations. When there are two or more storage units with the same number of historical erase and write operations, these storage units with the same historical number of erase and write operations are sorted according to the size of the valid data. The less valid data, the higher the ranking, and the earlier garbage collection is performed.

[0060] Optionally, the above method for determining the order in which storage units perform garbage collection may also be: first determine a first sorting result based on the historical number of erase and write operations, and then sort according to the magnitude of the effective data volume and the effective data volume. Specifically: first determine the historical number of erase and write operations of the storage unit, and then sort the storage units from small to large according to the magnitude of the historical number of erase and write operations. When there are two or more storage units with the same magnitude of the historical number of erase and write operations, these storage units with the same magnitude of the historical number of erase and write operations are sorted according to the specific number of historical erase and write operations. The fewer the historical erase and write operations, the higher the ranking. When there are two or more storage units with the same number of historical erase and write operations, these storage units with the same number of historical erase and write operations are sorted from small to large according to the magnitude of the effective data volume. When there are two or more storage units with the same magnitude of the effective data volume, the storage units with the same magnitude of the effective number are sorted from small to large according to the specific effective data volume. The higher the ranking, the earlier the garbage collection is performed.

[0061] Optionally, the above method for determining the order in which storage units perform garbage collection may also be: first determine a first sorting result based on the amount of valid data, and then sort according to the number of historical erase and write operations. Specifically: first determine the amount of valid data of the storage unit, and then sort the storage units from small to large according to the amount of valid data. When there are two or more storage units with the same amount of valid data, these storage units with the same amount of valid data are sorted according to the number of historical erase and write operations. The fewer the number of historical erase and write operations, the higher the ranking and the earlier garbage collection is performed.

[0062] Optionally, the above method for determining the order in which storage units execute garbage collection may also be: first determine a first sorting result based on the amount of valid data, and then sort according to the order of magnitude of historical erase and write times and the number of historical erase and write times. Specifically: first determine the order of the amount of valid data of the storage unit, and then sort the storage units from small to large according to the order of magnitude of the amount of valid data. When there are two or more storage units with the same order of valid data, these storage units with the same order of valid data are sorted according to the specific amount of valid data. The less valid data, the higher the order. When there are two or more storage units with the same order of valid data, these storage units with the same order of valid data are sorted from small to large according to the order of historical erase and write operation numbers. When there are two or more storage units with the same order of magnitude of historical erase and write operation numbers, the storage units with the same order of historical erase and write operation are sorted from small to large according to the specific number of historical erase and write operation numbers. The higher the order, the earlier the garbage collection is performed.

[0063] Then, step S208 may be executed to erase the data in the storage unit that has completed the data caching operation, so that the data in the storage unit is cleared.

[0064] Through the present application, the writable space of the memory can be obtained first. When the writable space is less than or equal to a preset threshold, it is necessary to perform garbage collection on the storage units in the memory to clear invalid data and increase the writable space of the memory. At this time, the effective data volume and historical erase and write times of the storage units are obtained, and the storage units are sorted according to the effective data volume and the historical erase and write times to ensure that the storage units with small effective data volume and historical erase and write times can be garbage collected first, and starting from the storage units at the front of the sorting, the operation of moving the effective data to the cache is performed, and then the storage units that have performed the data caching operation are erased to release the storage space so that the value of the writable space is higher than or equal to the preset threshold. Since in this embodiment, data transfer is preferentially performed on the storage units with small effective data volume and small historical erase and write times in the memory, and then, while taking into account the wear leveling of the memory, less effective data is rewritten from the cache area to the memory, which means smaller write amplification and less memory wear, thereby achieving the goal of extending the overall life of the memory while solving the technical problem of excessive write amplification in the related art.

[0065] In an optional implementation, determining the operation order corresponding to each of the plurality of storage units according to the first storage state parameter and the second storage state parameter includes:

[0066] S1, obtaining a first weight coefficient matching a first storage state parameter and a second weight coefficient matching a second storage state parameter;

[0067] S2, sequentially obtaining a first storage state parameter and a second storage state parameter of each of the plurality of storage units, and determining a priority index of each of the plurality of storage units according to a weighted summation result between the first weight coefficient, the second weight coefficient, and the first storage state parameter and the second storage state parameter;

[0068] S3, determining the operation order corresponding to each of the plurality of storage units based on the priority indexes matched by each of the plurality of storage units.

[0069] Optionally, in this implementation, first, the first weight coefficient and the second weight coefficient are obtained in S1, and then S2 may be executed to perform weighted summation on the first weight coefficient and the second weight coefficient as well as the number of historical erase operations and the amount of valid data obtained in S2 to determine the priority index. The weighted summation operation may be the following mathematical model:

[0070]

[0071] The above Q is the priority index, A is the first weight coefficient, B is the second weight coefficient, is the historical erase operation number, The effective data volume.

[0072] Then, S3 may be executed to determine the operation order of the storage unit by the priority index. Specifically, the priority indexes are sorted from small to large, and the sorting order of the priority indexes is determined as the execution order of the operations.

[0073] Through the above-mentioned implementation mode of the present application, a weighted sum is taken based on the obtained first weight coefficient, the second weight coefficient, the number of historical erase and write operations and the amount of valid data, thereby ensuring that storage units with small amounts of valid data and small numbers of historical erase and write operations can be sorted at the front, thereby ensuring that garbage collection can be performed first. Since in this implementation mode, data transfer is performed preferentially on storage units with small amounts of valid data and small numbers of historical erase and write operations in the memory, less valid data is rewritten from the cache area to the memory while taking into account the wear balance of the memory, which means smaller write amplification and less memory wear, thereby achieving the goal of extending the overall life of the memory while solving the technical problem of excessive write amplification in related technologies.

[0074] In an optional implementation, the priority indexes of the respective matches of the plurality of storage units are determined according to the weighted summation result between the first weight coefficient, the second weight coefficient, the first storage state parameter, and the second storage state parameter, including at least one of the following:

[0075] Method 1: obtaining a third weight coefficient, and sequentially obtaining a third storage state parameter of each of the plurality of storage units; determining a priority index matched by each of the plurality of storage units based on the third weight coefficient, the third storage state parameter of each of the plurality of storage units, and a weighted summation result, wherein the third storage state parameter is used to indicate the amount of invalid data stored in the storage unit, and the third weight coefficient is a weight coefficient matched with the third storage state parameter;

[0076] Method 2, obtaining a fourth weight coefficient, and obtaining the fourth storage state parameter of each of the multiple storage units in turn; based on the fourth weight coefficient, the fourth storage state parameter of each of the multiple storage units and the weighted summation result, determining the priority index matching each of the multiple storage units respectively, wherein the fourth storage state parameter is used to indicate the ratio between the first data amount of valid data and the second data amount of invalid data stored in the storage unit, and the fourth weight coefficient is a weight coefficient matching the fourth storage state parameter.

[0077] Optionally, in the above method 1, the amount of invalid data is obtained. The amount of invalid data can be obtained by subtracting the amount of valid data from the amount of storage space, or by direct query.

[0078] After obtaining the amount of invalid data and the corresponding weight coefficient, a weighted sum operation is further performed on the amount of invalid data and the corresponding weight coefficient, the amount of valid data and the corresponding weight coefficient, the number of historical erase operations and the corresponding weight coefficient, for example, the following weighted sum model operation is performed:

[0079]

[0080] The above Q is the priority index, A is the first weight coefficient, B is the second weight coefficient, and C is the third weight coefficient. is the historical erase operation number, is the effective data volume, Invalid data amount.

[0081] Optionally, in the second method, a ratio is calculated between the amount of valid data and the amount of invalid data obtained, and the obtained ratio is determined as a fourth weight parameter. After obtaining the ratio of the amount of valid data to the amount of invalid data and the corresponding fourth weight coefficient, a weighted sum operation is further performed on the ratio of the amount of valid data to the amount of invalid data and the corresponding weight coefficient, the amount of valid data and the corresponding weight coefficient, the number of historical erase and write operations and the corresponding weight coefficient, for example, the following weighted sum model operation is performed:

[0082]

[0083] The above Q is the priority index, A is the first weight coefficient, B is the second weight coefficient, and D is the fourth weight coefficient. is the historical erase operation number, is the effective data volume, Invalid data amount.

[0084] Through the above-mentioned implementation mode of the present application, a weighted sum is performed based on the obtained first weight coefficient, second weight coefficient, third weight coefficient, fourth weight coefficient, number of historical erase and write operations, amount of valid data, amount of invalid data, amount of valid data and amount of invalid data, to ensure that storage units with small amount of valid data and small number of historical erase and write operations can be sorted at the front, thereby ensuring that garbage collection can be performed first. Since in this implementation mode, data transfer is performed preferentially on storage units with small amount of valid data and small number of historical erase and write operations in the memory, less valid data is rewritten from the cache area to the memory while taking into account wear balancing of the memory, which means smaller write amplification and less wear on the memory, thereby achieving the goal of extending the overall life of the memory while solving the technical problem of excessive write amplification in related technologies.

[0085] In an optional implementation, obtaining a first weight coefficient matching the first storage state parameter and a second weight coefficient matching the second storage state parameter includes:

[0086] S1, determining an erase balance index matching the current storage set according to first state parameters of each of a plurality of storage units in the current storage set;

[0087] S2, obtaining a first weight coefficient and a second weight coefficient that match the erase balance index.

[0088] Optionally, in the above step S1, the erasure balance index may be the variance of the first state parameter. In the case where the erasure balance index is the variance of the first state parameter, the calculation of the erasure balance index is described by the following calculation model:

[0089] ;

[0090] First, by obtaining the first state parameter The average M of the first state parameter is calculated, and then the variance of the first state parameter is calculated using a variance calculation formula, and the calculated variance is determined as the erasure balance index Z.

[0091] Then execute step S2, and determine the first weight coefficient and the second weight coefficient by using the acquired erase balance index. The determination method can be, for example, when the erase balance index is 0.1, the first weight coefficient is determined to be 0.1, and the second weight coefficient is determined to be 0.2; when the erase balance index is 0.2, the first weight coefficient is determined to be 0.3, and the second weight coefficient is determined to be 0.1. It should be noted that the above-mentioned determination relationship is only for illustration and does not constitute a limitation of the present application.

[0092] Through the above-mentioned implementation mode of the application itself, the first weight coefficient and the second weight coefficient are determined by calculating the variance of the first state parameter, thereby ensuring that the weighted sum of the various parameters of the storage unit can be performed, and further determining the operation order of the storage unit, ensuring that the storage units with a small amount of valid data and the storage units with a small number of historical erase and write times are sorted first and operated first, thereby ensuring that the amount of valid data that needs to be moved during garbage collection is small, and the memory wear is smaller, thereby achieving the goal of extending the overall life of the memory while solving the technical problem of excessive write amplification in related technologies.

[0093] In an optional implementation manner, obtaining a first weight coefficient and a second weight coefficient that match the erasure balance index includes one of the following:

[0094] Method 1: according to the current index interval corresponding to the erase leveling index, obtaining a first weight coefficient and a second weight coefficient matching the current index interval, wherein the preset multiple index intervals correspond to a group of first weight coefficients and second weight coefficients respectively, and the multiple index intervals include the current index interval;

[0095] Method 2: Determine the product of the erasure correlation coefficient and the erasure balance index as the first weight coefficient, wherein the erasure correlation coefficient is used to indicate the positive correlation between the first weight coefficient and the erasure balance index; and obtain a preset second weight coefficient.

[0096] The implementation manner of the present application provides two optional calculation methods of the first weight coefficient and the second weight coefficient. In method one, a numerical interval is preset for the erase balance index, and each numerical interval has a corresponding first weight coefficient and a second weight coefficient. After calculating the erase balance index, the index interval corresponding to the erase balance index is determined, and further based on the determined index interval, the first weight coefficient and the second weight coefficient corresponding to the interval are determined.

[0097] In the above-mentioned method 2, the first weight coefficient is determined by the product of the erasure balance index and the erasure correlation coefficient, and the calculation formula is as follows:

[0098]

[0099] The above-mentioned A is the first weight coefficient, Z is the erasure balance index, and E is the erasure correlation coefficient. The erasure correlation coefficient can be a preset constant or a dependent variable determined by the erasure balance index. For example, when the erasure balance index is 0.2, round it up to get a value of 1 and determine the value as the erasure correlation coefficient. For example, when the erasure balance index is 0.6, round it up to get a value of 1 and determine the value as the erasure correlation coefficient. It should be noted that the above examples are only for illustrating the acquisition of the erasure correlation coefficient and do not constitute a limitation to the present application.

[0100] Furthermore, in the second method, the first weight coefficient is determined by multiplying the erase balance index and the erase correlation coefficient, and the preset second weight coefficient is obtained.

[0101] Through the above-mentioned implementation method of the application itself, the first weight coefficient is determined by calculating the product of the balance index and the erase correlation coefficient, and the preset second weight coefficient is obtained at the same time, so as to ensure that the weighted sum of various parameters of the storage unit can be performed, and the operation order of the storage unit is further determined, so as to ensure that the storage units with small effective data volume and storage units with small historical erase times are sorted first and operated first, so as to ensure that the amount of effective data that needs to be moved during garbage collection is small and the wear and tear on the memory is smaller.

[0102] In an optional implementation, when performing a data erasing operation on a storage unit that has performed a data caching operation, at least one of the following is further included:

[0103] Method 1: When the set writable space is greater than or equal to the reference threshold, suspending the sequential execution of cache operations on valid data stored in each of the multiple storage units;

[0104] Method 2: When the set writable space is greater than or equal to the reference threshold, suspending the data erasing operation on the storage unit that has performed the data caching operation;

[0105] The reference threshold is greater than or equal to the target threshold.

[0106] In the above two methods of this embodiment, the above reference threshold is a value greater than or equal to the target threshold used to trigger the activation of the garbage collection mechanism.

[0107] Optionally, in method one, a reference threshold that is greater than or equal to a target threshold for triggering the activation of a garbage collection mechanism is first preset. When the writable space in the storage area is lower than the target threshold, the memory activates the garbage collection mechanism, moves valid data to the cache, and erases data from the storage unit that performs the cache operation. If the garbage collection process reaches a point where the writable space in the storage area in the memory is greater than or equal to the reference threshold, the operation of moving the valid data to the cache is suspended. The time node for judging whether the writable space is greater than or equal to the reference threshold may be a time point with a preset fixed time interval. For example, after the operation of caching valid data is started, it is judged once every 10 nanoseconds whether the writable space is greater than or equal to the reference threshold. If it is judged that the writable space is greater than or equal to the reference threshold, the operation of caching the valid data in the storage unit is suspended. For another example, after each caching of valid data for a storage unit is completed, it is judged once whether the writable space is greater than or equal to the reference threshold. If it is judged that the writable space is greater than or equal to the reference threshold, the operation of caching the valid data in the storage unit is suspended.

[0108] Optionally, in method 2, a reference threshold that is greater than or equal to a target threshold for triggering the activation of a garbage collection mechanism is first preset. When the writable space in the storage area is lower than the target threshold, the memory activates the garbage collection mechanism, moves valid data to the cache, and erases data from the storage unit that performs the cache operation. If the garbage collection process reaches a point where the writable space in the storage area in the memory is greater than or equal to the reference threshold, the erasing operation on the storage unit is suspended. The time node for judging whether the writable space is greater than or equal to the reference threshold may be a time point with a preset fixed time interval. For example, after the operation of caching valid data is started, it is judged once every 10 nanoseconds whether the writable space is greater than or equal to the reference threshold. If it is judged that the writable space is greater than or equal to the reference threshold, the erasing operation on the storage unit is suspended. For another example, after each data erasure of a storage unit is completed, it is judged once whether the writable space is greater than or equal to the reference threshold. If it is judged that the writable space is greater than or equal to the reference threshold, the data erasing operation in the storage unit is suspended.

[0109] In the above implementation of the present application, the reference threshold is set to achieve Figure 7 Dynamic erasure of storage units in the to-be-erased pool, specifically: when erasing storage units in the to-be-erased pool, when the writable space of the storage set is greater than or equal to a reference threshold, data caching and data erasure are suspended, thereby avoiding erasing all storage units in the to-be-erased pool once, so that each time the memory performs garbage collection, storage units with small amounts of valid data and storage units with few historical erasure times are always given priority. For example: there are storage units A and B in the current pool to be erased, the data in A and B are invalid data, and the historical erase and write times of A is 36 times, and the historical erase and write times of B is 0 times. At this time, garbage collection is performed on storage unit B first. After the data of B is erased, the writable space of the storage set where A and B are located is greater than the preset reference threshold. At this time, garbage collection is suspended and garbage collection is no longer performed on A. When the writable space of the storage set is smaller than the target threshold again, if there are still only A and B in the pool to be erased at this time, the historical erase and write times of A is 36 times, and the historical erase and write times of B is 1 time. At this time, garbage collection is performed on storage unit B first to ensure that the data for garbage collection each time is the storage unit with the fewer historical erase and write times, thereby ensuring the wear balance of the memory.

[0110] Through the above-mentioned implementation mode of the present application, a reference threshold is first preset. During the garbage collection process, it is determined whether the writable space of the current memory is greater than or equal to the target threshold. If it is greater than or equal to the target threshold, the operations of valid data caching and data erasure are suspended to avoid that in each garbage collection process, each storage unit that needs to be garbage collected is subjected to a garbage collection operation. This ensures that the storage units subjected to garbage collection operations are those with a small amount of valid data and a small number of historical erasure times. Therefore, while ensuring the wear balance of the memory, the technical problem of excessive write amplification in the related technology is solved.

[0111] In an optional implementation, based on the operation order, cache operations are sequentially performed on valid data stored in each of the plurality of storage units, including:

[0112] S1, determining a current storage unit to be processed from a plurality of storage units based on an operation order;

[0113] S2, when a target storage unit matching the current storage unit is determined from the multiple storage units, cache the valid data stored in the current storage unit to the data cache area.

[0114] In the above step S1 of the present embodiment, all storage units that need to perform garbage collection operations are determined. The storage units that need to perform garbage collection operations are the storage units in the to-be-erased pool in the figure, and according to the determined operation sequence for performing the garbage collection operation, the current storage unit that currently needs to perform the garbage collection operation is determined.

[0115] Then, step S2 can be executed to determine the target storage unit that can perform data writing. The role of the target storage unit is to provide storage space for effective data migration for the current storage unit, so that the valid data in the current storage unit can complete the data transfer. Therefore, when the target storage unit is found in the memory, the valid data in the target storage unit that is currently performing the garbage collection operation is cached to the data cache area, preparing for further data transfer.

[0116] The following combination Figure 7 The determination of the target storage unit is described. Figure 7In the FDP structure hard disk shown, the storage units in each storage set are divided into four parts: free pool, open pool, waiting pool, and closed pool. The free pool is used to store storage units in an idle state where no data is written. Storage units that start writing data but are not full enter the open pool, storage units that are full of data enter the closed pool, and storage units waiting to be erased enter the waiting pool. When the hard disk writes data sent from the host side, it reads the location descriptor and handle descriptor contained in the host side write command, where: the location descriptor is used to specify the storage set identifier and the Placement Handle identifier for data placement, and the Placement The Handle identifier corresponds one-to-one with the handle identifier in the namespace; the handle descriptor is used to describe the type of the handle, including initial independence and continuous independence. Initial independence means that the user data written by the handle only needs to be stored independently when it is first written to the storage unit, and in subsequent possible operations involving data transfer (such as garbage collection, etc.), it can be transferred to the same target storage unit with user data written by different handles; continuous independence means that the user data written by the current handle only needs to be independent when it is first written to the storage unit, and in subsequent possible data transfer operations (such as garbage collection, etc.), it can only be transferred to the same target storage unit with user data written by the same handle.

[0117] When the data in the current storage unit is written by a handle of the initial independent type, the storage unit with the smallest number of historical erase operations is queried in the open pool as the target storage unit. If such a storage unit does not exist in the open pool, the storage unit with the smallest number of historical erase operations is selected in the free pool as the target storage unit. If the storage unit does not exist in the free pool, garbage collection is not performed.

[0118] When the data in the current storage unit is written by a persistent independent type handle, the storage unit with the largest writable space in the storage units associated with the handle in the open pool is first searched as the target unit. If such a storage unit does not exist in the open pool, the storage unit with the smallest number of historical erases is selected from the free pool as the target storage unit. If there is no idle storage unit, garbage collection is not performed.

[0119] Through the above-mentioned implementation mode of the present application, the target storage unit that currently needs to perform garbage collection operation can be determined from multiple storage units. Through the determined operation order, it is ensured that the storage unit that performs garbage collection operation first is a storage unit with a small amount of valid data, and the valid data in the current storage unit is written into the cache area to avoid the valid data being erased during the garbage collection process.

[0120] In an optional implementation, after the valid data stored in the current storage unit is cached in the data cache area, the method further includes:

[0121] Write the valid data in the data buffer into the target storage unit.

[0122] In this embodiment, the valid data cached in the data cache area is written to the target storage unit, thereby realizing the transfer of the valid data from the current storage unit to the target storage unit, ensuring that the valid data in the current storage unit can continue to be stored in the memory, thereby avoiding the loss of valid data due to data erasure.

[0123] In an optional implementation, caching valid data currently stored in the storage unit to a data cache area includes:

[0124] S1, obtaining a data bitmap matching the current storage unit, wherein the data bitmap is used to determine the storage position corresponding to the valid data stored in the current storage unit;

[0125] S2, obtaining a position sorting result of at least one storage position determined according to the data bitmap;

[0126] S3, according to the position sorting result, the valid data stored in at least one storage position in the current storage unit is sequentially cached into the data cache area.

[0127] Optionally, in the above steps S1-S3, first obtain the data bitmap of the current storage unit, calculate the logical block address of the valid data according to the position corresponding to the valid bit in the data bitmap, sort these logical block addresses in order from small to large, and then cache the valid data in the current storage unit in the data cache area in the determined order, and at the same time set the current storage unit to the erase state, and the current storage unit enters the free pool after completing the data erasure, and the storage units in the free pool are disconnected from all handles.

[0128] Through the above-mentioned implementation mode of the present application, the data bitmap of the current storage unit is obtained, and the storage address of the valid data in the memory is determined according to the data bitmap, and the valid data is further sorted from small to large according to the size of the storage address, thereby ensuring that the current storage unit can perform valid data in sequence when performing data caching operations, and because the sorting is performed in address order, it is ensured that the cache reads the valid data in the current storage unit in address order, thereby improving the efficiency of the current storage unit writing valid data to the cache.

[0129] In an optional manner, when a target storage unit matching the current storage unit is determined from a plurality of storage units, before caching the valid data stored in the current storage unit into the data cache area, the method further includes:

[0130] S1, obtaining a handle type of a current operation handle used to operate a current storage unit, wherein the handle type includes an initial independent type and a continuous independent type, the initial independent type is used to indicate that the current operation handle is allowed to operate on a reference storage unit, and the continuous independent type is used to indicate that the current operation handle is not allowed to operate on the reference storage unit, the reference storage unit is a storage unit operated according to a reference operation handle, and the reference operation handle is an operation handle different from the current operation handle;

[0131] S2, according to the handle type of the current operation handle, determine the target storage unit from multiple storage units included in the first storage subset and the second storage subset, wherein the current storage set includes the first storage subset and the second storage subset, the first storage unit in the first storage subset does not store the data object, the second storage unit in the second storage subset has stored the data object, and the writable space corresponding to the second storage unit is greater than 0.

[0132] In the above step S1, if Figure 5 As shown in the figure, different handles are responsible for centralized storage of different workload types. For example, the data of workload A is centrally stored in three storage units in storage set 0-2 through handles, so that different types of data can be stored as follows Figure 4 As shown, the data are centrally stored in different storage units.

[0133] In the above step S2, the first storage subset may be Figure 7 The second storage subset can be a collection of storage units in the free pool in Figure 7 The collection of storage units in the open pool.

[0134] Optional, in Figure 3 In the interaction diagram between the host and the storage device shown, when the host sends a data write command to the storage device, the command includes a location descriptor and a handle type, wherein: the location descriptor is used to specify the storage set identifier and the Placement Handle identifier for data placement, and the Placement Handle identifier corresponds one-to-one with the handle type identifier in the namespace; the handle type includes two types: an initial independent type and a continuous independent type. The initial independent type means that the user data written by the current handle only needs to be independent when it is first written to the storage unit, and can be moved to the same target storage unit with user data written by different handles in subsequent possible data transfers (such as garbage collection operations); continuous independence means that the user data written by the current handle only needs to be independent when it is first written to the storage unit, and can only be moved to the same target storage unit with user data written by the same handle in subsequent possible data transfers (such as garbage collection operations).

[0135] Optionally, in step S1, Figure 5As shown, assuming Figure 5 The handle types of handle 0 and handle 1 are initially independent. When data is moved from the storage unit where data is stored by handle 0 and handle 1, the valid data in the storage unit stored by handle 0 and handle 1 can be stored together in one target storage unit.

[0136] When the handle type of handle 0 is continuous and independent, when the current storage unit to which handle 0 writes data moves valid data, the valid data in the current storage unit can only be stored together with other data written by handle 0, and cannot be stored together with data written by other handles.

[0137] Then, step S2 is executed to determine the target storage unit according to the handle type. If the data in the current storage unit is written by the handle of the initial independent type, Figure 4 The storage unit with the smallest number of historical erase and write operations in the open pool shown in FIG. 1 is the target storage unit. If such a storage unit does not exist in the open pool, then Figure 4 In the free pool shown, a storage unit with the smallest number of historical erase and write times is selected as the target storage unit. If there is no storage unit in the free pool, garbage collection is not performed.

[0138] When the data in the current storage unit is written by a handle of the continuous independent type, the following is searched first: Figure 4 The storage unit with the largest writable space among the storage units associated with the handle in the open pool shown is used as the target storage unit. If such a storage unit does not exist in the open pool, Figure 4 In the free pool shown, a storage unit with the smallest number of historical erase and write times is selected as a target storage unit. If there is no free storage unit, garbage collection is not performed.

[0139] Through the above-mentioned implementation mode of the present application, the target storage unit that currently needs to perform garbage collection operation can be determined from multiple storage units. Through the determined operation order, it is ensured that the storage unit that performs garbage collection operation first is a storage unit with a small amount of valid data, and the valid data in the current storage unit is written into the cache area to avoid the valid data being erased during the garbage collection process.

[0140] In an optional manner, determining the target storage unit from a plurality of storage units included in the first storage subset and the second storage subset according to the handle type of the current operation handle includes:

[0141] S1, when the handle type of the current operation handle is an initial independent type and the second storage subset includes at least one second storage unit corresponding to the initial independent type, from the at least one second storage unit, determine the second storage unit with the lowest first storage state parameter as the target storage unit;

[0142] S2, when the handle type of the current operation handle is an initial independent type and the second storage subset does not include a second storage unit corresponding to the initial independent type, determine a first storage unit with the lowest first storage state parameter as a target storage unit from at least one first storage unit included in the first storage subset;

[0143] S3, when the handle type of the current operation handle is a persistent independent type and the second storage subset includes at least one second storage unit associated with the current operation handle, determine, from the at least one second storage unit, a second storage unit with the highest second storage state parameter as a target storage unit;

[0144] S4. When the handle type of the current operation handle is a persistent independent type and the second storage subset does not include a second storage unit associated with the current operation handle, a first storage unit with the lowest first storage state parameter is determined as a target storage unit from at least one first storage unit included in the first storage subset.

[0145] Optionally, in the above step S1, if the data in the current storage unit is written by the handle of the initial independent type, in the case Figure 4 The storage unit with the smallest number of historical erase and write operations in the open pool shown is used as the target storage unit.

[0146] Optionally, in the above step S2, if the data in the current storage unit is written by the handle of the initial independent type, if there is no storage unit with the smallest number of historical erase operations in the open pool, then Figure 4 The storage unit with the smallest number of historical erase and write times in the free pool shown is selected as the target storage unit. If there is no storage unit in the free pool, the data transfer operation is not performed.

[0147] Optionally, when the data transfer operation is not performed as in the above step S2, the memory waits for the data clearing command from the host side. After the hard disk executes the data clearing command from the host side, if Figure 4 If a storage unit with the smallest number of historical erase and write times appears in the free pool shown, the unit is used as the target storage unit and a data transfer operation is performed.

[0148] Optionally, in the above step S3, when the data in the current storage unit is written by a handle of a persistent independent type, searching for Figure 4 The storage unit with the largest writable space among the storage units associated with the handle in the open pool shown is used as the target storage unit.

[0149] Optionally, in the above step S4, when the data in the current storage unit is written by a handle of a persistent independent type, if there is no storage unit associated with the handle in the open pool, then Figure 4 In the free pool shown, the storage unit with the smallest number of historical erase and write times is selected as the target storage unit. If there is no free storage unit, no data transfer operation is performed.

[0150] Optionally, when the data transfer operation is not performed as in the above step S4, the memory waits for the data clearing command from the host side. After the hard disk executes the data clearing command from the host side, if Figure 4 If a storage unit with the smallest number of historical erase and write times appears in the free pool shown, the unit is used as the target storage unit and a data transfer operation is performed.

[0151] Through the above-mentioned implementation mode of the present application, the target storage unit that currently needs to perform garbage collection operation can be determined from multiple storage units. Through the determined operation order, it is ensured that the storage unit that performs garbage collection operation first is a storage unit with a small amount of valid data, and the valid data in the current storage unit is written into the cache area to avoid the valid data being erased during the garbage collection process.

[0152] The following combination Fig. 9 A method for processing data in a complete memory is described.

[0153] First, the valid data to be transferred can be determined through steps S902-S904, and then a target storage unit for storing the transferred valid data can be selected through S906. If the target storage unit exists in the memory, the data transfer operation can be performed through steps S910-S912. Specifically:

[0154] S902, determine the current storage unit; Figure 3 Taking the FDP structure hard disk in as an example, the application data in the host 301 is stored in the physical address range of the specified data storage. Inside the FDP solid state hard disk, the flash translation layer 308 is burned into the CPU. The flash translation layer completes functions such as address mapping, garbage collection and wear leveling, as well as the handle management function of the solid state hard disk that supports FDP, so as to support and manage the FDP structure.

[0155] Data storage such as Figure 4 As shown in FIG. 1 , in a hard disk without an FDP structure, data with different workloads are mixed in different storage units, while in a hard disk with an FDP structure, each storage unit stores data with different workloads in different data storage units in addition to storing the check bit XOR, as shown in FIG. Figure 4As shown, invalid data is centrally stored in storage unit 401, and workload B and workload C are centrally stored in other storage units respectively. When the host needs to write data with workload A to the memory, in the hard disk using the FDP structure, garbage collection is performed on storage unit 401, and the invalid data in storage unit 401 is cleared. Storage unit 401 after the invalid data clearing operation is completed is used as storage unit 402 where data can be written, and the data of workload A is stored in storage unit 402. Since in the solid-state hard disk of FDP, invalid data appear in one storage unit, it is rarely necessary to move valid data, thereby effectively reducing the write amplification of the memory.

[0156] The above-mentioned storing of data of different loads in different storage units can be as follows: Figure 5 As shown, Figure 5 As shown, there is a handle module in the hard disk, which is responsible for writing data with different workloads into one or more storage units in a centralized manner.

[0157] For example, handle 0 is responsible for writing the data set with workload A into multiple storage units in storage sets 0-3, such as Figure 8 As shown, after receiving the write command for data with workload A sent by the host, the hard disk determines the storage unit to be written through handle 0. Each handle is associated with a storage unit in each storage set, such as Figure 8 The handle 0 in is associated with storage sets 0-2, and the data with workload A is written to different storage sets through handle 0.

[0158] It should be noted that the storage unit in the above step S902 refers to Figure 6 The storage area of ​​the flash memory shown is composed of blocks with the same physical location on every 32 wafers.

[0159] In the above step S902, the storage units in each storage set have four states: idle, open, closed, and ready to be erased, and are located at Figure 7 In the free pool, open pool, closed pool and to-be-erased pool shown, it should be noted that the free pool, open pool, closed pool and to-be-erased pool here are not physical entities with fixed storage addresses, but can be a collection for recording storage units. For example, the free pool can be a collection of addresses, which records the addresses of all storage units in an idle state. The above-mentioned idle state refers to a state in which no data is written. Similarly, the address of a storage unit that is in a state of starting to write data but not fully written is recorded in the open pool, the address of a storage unit in a state of fully written data is recorded in the closed pool, and the address of a storage unit waiting to be erased is recorded in the to-be-erased pool.

[0160] In the above step S902, the host continuously sends write commands to the solid-state drive, and the command needs to include a location descriptor and a handle descriptor, wherein: the location descriptor is used to specify the storage set identifier and the PlacementHandle identifier for data placement, and the Placement Handle identifier corresponds one-to-one with the handle identifier in the namespace; the handle descriptor is used for the type of handle, including initial independence and continuous independence. Initial independence means that the user data written by the current handle only needs to be independent when it is first written to the RU, and in subsequent possible data transfers (such as garbage collection operations), it can be transferred to the same storage unit with user data written by different handles; continuous independence means that the user data written by the current handle only needs to be independent when it is first written to the RU, and in subsequent possible data transfers (such as garbage collection operations), it can only be transferred to the same storage unit with user data written by the same handle.

[0161] When the solid state drive receives a write command from the host, the controller determines the storage unit to be written through the handle. The handle is associated with the storage unit in each storage set. Figure 8 As shown, the data of workload A is continuously written to storage set 0-2 through handle 0.

[0162] As data is continuously written into the current storage unit, the available space of the current storage unit is continuously reduced, and the storage unit currently pointed to by the storage unit Handle will eventually be filled up. Figure 8 The data of workload A in the storage set is continuously written to storage set 2, causing the space of the corresponding storage unit to be almost exhausted. At this time, the controller needs to find other storage units with available space for writing. Therefore, the handle needs to regularly check the writable space of the storage unit. The writable space of the storage unit is defined as the number of logical blocks required to fill the storage unit. When the writable space of the storage unit drops to a certain threshold, the handle first selects a storage unit from the free pool and re-indexes it. If there is no storage unit in the free pool of the storage set, the storage unit with the largest writable space among the storage units written by the same handle in the open pool is selected for indexing. The unwritten data will continue to be written to the re-indexed storage unit, and the full Reclaim Unit will enter the closed pool.

[0163] When the writable space of a storage set is less than the target threshold, the storage set located at Figure 7The storage unit in the to-be-erased pool shown in the figure performs a garbage collection operation. First, the historical number of erase operations and the amount of valid data of the storage unit in the to-be-erased pool and their corresponding coefficients are obtained. Then, the historical number of erase operations and the amount of valid data of each storage unit in the to-be-erased pool and their corresponding coefficients are weighted summed. The units are sorted according to the calculation results, and the storage unit at the front of the sort is determined as the current storage unit.

[0164] S904, valid data query: first obtain the data bitmap of the current storage unit, and the data with the valid bit marked as 1 in the data bitmap is the valid data.

[0165] S906, search for the target storage unit; when the data in the current storage unit is written by a handle of the initial independent type, query the storage unit with the smallest number of historical erase operations in the open pool as the target storage unit. If such storage unit does not exist in the open pool, select the storage unit with the smallest number of historical erase operations in the free pool as the target storage unit.

[0166] When the data in the current storage unit is written by a handle of a continuous independent type, the storage unit with the largest writable space in the storage units associated with the handle in the open pool is first searched as the target unit. If such a storage unit does not exist in the open pool, the storage unit with the smallest number of historical erases is selected in the free pool as the target storage unit.

[0167] S908, determine whether a target storage unit that meets the conditions is found, if found, execute step S910, if not found, execute step S920 and stop garbage collection.

[0168] S910, logical block address sorting, calculating the logical block address of the valid data according to the position corresponding to the valid bit in the data bitmap of the current storage unit, and sorting these logical block addresses in order from small to large, in preparation for sequential caching of the valid data.

[0169] S912, data transfer; cache the valid data in the current storage unit in the order determined in S910, write the valid data into the data cache area, and set the cached current storage unit to the erase state. After the erasure is completed, it enters the free pool, and the storage units in the free pool are disconnected from all handles. The valid data cached in the data cache area is written into the target storage unit, thereby realizing the transfer of valid data from the current storage unit to the target storage unit.

[0170] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0171] According to another aspect of the embodiments of the present application, a data processing device in a memory for implementing the above-mentioned data processing method in a memory is also provided. Fig.10 As shown, the device comprises:

[0172] A first acquisition unit 1002 is used to acquire a set writable space corresponding to a current storage set in a target storage device, wherein the current storage set includes multiple storage units, the set writable space is the sum of the writable spaces of the multiple storage units, and the writable space is the data storage space not occupied by data objects, and the data objects include valid data objects and invalid data objects;

[0173] A second acquisition unit 1004 is used to acquire a first storage state parameter and a second storage state parameter of each of the plurality of storage units when the set writable space is less than or equal to the target threshold, wherein the first storage state parameter is used to indicate the number of historical erase and write operations of the storage unit, and the second storage state parameter is used to indicate the amount of valid data stored in the storage unit;

[0174] A determining unit 1006, configured to determine an operation order corresponding to each of the plurality of storage units according to the first storage state parameter and the second storage state parameter, and sequentially perform a cache operation on valid data stored in each of the plurality of storage units based on the operation order;

[0175] The erasing unit 1008 is used to perform a data erasing operation on the storage unit that has performed the data caching operation, wherein the data erasing operation is used to configure the data storage space in the storage unit as a writable space.

[0176] Optionally, the above-mentioned determination unit 1006 includes: a first acquisition module, used to obtain a first weight coefficient matching the first storage state parameter, and a second weight coefficient matching the second storage state parameter; a second acquisition module, used to sequentially acquire the first storage state parameter and the second storage state parameter of each of the multiple storage units, and determine the priority indexes that match each of the multiple storage units respectively according to the weighted summation result between the first weight coefficient, the second weight coefficient, the first storage state parameter and the second storage state parameter; a determination module, used to determine the operation order corresponding to each of the multiple storage units based on the priority indexes that match each of the multiple storage units.

[0177] Optionally, the above-mentioned second acquisition module is also used to: obtain a third weight coefficient, and obtain the third storage state parameter of each of the multiple storage units in turn; based on the third weight coefficient, the third storage state parameter of each of the multiple storage units and the weighted summation result, determine the priority index matching each of the multiple storage units respectively, wherein the third storage state parameter is used to indicate the data amount of invalid data stored in the storage unit, and the third weight coefficient is a weight coefficient matching the third storage state parameter; obtain a fourth weight coefficient, and obtain the fourth storage state parameter of each of the multiple storage units in turn; based on the fourth weight coefficient, the fourth storage state parameter of each of the multiple storage units and the weighted summation result, determine the priority index matching each of the multiple storage units respectively, wherein the fourth storage state parameter is used to indicate the ratio between the first data amount of valid data and the second data amount of invalid data stored in the storage unit, and the fourth weight coefficient is a weight coefficient matching the fourth storage state parameter.

[0178] Optionally, the first acquisition module is further used to: determine an erase balance index matching the current storage set according to respective first state parameters of a plurality of storage units in the current storage set; and acquire a first weight coefficient and a second weight coefficient matching the erase balance index.

[0179] Optionally, the above-mentioned first acquisition module is also used to: obtain a first weight coefficient and a second weight coefficient matching the current index interval according to the current index interval corresponding to the erasure balance index, wherein the preset multiple index intervals correspond to a group of first weight coefficients and second weight coefficients respectively, and the multiple index intervals include the current index interval; determine the product of the erasure correlation coefficient and the erasure balance index as the first weight coefficient, wherein the erasure correlation coefficient is used to indicate the positive correlation between the first weight coefficient and the erasure balance index; obtain the preset second weight coefficient.

[0180] Optionally, the above-mentioned erasing unit 1008 is also used to: when the set writable space is greater than or equal to a reference threshold, suspend the sequential execution of caching operations on valid data stored in each of the multiple storage units; when the set writable space is greater than or equal to a reference threshold, suspend the execution of data erasing operations on the storage units that have executed data caching operations; wherein the reference threshold is greater than or equal to the target threshold.

[0181] Optionally, the above-mentioned determination 1006 unit is also used to: determine the current storage unit to be processed from multiple storage units based on the operation order; when a target storage unit matching the current storage unit is determined from multiple storage units, cache the valid data stored in the current storage unit to the data cache area.

[0182] Optionally, the determination unit 1006 is further configured to write valid data in the data cache area into a target storage unit.

[0183] Optionally, the above-mentioned determination unit 1006 is also used to: obtain a data bitmap matching the current storage unit, wherein the data bitmap is used to determine the storage position corresponding to the valid data stored in the current storage unit; obtain a position sorting result of at least one storage position determined according to the data bitmap; and according to the position sorting result, cache the valid data stored in at least one storage position in the current storage unit in sequence to the data cache area.

[0184] Optionally, the above-mentioned determination unit 1006 is also used to: obtain the handle type of the current operation handle used to operate the current storage unit, wherein the handle type includes an initial independent type and a continuous independent type, the initial independent type is used to indicate that the current operation handle is allowed to operate on the reference storage unit, and the continuous independent type is used to indicate that the current operation handle is not allowed to operate on the reference storage unit, the reference storage unit is a storage unit operated according to the reference operation handle, and the reference operation handle is an operation handle different from the current operation handle; according to the handle type of the current operation handle, determine the target storage unit from the multiple storage units included in the first storage subset and the second storage subset, wherein the current storage set includes the first storage subset and the second storage subset, the first storage unit in the first storage subset does not store the data object, the second storage unit in the second storage subset has stored the data object, and the writable space corresponding to the second storage unit is greater than 0.

[0185] Optionally, the above-mentioned determination unit 1006 is also used to: when the handle type of the current operation handle is the initial independent type and the second storage subset includes at least one second storage unit corresponding to the initial independent type, determine the second storage unit with the lowest first storage state parameter as the target storage unit from the at least one second storage unit; when the handle type of the current operation handle is the initial independent type and the second storage subset does not include the second storage unit corresponding to the initial independent type, determine the first storage unit with the lowest first storage state parameter as the target storage unit from the at least one first storage unit included in the first storage subset; when the handle type of the current operation handle is the persistent independent type and the second storage subset includes at least one second storage unit associated with the current operation handle, determine the second storage unit with the highest second storage state parameter as the target storage unit from the at least one second storage unit; when the handle type of the current operation handle is the persistent independent type and the second storage subset does not include the second storage unit associated with the current operation handle, determine the first storage unit with the lowest first storage state parameter as the target storage unit from the at least one first storage unit included in the first storage subset.

[0186] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned method for processing data in a memory is also provided. The electronic device may be Figure 1 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a mobile phone or a computer as an example. Fig.11 As shown, the electronic device includes a memory 1102 and a processor 1104. The memory 1102 stores a computer program, and the processor 1104 is configured to execute the steps in any of the above method embodiments through the computer program.

[0187] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0188] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0189] S1, obtaining a set writable space corresponding to a current storage set in a target storage, wherein the current storage set includes a plurality of storage units, the set writable space is the sum of the writable spaces of the plurality of storage units, the writable space is the data storage space not occupied by data objects, and the data objects include valid data objects and invalid data objects;

[0190] S2, when the set writable space is less than or equal to the target threshold, obtaining a first storage state parameter and a second storage state parameter of each of the plurality of storage units, wherein the first storage state parameter is used to indicate the number of historical erase and write operations of the storage unit, and the second storage state parameter is used to indicate the amount of valid data stored in the storage unit;

[0191] S3, determining the operation order corresponding to each of the plurality of storage units according to the first storage state parameter and the second storage state parameter, and performing cache operations on the valid data stored in each of the plurality of storage units in sequence based on the operation order;

[0192] S4, performing a data erasing operation on the storage unit that has performed the data caching operation, wherein the data erasing operation is used to configure the data storage space in the storage unit as a writable space.

[0193] Alternatively, a person skilled in the art may understand that: Fig.11 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Fig.10 The structure of the electronic device is not limited. Fig.10 More or fewer components (such as network interfaces, etc.) as shown in, or with Fig.10 Different configurations shown.

[0194] Among them, the memory 1102 can be used to store software programs and modules, such as the program instructions / modules corresponding to the data processing method and device in the memory in the embodiment of the present application. The processor 1104 executes various functional applications and data processing by running the software programs and modules stored in the memory 1102, that is, realizing the above-mentioned data processing method in the memory. The memory 1102 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1102 may further include a memory remotely arranged relative to the processor 1104, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 1102 can be specifically, but not limited to, used to store information such as page elements and page styles. As an example, such as Fig.11As shown, the memory 1102 may include, but is not limited to, the first acquisition unit 1002, the second acquisition unit 1004, the determination unit 1006, and the erasing unit 1008 in the data processing device in the memory. In addition, other module units in the data processing device in the memory may also be included but are not limited to, which will not be repeated in this example.

[0195] Optionally, the transmission device 1106 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 1106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 1106 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0196] In addition, the electronic device further includes: a display 1108 for displaying the target page; and a connection bus 1110 for connecting various module components in the electronic device.

[0197] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. The nodes may form a point-to-point network, and any form of computing device, such as a server, terminal or other electronic device, may become a node in the blockchain system by joining the point-to-point network.

[0198] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in the above various optional implementations;

[0199] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0200] S1, obtaining a set writable space corresponding to a current storage set in a target storage, wherein the current storage set includes a plurality of storage units, the set writable space is the sum of the writable spaces of the plurality of storage units, the writable space is the data storage space not occupied by data objects, and the data objects include valid data objects and invalid data objects;

[0201] S2, when the set writable space is less than or equal to the target threshold, obtaining a first storage state parameter and a second storage state parameter of each of the plurality of storage units, wherein the first storage state parameter is used to indicate the number of historical erase and write operations of the storage unit, and the second storage state parameter is used to indicate the amount of valid data stored in the storage unit;

[0202] S3, determining the operation order corresponding to each of the plurality of storage units according to the first storage state parameter and the second storage state parameter, and performing cache operations on the valid data stored in each of the plurality of storage units in sequence based on the operation order;

[0203] S4, performing a data erasing operation on the storage unit that has performed the data caching operation, wherein the data erasing operation is used to configure the data storage space in the storage unit as a writable space.

[0204] Optionally, in the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0205] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0206] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the methods of each embodiment of the present application.

[0207] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0208] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0209] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0210] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0211] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

[0212] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0213] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0214] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0215] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0216] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0217] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0218] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0219] The embodiments of the present application also provide a computer program, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in any one of the above method embodiments.

[0220] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0221] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0222] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A data processing method in a memory, characterized in that: include: Acquire a set writable space corresponding to a current storage set in a target memory, wherein the current storage set includes a plurality of storage units, the set writable space is the sum of the writable spaces of the plurality of storage units, the writable space is a data storage space not occupied by data objects, and the data objects include valid data objects and invalid data objects; When the set writable space is less than or equal to the target threshold, obtaining a first storage state parameter and a second storage state parameter of each of the plurality of storage units, wherein the first storage state parameter is used to indicate the number of historical erase and write operations of the storage unit, and the second storage state parameter is used to indicate the amount of valid data stored in the storage unit; Acquire a first weight coefficient matching the first storage state parameter and a second weight coefficient matching the second storage state parameter; Determine the priority indexes of the respective matches of the plurality of storage units according to the first weight coefficient, the second weight coefficient, a weighted sum of the first storage state parameter and the second storage state parameter; Based on the priority indexes matched by the plurality of storage units, respectively, determining the operation order corresponding to the plurality of storage units, and based on the operation order, sequentially performing cache operations on the valid data stored in the plurality of storage units respectively; A data erasing operation is performed on the storage unit that has performed a data caching operation, wherein the data erasing operation is used to configure the data storage space in the storage unit as the writable space, and when the data erasing operation is performed on the storage unit, if the set writable space is greater than or equal to a reference threshold, the caching operation is suspended for sequentially performing the caching operation on the valid data stored in each of the plurality of storage units.

2. The method according to claim 1, characterized in that Determining the priority indexes of the plurality of storage units respectively according to the first weight coefficient, the second weight coefficient, the weighted sum of the first storage state parameter and the second storage state parameter comprises at least one of the following: Obtaining a third weight coefficient, and sequentially obtaining a third storage state parameter of each of the plurality of storage units; determining a priority index that matches each of the plurality of storage units based on the third weight coefficient, the third storage state parameter of each of the plurality of storage units, and the weighted summation result, wherein the third storage state parameter is used to indicate the amount of invalid data stored in the storage unit, and the third weight coefficient is a weight coefficient that matches the third storage state parameter; Obtain a fourth weight coefficient, and sequentially obtain a fourth storage state parameter of each of the plurality of storage units; based on the fourth weight coefficient, the fourth storage state parameter of each of the plurality of storage units and the weighted summation result, determine a priority index that matches each of the plurality of storage units, respectively, wherein the fourth storage state parameter is used to indicate a ratio between a first data amount of the valid data and a second data amount of the invalid data stored in the storage unit, and the fourth weight coefficient is a weight coefficient that matches the fourth storage state parameter.

3. The method according to claim 1, characterized in that The acquiring a first weight coefficient matching the first storage state parameter and a second weight coefficient matching the second storage state parameter comprises: Determining an erase balance index matching the current storage set according to the first storage state parameters of each of the plurality of storage units in the current storage set; The first weight coefficient and the second weight coefficient matching the erase balance index are obtained.

4. The method according to claim 3, characterized in that The acquiring of the first weight coefficient and the second weight coefficient matching the erasure balance index comprises one of the following: According to the current index interval corresponding to the erase leveling index, acquiring the first weight coefficient and the second weight coefficient matching the current index interval, wherein the preset multiple index intervals correspond to a group of the first weight coefficient and the second weight coefficient respectively, and the multiple index intervals include the current index interval; The product of the erasure correlation coefficient and the erasure balance index is determined as the first weight coefficient, wherein the erasure correlation coefficient is used to indicate a positive correlation between the first weight coefficient and the erasure balance index; and the preset second weight coefficient is obtained.

5. The method according to claim 1, characterized in that When performing a data erasing operation on the storage unit that has performed a data caching operation, the method further includes: When the set writable space is greater than or equal to a reference threshold, suspending the data erasing operation on the storage unit that has performed the data caching operation; Wherein, the reference threshold is greater than or equal to the target threshold.

6. The method according to claim 1, characterized in that The sequentially performing cache operations on the valid data stored in the plurality of storage units based on the operation order includes: Based on the operation order, determining a current storage unit to be processed from the plurality of storage units; When a target storage unit matching the current storage unit is determined from the plurality of storage units, the valid data stored in the current storage unit is cached in a data cache area.

7. The method according to claim 6, characterized in that After caching the valid data stored in the current storage unit to the data cache area, the method further includes: The valid data in the data buffer area is written into the target storage unit.

8. The method according to claim 6, characterized in that The step of caching the valid data stored in the current storage unit to a data cache area comprises: Acquire a data bitmap matching the current storage unit, wherein the data bitmap is used to determine a storage location corresponding to the valid data stored in the current storage unit; Obtaining a position sorting result of at least one of the storage locations determined according to the data bitmap; According to the position sorting result, the valid data stored in at least one of the storage positions in the current storage unit is sequentially cached in the data cache area.

9. The method according to claim 6, characterized in that In the case where a target storage unit matching the current storage unit is determined from the plurality of storage units, before caching the valid data stored in the current storage unit to the data cache area, the method further includes: Obtaining a handle type of a current operation handle used to operate the current storage unit, wherein the handle type includes an initial independent type and a continuous independent type, the initial independent type is used to indicate that the current operation handle is allowed to operate on a reference storage unit, and the continuous independent type is used to indicate that the current operation handle is not allowed to operate on the reference storage unit, the reference storage unit is a storage unit operated according to a reference operation handle, and the reference operation handle is an operation handle different from the current operation handle; According to the handle type of the current operation handle, the target storage unit is determined from the multiple storage units included in the first storage subset and the second storage subset, wherein the current storage set includes the first storage subset and the second storage subset, the data object is not stored in the first storage unit in the first storage subset, the data object is stored in the second storage unit in the second storage subset, and the writable space corresponding to the second storage unit is greater than 0.

10. The method according to claim 9, characterized in that The step of determining the target storage unit from the plurality of storage units included in the first storage subset and the second storage subset according to the handle type of the current operation handle comprises: In a case where the handle type of the current operation handle is the initial independent type and the second storage subset includes at least one second storage unit corresponding to the initial independent type, determining, from the at least one second storage unit, the second storage unit having the lowest first storage state parameter as the target storage unit; In a case where the handle type of the current operation handle is the initial independent type and the second storage subset does not include the second storage unit corresponding to the initial independent type, determining the first storage unit having the lowest first storage state parameter from at least one first storage unit included in the first storage subset as the target storage unit; In a case where the handle type of the current operation handle is the persistent independent type and the second storage subset includes at least one second storage unit associated with the current operation handle, determining the second storage unit with the highest second storage state parameter from the at least one second storage unit as the target storage unit; When the handle type of the current operation handle is the persistent independent type and the second storage unit associated with the current operation handle is not included in the second storage subset, the first storage unit with the lowest first storage state parameter is determined as the target storage unit from at least one of the first storage units included in the first storage subset.

11. A data processing device in a memory, characterized in that: include: A first acquisition unit is used to acquire a set writable space corresponding to a current storage set in a target memory, wherein the current storage set includes a plurality of storage units, the set writable space is the sum of the writable spaces of the plurality of storage units, the writable space is a data storage space not occupied by a data object, and the data object includes a valid data object and an invalid data object; A second acquisition unit is used to acquire, when the set writable space is less than or equal to a target threshold, a first storage state parameter and a second storage state parameter of each of the plurality of storage units, wherein the first storage state parameter is used to indicate the number of historical erase and write operations of the storage unit, and the second storage state parameter is used to indicate the amount of valid data stored in the storage unit; a determining unit, configured to determine priority indexes matched by each of the plurality of storage units, respectively, according to a first weight coefficient, a second weight coefficient, a weighted sum of the first storage state parameter, and the second storage state parameter; determine operation orders corresponding to each of the plurality of storage units based on the priority indexes matched by each of the plurality of storage units, and sequentially perform cache operations on the valid data stored by each of the plurality of storage units based on the operation order; an erasing unit, configured to perform a data erasing operation on the storage unit that has performed a data caching operation, wherein the data erasing operation is used to configure the data storage space in the storage unit as the writable space, and when the storage unit performs the data erasing operation, if the set writable space is greater than or equal to a reference threshold, suspending the sequential execution of the caching operation on the valid data stored in each of the plurality of storage units; The device is also used to obtain a first weight coefficient that matches the first storage state parameter and a second weight coefficient that matches the second storage state parameter.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 10 when executed by a processor.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 10 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 10 are implemented.

Images