All-flash data writing method, device, electronic device and storage medium

Abstract

The present application discloses a method, device, electronic device and storage medium for writing data to an all-flash memory, the method comprising: dividing the data to be written into N first data blocks and 1 second data block, wherein each data block does not overlap with each other, and the size of the first data block is consistent with the size of a single storage unit, and the size of the second data block is smaller than the size of a single storage unit; compressing the second data block and at least one first data block, a total of M data blocks, into a third data block, so that the size of the third data block is smaller than the size of M-1 storage units; writing the N-M+1 first data blocks that are not compressed and the third data block as a whole into the storage space in an aligned write manner. Through the present application, the data is cut and compressed before performing aligned write, and aligned write is performed on this basis, while taking into account both disk space utilization and data writing efficiency.

Description

Technical Field

[0001] The present application relates to the technical field of data writing, and in particular to a method, device, electronic device and storage medium for writing data to an all-flash memory. Background Art

[0002] All-flash storage means that all storage media in the storage system are composed of flash memory (SSD). In other words, in all-flash storage, all data is stored on SSDs, and mechanical hard disks (HDDs) or other types of storage devices are not supported. All storage devices use flash media, which greatly improves the speed of data access and supports high-concurrency and high-throughput storage operations.

[0003] Stacking technology is a design that "builds tall buildings" in a limited space, making storage devices smaller but with larger capacity, faster access speed, and lower cost. The Storage Performance Development Kit can directly interact with these devices efficiently, bypassing the operating system kernel layer, reducing latency and improving performance.

[0004] Although stacked SSD can store a lot of data, it is not suitable for storing metadata. It is not good at processing these small-scale, frequent updates, because frequently modified small blocks of data will slow down the stacked SSD. Therefore, as shown in Figure 1, storing metadata in persistent memory, using persistent memory to provide low-latency access, and storing user data in stacked SSD, using the large capacity and cost-effectiveness of stacked SSD for large-scale data storage, is a more preferred storage method.

[0005] In all-flash storage, all data is processed in user mode, that is, the user divides the data into the required format, and then calls the storage performance tool development kit to write the data to the storage device. In the prior art, there are two writing modes: sequential writing and aligned writing.

[0006] Sequential writing makes full use of storage space, but data writing performance is low. Figure 2 As shown, the steps for sequential writing are as follows:

[0007] The first step is to obtain the status information of the space at the end of the last data write. If there is any remaining unused space, read the data in the storage space. Figure 2The strip on the right side of the middle composed of multiple grid blocks represents the storage space, and the strip includes several blocks of the same size arranged in sequence in the space, and each block represents a storage unit, that is, the strip represents several sequentially arranged storage units. According to the order of storage from left to right, after the last data writing, the first grid and the second grid of the strip have data stored, wherein the storage unit of the first grid has no remaining space, and the storage unit of the second grid has a part of the data stored but the storage unit is not full, so that there is still some remaining space in the storage unit. At this time, the data read from the storage unit of the second grid is the red data block indicated by the dotted line 2 in the figure.

[0008] Step 2: If data is read in step 1, merge the read data with the current user data and split it according to the block size. Figure 2 For example, what is to be stored this time is Figure 2 The green data block above is divided into Figure 2 The yellow, green and purple parts behind the red data are shown in the middle of the left side. Specifically, the green data block is spliced ​​behind the red data block read in the first step to form a data block to be stored, and the data block to be stored is divided into a number of data to be stored corresponding to the storage unit according to the size of the storage unit. Among them, the yellow part comes from Figure 2 The green data block above, the red and yellow parts together form a to-be-stored data that just corresponds to the storage unit of the second grid, so that the storage unit of the second grid can be filled. The green to-be-stored data between the yellow and purple just corresponds to the storage unit of the third grid, so that the storage unit of the third grid can be filled. Finally, the purple to-be-stored data is less and cannot fill the storage unit of the fourth grid, so a certain amount of remaining space is left in the storage unit of the fourth grid.

[0009] Step 3: Split the data blocks to be stored obtained in step 2 and write them into the storage space. According to the division of the data blocks to be stored in step 2, store the red and yellow parts into the storage cells of the second grid, the green part into the storage cells of the third grid, and the purple part into the storage cells of the fourth grid.

[0010] The above process involves reading data first and then writing it, which will slow down the speed of writing data to the disk, that is, reduce the performance of writing data to the disk.

[0011] In the aligned write mode, data writing performance is higher, but more storage space will be wasted. Figure 3 As shown, the steps for aligning write are as follows:

[0012] The first step is to split the data. Figure 3 The green data block in the upper left corner is the data to be written. Figure 3The strip with multiple storage units on the right in the middle is used as storage space. The green data block is divided according to the size of the storage unit to form Figure 3 The yellow, green and purple parts on the left side of the middle. Among them, the size of the yellow part corresponds to one storage unit, the size of the green part is the same as one storage unit, and the size of the purple part is smaller than one storage unit.

[0013] Step 2: Write the user data split in the first step to the storage device. If the size of the last split data is smaller than the storage space size, the unused space is filled with data. Figure 3 For example, according to the order of storage space from left to right, after the last write operation, the first storage unit has been filled, and the second storage unit has been partially stored with some remaining space. During this storage, the remaining space in the second storage unit is filled with other data, the yellow part is written into the third storage unit, the green part is written into the fourth storage unit, and the purple part is written into the fifth storage unit. At this time, the fifth storage unit is not full, and its unused space will be filled with other data so that the next write operation starts from the sixth storage unit.

[0014] In the above process, each write will generate residual space, and the size of the residual space is uncontrollable, and data is written from the next empty storage unit each time, which causes the overall available space of the disk to decrease rapidly as the number of writes increases, wasting a lot of disk space and reducing disk utilization.

[0015] Therefore, a method of writing data that takes both storage space and data writing efficiency into consideration is needed. Summary of the invention

[0016] The embodiments of the present application provide an all-flash data writing method, device, electronic device and storage medium, which are used to solve the defects of existing data writing methods in balancing storage space and data writing efficiency.

[0017] According to one aspect of the present application, a method for writing data to an all-flash memory is provided, comprising:

[0018] Divide the data to be written into N first data blocks and 1 second data block, wherein each of the data blocks does not overlap with each other, and the size of the first data block is consistent with the size of a single storage unit, and the size of the second data block is smaller than the size of a single storage unit;

[0019] compressing the second data block and at least one first data block (a total of M data blocks) into a third data block, so that the size of the third data block is smaller than the size of M-1 storage units;

[0020] The N-M+1 first data blocks that are not compressed and the third data block are written as a whole into the storage space in an aligned write manner.

[0021] Further, compressing the second data block and at least one first data block (a total of M data blocks) into a third data block, so that the size of the third data block is smaller than the size of M-1 storage units, includes:

[0022] Compression estimation is performed in order of the number of the first data blocks from small to large to obtain an estimated value of the third data block size, until the estimated value of the third data block size satisfies a size less than M-1 storage units, and compression is performed according to the number of data blocks participating in compression corresponding to the estimated value of the third data block size obtained at this time.

[0023] Further, writing the N-M+1 first data blocks not involved in the compression and the third data block as a whole into the storage space in an aligned write manner includes:

[0024] Obtaining a first vacant storage unit in the storage space, where the first storage unit is adjacent to other storage units storing data in the storage space;

[0025] Sequentially write N-M+1 of the first data blocks and the third data block into the first storage unit and other storage units after the first storage unit one by one, with the first storage unit as the starting point for writing data;

[0026] It is defined that the third data block is written into the second storage unit, and padding data is written into the remaining space of the second storage unit until the second storage unit is full.

[0027] Furthermore, before dividing the data to be written into N first data blocks and 1 second data block, the method includes:

[0028] Get the average size of a single file in the data to be written;

[0029] Using the average value as a reference, generating a capacity parameter of the single storage unit having a size equivalent to the average value;

[0030] The size of the single storage unit is adjusted according to the capacity parameter.

[0031] Further, compressing the second data block and at least one first data block, a total of M data blocks, into a third data block includes:

[0032] The number of M is determined according to the size of the single storage unit. The larger the single storage unit is, the larger M is.

[0033] Further, determining the number of M according to the size of the single storage unit includes:

[0034] When the size of the single storage unit is 4K or 8K, M=3;

[0035] When the size of the single storage unit is 16K, M=4;

[0036] When the size of the single storage unit is 32K or 64K, M=5;

[0037] When the size of the single storage unit is 128K, M=6;

[0038] The corresponding relationship between the size of the single storage unit and the number of M is pre-saved in a configuration file.

[0039] Furthermore, the compression algorithm used includes one or more of the Lz4 algorithm, the Snappy algorithm, the Zstd algorithm, and the Zlib algorithm, and the compression algorithm is pre-saved in the configuration file.

[0040] A second aspect of the present application provides an all-flash data writing device, comprising:

[0041] A data splitting module, used for dividing the data to be written into N first data blocks and 1 second data block, wherein each of the data blocks does not overlap with each other, and the size of the first data block is consistent with the size of a single storage unit, and the size of the second data block is smaller than the size of a single storage unit;

[0042] a compression module, configured to compress the second data block and at least one first data block (a total of M data blocks) into a third data block, so that the size of the third data block is smaller than the size of M-1 storage units;

[0043] A writing module is used to write the N-M+1 first data blocks that are not compressed and the third data block into the storage space as a whole in an aligned write manner.

[0044] The third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the method described in the first aspect of the present application.

[0045] The fourth aspect of the present application provides a computer-readable storage medium, which is used to store a computer program. When the computer program runs on a computer, the computer executes the method described in the first aspect of the present application.

[0046] In an embodiment of the present application, an all-flash data writing method is adopted, including: dividing the data to be written into N first data blocks and 1 second data block, each of which does not overlap with each other, and the size of the first data block is consistent with the size of a single storage unit, and the size of the second data block is smaller than the size of a single storage unit; compressing the second data block and at least one first data block, totaling M data blocks, into a third data block, so that the size of the third data block is smaller than the size of M-1 storage units; writing the N-M+1 first data blocks that are not compressed and the third data block as a whole into the storage space in an aligned write manner.

[0047] Through this application, the data is cut and compressed before executing the aligned write, and the aligned write is performed on this basis, while taking into account both the disk space utilization and the data writing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0049] Figure 1 It is a schematic diagram of a data storage distribution method in the prior art.

[0050] Figure 2 It is a schematic diagram of data writing using sequential writing in the prior art.

[0051] Figure 3 It is a schematic diagram of data writing using aligned writing in the prior art.

[0052] Figure 4 It is a schematic diagram of all-flash data writing according to an embodiment of the present application.

[0053] Figure 5 It is a flowchart of a method for writing all-flash data according to an embodiment of the present application.

[0054] Figure 6 It is a schematic diagram of the composition of an all-flash data writing device according to an embodiment of the present application.

[0055] Figure 7 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0056] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0057] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0058] An embodiment of the present application provides an all-flash data writing method, which is different from the sequential writing and aligned writing methods in the prior art. Instead, the method comprehensively considers the size relationship between a single storage unit of the disk and the data block to be written, and adaptively compresses the data block according to the size of each data block. The compression goal is to make the number of vacant storage units occupied by the compressed data block when writing it to the disk using the aligned writing method is less than when it is not compressed. Moreover, since the compression mechanism adopted in the present application will cause the utilization rate of the last storage unit when writing to the disk to be greater than the utilization rate of the last storage unit when writing without compression, the method of the present application can improve the utilization rate of the disk and ensure higher writing performance by using the aligned writing method.

[0059] like Figure 4 FIG. 1 is a schematic diagram of a method for writing data to an all-flash memory according to an embodiment of the present application. Figure 4 The green bar in the upper middle represents the data to be written. Figure 4 The long strip composed of multiple grid blocks on the right side of the middle represents the storage space, which includes multiple sequentially arranged storage units, each of which has the same size. Figure 4As shown, the storage space on the right side of the middle shows that the current storage unit of the first grid has been fully written, and the storage unit of the second grid has been partially written. According to the size of the above storage unit, the data to be written represented by the green strip is divided into three adjacent yellow, green and purple parts, as indicated by the arrow marked 1 in the figure, where the size of the yellow and green parts are the same as the size of a single storage unit, and the size of the purple part is smaller than a single storage unit. Then, according to the arrow marked 2, the two adjacent parts located at the end of the data to be written, namely the purple part and the green part, are first compressed and estimated. With reference to the compression rate range of the specific compression algorithm, the estimation is performed according to the minimum compression rate. As shown in the figure, the green part and the purple part are both reduced to a certain extent, which is used to illustrate the corresponding size after the compression estimation. At this time, it is determined whether the overall data size of the reduced green part and the purple part is less than one storage unit. If so, the compression has achieved the above compression goal in one compression. At this time, the green part and the purple part can be compressed together; if it is still larger than one storage unit at this time, the uncompressed yellow part, the green part and the purple part are compressed for the second time together to determine whether the overall data size of the three reduced parts is less than two storage units. If so, the yellow part, the green part and the purple part can be compressed together, and so on. Taking the compression in the figure as an example to achieve the compression target, the data to be written before compression requires 3 storage units to accommodate, and the storage unit that accommodates the purple part will be vacant, while the compressed data only requires 2 storage units to accommodate, and the overall size of the compressed yellow and purple data accounts for a larger proportion of a single storage unit than the purple part before compression. Therefore, the space utilization of the storage unit can be further improved. When writing in the aligned write mode, the last storage unit to be written must also be filled. If the data itself cannot be filled, some filling data is constructed to fill it until the storage unit is filled. That is, Figure 4 The black portion shown in represents the fill data.

[0060] Therefore, it can be seen from the above examples that an embodiment of the present invention provides a method for writing data to an all-flash memory, such as Figure 5 As shown, the method comprises the following steps:

[0061] Step S101, divide the data to be written into N first data blocks and 1 second data block, wherein each data block does not overlap with each other, and the size of the first data block is consistent with the size of a single storage unit, and the size of the second data block is smaller than the size of a single storage unit.

[0062] In the above example, the yellow part and the green part formed after the division are the first data block, that is, N=2, and the purple part is the second data block.

[0063] Step S102: compress the second data block and at least one first data block (a total of M data blocks) into a third data block, so that the size of the third data block is smaller than the size of M-1 storage units.

[0064] In the above example, the green part and the purple part are compressed together to form the third data block, that is, M=2, so the compression target is that the size of the third data block is smaller than the size of 1 storage unit.

[0065] Step S103: Write the N-M+1 first data blocks that are not compressed and the third data block as a whole into the storage space in an aligned write manner.

[0066] In the above example, the first data block that is not involved in the compression, that is, the yellow data block, that is, the whole formed by compressing the yellow data block, the color part and the purple part together, is written into the storage space in an aligned write manner.

[0067] In some preferred embodiments, compressing the second data block and at least one first data block (a total of M data blocks) into a third data block, so that the size of the third data block is smaller than the size of M-1 storage units, includes:

[0068] Compression estimation is performed in order of the number of the first data blocks from small to large to obtain an estimated value of the third data block size, until the estimated value of the third data block size satisfies a size less than M-1 storage units, and compression is performed according to the number of data blocks participating in compression corresponding to the estimated value of the third data block size obtained at this time.

[0069] Through the method of this embodiment, the data is divided into slices with reference to the size of the storage unit, and then the compression estimation method is used to calculate how many slices must be compressed at least to achieve the compression target. This estimation method is time-saving and will not cause excessive compression. It can improve storage space utilization while taking into account processing efficiency.

[0070] In some preferred embodiments, writing the N-M+1 first data blocks not involved in compression and the third data block as a whole into the storage space in an aligned write manner includes:

[0071] Step S1031: Obtain a first vacant storage unit in the storage space, where the first storage unit is adjacent to other storage units storing data in the storage space.

[0072] Step S1032: sequentially write N-M+1 first data blocks and the third data block into the first storage unit and other storage units after the first storage unit, block by block, with the first storage unit as the starting point for writing data.

[0073] Step S1033: define that the third data block is written into the second storage unit, and write the filling data into the remaining space of the second storage unit until the second storage unit is full.

[0074] Through the above-mentioned aligned writing method, data writing can be completed efficiently and all storage units from the first storage unit to before the second storage unit are filled, and the second storage unit is also filled with the third data block and the filling data.

[0075] In some optional embodiments, the padding data may be a combination of 0 and / or 1.

[0076] In some preferred embodiments, the compression algorithm used includes one or more of the Lz4 algorithm, the Snappy algorithm, the Zstd algorithm, and the Zlib algorithm, and the compression algorithm is pre-stored in the configuration file. The specific parameters and characteristics of each compression algorithm are shown in Table 1, and those skilled in the art can select according to their needs.

[0077] Table 1

[0078]

[0079] In some preferred embodiments, compressing the second data block and at least one first data block (a total of M data blocks) into a third data block comprises:

[0080] The number of M is determined according to the size of the single storage unit. The larger the single storage unit is, the larger M is.

[0081] In some preferred embodiments, determining the number of M according to the size of the single storage unit includes:

[0082] When the size of the single storage unit is 4K or 8K, M=3;

[0083] When the size of the single storage unit is 16K, M=4;

[0084] When the size of the single storage unit is 32K or 64K, M=5;

[0085] When the size of the single storage unit is 128K, M=6;

[0086] The corresponding relationship between the size of the single storage unit and the number of M is pre-saved in a configuration file.

[0087] In the embodiment of the present invention, the system assigns initial values ​​to the size of a single storage unit, the compression algorithm, and the number of M as compression parameters and saves them in the persistent memory. Although these data also occupy a certain space, they are negligible compared to the space wasted by traditional aligned writes. The initial value of the size of a single storage unit is 32K, the initial value of the compression algorithm is lz4, and the initial value of the number of M is 5.

[0088] Since the data to be written is divided with reference to the storage unit, once the size of the storage unit is determined, it will affect the specific division method of the data to be written and the subsequent compression measurement speed. In this embodiment, it is preferred to have fewer measurements to make the compression more efficient. In the case of different users and different data objects, the data to be written may be of different sizes. Therefore, based on this principle, in order to make most of the data to be written better fit a certain storage unit, in some preferred embodiments, before dividing the data to be written into N first data blocks and 1 second data block, it includes:

[0089] Step S1011, obtaining an average value of the size of a single file in the data to be written.

[0090] Step S1012: Using the average value as a reference, generate a capacity parameter of the single storage unit having a size equivalent to the average value.

[0091] Step S1013: adjusting the size of the single storage unit according to the capacity parameter.

[0092] Therefore, when the user is using it, during the data upload process, he can dynamically adjust the compression parameters through commands according to the data to be written, including adjusting the size of the above-mentioned storage unit, so as to further adjust the data segmentation, and further determine the size of M according to the storage unit size.

[0093] Table 2 below lists the test structures of the success ratio of compression under different storage unit sizes and different M values, where success means achieving the compression target. As can be seen from Table 2, for storage units of the same size, the larger the M value, the higher the compression success ratio, that is, the higher the utilization rate of the corresponding storage space. Therefore, for users who have higher requirements for the utilization rate of storage space, the M value that can make the success ratio at 100% should be given priority as the appropriate compression parameter.

[0094] Table 2

[0095]

[0096] Based on the same inventive concept as the above method embodiment, the present application also provides an all-flash data writing device, such as Figure 6FIG. 1 is a schematic diagram of an all-flash data writing device, comprising Figure 6 It can be known that the all-flash memory data writing device comprises:

[0097] A data splitting module, used for dividing the data to be written into N first data blocks and 1 second data block, wherein each of the data blocks does not overlap with each other, and the size of the first data block is consistent with the size of a single storage unit, and the size of the second data block is smaller than the size of a single storage unit;

[0098] a compression module, configured to compress the second data block and at least one first data block (a total of M data blocks) into a third data block, so that the size of the third data block is smaller than the size of M-1 storage units;

[0099] A writing module is used to write the N-M+1 first data blocks that are not compressed and the third data block into the storage space as a whole in an aligned write manner.

[0100] In some preferred embodiments, compressing the second data block and at least one first data block (a total of M data blocks) into a third data block, so that the size of the third data block is smaller than the size of M-1 storage units, includes:

[0101] Compression estimation is performed in order of the number of the first data blocks from small to large to obtain an estimated value of the third data block size, until the estimated value of the third data block size satisfies a size less than M-1 storage units, and compression is performed according to the number of data blocks participating in compression corresponding to the estimated value of the third data block size obtained at this time.

[0102] In some preferred embodiments, the writing module is used to write the N-M+1 first data blocks that are not compressed and the third data block as a whole into the storage space in an aligned write manner, including:

[0103] A first obtaining module, configured to obtain a first vacant storage unit in the storage space, wherein the first storage unit is adjacent to other storage units storing data in the storage space;

[0104] A sequential writing module, used for sequentially writing N-M+1 first data blocks and the third data block into the first storage unit and other storage units after the first storage unit one by one, with the first storage unit as the starting point for writing data;

[0105] A filling module is used to define that the third data block is written into the second storage unit, and write the filling data into the remaining space of the second storage unit until the second storage unit is full.

[0106] In some preferred embodiments, the data splitting module, before being used to divide the data to be written into N first data blocks and 1 second data block, includes:

[0107] The mean calculation module is used to obtain the average value of the size of a single file in the data to be written;

[0108] A parameter calculation module, used for generating a capacity parameter of the single storage unit having a size equivalent to the average value by taking the average value as a reference;

[0109] A capacity adjustment module is used to adjust the size of the single storage unit according to the capacity parameter.

[0110] In some preferred embodiments, the compression module is used to compress the second data block and at least one first data block, totaling M data blocks, into a third data block, including:

[0111] The quantity determination module is used to determine the number of M according to the size of the single storage unit. The larger the single storage unit is, the larger the M is.

[0112] In some preferred embodiments, the quantity determination module is used to determine the quantity of M according to the size of the single storage unit, including:

[0113] When the size of the single storage unit is 4K or 8K, M=3;

[0114] When the size of the single storage unit is 16K, M=4;

[0115] When the size of the single storage unit is 32K or 64K, M=5;

[0116] When the size of the single storage unit is 128K, M=6;

[0117] The corresponding relationship between the size of the single storage unit and the number of M is pre-saved in a configuration file.

[0118] In some preferred embodiments, the compression algorithm used includes one or more of the Lz4 algorithm, the Snappy algorithm, the Zstd algorithm, and the Zlib algorithm, and the compression algorithm is pre-saved in a configuration file.

[0119] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0120] Based on the same inventive concept as the above-mentioned method embodiment, an electronic device is also provided in an embodiment of the present application, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the all-flash data method in the above-mentioned embodiment.

[0121] In one embodiment, the electronic device may be a server. In this embodiment, the structure of the electronic device may be as follows: Figure 7 As shown, it includes a memory 2001 , a communication module 2003 and one or more processors 2002 .

[0122] The memory 2001 is used to store computer programs executed by the processor 2002. The memory 2001 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0123] The memory 2001 may be a volatile memory, such as a random-access memory (RAM); the memory 2001 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 2001 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2001 may be a combination of the above memories.

[0124] The processor 2002 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 2002 is configured to implement the above-mentioned audio data processing method when calling the computer program stored in the memory 2001 .

[0125] The communication module 2003 is used to communicate with terminal devices and other servers.

[0126] The specific connection medium between the memory 2001, the communication module 2003 and the processor 2002 is not limited in the embodiment of the present application. Figure 7 In the embodiment, the memory 2001 and the processor 2002 are connected via a bus 2004. The bus 2004 is Figure 7 The arrows in the figure are used to describe the connection between other components, which is only for illustrative purposes and is not intended to be limiting. Bus 2004 can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 7 The diagram shows that only one arrow is used, but this does not mean that there is only one bus or only one type of bus.

[0127] Based on the same inventive concept as the above method embodiment, an embodiment of the present invention also provides a computer-readable storage medium, which is used to store a computer program. When the computer program is run on a computer, the electronic device implements the control method in the above embodiment. The computer-readable storage medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0128] Based on the same inventive concept as the above-mentioned method embodiment, an embodiment of the present invention further provides a computer program product, the computer program product including a computer program. When the program product is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of the control method according to various exemplary embodiments of the present application described above in this specification. The program product may adopt any combination of one or more readable media. These computer program commands may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the commands executed by the processor of the computer or other programmable data processing device generate commands for implementing the steps in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0129] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

Claims

1. A method for writing data to an all-flash memory, characterized in that: Divide the data to be written into N first data blocks and 1 second data block, wherein each of the data blocks does not overlap with each other, and the size of the first data block is consistent with the size of a single storage unit, and the size of the second data block is smaller than the size of a single storage unit; compressing the second data block and at least one first data block (a total of M data blocks) into a third data block, so that the size of the third data block is smaller than the size of M-1 storage units; Writing the N-M+1 first data blocks that are not compressed and the third data block as a whole into the storage space in an aligned write manner; When the size of the single storage unit is 4K or 8K, M=3; When the size of the single storage unit is 16K, M=4; When the size of the single storage unit is 32K or 64K, M=5; When the size of the single storage unit is 128K, M=6.

2. The method according to claim 1, characterized in that The step of compressing the second data block and at least one first data block (a total of M data blocks) into a third data block, so that the size of the third data block is smaller than the size of M-1 storage units, comprises: Compression estimation is performed in order of the number of the first data blocks from small to large to obtain an estimated value of the third data block size, until the estimated value of the third data block size satisfies a size less than M-1 storage units, and compression is performed according to the number of data blocks participating in compression corresponding to the estimated value of the third data block size obtained at this time.

3. The method according to claim 1, characterized in that The step of writing the N-M+1 first data blocks not involved in the compression and the third data block as a whole into the storage space in an aligned write manner includes: Obtaining a first vacant storage unit in the storage space, where the first storage unit is adjacent to other storage units storing data in the storage space; Sequentially write N-M+1 of the first data blocks and the third data block into the first storage unit and other storage units after the first storage unit one by one, with the first storage unit as the starting point for writing data; It is defined that the third data block is written into the second storage unit, and padding data is written into the remaining space of the second storage unit until the second storage unit is full.

4. The method according to claim 1, characterized in that: Before dividing the data to be written into N first data blocks and 1 second data block, the method includes: Get the average size of a single file in the data to be written; Using the average value as a reference, generating a capacity parameter of the single storage unit having a size equivalent to the average value; The size of the single storage unit is adjusted according to the capacity parameter.

5. The method according to claim 4, characterized in that The compressing the second data block and at least one first data block (a total of M data blocks) into a third data block comprises: The number of M is determined according to the size of the single storage unit. The larger the single storage unit is, the larger M is.

6. The method according to claim 5, characterized in that The determining the number of M according to the size of the single storage unit includes: The corresponding relationship between the size of the single storage unit and the number of M is pre-saved in a configuration file.

7. The method according to any one of claims 1 to 6, characterized in that: The compression algorithm used includes one or more of the Lz4 algorithm, the Snappy algorithm, the Zstd algorithm, and the Zlib algorithm, and the compression algorithm is pre-saved in the configuration file.

8. An all-flash data writing device, characterized in that: include: A data splitting module, used for dividing the data to be written into N first data blocks and 1 second data block, wherein each of the data blocks does not overlap with each other, and the size of the first data block is consistent with the size of a single storage unit, and the size of the second data block is smaller than the size of a single storage unit; a compression module, configured to compress the second data block and at least one first data block (a total of M data blocks) into a third data block, so that the size of the third data block is smaller than the size of M-1 storage units; A writing module, used for writing the N-M+1 first data blocks not involved in compression and the third data block as a whole into the storage space in an aligned writing manner; When the size of the single storage unit is 4K or 8K, M=3; When the size of the single storage unit is 16K, M=4; When the size of the single storage unit is 32K or 64K, M=5; When the size of the single storage unit is 128K, M=6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the electronic device implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images