Data writing method and device, storage medium and electronic device

Abstract

An embodiment of the present application provides a data writing method and device, a storage medium and an electronic device, including: determining a writing mode for writing target data to a storage device, wherein the writing mode includes: create writing and overwrite writing; determining a target data allocator corresponding to the target data according to the writing mode, wherein the target data allocator includes: a primary allocator and / or a secondary allocator, the primary allocator is used to allocate a first storage space in the storage device, the secondary allocator is used to allocate a second storage space in the storage device, the first storage space is used to indicate free space in the storage device, and the second storage space is used to indicate storage space after data has been released in the storage device; based on the target data allocator, writing the target data to the target storage space in the storage device.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of big data, and more specifically, to a method and device for writing data, a storage medium, and an electronic device. Background Art

[0002] At present, the existing writing process of distributed storage adopts the append writing method. In the append writing method, both creation writing and overwriting writing are first applied from the unused space. The entire space address will be used up in a round in the process of writing data. After it is used up, the space for released data will be searched again from the initial position. At this time, the space address released in the overwriting process will be used.

[0003] The problem of low data writing performance caused by applying for space from unused storage space for both creation and overwriting in the related technology and then releasing invalid data in the storage device after the storage space in the storage device is used up has not been effectively solved. Summary of the invention

[0004] The embodiments of the present application provide a data writing method and system, a storage medium and an electronic device, so as to at least solve the problem of low data writing performance caused by applying for space from unused storage space for both creation writing and overwriting writing in the related art, and then releasing invalid data in the storage device after the storage space in the storage device is used up for a round.

[0005] According to one embodiment of the present application, a method for writing data is provided, the method comprising: determining a writing mode for writing target data to a storage device, wherein the writing mode comprises: create writing and overwrite writing; determining a target data allocator corresponding to the target data according to the writing mode, wherein the target data allocator comprises: a primary allocator and / or a secondary allocator, the primary allocator being used to allocate a first storage space in the storage device, the secondary allocator being used to allocate a second storage space in the storage device, the first storage space being used to indicate free space in the storage device, and the second storage space being used to indicate storage space after data has been released in the storage device; and writing the target data to the target storage space in the storage device based on the target data allocator.

[0006] In an exemplary embodiment, determining the target data allocator corresponding to the target data according to the write mode includes: when determining that the write mode is the create write, determining the target data allocator corresponding to the target data is the primary allocator; when determining that the write mode is the overwrite write, determining the target data allocator corresponding to the target data is the secondary allocator.

[0007] In an exemplary embodiment, allocating the target data to the target storage space in the storage device based on the target data allocator includes: determining a first space amount corresponding to the target data and determining a second space amount of the first storage space when the write mode is determined to be the create write; determining a first size relationship between the first space amount and the second space amount; and allocating the target data to the target storage space according to the first size relationship and the target data allocator.

[0008] In an exemplary embodiment, the target data is allocated to the target storage space according to the first size relationship and the target data allocator, including: when the first size relationship indicates that the first space amount is less than or equal to the second space amount, the target data is written to the first storage space based on the main allocator; when the first size relationship indicates that the first space amount is greater than the second space amount, a first difference between the first space amount and the second space amount is determined; based on the main allocator, the first data in the target data is written to the first storage space, and based on the secondary allocator, the second data in the target data is written to the second storage space, wherein the space amount corresponding to the first data is the second space amount, and the space amount corresponding to the second data is the first difference.

[0009] In an exemplary embodiment, allocating the target data to the target storage space in the target storage device based on the target data allocator includes: when determining that the write mode is the overwrite write, determining a first space amount corresponding to the target data, and determining a third space amount of the second storage space; determining a second size relationship between the first space amount and the third space amount; and allocating the target data to the target storage space according to the second size relationship.

[0010] In an exemplary embodiment, allocating the target data to the target storage space according to the second size relationship includes: when the second size relationship indicates that the first amount of space is less than or equal to the third amount of space, writing the target data to the second storage space based on the secondary allocator; when the second size relationship indicates that the first amount of space is greater than the third amount of space, determining a second difference between the first amount of space and the third amount of space; writing the third data in the target data to the second storage space based on the secondary allocator, and writing the fourth data in the target data to the first storage space based on the primary allocator, wherein the amount of space corresponding to the third data is the third amount of space, and the amount of space corresponding to the fourth data is the second difference.

[0011] In an exemplary embodiment, after allocating the target data to the target storage space according to the second size relationship, the method also includes: determining a fourth space amount in which data has been written in the storage device; determining a third size relationship between the fourth space amount and the first preset space amount; and updating the second storage space according to the third size relationship.

[0012] In an exemplary embodiment, updating the second storage space according to the third size relationship includes: determining a first number of write commands received by the storage device within a target time period, and determining a second number of storage space allocated in the storage device based on the first number of write commands, when the third size relationship indicates that the fourth space amount is less than or equal to the first preset space amount; calculating a second ratio of the second number to the first number, and determining a fourth size relationship between the second ratio and the first preset ratio; and updating the second storage space according to the fourth size relationship.

[0013] In an exemplary embodiment, updating the second storage space according to the fourth size relationship includes: when the fourth size relationship indicates that the second ratio is less than or equal to the first preset ratio, determining the maximum space amount of the second storage space according to the sum of the fourth space amount and the third space amount, and determining sixth data in the storage device according to the target data, wherein the sixth data is data that is partially or completely the same as the target data; determining the third storage space for the sixth data in the storage device; merging the second storage space and the third storage space to generate a fourth storage space; determining a fifth space amount of the fourth storage space; and updating the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount.

[0014] In an exemplary embodiment, updating the second storage space according to the fourth size relationship also includes: determining the maximum space amount of the second storage space according to the fourth space amount when the fourth size relationship indicates that the second ratio is greater than the first preset ratio; and updating the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount.

[0015] In an exemplary embodiment, updating the second storage space according to the third size relationship includes: determining a first number of write commands received by the storage device within a target time period, and determining a second number of storage space allocated in the storage device based on the first number of write commands, when the third size relationship indicates that the fourth space amount is greater than the first preset space amount and the fourth space amount is less than the second preset space amount; calculating a second ratio of the second number to the first number and determining a sixth size relationship between the second ratio and the second preset ratio; and updating the second storage space according to the sixth size relationship.

[0016] In an exemplary embodiment, updating the second storage space according to the sixth size relationship includes: when the sixth size relationship indicates that the second ratio is less than or equal to the second preset ratio, determining the maximum space amount of the second storage space according to the fourth space amount, and determining sixth data consistent with the target data in the storage device; determining a third storage space for the sixth data in the storage device; merging the second storage space and the third storage space to generate a fourth storage space; determining a fifth space amount of the fourth storage space; and updating the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount.

[0017] In an exemplary embodiment, updating the second storage space according to the sixth size relationship includes: determining the maximum space amount of the second storage space according to the sum of the fourth space amount and the third space amount when the sixth size relationship indicates that the second ratio is greater than the second preset ratio; and updating the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount.

[0018] In an exemplary embodiment, updating the second storage space according to the third size relationship includes: when the third size relationship indicates that the fourth space amount is greater than the first preset space amount, and the fourth space amount is greater than the second preset space amount, determining the target preset space amount as the maximum space amount of the second storage space, and determining sixth data consistent with the target data in the storage device; determining a third storage space in the storage device for the sixth data; merging the second storage space and the third storage space to generate a fourth storage space; determining a fifth space amount of the fourth storage space; and updating the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount.

[0019] In an exemplary embodiment, updating the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount includes: when the fifth size relationship indicates that the fifth space amount is greater than the maximum space amount, dividing the fourth storage space into a fifth storage space and a sixth storage space, wherein the space amount of the fifth storage space is the maximum space amount, and the space amount of the sixth storage space is the third difference between the fifth space amount and the maximum space amount; updating the second storage space according to the fifth storage space, and updating the first storage space according to the sixth storage space; and updating the second storage space according to the fourth storage space when the fifth size relationship indicates that the fifth space amount is less than or equal to the maximum space amount.

[0020] In an exemplary embodiment, after the target data allocator writes the target data to the target storage space in the storage device, the method further includes: when the storage device receives a delete command, deleting the seventh data in the storage space according to the delete command, wherein the delete command is used to instruct the deletion of the seventh data in the storage device; determining the seventh storage space allowed to be allocated by the secondary allocator, and determining the sixth space amount of the seventh storage space; determining whether the sixth space amount is greater than the third preset space amount; when it is determined that the sixth space amount is greater than or equal to the third preset space amount, determining the eighth storage space corresponding to the seventh data as the storage space to be allocated by the primary allocator; when it is determined that the sixth space amount is less than the third preset space amount, determining the eighth storage space as the storage space to be allocated by the secondary allocator.

[0021] According to another embodiment of the present application, a data writing device is provided, comprising: a first determination module, used to determine a writing method of target data into a storage device, wherein the writing method comprises: create write and overwrite write; a second determination module, used to determine a target data allocator corresponding to the target data according to the writing method, wherein the target data allocator comprises: a primary allocator and / or a secondary allocator, the primary allocator being used to allocate a first storage space in the storage device, the secondary allocator being used to allocate a second storage space in the storage device, the first storage space being used to indicate free space in the storage device, and the second storage space being used to indicate storage space after data has been released in the storage device; an allocation module, used to write the target data to a target storage space in the storage device based on the target data allocator.

[0022] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer process is stored, wherein the computer process is configured to execute the steps in the above method embodiment when running.

[0023] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer process, and the processor is configured to run the computer process to execute the steps in the above method embodiment.

[0024] According to another embodiment of the present application, a computer process product is provided, including a computer process, wherein the computer process implements the steps in the above method embodiment when executed by a processor.

[0025] Through the data writing method of the present application, the writing mode of the target data is determined to be create write or overwrite write, and the target data distributor corresponding to the target data is determined according to different writing modes. The target data distributor includes: a main distributor and / or a secondary distributor, the main distributor is used to allocate free space in the storage device, and the secondary distributor is used to allocate the storage device after the data has been released in the storage device. The target data is written to the target storage space in the storage device based on the main distributor and / or the secondary distributor. That is to say, the embodiment of the present application sets two target data distributors: a main distributor and a secondary distributor, and the two target data distributors respectively allocate different storage spaces in the storage device, wherein the secondary distributor can allocate the space released after the data has been written but the data is invalid, and then when the data is written, the data can be directly written from the unused free space in the storage device, or the data can be written from the released space. The problem of low data writing performance caused by the creation write and overwrite write in the related art is solved. It makes it possible to release the garbage data as soon as possible and improve the data writing performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a hardware structure block diagram of a computer terminal of a method for writing data in an embodiment of the present application;

[0027] Figure 2 is a flow chart of a method for writing data according to an embodiment of the present application;

[0028] Figure 3 It is a schematic diagram created and written according to the related technology of the optional embodiment of the present application;

[0029] Figure 4 It is a schematic diagram of overwriting in the related art according to an optional embodiment of the present application;

[0030] Figure 5 It is a schematic diagram of creating a writing according to an optional embodiment of the present application;

[0031] Figure 6 is a schematic diagram of overlay writing according to an optional embodiment of the present application;

[0032] Figure 7 is a schematic diagram (I) of allocation of free space size of a secondary allocator according to an optional embodiment of the present application;

[0033] Figure 8 is a schematic diagram (II) of allocation of free space size of a secondary allocator according to an optional embodiment of the present application;

[0034] Fig. 9 is a schematic diagram (III) of allocation of free space size of a secondary allocator according to an optional embodiment of the present application;

[0035] Fig.10 It is a structural block diagram of a data writing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0038] The method embodiments provided in the embodiments of the present application can be applied to an operating system, and the operating system can be located in a computer terminal or a similar computing device for execution. Taking running on a computer terminal as an example, Figure 1 1 is a hardware structure block diagram of a computer terminal of a data writing method according to an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned computer terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0039] The memory 104 can be used to store computer processes, for example, software processes and modules of application software, such as the computer process corresponding to the method for determining the interactive state in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer process stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0040] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0041] Figure 2 is a flow chart of a method for writing data according to an embodiment of the present application, which is executed in Figure 1 In a computer terminal, such as Figure 2 As shown, the process includes the following steps:

[0042] Step S202, determining a writing mode for writing target data into a storage device, wherein the writing mode includes: creating writing and overwriting writing;

[0043] Step S204, determining a target data allocator corresponding to the target data according to the write mode, wherein the target data allocator includes: a primary allocator and / or a secondary allocator, the primary allocator is used to allocate a first storage space in the storage device, the secondary allocator is used to allocate a second storage space in the storage device, the first storage space is used to indicate free space in the storage device, and the second storage space is used to indicate storage space after data has been released in the storage device;

[0044] Step S206: writing the target data into the target storage space in the storage device based on the target data allocator.

[0045] It can be understood that the target data can be written to the target storage space in the storage device only through the primary allocator, or only through the secondary allocator; the target data can also be written to the target storage space in the storage device through both the primary allocator and the secondary allocator.

[0046] Through the data writing method of the present application, the writing mode of the target data is determined to be create write or overwrite write, and the target data distributor corresponding to the target data is determined according to different writing modes. The target data distributor includes: a main distributor and / or a secondary distributor, the main distributor is used to allocate free space in the storage device, and the secondary distributor is used to allocate the storage device after the data has been released in the storage device. The target data is written to the target storage space in the storage device based on the main distributor and / or the secondary distributor. That is to say, the embodiment of the present application sets two target data distributors: a main distributor and a secondary distributor, and the two target data distributors respectively allocate different storage spaces in the storage device, wherein the secondary distributor can allocate the space released after the data has been written but the data is invalid, and then when the data is written, the data can be directly written from the unused free space in the storage device, or the data can be written from the released space. The problem of low data writing performance caused by the creation write and overwrite write in the related art is solved. It makes it possible to release the garbage data as soon as possible and improve the data writing performance.

[0047] Optionally, the above step S204 determines the target data allocator corresponding to the target data according to the writing method, including: when it is determined that the writing method is the create write, determining that the target data allocator corresponding to the target data is the primary allocator; when it is determined that the writing method is the overwrite write, determining that the target data allocator corresponding to the target data is the secondary allocator.

[0048] It can be understood that different writing methods correspond to different allocators, among which: create writing corresponds to the main allocator. When the space allocable by the main allocator is insufficient, space can be allocated from the secondary allocator; overwrite writing corresponds to the secondary allocator. When the space allocable by the secondary allocator is insufficient, space can be allocated from the main allocator.

[0049] Through the above technical solution, the allocators corresponding to the creation write and the overwrite write are different, which can improve the replication rate of the storage device.

[0050] Optionally, the above-mentioned step S206 of allocating the target data to the target storage space in the storage device based on the target data allocator includes: when determining that the write mode is the create write, determining a first space amount corresponding to the target data, and determining a second space amount of the first storage space; determining a first size relationship between the first space amount and the second space amount; and allocating the target data to the target storage space according to the first size relationship and the target data allocator.

[0051] Among them, allocating the target data to the target storage space according to the first size relationship and the target data allocator includes: when the first size relationship indicates that the first space amount is less than or equal to the second space amount, writing the target data to the first storage space based on the main allocator; when the first size relationship indicates that the first space amount is greater than the second space amount, determining a first difference between the first space amount and the second space amount; writing the first data in the target data to the first storage space based on the main allocator, and writing the second data in the target data to the second storage space based on the secondary allocator, wherein the space amount corresponding to the first data is the second space amount, and the space amount corresponding to the second data is the first difference.

[0052] It is understandable that, when the write mode is create write, the first space amount of the target data and the second space amount of the first storage space can be determined. Then, it is determined whether the first storage space has sufficient space to write the target data by the first space amount and the second space amount. When it is determined that the first storage space does not have sufficient space, the second data of the target data that is not written into the first storage space can be written into the second storage space.

[0053] That is to say, when the write mode is determined to be create write, there may be a situation where the target data is written to the target storage space only through the primary allocator. At this time, the target storage space is the first storage space; there may also be a situation where the target data is written to the target storage space through the primary allocator and the secondary allocator. At this time, the target storage space is the first storage space and the second storage space.

[0054] Optionally, the above-mentioned step S206 of allocating the target data to the target storage space in the target storage device based on the target data allocator includes: when determining that the write mode is the overwrite write, determining the first space amount corresponding to the target data, and determining the third space amount of the second storage space; determining a second size relationship between the first space amount and the third space amount; and allocating the target data to the target storage space according to the second size relationship.

[0055] Among them, allocating the target data to the target storage space according to the second size relationship includes: when the second size relationship indicates that the first space amount is less than or equal to the third space amount, writing the target data to the second storage space based on the secondary allocator; when the second size relationship indicates that the first space amount is greater than the third space amount, determining the second difference between the first space amount and the third space amount; writing the third data in the target data to the second storage space based on the secondary allocator, and writing the fourth data in the target data to the first storage space based on the main allocator, wherein the space amount corresponding to the third data is the third space amount, and the space amount corresponding to the fourth data is the second difference.

[0056] Among them, after allocating the target data to the target storage space according to the second size relationship, the method also includes: determining a fourth space amount in which data has been written in the storage device; determining a third size relationship between the fourth space amount and the first preset space amount; and updating the second storage space according to the third size relationship.

[0057] It is understandable that, when the writing mode is overwriting, the first space amount of the target data and the third space amount of the second storage space can be determined. Then, it is determined whether the second storage space has sufficient space to write the target data by the first space amount and the third space amount. When it is determined that the second storage space does not have sufficient space, the fourth data of the target data that has not been written into the second storage space can be written into the first storage space.

[0058] That is to say, when the write method is determined to be overwrite, there may be a situation where the target data is written to the target storage space only through the secondary allocator. At this time, the target storage space is the second storage space; there may also be a situation where the target data is written to the target storage space through the main allocator and the secondary allocator. At this time, the target storage space is the first storage space and the second storage space.

[0059] Through the above technical solution, in the case of create write and overwrite write, space in different storage devices is allocated through different allocators respectively, thereby realizing fast writing and overwriting of data and improving system performance and stability.

[0060] Optionally, the updating of the second storage space according to the third size relationship includes the following three updating schemes, which may include a first scheme in which the fourth space amount is less than or equal to the first preset space amount, a second scheme in which the fourth space amount is greater than the first preset space and less than the second preset space, and a third scheme in which the fourth space amount is greater than or equal to the second preset space amount, specifically:

[0061] (1) When the third size relationship indicates that the fourth amount of space is less than or equal to the first preset amount of space, determining a first number of write commands received by the storage device within a target time period, and determining a second amount of storage space allocated in the storage device based on the first number of write commands; calculating a second ratio of the second number to the first number, and determining a fourth size relationship between the second ratio and the first preset ratio; and updating the second storage space according to the fourth size relationship.

[0062] Among them, updating the second storage space according to the fourth size relationship includes: when the fourth size relationship indicates that the second ratio is less than or equal to the first preset ratio, determining the maximum space amount of the second storage space according to the sum of the fourth space amount and the third space amount, and determining sixth data in the storage device according to the target data, wherein the sixth data is data that is partially or completely the same as the target data; determining the third storage space for the sixth data in the storage device; merging the second storage space and the third storage space to generate a fourth storage space; determining the fifth space amount of the fourth storage space; and updating the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount.

[0063] Among them, updating the second storage space according to the fourth size relationship also includes: when the fourth size relationship indicates that the second ratio is greater than the first preset ratio, determining the maximum space amount of the second storage space according to the fourth space amount; updating the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount.

[0064] It can be understood that the third amount of space is the amount of space of the second storage space; the fourth amount of space is the amount of space in the storage device where data has been written; and the fifth amount of space is the amount of space of the fourth storage space after merging the third storage space corresponding to the overwritten old data with the second storage space.

[0065] In the case where the fourth space amount is less than or equal to the first preset space amount, since the amount of data to be written corresponding to a write command may not be completely written in one storage space of the storage device, for example: the write command is to write 10 space amounts of data, but there is no area greater than or equal to 10 space amounts in the storage device, it is necessary to determine fragments in the storage device and write the 10 space amounts of data into multiple fragments (for example: write into three fragments), and the second amount of storage space to be allocated is 3. That is, the first number of write commands received by the storage device and the second number of storage spaces allocated in the storage device can be determined.

[0066] For example, the storage device receives 2 write commands within 10 minutes, the amount of storage space to be allocated is 10, and the first preset ratio is 3. It can be determined that the second ratio is greater than 3.

[0067] When the second ratio is greater than 3, the maximum space amount of the second storage space may be determined according to the fourth space amount of written data. For example, half of the fourth space amount may be determined as the maximum space amount of the second storage space.

[0068] For example, if the storage device receives 5 write commands within 10 minutes, the amount of storage space to be allocated is 10, and the first preset ratio is 3, it can be determined that the second ratio is less than 3.

[0069] When the second ratio is less than 3, the maximum space amount of the second storage space can be determined according to the sum of the fourth space amount and the third space amount. For example, the sum of the fourth space amount and the third space amount can be directly determined as the maximum space amount of the second storage device.

[0070] Controlling the second storage space corresponding to the secondary allocator can effectively manage the storage space, improve the performance and utilization of the storage device, and ensure the security and reliability of data.

[0071] When the maximum space amount of the second storage space is determined, the third storage space occupied by the old data in the corresponding storage space can be determined, and the third storage space and the second storage space are combined to obtain a fourth storage space. Then, a fifth size relationship between the fifth space amount of the fourth storage space and the maximum space amount is determined, so as to update the second storage space according to the fifth size relationship.

[0072] (2) When the third size relationship indicates that the fourth space amount is greater than the first preset space amount and the fourth space amount is less than the second preset space amount, determine a first number of write commands received by the storage device within a target time period, and determine a second number of storage space allocated in the storage device based on the first number of write commands; calculate a second ratio of the second number to the first number, and determine a sixth size relationship between the second ratio and a second preset ratio; and update the second storage space according to the sixth size relationship.

[0073] Among them, updating the second storage space according to the sixth size relationship includes: when the sixth size relationship indicates that the second ratio is less than or equal to the second preset ratio, determining the maximum space amount of the second storage space according to the fourth space amount, and determining sixth data consistent with the target data in the storage device; determining the third storage space of the sixth data in the storage device; merging the second storage space and the third storage space to generate a fourth storage space; determining the fifth space amount of the fourth storage space; and updating the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount.

[0074] Among them, updating the second storage space according to the sixth size relationship includes: when the sixth size relationship indicates that the second ratio is greater than the second preset ratio, determining the maximum space amount of the second storage space according to the sum of the fourth space amount and the third space amount; updating the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount.

[0075] In the case where the fourth amount of space is greater than the first preset amount of space, it can be determined whether the fourth amount of space is less than the second preset amount of space. In the case where it is determined that the fourth amount of space is less than the second preset amount of space, the first quantity and the second quantity can be determined, and then the storage space is updated. For example: the write command is to write 10 amounts of space data, and there is no area greater than or equal to 10 amounts of space in the storage device. At this time, it is necessary to determine the fragments in the storage device and write the 10 amounts of space data into multiple fragments (for example, write 2 fragments) respectively. At this time, the second amount of storage space to be allocated is 2. That is, the first number of write commands received by the storage device and the second number of storage space allocated in the storage device can be determined.

[0076] For example, if the storage device receives one write command within 10 minutes, the amount of storage space to be allocated is 10, and the second preset ratio is 7, it can be determined that the second ratio is greater than 7.

[0077] When the second ratio is greater than 7, the preset space amount of the second storage space may be determined according to the sum of the fourth space amount and the third space amount of written data. For example, half of the sum of the third space amount and the fourth space amount may be determined as the maximum space amount of the second storage space.

[0078] For example, the storage device receives 4 write commands within 10 minutes, the amount of storage space to be allocated is 10, and the second preset ratio is 7. It can be determined that the second ratio is less than 7.

[0079] When the second ratio is less than 7, the maximum space amount of the second storage space may be determined according to the fourth space amount of written data. For example, one third of the fourth space amount may be determined as the maximum space amount of the second storage space.

[0080] When the maximum space amount of the second storage space is determined, the third storage space occupied by the old data in the corresponding storage space can be determined, and the third storage space and the second storage space are combined to obtain a fourth storage space. Then, a fifth size relationship between the fifth space amount of the fourth storage space and the maximum space amount is determined, so as to update the second storage space according to the fifth size relationship.

[0081] (3) When the third size relationship indicates that the fourth space amount is greater than the first preset space amount, and the fourth space amount is greater than the second preset space amount, the target preset space amount is determined as the maximum space amount of the second storage space, and sixth data consistent with the target data is determined in the storage device; a third storage space for the sixth data in the storage device is determined; the second storage space and the third storage space are merged to generate a fourth storage space; a fifth space amount of the fourth storage space is determined; and the second storage space is updated according to the fifth size relationship between the fifth space amount and the maximum space amount.

[0082] When the fourth space amount is greater than the second preset space amount, the target preset space amount may be directly determined as the maximum space amount of the second storage space, for example, 2% of the total space amount of the storage device.

[0083] When the maximum space amount of the second storage space is determined, the third storage space occupied by the old data in the corresponding storage space can be determined, and the third storage space and the second storage space are combined to obtain a fourth storage space. Then, a fifth size relationship between the fifth space amount of the fourth storage space and the maximum space amount is determined, so as to update the second storage space according to the fifth size relationship.

[0084] Among them, the "updating the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount" in the above three update schemes includes: when the fifth size relationship indicates that the fifth space amount is greater than the maximum space amount, dividing the fourth storage space into a fifth storage space and a sixth storage space, wherein the space amount of the fifth storage space is the maximum space amount, and the space amount of the sixth storage space is the third difference between the fifth space amount and the maximum space amount; updating the second storage space according to the fifth storage space, and updating the first storage space according to the sixth storage space; when the fifth size relationship indicates that the fifth space amount is less than or equal to the maximum space amount, updating the second storage space according to the fourth storage space.

[0085] It can be understood that after the fifth size relationship is determined by the above three methods, the second storage space can be updated according to the fifth size relationship:

[0086] When the fifth amount of space is greater than the maximum amount of space, it can be determined that the amount of space that can be allocated by the secondary allocator is greater than the maximum amount of space. At this time, only the fifth storage space with the maximum amount of space is retained as the second storage space of the secondary allocator, and the sixth storage space will be merged with the first storage space to update the first storage space.

[0087] When the fifth space amount is less than or equal to the maximum space amount, it can be determined that the space amount that can be allocated by the secondary allocator is less than or equal to the maximum space amount, and the fourth storage space can be determined as the second storage space of the secondary allocator.

[0088] Through the embodiments of the present application, the main allocator can directly use the excess space of the secondary allocator for allocation, saving the time and overhead of reapplying for and releasing memory, thereby improving the speed and efficiency of memory allocation.

[0089] Optionally, after the target data allocator writes the target data to the target storage space in the storage device in the above step S206, the method further includes: when the storage device receives a delete command, deleting the seventh data in the storage space according to the delete command, wherein the delete command is used to instruct the deletion of the seventh data in the storage device; determining the seventh storage space allowed to be allocated by the secondary allocator, and determining the sixth space amount of the seventh storage space; determining whether the sixth space amount is greater than the third preset space amount; when it is determined that the sixth space amount is greater than or equal to the third preset space amount, determining the eighth storage space corresponding to the seventh data as the storage space to be allocated by the primary allocator; when it is determined that the sixth space amount is less than the third preset space amount, determining the eighth storage space as the storage space to be allocated by the secondary allocator.

[0090] It is understandable that, in addition to the above three solutions, it is also necessary to consider whether the first storage space and the second storage space corresponding to the primary allocator and the secondary allocator need to be updated when deleting the data in the storage space. Specifically:

[0091] In the case of deleting data in the storage space, it can be determined that the data in the storage space is constantly decreasing. At this time, the second storage space can be updated according to the third size relationship, including the following three updating schemes, which can include a first scheme in which the fourth space amount is less than or equal to the first preset space amount, a second scheme in which the fourth space amount is greater than the first preset space and less than the second preset space, and a third scheme in which the fourth space amount is greater than or equal to the second preset space amount, specifically:

[0092] (1) When the third size relationship indicates that the fourth amount of space is less than or equal to the first preset amount of space, determine a third number of deletion commands received by the storage device within a target time period, and determine a fourth amount of storage space added in the storage device based on the third number of deletion commands; calculate a third ratio of the fourth number to the third number, and determine a seventh size relationship between the third ratio and the third preset ratio; and update the second storage space according to the seventh size relationship.

[0093] Among them, updating the second storage space according to the seventh size relationship includes: when the seventh size relationship indicates that the third ratio is less than or equal to the third preset ratio, determining the seventh space amount of the written data of the storage device after deleting the seventh data; determining the maximum space amount of the second storage space according to the sum of the sixth space amount and the seventh space amount, and determining the eighth space amount of the eighth storage space; merging the eighth storage space and the second storage space to generate a ninth storage space; determining the ninth space amount of the ninth storage space; and updating the second storage space according to the eighth size relationship between the ninth space amount and the maximum space amount.

[0094] Among them, updating the second storage space according to the seventh size relationship also includes: when the seventh size relationship indicates that the third ratio is greater than the third preset ratio, determining the maximum space amount of the second storage space according to the seventh space amount; updating the second storage space according to the eighth size relationship between the ninth space amount and the maximum space amount.

[0095] (2) When the third size relationship indicates that the fourth amount of space is greater than the first preset amount of space, and the fourth amount of space is less than the second preset amount of space, determine a third number of deletion commands received by the storage device within a target time period, and determine a fourth amount of storage space added to the storage device based on the third number of deletion commands; calculate a third ratio of the fourth number to the third number, and determine a ninth size relationship between the third ratio and the fourth preset ratio; and update the second storage space according to the ninth size relationship.

[0096] Among them, updating the second storage space according to the ninth size relationship includes: when the ninth size relationship indicates that the third ratio is less than or equal to the third preset ratio, determining the maximum space amount of the second storage space according to the seventh space amount, and determining the eighth space amount of the eighth storage space; merging the eighth storage space and the second storage space to generate a ninth storage space; determining the ninth space amount of the ninth storage space; and updating the second storage space according to the eighth size relationship between the ninth space amount and the maximum space amount.

[0097] Among them, updating the second storage space according to the ninth size relationship includes: when the ninth size relationship indicates that the third ratio is greater than the third preset ratio, determining the maximum space amount of the second storage space according to the sum of the seventh space amount and the sixth space amount; updating the second storage space according to the eighth size relationship between the ninth space amount and the maximum space amount.

[0098] (3) When the third size relationship indicates that the fourth space amount is greater than the first preset space amount, and the fourth space amount is greater than the second preset space amount, the fourth preset space amount is determined as the maximum space amount of the second storage space, and the eighth space amount of the eighth storage space is determined; the eighth storage space and the second storage space are merged to generate a ninth storage space; a ninth space amount of the ninth storage space is determined; and the second storage space is updated according to the eighth size relationship between the ninth space amount and the maximum space amount.

[0099] Among them, updating the second storage space according to the eighth size relationship between the ninth space amount and the maximum space amount includes: when the eighth size relationship indicates that the ninth space amount is greater than the maximum space amount, dividing the ninth storage space into a tenth storage space and an eleventh storage space, wherein the space amount of the tenth storage space is the maximum space amount, and the space amount of the eleventh storage space is a fourth difference between the ninth space amount and the maximum space amount; updating the second storage space according to the tenth storage space, and updating the first storage space according to the eleventh storage space; and updating the second storage space according to the ninth storage space when the eighth size relationship indicates that the ninth space amount is less than or equal to the maximum space amount.

[0100] Through the above technical solution, the fragmentation of storage space in the storage device can be reduced, the utilization rate of the storage space can be improved, and thus the performance and response speed of the system can be improved.

[0101] In order to better understand the process of the above data writing method, the implementation process of the above data writing method is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of the present application.

[0102] In distributed storage software based on distributed storage, data is stored on the disk through the storage engine. It needs to be allocated a logical space address by the allocator first, and then write data on the allocated space address to correspond to the physical block address (PBA) in the solid state drive (SSD). The allocation principle of the allocator in the related technology is:

[0103] (1) The allocator maps out the address space of the entire disk. The logical addresses in the address space correspond to the logical block addresses (LBA) in the disk.

[0104] (2) When the IO reaches the storage engine, it needs to apply for the required space from the allocator based on the IO size;

[0105] (3) After applying for the space, write the data to the corresponding space address;

[0106] (4) The operating system converts the space address allocated by the allocator into the LBA address on the actual disk and passes it to the disk;

[0107] (5) The disk converts the LBA address to PBA and actually writes the data to the flash memory.

[0108] In the related art, distributed storage usually writes data by appending writes, and the characteristics of the appending write method include:

[0109] (1) The allocator allocates space in order from low address to high address, and uses a cursor to record the address offset of the last allocation;

[0110] (2) Figure 3 It is a schematic diagram created and written according to the related technology of the optional embodiment of the present application, such as Figure 3 As shown in the figure, for the creation of an object, the space of the corresponding IO size is allocated and then written directly to the disk. That is, the creation uses the space address that has not been written before;

[0111] (3) Figure 4 is a schematic diagram of overwriting in the related art according to an optional embodiment of the present application, such as Figure 4 As shown in the figure, for an object to be overwritten, the new data first applies for a space of the corresponding IO size and writes into the space. After the writing is successful, the space where the old data of the object is located is released. That is, the new data applies for a new space that has not been written before, and the old data space is released.

[0112] (4) According to the writing method of (2) and (3), both creation and overwriting are first applied from unused space. The entire space address will be used up quickly during the data writing process. After it is used up, the cursor starts again from the initial position. At this time, the space address released during the overwriting process will be used;

[0113] (5) When the software layer runs out of address space, the corresponding disk, LBA is all used up, and the valid data in the disk is the size of the entire user space;

[0114] (6) In the existing append-write mode of distributed storage, no matter how much data is written, the actual valid data in the disk is the size of the entire user space. For example: for a disk with 3TB visible to the user, 1.5TB of data is created and written, and then 1.5TB of data is modified and written. The data volume on the storage side is 50%. But for the disk, the actual valid data in the disk is 3TB at this time;

[0115] The reason for the above problem is that the disk needs to write new data to the LBA corresponding to the data in order to mark the data invalid. However, the storage engine writes data in append mode and does not write data to the old LBA. Therefore, all data is valid for the disk. If data is continued to be written and garbage collection (GC) is triggered, 3TB of data needs to be moved, and the write amplification is 2, which means low performance.

[0116] In view of the above problems, the optional embodiment of the present application provides a distributed storage space allocation optimization method, which provides a secondary allocator to reclaim the space released by old data during the overwrite process. When overwriting, the space is directly requested from the secondary allocator, so that the LBA address is reused in advance, the data in the disk is invalidated as soon as possible, and the write amplification is reduced, thereby improving the performance. Specifically:

[0117] Step S1, the allocator allocates space in order from low address to high address.

[0118] Step S2, Figure 5 is a schematic diagram of creating a write according to an optional embodiment of the present application, such as Figure 5 As shown: The process of creating a write is to allocate space of the corresponding IO size and then write the data directly.

[0119] Step S3, at the storage engine layer, a secondary allocator is added based on the original allocator (ie, the main allocator of this application) to collect the space released by old data during the overwrite process (the secondary allocator initially has no free space).

[0120] Step S4, when there is no free space in the secondary allocator, in the overwrite process, the new data searches for a new space address in the main allocator to be written in. After completion, the space occupied by the old data in the main allocator is released and added to the secondary allocator.

[0121] Step S5, Figure 6 is a schematic diagram of overwriting according to an optional embodiment of the present application, such as Figure 6 As shown, when the free space capacity in the secondary allocator is sufficient (the free space needs to reach the set value before it can be used), in the process of overwriting, the new data applies for the free space address in the secondary allocator and writes it. Compared with the method of applying for space only in the main allocator in the prior art, the optional embodiment of the present application can reuse the released space as early as possible, and invalidate the old data on the PBA corresponding to the released space address in the disk.

[0122] Step S6, when the amount of data written by the user is less than the user space, from the pre-embedded data to the steady state process, due to the existence of the secondary allocator, the overwrite process no longer applies for space from the main allocator, resulting in some unused space addresses (LBA) in the main allocator, so the effective data in the disk is less than the size of the user space. The less effective data, the smaller the write amplification and the higher the performance.

[0123] In addition, the setting of the free space size of the secondary allocator in the optional embodiment of the present application can be dynamically adjusted according to the amount of data written by the user. Among them, it can be divided into four scenarios according to the amount of user space occupied by user-written data: low usage (below 40%), medium usage (40%-70%), high usage (above 70%), and constantly changing space usage. And the allocator is divided into a primary allocator (for managing the initial space) and a secondary allocator (for managing the released space). In the process of creating a write, the new data applies for space from the primary allocator. In the process of overwriting the write, the new data first applies for space from the secondary allocator, and the space released by the old data is placed in the secondary allocator. Specifically, the allocation process of the free space size of the secondary allocator is as follows:

[0124] (1) Figure 7 is a schematic diagram (I) of allocating the free space size of the secondary allocator according to an optional embodiment of the present application, such as Figure 7 As shown, the space allocation scheme for low usage (less than 40%, i.e., the first preset space amount of this application) is:

[0125] 1) If Figure 7 (a) During the creation of a write, the secondary allocator has no free space and requests data space from the primary allocator.

[0126] 2) If Figure 7 (b) When the water level is 30% (i.e., the user-written data occupies 30% of the user space), during the overwrite write process without fragmentation, the secondary allocator has 15% of the space. The new data (i.e., the target data of this application) applies for data space from the secondary allocator, and the old data (i.e., the sixth data of this application) is released from the disk (i.e., the storage device of this application). The space after the old data is released is the space that can be allocated by the secondary allocator.

[0127] 3) If Figure 7 (c) When the water level is 30% (i.e., the user-written data occupies 30% of the user space), during the overwrite write process with more fragmentation, the secondary allocator has 30% of the space. The new data (i.e., the target data of this application) applies for data space from the secondary allocator, and the old data (i.e., the sixth data of this application) is released from the disk (i.e., the storage device of this application). The space after the old data is released is the space that can be allocated by the secondary allocator.

[0128] In summary, in the case of low usage, the impact of fragmentation needs to be considered, that is, the ratio of the number of IOs issued within a period of time (for example: 10 minutes, which is the target time period of this application) (that is, the first number of write commands received by the storage device of this application) to the number of allocated spaces (the second number of storage spaces allocated in the storage device based on the first number of write commands) can be counted (that is, the second ratio of this application).

[0129] When the number of allocated spaces / the number of issued IOs>3 (ie, the first preset ratio of the present application), the secondary allocator manages the free space capacity (ie, the maximum space amount of the present application) to be the same as the water level.

[0130] When the number of allocated spaces / number of issued IOs ≤ 3, the maximum amount of free space managed by the secondary allocator is set to half the amount of user data (i.e. the sum of the fourth space amount and the third space amount in this application), and space is applied for from the main allocator when the space capacity of the secondary allocator is insufficient.

[0131] (2) Figure 8 is a schematic diagram (II) of the allocation of free space size of a secondary allocator according to an optional embodiment of the present application, such as Figure 8 As shown, the space allocation scheme for medium usage (40%-70%, 70% is the second preset space amount of this application) is:

[0132] When the water level is greater than 40% and less than 70%, for example, 55%, and there is a lot of debris, the space of the secondary distributor is 25%.

[0133] That is, in the case of medium usage, the impact of fragmentation needs to be considered, that is, the ratio of the number of IOs issued within a period of time (for example: 10 minutes, which is the target time period of this application) (that is, the first number of write commands received by the storage device of this application) to the number of allocated spaces (the second number of storage spaces allocated in the storage device based on the first number of write commands) can be counted (that is, the second ratio of this application).

[0134] When the number of allocated spaces / number of issued IOs>7 (i.e., the second preset ratio of the present application), the free space capacity managed by the secondary allocator (i.e., the maximum space amount of the present application) is adjusted to half of the amount of data written by the user (e.g.: Figure 8 half of the 55% in the

[0135] When the number of allocated spaces / number of issued IOs is ≤ 7, the maximum amount of free space managed by the secondary allocator is set to 1 / 3 of the water level.

[0136] (3) Fig. 9 is a schematic diagram (III) of the allocation of free space size of a secondary allocator according to an optional embodiment of the present application, such as Fig. 9As shown in the figure, the space allocation scheme for high usage (greater than 70%) is:

[0137] When the water level is greater than 70%, for example, 80%, the space of the secondary distributor is 2% (ie, the target preset space amount of the present application).

[0138] That is, in the case of high usage, the performance of space fragmentation is almost the same as the amount of data increases, so there is no need to consider the impact of fragmentation. The amount of free space managed by the secondary allocator is set to 2% of the total space capacity.

[0139] (4) Batch plan when space usage changes continuously.

[0140] As the amount of data written by users gradually increases (i.e., as data is continuously written), the space of the secondary batcher can be allocated according to the allocation schemes (1), (2), and (3).

[0141] As the amount of data written by users gradually decreases (i.e., as data is continuously deleted), the space of the secondary batcher can be allocated according to the reverse allocation scheme of (1), (2), and (3). That is, when the secondary batcher manages a large amount of free space, the released space is trimmed (integrated) and directly placed in the primary batcher until the space of the secondary batcher is lower than the set amount and then placed in the secondary batcher.

[0142] In summary, the optional embodiment of the present application reclaims the space released by old data by adding a secondary allocator, and gives priority to applying from the secondary allocator when applying for space for overwrite writing. When the amount of data written by the user is less than 100%, it ensures that there is a margin for the LBA address, thereby reducing the amount of valid data in the disk and improving performance. Specifically, it has the following advantages: optimizing the storage engine layer space allocator, adding a secondary allocator to collect the space released by old data in the overwrite writing process, allocating space from the secondary allocator when overwrite writing, and releasing the old data address as soon as possible, avoiding the problem of delayed release of data in the disk caused by the append writing method in the related technology; at the same time, according to the different amounts of data written by the user, the secondary allocator dynamically sets different space sizes to ensure that the performance is improved at different water levels while reducing the problem of aggravated disk fragmentation caused by the use of the secondary allocator.

[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0144] In the present embodiment, a data verification device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0145] Fig.10 is a structural block diagram of a data writing device according to an embodiment of the present application, such as Fig.10 As shown, the system includes:

[0146] The first determination module 1002 is used to determine a writing mode of writing target data into a storage device, wherein the writing mode includes: creating writing and overwriting writing;

[0147] A second determination module 1004 is used to determine a target data allocator corresponding to the target data according to the write mode, wherein the target data allocator includes: a primary allocator and / or a secondary allocator, the primary allocator is used to allocate a first storage space in the storage device, the secondary allocator is used to allocate a second storage space in the storage device, the first storage space is used to indicate free space in the storage device, and the second storage space is used to indicate storage space after data has been released in the storage device;

[0148] The allocation module 1006 is configured to write the target data into the target storage space in the storage device based on the target data allocator.

[0149] Through the data writing device of the present application, the writing mode of the target data is determined to be create write or overwrite write, and the target data distributor corresponding to the target data is determined according to different writing modes. The target data distributor includes: a main distributor and / or a secondary distributor, the main distributor is used to allocate free space in the storage device, and the secondary distributor is used to allocate the storage device after the data has been released in the storage device. The target data is written to the target storage space in the storage device based on the main distributor and / or the secondary distributor. That is to say, the embodiment of the present application sets two target data distributors: a main distributor and a secondary distributor, and the two target data distributors respectively allocate different storage spaces in the storage device, wherein the secondary distributor can allocate the space released after the data has been written but the data is invalid, and then when the data is written, the data can be directly written from the unused free space in the storage device, or the data can be written from the released space. The problem of low data writing performance caused by the creation write and overwrite write in the related art is solved. It makes it possible to release the garbage data as soon as possible and improve the data writing performance.

[0150] In an exemplary embodiment, the second determination module 1004 is further used to determine that the target data allocator corresponding to the target data is the primary allocator when the write mode is determined to be the create write; and to determine that the target data allocator corresponding to the target data is the secondary allocator when the write mode is determined to be the overwrite write.

[0151] In an exemplary embodiment, the allocation module 1006 is also used to determine the first space amount corresponding to the target data and the second space amount of the first storage space when the write mode is determined to be the create write; determine a first size relationship between the first space amount and the second space amount; and allocate the target data to the target storage space according to the first size relationship and the target data allocator.

[0152] In an exemplary embodiment, the allocation module 1006 is also used to write the target data to the first storage space based on the main allocator when the first size relationship indicates that the first space amount is less than or equal to the second space amount; determine a first difference between the first space amount and the second space amount when the first size relationship indicates that the first space amount is greater than the second space amount; write the first data in the target data to the first storage space based on the main allocator, and write the second data in the target data to the second storage space based on the secondary allocator, wherein the space amount corresponding to the first data is the second space amount, and the space amount corresponding to the second data is the first difference.

[0153] In an exemplary embodiment, the allocation module 1006 is also used to determine the first space amount corresponding to the target data and determine the third space amount of the second storage space when it is determined that the write mode is the overwrite write; determine the second size relationship between the first space amount and the third space amount; and allocate the target data to the target storage space according to the second size relationship.

[0154] In an exemplary embodiment, the allocation module 1006 is also used to write the target data to the second storage space based on the secondary allocator when the second size relationship indicates that the first space amount is less than or equal to the third space amount; determine the second difference between the first space amount and the third space amount when the second size relationship indicates that the first space amount is greater than the third space amount; write the third data in the target data to the second storage space based on the secondary allocator, and write the fourth data in the target data to the first storage space based on the main allocator, wherein the space amount corresponding to the third data is the third space amount, and the space amount corresponding to the fourth data is the second difference.

[0155] In an exemplary embodiment, the allocation module 1006 is further used to determine a fourth amount of space in the storage device where data has been written; determine a third size relationship between the fourth amount of space and the first preset amount of space; and update the second storage space according to the third size relationship.

[0156] In an exemplary embodiment, the allocation module 1006 is also used to determine a first number of write commands received by the storage device within a target time period, and determine a second number of storage space allocated in the storage device based on the first number of write commands, when the third size relationship indicates that the fourth amount of space is less than or equal to the first preset amount of space; calculate a second ratio of the second number to the first number, and determine a fourth size relationship between the second ratio and the first preset ratio; and update the second storage space according to the fourth size relationship.

[0157] In an exemplary embodiment, the allocation module 1006 is also used to determine the maximum space amount of the second storage space according to the sum of the fourth space amount and the third space amount when the fourth size relationship indicates that the second ratio is less than or equal to the first preset ratio, and determine sixth data in the storage device according to the target data, wherein the sixth data is data that is partially or completely the same as the target data; determine the third storage space of the sixth data in the storage device; merge the second storage space and the third storage space to generate a fourth storage space; determine a fifth space amount of the fourth storage space; and update the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount.

[0158] In an exemplary embodiment, the allocation module 1006 is also used to determine the maximum space amount of the second storage space according to the fourth space amount when the fourth size relationship indicates that the second ratio is greater than the first preset ratio; and to update the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount.

[0159] In an exemplary embodiment, the allocation module 1006 is also used to determine a first number of write commands received by the storage device within a target time period, and determine a second number of storage space allocated in the storage device based on the first number of write commands, when the third size relationship indicates that the fourth space amount is greater than the first preset space amount and the fourth space amount is less than the second preset space amount; calculate a second ratio of the second number to the first number, and determine a sixth size relationship between the second ratio and the second preset ratio; and update the second storage space according to the sixth size relationship.

[0160] In an exemplary embodiment, the allocation module 1006 is also used to determine the maximum space amount of the second storage space based on the fourth space amount and determine sixth data consistent with the target data in the storage device when the sixth size relationship indicates that the second ratio is less than or equal to the second preset ratio; determine a third storage space for the sixth data in the storage device; merge the second storage space and the third storage space to generate a fourth storage space; determine a fifth space amount of the fourth storage space; and update the second storage space based on the fifth size relationship between the fifth space amount and the maximum space amount.

[0161] In an exemplary embodiment, the allocation module 1006 is also used to determine the maximum space amount of the second storage space according to the sum of the fourth space amount and the third space amount when the sixth size relationship indicates that the second ratio is greater than the second preset ratio; and to update the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount.

[0162] In an exemplary embodiment, the allocation module 1006 is also used to determine the target preset space amount as the maximum space amount of the second storage space when the third size relationship indicates that the fourth space amount is greater than the first preset space amount and the fourth space amount is greater than the second preset space amount, and determine sixth data consistent with the target data in the storage device; determine a third storage space in the storage device for the sixth data; merge the second storage space and the third storage space to generate a fourth storage space; determine a fifth space amount of the fourth storage space; and update the second storage space according to the fifth size relationship between the fifth space amount and the maximum space amount.

[0163] In an exemplary embodiment, the allocation module 1006 is also used to divide the fourth storage space into a fifth storage space and a sixth storage space when the fifth size relationship indicates that the fifth space amount is greater than the maximum space amount, wherein the space amount of the fifth storage space is the maximum space amount, and the space amount of the sixth storage space is the third difference between the fifth space amount and the maximum space amount; update the second storage space according to the fifth storage space, and update the first storage space according to the sixth storage space; and update the second storage space according to the fourth storage space when the fifth size relationship indicates that the fifth space amount is less than or equal to the maximum space amount.

[0164] In an exemplary embodiment, the allocation module 1006 is also used to delete the seventh data in the storage space according to the deletion command when the storage device receives a deletion command, wherein the deletion command is used to instruct the deletion of the seventh data in the storage device; determine the seventh storage space allowed to be allocated by the secondary allocator, and determine the sixth space amount of the seventh storage space; determine whether the sixth space amount is greater than the third preset space amount; when it is determined that the sixth space amount is greater than or equal to the third preset space amount, determine the eighth storage space corresponding to the seventh data as the storage space to be allocated by the main allocator; when it is determined that the sixth space amount is less than the third preset space amount, determine the eighth storage space as the storage space to be allocated by the secondary allocator.

[0165] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer processes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0166] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer process, and the processor is configured to run the computer process to execute the steps in the above method embodiment.

[0167] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0168] An embodiment of the present application further provides a computer process product, including a computer process, wherein when the computer process is executed by a processor, the steps in the above method embodiment are implemented.

[0169] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0170] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a process code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the present application is not limited to any specific combination of hardware and software.

[0171] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for writing data, characterized in that: include: Determine a writing mode for writing target data into a storage device, wherein the writing mode includes: creating writing and overwriting writing; Determine a target data allocator corresponding to the target data according to the write mode, wherein the target data allocator includes: a primary allocator and / or a secondary allocator, the primary allocator is used to allocate a first storage space in the storage device, the secondary allocator is used to allocate a second storage space in the storage device, the first storage space is used to indicate free space in the storage device, and the second storage space is used to indicate storage space after data has been released in the storage device; writing the target data to a target storage space in the storage device based on the target data allocator; Wherein, allocating the target data to the target storage space in the target storage device based on the target data allocator includes: In the case where it is determined that the writing mode is the overwrite writing, determining a first space amount corresponding to the target data, and determining a third space amount of the second storage space; Determining a second size relationship between the first spatial amount and the third spatial amount; The target data is allocated to the target storage space according to the second size relationship.

2. The method according to claim 1, characterized in that Determining a target data distributor corresponding to the target data according to the writing mode includes: In the case where the write mode is determined to be the create write, determining the target data distributor corresponding to the target data to be the main distributor; When it is determined that the writing mode is the overwrite writing, it is determined that the target data distributor corresponding to the target data is the secondary distributor.

3. The method according to claim 1, characterized in that Allocating the target data to a target storage space in the storage device based on the target data allocator includes: In the case where it is determined that the write mode is the create write, determining a first space amount corresponding to the target data, and determining a second space amount of the first storage space; Determining a first size relationship between the first spatial amount and the second spatial amount; The target data is allocated to the target storage space according to the first size relationship and the target data allocator.

4. The method according to claim 3, characterized in that Allocating the target data to the target storage space according to the first size relationship and the target data allocator includes: writing the target data to the first storage space based on the master allocator if the first size relationship indicates that the first amount of space is less than or equal to the second amount of space; If the first size relationship indicates that the first amount of space is greater than the second amount of space, determining a first difference between the first amount of space and the second amount of space; Based on the main allocator, the first data in the target data is written to the first storage space, and based on the secondary allocator, the second data in the target data is written to the second storage space, wherein the space amount corresponding to the first data is the second space amount, and the space amount corresponding to the second data is the first difference.

5. The method according to claim 1, characterized in that Allocating the target data to the target storage space according to the second size relationship includes: If the second size relationship indicates that the first amount of space is less than or equal to the third amount of space, writing the target data to the second storage space based on the secondary allocator; If the second size relationship indicates that the first amount of space is greater than the third amount of space, determining a second difference between the first amount of space and the third amount of space; Based on the secondary allocator, the third data in the target data is written to the second storage space, and based on the main allocator, the fourth data in the target data is written to the first storage space, wherein the space amount corresponding to the third data is the third space amount, and the space amount corresponding to the fourth data is the second difference.

6. The method according to claim 1, characterized in that After allocating the target data to the target storage space according to the second size relationship, the method further includes: Determining a fourth amount of space in the storage device where data has been written; Determining a third size relationship between the fourth space amount and the first preset space amount; The second storage space is updated according to the third size relationship.

7. The method according to claim 6, characterized in that Updating the second storage space according to the third size relationship includes: In a case where the third size relationship indicates that the fourth amount of space is less than or equal to the first preset amount of space, determining a first number of write commands received by the storage device within a target time period, and determining a second amount of storage space allocated in the storage device based on the first number of write commands; Calculating a second ratio of the second quantity to the first quantity, and determining a fourth magnitude relationship between the second ratio and the first preset ratio; The second storage space is updated according to the fourth size relationship.

8. The method according to claim 7, characterized in that Updating the second storage space according to the fourth size relationship includes: In a case where the fourth size relationship indicates that the second ratio is less than or equal to the first preset ratio, determining a maximum space amount of the second storage space according to a sum of the fourth space amount and the third space amount, and determining sixth data in the storage device according to the target data, wherein the sixth data is data that is partially or completely the same as the target data; Determine a third storage space for the sixth data in the storage device; merging the second storage space and the third storage space to generate a fourth storage space; determining a fifth amount of space of the fourth storage space; The second storage space is updated according to a fifth size relationship between the fifth space amount and the maximum space amount.

9. The method according to claim 8, characterized in that Updating the second storage space according to the fourth size relationship further includes: In a case where the fourth size relationship indicates that the second ratio is greater than the first preset ratio, determining a maximum space amount of the second storage space according to the fourth space amount; The second storage space is updated according to a fifth size relationship between the fifth space amount and the maximum space amount.

10. The method according to claim 6, characterized in that Updating the second storage space according to the third size relationship includes: In a case where the third size relationship indicates that the fourth amount of space is greater than the first preset amount of space, and the fourth amount of space is less than the second preset amount of space, determining a first number of write commands received by the storage device within a target time period, and determining a second amount of storage space allocated in the storage device based on the first number of write commands; Calculating a second ratio of the second quantity to the first quantity, and determining a sixth magnitude relationship between the second ratio and a second preset ratio; The second storage space is updated according to the sixth size relationship.

11. The method according to claim 10, characterized in that Updating the second storage space according to the sixth size relationship includes: In a case where the sixth size relationship indicates that the second ratio is less than or equal to the second preset ratio, determining a maximum space amount of the second storage space according to the fourth space amount, and determining sixth data in the storage device according to the target data, wherein the sixth data is data that is partially or completely the same as the target data; Determine a third storage space for the sixth data in the storage device; merging the second storage space and the third storage space to generate a fourth storage space; determining a fifth amount of space of the fourth storage space; The second storage space is updated according to a fifth size relationship between the fifth space amount and the maximum space amount.

12. The method according to claim 11, characterized in that Updating the second storage space according to the sixth size relationship includes: When the sixth size relationship indicates that the second ratio is greater than the second preset ratio, determining a maximum space amount of the second storage space according to a sum of the fourth space amount and the third space amount; The second storage space is updated according to a fifth size relationship between the fifth space amount and the maximum space amount.

13. The method according to claim 6, characterized in that Updating the second storage space according to the third size relationship includes: In a case where the third size relationship indicates that the fourth space amount is greater than the first preset space amount, and the fourth space amount is greater than the second preset space amount, determining the target preset space amount as the maximum space amount of the second storage space, and determining sixth data in the storage device according to the target data, wherein the sixth data is data that is partially or completely the same as the target data; Determine a third storage space for the sixth data in the storage device; merging the second storage space and the third storage space to generate a fourth storage space; determining a fifth amount of space of the fourth storage space; The second storage space is updated according to a fifth size relationship between the fifth space amount and the maximum space amount.

14. The method according to any one of claims 8 to 9 or claims 11 to 13, characterized in that Updating the second storage space according to a fifth size relationship between the fifth space amount and the maximum space amount includes: In a case where the fifth size relationship indicates that the fifth space amount is greater than the maximum space amount, dividing the fourth storage space into a fifth storage space and a sixth storage space, wherein the space amount of the fifth storage space is the maximum space amount, and the space amount of the sixth storage space is a third difference between the fifth space amount and the maximum space amount; updating the second storage space according to the fifth storage space, and updating the first storage space according to the sixth storage space; In a case where the fifth size relationship indicates that the fifth amount of space is less than or equal to the maximum amount of space, the second storage space is updated according to the fourth storage space.

15. The method according to claim 1, characterized in that After the target data allocator writes the target data into the target storage space in the storage device, the method further includes: When the storage device receives a deletion command, deleting the seventh data in the storage space according to the deletion command, wherein the deletion command is used to instruct to delete the seventh data in the storage device; determining a seventh storage space that the secondary allocator is allowed to allocate, and determining a sixth space amount of the seventh storage space; determining whether the sixth amount of space is greater than a third preset amount of space; In the case where it is determined that the sixth space amount is greater than or equal to the third preset space amount, determining the eighth storage space corresponding to the seventh data as the storage space to be allocated by the main allocator; In the case where it is determined that the sixth space amount is less than the third preset space amount, the eighth storage space is determined as the storage space to be allocated by the secondary allocator.

16. A data writing device, characterized in that: include: A first determining module is used to determine a writing mode of writing target data into a storage device, wherein the writing mode includes: creating writing and overwriting writing; A second determination module is used to determine a target data allocator corresponding to the target data according to the write mode, wherein the target data allocator includes: a primary allocator and / or a secondary allocator, the primary allocator is used to allocate a first storage space in the storage device, the secondary allocator is used to allocate a second storage space in the storage device, the first storage space is used to indicate a free space in the storage device, and the second storage space is used to indicate a storage space in the storage device after data has been released; an allocating module, configured to write the target data into a target storage space in the storage device based on the target data allocator; Among them, the allocation module is also used to determine the first space amount corresponding to the target data and determine the third space amount of the second storage space when it is determined that the write method is the overwrite write; determine the second size relationship between the first space amount and the third space amount; and allocate the target data to the target storage space according to the second size relationship.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer process, wherein the computer process implements the steps of the method according to any one of claims 1 to 15 when executed by a processor.

18. An electronic device comprising a memory, a processor, and a computer process stored in the memory and running on the processor, characterized in that: When the processor executes the computer process, the steps of the method according to any one of claims 1 to 15 are implemented.

19. A computer program product comprising a computer program, characterized in that The computer program is used by a processor to execute the steps of the method described in any one of claims 1 to 15.

Images