Data writing method and device, computer device, and storage medium

Abstract

The application relates to a data writing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring the number of page data contained in to-be-written data; calculating the residual number of residual page data corresponding to the to-be-written data according to the number of page data contained in the to-be-written data and the number of pages contained in a preset strip, wherein the preset strip comprises a plurality of page strips; in the case that the residual number meets a target padding condition, determining the padding number of a target page strip corresponding to the residual page data in a storage space comprising a plurality of preset strips, and writing the to-be-written data into the storage space based on the padding number of the target page strip. By using the method, the redundant data contained in the storage space can be reduced, unnecessary redundant zero data can be avoided, and the utilization rate of the storage space can be improved under the condition of ensuring the parallel writing efficiency.

Description

Technical Field

[0001] This application relates to the field of storage, and in particular to a data writing method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] With the development of storage technology, erasure coding algorithms have emerged. These algorithms can encode N blocks of data into M blocks of redundant parity data, ensuring data reliability. Therefore, erasure coding algorithms are widely used in the storage field. When calculating the parity data, the data to be written needs to meet a preset stripe size.

[0003] In related technologies, when the data to be written is not full within the stripe size, the offset and length of the padding data required for the data to be written are calculated based on the length of the data to be written and the offset within a preset stripe. Then, the zero data is used as padding data, and the padding data and the data to be written are merged into full-strip data. This allows the full-strip data and its corresponding parity data to be stored in the storage space. However, the storage space contains redundant zero data, resulting in low storage space utilization. Summary of the Invention

[0004] Therefore, it is necessary to provide a data writing method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve storage space utilization in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a data writing method. The method includes:

[0006] Get the number of page data contained in the data to be written;

[0007] Based on the number of page data contained in the data to be written and the number of pages contained in the preset strip, calculate the remaining number of remaining page data corresponding to the data to be written, wherein the preset strip includes multiple page strips;

[0008] If the remaining number meets the target padding condition, in a storage space containing multiple preset stripes, the padding number of the target page strip corresponding to the remaining page data is determined, and the data to be written is written to the storage space based on the padding number of the target page strip.

[0009] Based on the above solutions, redundant data contained in the storage space can be reduced, and unnecessary redundant data writing can be avoided. While ensuring parallel writing efficiency, the utilization rate of storage space can be improved. By using different padding methods according to the length of the data to be written, it can be ensured that data does not need to be read from the disk when writing data, and different data to be written can be distributed in different stripes or page stripes, thereby ensuring concurrent writing of different object data and improving writing efficiency.

[0010] In one embodiment, calculating the remaining number of remaining page data corresponding to the data to be written, based on the number of page data contained in the data to be written and the number of pages contained in the preset stripe, includes:

[0011] The number of page data contained in the data to be written is divided by the number of pages contained in the preset stripe to obtain the quotient and the remainder.

[0012] The remainder value is determined to be the remaining number of remaining page data corresponding to the data to be written.

[0013] Based on the above scheme, the number of remaining page data in a non-full strip can be accurately calculated.

[0014] In one embodiment, the method further includes:

[0015] Obtain the number of page strips included in the preset strip;

[0016] If the remaining number is greater than the target value and the remaining number is less than the number of page strips contained in the preset strip, then the remaining number is determined to meet the target completion condition.

[0017] If the remaining number is equal to the target value, or if the remaining number is greater than or equal to the number of page strips contained in the preset strip, then it is determined that the remaining number does not meet the target completion condition.

[0018] Based on the above approach, it is possible to comprehensively determine whether the remaining page data to be written meets the target padding conditions.

[0019] In one embodiment, determining the number of target page strips to be padded in a storage space containing multiple preset stripes includes:

[0020] In a storage space containing multiple preset stripes, determine the number of target page stripes where the remaining page data is located;

[0021] Based on the number of target page stripes, determine the total number of pages contained in the target page stripes;

[0022] The number of pages to be filled is calculated based on the total number and the remaining number of pages corresponding to the remaining page data.

[0023] Based on the above scheme, the number of missing pieces can be calculated efficiently.

[0024] In one embodiment, writing the data to be written into the storage space based on the padding number of the target page stripe includes:

[0025] Based on the number of padding strips on the target page, calculate the first data length of the data to be written;

[0026] Obtain the offset of the data to be written in the storage space;

[0027] Based on the first data length of the data to be written and the offset of the data to be written in the storage space, the data to be written is written to the storage space.

[0028] Based on the above scheme, the data to be written can be accurately written to the storage space. While ensuring parallel data writing, the data writing time is reduced, the data writing efficiency is improved, and the utilization rate of storage space is increased, while avoiding too much redundant zero data writing.

[0029] In one embodiment, writing the data to be written into the storage space includes:

[0030] Based on the number of missing data in the target page strip, determine the missing data;

[0031] Based on the data to be written and the padding data, calculate the verification data;

[0032] Write the data to be written, the padding data, and the verification data into the storage space.

[0033] Based on the above scheme, the data to be written can be accurately written to the storage space, ensuring parallel data writing while avoiding the writing of too much redundant data and improving the utilization rate of storage space.

[0034] Secondly, this application also provides a data writing device. The device includes:

[0035] The first acquisition module is used to acquire the number of page data contained in the data to be written;

[0036] The calculation module is used to calculate the remaining number of remaining page data corresponding to the data to be written based on the number of page data contained in the data to be written and the number of pages contained in the preset stripe, wherein the preset stripe includes multiple page stripes.

[0037] The first determining module is configured to, when the remaining number satisfies the target padding condition, determine the padding number of the target page strip corresponding to the remaining page data in a storage space containing multiple preset stripes, and write the data to be written into the storage space based on the padding number of the target page strip.

[0038] In one embodiment, the computing module includes:

[0039] The calculation unit is used to perform a division operation on the number of page data contained in the data to be written and the number of pages contained in the preset strip to obtain a quotient and a remainder.

[0040] A determining unit is used to determine that the remainder value is the remaining number of remaining page data corresponding to the data to be written.

[0041] In one embodiment, the device further includes:

[0042] The second acquisition module is used to acquire the number of page strips contained in the preset strip;

[0043] The second determining module is used to determine that the remaining number satisfies the target completion condition if the remaining number is greater than the target value and the remaining number is less than the number of page strips contained in the preset strip.

[0044] The third determining module is used to determine that the remaining number does not meet the target completion condition if the remaining number is equal to the target value, or if the remaining number is greater than or equal to the number of page strips contained in the preset strip.

[0045] In one embodiment, the first determining module is specifically used for:

[0046] In a storage space containing multiple preset stripes, determine the number of target page stripes where the remaining page data is located; based on the number of target page stripes, determine the total number of pages contained in the target page stripes; based on the total number and the remaining number corresponding to the remaining page data, calculate the number of pages to be padded.

[0047] In one embodiment, the first determining module is further specifically used for:

[0048] Based on the number of padding strips on the target page, calculate the first data length of the data to be written;

[0049] Obtain the offset of the data to be written in the storage space;

[0050] Based on the first data length of the data to be written and the offset of the data to be written in the storage space, the data to be written is written to the storage space.

[0051] In one embodiment, the first determining module is further specifically used for:

[0052] Based on the number of missing data in the target page strip, determine the missing data;

[0053] Based on the data to be written and the padding data, calculate the verification data;

[0054] Write the data to be written, the padding data, and the verification data into the storage space.

[0055] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0056] Get the number of page data contained in the data to be written;

[0057] Based on the number of page data contained in the data to be written and the number of pages contained in the preset strip, calculate the remaining number of remaining page data corresponding to the data to be written, wherein the preset strip includes multiple page strips;

[0058] If the remaining number meets the target padding condition, in a storage space containing multiple preset stripes, the padding number of the target page strip corresponding to the remaining page data is determined, and the data to be written is written to the storage space based on the padding number of the target page strip.

[0059] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0060] Get the number of page data contained in the data to be written;

[0061] Based on the number of page data contained in the data to be written and the number of pages contained in the preset strip, calculate the remaining number of remaining page data corresponding to the data to be written, wherein the preset strip includes multiple page strips;

[0062] If the remaining number meets the target padding condition, in a storage space containing multiple preset stripes, the padding number of the target page strip corresponding to the remaining page data is determined, and the data to be written is written to the storage space based on the padding number of the target page strip.

[0063] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0064] Get the number of page data contained in the data to be written;

[0065] Based on the number of page data contained in the data to be written and the number of pages contained in the preset strip, calculate the remaining number of remaining page data corresponding to the data to be written, wherein the preset strip includes multiple page strips;

[0066] If the remaining number meets the target padding condition, in a storage space containing multiple preset stripes, the padding number of the target page strip corresponding to the remaining page data is determined, and the data to be written is written to the storage space based on the padding number of the target page strip.

[0067] The aforementioned data writing method, apparatus, computer device, storage medium, and computer program product include the following steps: obtaining the number of page data contained in the data to be written; calculating the remaining number of remaining page data corresponding to the data to be written based on the number of page data contained in the data to be written and the number of pages contained in a preset stripe, wherein the preset stripe includes multiple page stripes; if the remaining number satisfies a target padding condition, determining the padding number of the target page stripe corresponding to the remaining page data in a storage space containing multiple preset stripes, and writing the data to be written into the storage space based on the padding number of the target page stripe. By employing this method, the proportion of redundant zero data in the storage space can be reduced, unnecessary redundant zero data writing can be avoided, and the utilization rate of storage space can be improved while ensuring parallel writing efficiency. Attached Figure Description

[0068] Figure 1 This is a flowchart illustrating a data writing method in one embodiment;

[0069] Figure 2 This is a flowchart illustrating the steps for calculating the remaining number in one embodiment;

[0070] Figure 3 This is a flowchart illustrating the step of determining whether the padding conditions are met in one embodiment.

[0071] Figure 4 This is a flowchart illustrating the steps for calculating the number of paddings in one embodiment;

[0072] Figure 5 This is a flowchart illustrating the step of writing to storage space in one embodiment;

[0073] Figure 6This is a flowchart illustrating the step of writing to storage space in one embodiment;

[0074] Figure 7 This is a schematic diagram of the storage space data structure and the data structure of the data to be written, as shown in one embodiment.

[0075] Figure 8 This is a structural block diagram of a data writing device in one embodiment;

[0076] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0078] In one embodiment, a data writing method is provided. This embodiment uses the application of this method to a storage device as an example for illustration. It is understood that this method can also be applied to a terminal or a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The aforementioned terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc., and the server can be implemented using a standalone server or a server cluster composed of multiple servers. Figure 1 As shown, in this embodiment, the data writing method includes the following steps:

[0079] Step 102: Obtain the number of page data contained in the data to be written.

[0080] The data to be written can consist of multiple pages, such as data generated by user actions on the storage device (terminal). In one example, a user might save an image on the terminal; in this case, the storage device can use the image as the data to be written. In another example, a user might upload an image to another storage space on the terminal; in this case, the storage device can also use the image as the data to be written, and so on. Optionally,

[0081] In practice, the storage device responds to a data write command and obtains the data to be written carried by the data write command. After obtaining the data to be written, the storage device can determine the number of page data contained in the data to be written based on the data size of the data to be written.

[0082] Optionally, in response to the data write command, the storage device can obtain the data to be written carried in the data write command. The data to be written may be data transmitted to the storage device by other terminals through a data transmission network.

[0083] Step 104: Calculate the remaining number of page data corresponding to the data to be written based on the number of page data contained in the data to be written and the number of pages contained in the preset stripe.

[0084] The preset stripe comprises multiple page stripes. A preset stripe can be a unit for dividing storage space. A storage space can contain multiple logical disks of the same size. The pages, page stripes, and stripe divisions on each logical disk are identical. Each logical disk contains multiple pages. A preset stripe can contain multiple pages with the same index located on different logical disks, and a page stripe can contain a single page with the same index located on different logical disks. In other words, a preset stripe contains multiple page stripes. Remaining page data can be the page data in the data to be written that does not fill the preset stripe; the remaining number is the number of page data that does not fill the preset stripe. Each page can store one page data, and there is a one-to-one correspondence between pages and page data.

[0085] In practice, the storage device can determine the number of pages contained in the data to be written, and the storage device can also determine the number of pages contained in the preset stripe in the storage space, that is, the number of pages that the preset stripe can store. Based on this, the storage device can calculate the number of pages that the data to be written needs to occupy in the entire preset stripe, and the number of pages that are not a whole preset stripe, based on the number of pages contained in the data to be written and the number of pages that the preset stripe can store.

[0086] Step 106: If the remaining number meets the target padding condition, determine the padding number of the target page strip corresponding to the remaining page data in the storage space containing multiple preset stripes, and write the data to be written into the storage space based on the padding number of the target page strip.

[0087] The target completion condition can be that the number of remaining page data belongs to the target range, the target range can be determined based on the number of page strips contained in the preset strip, and the completion number is the number of page data that needs to be completed.

[0088] In implementation, the storage device, based on the calculated remaining number of pages, can determine whether the remaining data meets the target padding condition. If the storage device determines that the remaining page data meets the target padding condition, it can determine the target page stripe in which the remaining page data would be written to the storage space according to the standard data write order. Based on this, the storage device can determine the number of target page stripes containing the remaining page data. Based on the number of target page stripes, the number of pages contained in the target page stripe, and the number of pages contained in the remaining page data, the padding number corresponding to that target page stripe can be determined. Thus, the storage device can write the data to be written to the storage space based on the calculated padding number.

[0089] In the above data writing method, the number of page data contained in the data to be written is obtained. Based on the number of page data contained in the data to be written and the number of pages contained in a preset stripe, the remaining number of page data corresponding to the data to be written is calculated. The preset stripe includes multiple page stripes. If the remaining number meets the target padding condition, the padding number of the target page stripe corresponding to the remaining page data is determined in the storage space containing multiple preset stripes. Based on the padding number of the target page stripe, the data to be written is written to the storage space. By adopting this method, redundant data contained in the storage space can be reduced, avoiding the writing of unnecessary redundant data. It can improve the utilization of storage space while ensuring parallel writing efficiency. By using different padding methods according to the data length of the data to be written, it can be ensured that data does not need to be read from the disk during writing, and that different data to be written is distributed in different stripes or page stripes, thereby ensuring concurrent writing of different object data and improving writing efficiency.

[0090] In one embodiment, such as Figure 2 As shown, the specific processing steps of step 104, "calculating the remaining number of remaining page data corresponding to the data to be written based on the number of page data contained in the data to be written and the number of pages contained in the preset stripe," include:

[0091] Step 202: Divide the number of page data contained in the data to be written by the number of pages contained in the preset stripe to obtain the quotient and the remainder.

[0092] Step 204: Determine the remainder value as the remaining number of pages of data corresponding to the data to be written.

[0093] In implementation, the storage device can determine the size of the data to be written, which can be in KB. The storage device can also determine the size of the page data. Based on this, the storage device can perform a division operation based on the size of the data to be written and the size of the page data to obtain the number of page data items contained in the data to be written. Then, the storage device can divide the number of page data items contained in the data to be written by the number of pages contained in a preset stripe to obtain a quotient and a remainder. The storage device can use the calculated remainder as the remaining number of page data items to be written.

[0094] The calculated quotient represents the number of full preset stripes occupied by the data to be written, and the calculated remainder represents the number of page data that are not full stripes of the data to be written.

[0095] In one example, the data to be written could be a 264KB object, and the page data could be 4KB. Thus, the storage device can determine that the number of pages in the data to be written is 66, meaning the data to be written can be 66 pages. The preset stripe can contain 64 pages. Therefore, the calculated quotient can be 1, and the remainder can be 2. Based on this, the storage device can determine that the number of full stripes occupied by the data to be written is 1, and the number of remaining non-full stripes of page data is 2, meaning the remaining number of page data is 2.

[0096] In this embodiment, the number of remaining page data that is not in full stripe can be accurately calculated.

[0097] In one embodiment, such as Figure 3 As shown, the data writing method also includes:

[0098] Step 302: Obtain the number of page strips contained in the preset strip.

[0099] In implementation, the storage device can determine the storage space where the data to be written will be written, and determine the number of page stripes contained in the preset stripes configured in that storage space. The storage device can also determine the number of chunks contained in that storage space. That is, the number of page stripes contained in the preset stripes in the storage space is the same as the number of pages contained in a chunk. A chunk is a storage unit containing multiple contiguous pages located on the same logical disk, and a preset stripe consists of a group of chunks with the same index on a logical disk.

[0100] Step 304: If the remaining number is greater than the target value and less than the number of page strips contained in the preset strip, then the remaining number is determined to meet the target completion condition.

[0101] The target value can be zero.

[0102] In implementation, the storage device can compare the calculated remaining number with the target value, and also with the number of page stripes contained in the preset stripe. If the storage device determines that the remaining number is greater than the target value and less than the number of page stripes contained in the preset stripe, then the storage device determines that the remaining number meets the target padding condition. In other words, the storage device can determine the target range based on the target value and the number of page stripes contained in the preset stripe. Thus, the storage device can determine whether the remaining number meets the target padding condition by judging whether it falls within the target range.

[0103] If the storage device determines that the remaining quantity is within the target range, then the storage device can determine that the remaining quantity meets the target completion condition.

[0104] Step 306: If the remaining number is equal to the target value, or the remaining number is greater than or equal to the number of page strips contained in the preset strip, then it is determined that the remaining number does not meet the target completion condition.

[0105] In implementation, if the storage device determines that the remaining number equals the target value, or that the remaining number is greater than or equal to the number of page stripes contained in the preset stripe, then the storage device determines that the remaining number does not meet the target padding condition. Alternatively, if the storage device determines that the remaining number is not within the target range, then the storage device can determine that the remaining number does not meet the target padding condition.

[0106] In this embodiment, it is possible to comprehensively determine whether the remaining page data of the data to be written meets the target padding conditions.

[0107] In one embodiment, such as Figure 4 As shown, the specific processing steps for the step "determining the number of target page stripes to be padded in a storage space containing multiple preset stripes" include:

[0108] Step 402: In the storage space containing multiple preset stripes, determine the number of target page stripes where the remaining page data is located.

[0109] In practice, the storage device can determine the number of page stripes containing the remaining page data, i.e., the number of target page stripes, according to a preset write order.

[0110] Step 404: Determine the total number of pages contained in the target page strip based on the number of target page strips.

[0111] In practice, the storage device can calculate the total number of pages contained in multiple target page stripes based on the number of target page stripes and the number of pages contained in each page stripe.

[0112] In one example, a page strip can contain 'a' pages. The storage device can determine the number of pages in the page strip that store the data to be written (raw data) and the number of pages that store the verification data of the data to be written using a write strategy. Based on this, the storage device can perform a product operation based on the number of pages containing the raw data in each page strip and the number of target page strips, and use the resulting product as the total number of pages.

[0113] Step 406: Calculate the number of pages to fill in based on the total number and the remaining number of pages corresponding to the remaining data.

[0114] In practice, the storage device can calculate the difference between the total number and the remaining number of pages occupied by the remaining page data, and use the difference as the number of pages that need to be padded, i.e., the padding number.

[0115] In this embodiment, the number of fillers can be calculated efficiently.

[0116] In one embodiment, such as Figure 5 As shown, the specific processing steps of the step "writing the data to be written to the storage space based on the number of padding stripes of the target page" include:

[0117] Step 502: Calculate the first data length of the data to be written based on the number of padding strips on the target page.

[0118] The first data length is the number of pages occupied by the data to be written when it is written to a logical disk.

[0119] In implementation, the storage device can determine the number of target page stripes based on the number of padding stripes. The storage device can also obtain the number of full stripes occupied by the data to be written. In this way, the storage device can calculate the first data length of the data to be written on each logical disk based on the number of full stripes and the number of target page stripes.

[0120] Step 504: Obtain the offset of the data to be written in the storage space.

[0121] In practice, the storage device can obtain the offset of the data to be written from the database or the storage device's log. The offset of the data to be written can be calculated by the storage device after completing the write operation of the previous data to be written, based on the offset of the previous data to be written, the data length of the previous data to be written, and the data length of the padding data corresponding to the previous data to be written.

[0122] In one example, the storage device can calculate the quotient of the offset of the data to be written and the size of the page stripe, and multiply the quotient by the page size to determine the product as the offset of the data to be written on each logical disk.

[0123] For example, the page size can be 4kb, meaning the maximum amount of data that can be stored in a page is 4kb. Thus, the page stripe size can be 16kb, and the data to be written can be 32kb. The offset of the data to be written in each logical disk can be calculated using 32kb / 16kb * 4kb, meaning the offset can be 8kb, which corresponds to the second page stripe in the logical disk.

[0124] Step 506: Write the data to be written to the storage space based on the first data length of the data to be written and the offset of the data to be written in the storage space.

[0125] In practice, the storage device can determine the starting storage location of the data to be written in the storage space based on the offset of the data to be written in the storage space, and write the data to be written into the storage space based on the starting storage location and the first data length of the data to be written.

[0126] In this embodiment, the data to be written can be accurately written to the storage space. While ensuring parallel data writing, the writing time is reduced, the writing efficiency is improved, and the utilization rate of storage space is increased, while avoiding the writing of too much redundant zero data.

[0127] In one embodiment, such as Figure 6 As shown, the specific processing steps for the step "writing the data to be written to the storage space" include:

[0128] Step 602: Determine the padding data based on the number of padding strips on the target page.

[0129] In implementation, the storage device can determine the page data that needs to be padded in the target page strip based on the padding number; that is, the padding data. In one example, the storage device can pad the end of the data to be written with data matching the padding number, and use this as padding data. In another example, the storage device can also determine zero data matching the padding number as padding data.

[0130] Step 604: Calculate the verification data based on the data to be written and the supplementary data.

[0131] In practice, the storage device can arrange the data to be written and the padding data according to the data to be written, the padding data, and the pre-configured write rules, and calculate the check data of the data located in the same page strip.

[0132] Step 606: Write the data to be written, the supplementary data, and the verification data into the storage space.

[0133] In practice, the storage device can write the data to be written, the padding data, and the calculated verification data into the storage space according to the pre-configured writing rules.

[0134] In this embodiment, the data to be written can be accurately written to the storage space, ensuring parallel data writing while avoiding the writing of too much redundant data and improving the utilization rate of storage space.

[0135] The specific implementation process of the above embodiments can be described in detail below with reference to the schematic diagrams:

[0136] like Figure 7 As shown, the storage space includes multiple logical disks, such as ldisk0, ldisk1, ldisk2, ldisk3, ldisk4, and ldisk5. ldisk0, ldisk1, ldisk2, and ldisk3 store the raw data (data to be written), while ldisk4 and ldisk5 store verification data calculated based on the raw data. The index order of the pages contained in each logical disk within the storage space is the same. A storage subspace (fixed slice) consisting of a fixed number of consecutive pages on each logical disk can be a chunk. A storage subspace containing chunks with the same index on all logical disks can be a stripe (preset stripe). A storage subspace containing pages with the same index on all logical disks can be a page stripe. The number of page stripes contained in a preset stripe is the same as the number of pages contained in a chunk.

[0137] The data to be written can be object data, such as 66 pages (pages 0-65). The storage device can calculate the remaining number of page data based on the number of pages contained in the data to be written and the number of pages contained in a preset stripe. For example, the remaining page data could be the data contained in page 64 and page 65, and the remaining number could be 2. The storage device can then compare the remaining number of the data to be written with the number of page stripes contained in the preset stripe, or with the number of pages contained in a chunk, to obtain a comparison result. If the storage device determines that the remaining number is greater than zero and less than the number of page stripes contained in the preset stripe, then the storage device can determine that the remaining number of the data to be written satisfies the target padding condition, i.e., 2 is less than 16.

[0138] In this way, the storage device can determine that the data to be written requires a full stripe, that is, the first chunk of each logical disk and two separate pages. Based on this, the storage device can determine that the positions that need to be padded are the remaining positions in page stripe 16 and page stripe 17 excluding ldisk0. That is, all logical disks except ldisk0, ldisk4, and ldisk5 need to be padded with two pages of data, that is, page stripe 16 and page stripe 17 are padded to a full page stripe. The padded data can be zeros.

[0139] In one example, pages 0-15 of the object data can be written to the first chunk of ldisk0 in storage space; pages 16-31 of the object data can be written to the first chunk of ldisk1 in storage space; pages 32-47 of the object data can be written to the first chunk of ldisk2 in storage space; pages 48-63 of the object data can be written to the first chunk of ldisk3 in storage space; the checksum p0 calculated from pages 0, 16, 32, and 48 of the object data can be written to the first page of ldisk4; and the checksum q0 calculated from pages 0, 16, 32, and 48 of the object data can be written to the first page of ldisk5. The writing process for other data to be written, as well as the padding data and the calculated checksum data, is similar to the writing process for the data mentioned above and will not be repeated here; and the offset of the next data to be written on each logical disk is the 18th page, as shown in the example. Figure 7 The offset shown in the storage space is the 72nd offset. The next data to be written can be calculated based on this offset and the data can be written accordingly, which will not be elaborated here.

[0140] The data writing method provided in this application can determine the padding method based on the data length after receiving the data. That is, if the data length is divisible by the stripe, no padding is needed. If it is not divisible and the remainder is greater than or equal to the chunk, the tail data (remainder) is padded to the full stripe. If it is not divisible and the remainder is less than the chunk, it is padded according to the page stripe. Then, the check data is calculated, the offset and length of the written data on each disk are calculated, and the data is written to each disk.

[0141] The following is an example of page stripe completion: In erasure coding, the number of data blocks N is 4, the number of check blocks M is 2, and the chunk size is 64KB (i.e., 16 pages). Therefore, the stripe size is 256KB (i.e., 64 pages). Figure 7As shown in the diagram, the numbers are page numbers, starting from 0. ldisk0, ldisk1, ldisk2, and ldisk3 store data, while ldisk4 and ldisk5 store parity data. If a 264KB object (66 pages) is written, the calculated tail data is 2 pages. The tail data is not a full stripe and is smaller than a chunk. Padding is done according to the page stripes. Only page stripes 16 and 17 need to be padded, that is, only 6 pages with page numbers 66-71 need to be padded. The parity data blocks p16 and q16 on page stripe 16 are calculated by zero-padding data on page numbers 64 and 66, 68, and 70. The calculation process for page stripe 17 is similar to that of page stripe 16 and will not be repeated here.

[0142] The data writing method provided in this application can use different padding methods according to the length of the data being written. This ensures that data does not need to be read from the disk during writing and that different object data is distributed in different stripes or page stripes, thereby ensuring concurrent writing of different object data and improving writing efficiency. When using page stripe padding, write amplification and wasted storage space can be reduced. Since there is no need to limit the size of the chunk to avoid wasting storage space, it has no impact on read performance.

[0143] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0144] Based on the same inventive concept, this application also provides a data writing device for implementing the data writing method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more data writing device embodiments provided below can be found in the limitations of the data writing method described above, and will not be repeated here.

[0145] In one embodiment, such as Figure 8As shown, a data writing device 800 is provided, including: a first acquisition module 801, a calculation module 802, and a first determination module 803, wherein:

[0146] The first acquisition module 801 is used to acquire the number of page data contained in the data to be written;

[0147] The calculation module 802 is used to calculate the remaining number of remaining page data corresponding to the data to be written based on the number of page data contained in the data to be written and the number of pages contained in the preset stripe. The preset stripe includes multiple page stripes.

[0148] The first determining module 803 is used to determine the number of target page strips to be filled in the storage space containing multiple preset stripes when the remaining number meets the target filling condition, and to write the data to be written into the storage space based on the number of target page strips to be filled.

[0149] In one embodiment, the computing module 802 includes:

[0150] The calculation unit is used to divide the number of page data contained in the data to be written by the number of pages contained in the preset stripe to obtain the quotient and the remainder.

[0151] The determination unit is used to determine the remaining number of pages of data corresponding to the data to be written.

[0152] In one embodiment, the data writing device 800 further includes:

[0153] The second acquisition module is used to acquire the number of page strips contained in the preset strip;

[0154] The second determining module is used to determine that the remaining number satisfies the target completion condition if the remaining number is greater than the target value and less than the number of page strips contained in the preset strip.

[0155] The third determining module is used to determine that the remaining number does not meet the target completion condition if the remaining number is equal to the target value, or if the remaining number is greater than or equal to the number of page strips contained in the preset strip.

[0156] In one embodiment, the first determining module 803 is specifically used for:

[0157] In a storage space containing multiple preset stripes, determine the number of target page stripes containing the remaining page data; based on the number of target page stripes, determine the total number of pages contained in the target page stripes; based on the total number and the remaining number corresponding to the remaining page data, calculate the number of pages to be padded.

[0158] In one embodiment, the first determining module 803 is further specifically used for:

[0159] Calculate the first data length of the data to be written based on the number of padding strips on the target page;

[0160] Get the offset of the data to be written in the storage space;

[0161] The data to be written is written to the storage space based on the first data length of the data to be written and the offset of the data to be written in the storage space.

[0162] In one embodiment, the first determining module 803 is further specifically used for:

[0163] The data to be filled in is determined based on the number of strips to be filled in on the target page;

[0164] Calculate the verification data based on the data to be written and the padding data;

[0165] Write the data to be written, the supplementary data, and the verification data to the storage space.

[0166] Each module in the aforementioned data writing device 800 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0167] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data corresponding to write processes. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a data writing method.

[0168] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0169] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0170] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0171] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0172] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0173] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0174] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0175] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data writing method, characterized in that, The method includes: The number of page data contained in the data to be written is obtained, and each page stores one page data; the page data and the page have a one-to-one correspondence. Based on the number of page data contained in the data to be written and the number of pages contained in the preset stripe, calculate the total number of the preset stripe that the data to be written needs to occupy and the remaining number of the corresponding remaining page data. The preset stripe includes multiple page stripes with the same index located on different logical disks. The remaining number is the number of page data that is not a whole preset stripe. If the remaining number satisfies the target padding condition, in a storage space containing multiple preset stripes, the padding number of the target page strip corresponding to the remaining page data is determined, and based on the padding number of the target page strip, the data to be written is written into the storage space. The method further includes: Obtain the number of page strips contained in the preset strip; if the remaining number is greater than the target value and the remaining number is less than the number of page strips contained in the preset strip, then determine that the remaining number satisfies the target completion condition. The step of determining the number of target page strips to be padded in a storage space containing multiple preset stripes includes: In a storage space containing multiple preset stripes, determine the number of target page stripes where the remaining page data is written to the storage space; based on the number of target page stripes, determine the total number of pages contained in the target page stripes; based on the total number and the remaining number corresponding to the remaining page data, calculate the number of padding pages.

2. The method according to claim 1, characterized in that, The step of calculating the remaining number of remaining page data corresponding to the data to be written, based on the number of page data contained in the data to be written and the number of pages contained in the preset stripe, includes: The number of page data contained in the data to be written is divided by the number of pages contained in the preset stripe to obtain the quotient and the remainder. The remainder value is determined to be the remaining number of remaining page data corresponding to the data to be written.

3. The method according to claim 1, characterized in that, The method further includes: If the remaining number is equal to the target value, or if the remaining number is greater than or equal to the number of page strips contained in the preset strip, then it is determined that the remaining number does not meet the target completion condition.

4. The method according to claim 1, characterized in that, The step of writing the data to be written into the storage space based on the number of padding stripes for the target page includes: Based on the number of padding strips on the target page, calculate the first data length of the data to be written; Obtain the offset of the data to be written in the storage space; Based on the first data length of the data to be written and the offset of the data to be written in the storage space, the data to be written is written to the storage space.

5. The method according to claim 4, characterized in that, The step of writing the data to be written into the storage space includes: Based on the number of missing data in the target page strip, determine the missing data; Based on the data to be written and the padding data, calculate the verification data; Write the data to be written, the padding data, and the verification data into the storage space.

6. A data writing device, characterized in that, The device includes: The first acquisition module is used to acquire the number of page data contained in the data to be written, with each page storing one page data; the page data and the page have a one-to-one correspondence. The calculation module is used to calculate the number of pages that the data to be written needs to occupy in the entire preset stripe and the remaining number of pages in the preset stripe, based on the number of pages in the data to be written and the number of pages in the preset stripe. The preset stripe includes multiple page stripes with the same index located on different logical disks, and the remaining number is the number of pages that are not a whole preset stripe. The first determining module is used to determine the number of target page strips to be filled in the storage space containing multiple preset stripes when the remaining number meets the target filling condition, and to write the data to be written into the storage space based on the number of target page strips to be filled. The device further includes: The second acquisition module is used to acquire the number of page strips contained in the preset strip; The second determining module is used to determine that the remaining number satisfies the target completion condition if the remaining number is greater than the target value and the remaining number is less than the number of page strips contained in the preset strip. The first determining module is specifically used to determine, in a storage space containing multiple preset stripes, the number of target page stripes where the remaining page data is written into the storage space; based on the number of target page stripes, determine the total number of pages contained in the target page stripes; and based on the total number and the remaining number corresponding to the remaining page data, calculate the number of padding elements.

7. The apparatus according to claim 6, characterized in that, The device further includes: The third determining module is used to determine that the remaining number does not meet the target completion condition if the remaining number is equal to the target value, or if the remaining number is greater than or equal to the number of page strips contained in the preset strip.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images