A high-efficiency coding storage method and device

Abstract

This invention provides a highly efficient encoding and storage method and apparatus. This method horizontally groups stripes to be encoded to obtain multiple substrips. Information about at least one target substrip for current data storage is sent to an encoding manager. The encoding manager then allocates an idle target CPU core from multiple CPU cores to each target substrip, which then encodes and stores the target substrip. This invention enables simultaneous encoding and storage across multiple CPU cores, maximizing CPU utilization and significantly improving encoding and storage efficiency, reducing user wait time, and enhancing the user experience.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more particularly to a high-efficiency coding storage method and device. Background Art

[0002] In a distributed storage system, all disks in a server cluster are centrally managed and grouped in 4+2, 6+3, 8+4, and other configurations. For example, in a 4+2 configuration, four of the six disks in each group are used to store the original file data, and two are used to store the encoded data. Each disk in each group is divided into N data blocks, and a stripe is formed from one data block from each disk. Therefore, each group corresponds to N stripes.

[0003] When a data block in a stripe that stores raw data is written to, the original data must be encoded and the resulting encoded data stored in the data block that stores the encoded data. However, currently, a stripe exclusively uses one CPU core in the server for encoding and storage. Even if other CPU cores are available, they cannot participate. This results in extremely low encoding and storage efficiency, long user wait times, and a poor user experience. Summary of the Invention

[0004] In view of this, in order to solve the above problems, the present invention provides an efficient encoding storage method and device.

[0005] The technical solution is as follows:

[0006] An efficient coding storage method, the method comprising:

[0007] Obtaining a stripe to be encoded; wherein the stripe to be encoded includes a first data block for storing original data and a second data block for storing encoded data, and the storage space size of the first data block and the second data block is the same;

[0008] The first data block and the second data block are horizontally grouped to obtain a plurality of sub-strips; wherein each of the sub-strips includes a first sub-data block for storing original data and a second sub-data block for storing encoded data, and the storage space size of the first sub-data block and the second sub-data block of each sub-strip is the same;

[0009] Determining at least one target sub-strip for storing data this time from the multiple sub-strips;

[0010] The information of the at least one target sub-stripe is sent to the encoding manager, so that the encoding manager allocates an idle target CPU core from multiple CPU cores to each target sub-stripe, and the target CPU core encodes the data in the first sub-data block of the corresponding target sub-stripe and stores the encoding result in the second sub-data block of the corresponding target sub-stripe.

[0011] Preferably, the determining of at least one target sub-strip for storing data this time from the multiple sub-strips includes:

[0012] Determining whether the space occupied by the currently stored data is smaller than the storage space of the first data block;

[0013] If the space occupied by the currently stored data is smaller than the storage space of the first data block, determining the first sub-data block and offset position last occupied by the previously stored data, and determining the first sub-data block and offset position last occupied by the currently stored data based on the space occupied by the currently stored data;

[0014] Determine whether the first data block corresponding to the first sub-data block last occupied by the last stored data is the same as the first data block corresponding to the first sub-data block last occupied by the current stored data;

[0015] If the first data block corresponding to the first sub-data block last occupied by the data stored last time is the same as the first data block corresponding to the first sub-data block last occupied by the data stored this time, a sub-strip between the offset position last occupied by the data stored last time and the offset position last occupied by the data stored this time among the multiple sub-strips is used as at least one target sub-strip for the data stored this time;

[0016] If the first data block corresponding to the first sub-data block last occupied by the last stored data is different from the first data block corresponding to the first sub-data block last occupied by the current stored data, the remaining sub-strips among the multiple sub-strips except the sub-strips located between the offset position last occupied by the current stored data and the offset position last occupied by the last stored data are used as at least one target sub-strip for the current stored data.

[0017] Preferably, the method further comprises:

[0018] If the space occupied by the currently stored data is not less than the storage space of the first data block, each of the multiple sub-strips is used as a target sub-strip.

[0019] Preferably, if the first sub-data block last occupied by the data stored last time is different from the first sub-data block last occupied by the data stored this time, the method further includes:

[0020] In the case that the offset position last occupied by the data stored this time and the offset position last occupied by the data stored last time correspond to the same sub-strip, the corresponding sub-strip is divided into two new sub-strips; wherein, one new sub-strip is located between the offset position last occupied by the data stored this time and the minimum offset position of the corresponding sub-strip, and the other new sub-strip is located between the offset position last occupied by the data stored last time and the maximum offset position of the corresponding sub-strip.

[0021] Preferably, the sending the information of the at least one target sub-strip to the encoding manager includes:

[0022] The storage location of the currently stored data in each sub-strip is sent to the encoding manager.

[0023] A high-efficiency coding storage device, comprising:

[0024] An acquisition module, configured to acquire a stripe to be encoded; wherein the stripe to be encoded includes a first data block for storing original data and a second data block for storing encoded data, and the storage space size of the first data block and the second data block is the same;

[0025] a grouping module, configured to horizontally group the first data block and the second data block to obtain a plurality of sub-strips; wherein each of the sub-strips includes a first sub-data block for storing original data and a second sub-data block for storing encoded data, and the storage space size of the first sub-data block and the second sub-data block of each sub-strip is the same;

[0026] a determination module, configured to determine at least one target sub-strip for storing data this time from the plurality of sub-strips;

[0027] and a sending module, configured to send information about the at least one target sub-strip to an encoding manager, so that the encoding manager allocates an idle target CPU core from a plurality of CPU cores to each target sub-strip, and the target CPU core encodes data in a first sub-data block corresponding to the target sub-strip and stores the encoding result in a second sub-data block corresponding to the target sub-strip.

[0028] Preferably, the determining module is specifically configured to:

[0029] Determine whether the size of the space occupied by the currently stored data is smaller than the size of the storage space of the first data block; if the size of the space occupied by the currently stored data is smaller than the size of the storage space of the first data block, determine the first sub-data block and the offset position last occupied by the last stored data, and determine the first sub-data block and the offset position last occupied by the currently stored data according to the size of the space occupied by the currently stored data; determine whether the first data block corresponding to the first sub-data block last occupied by the last stored data is the same as the first data block corresponding to the first sub-data block last occupied by the currently stored data; if the first data block corresponding to the first sub-data block last occupied by the last stored data is the same as the first data block corresponding to the first sub-data block last occupied by the currently stored data If the first data block corresponding to the first sub-data block last occupied by the data is the same, the sub-strip located between the offset position last occupied by the last stored data and the offset position last occupied by the current stored data among the multiple sub-strips is used as at least one target sub-strip for the current stored data; if the first data block corresponding to the first sub-data block last occupied by the last stored data is different from the first data block corresponding to the first sub-data block last occupied by the current stored data, the remaining sub-strips among the multiple sub-strips except the sub-strip located between the offset position last occupied by the current stored data and the offset position last occupied by the last stored data are used as at least one target sub-strip for the current stored data.

[0030] Preferably, the determining module is further configured to:

[0031] If the space occupied by the currently stored data is not less than the storage space of the first data block, each of the multiple sub-strips is used as a target sub-strip.

[0032] Preferably, the determining module is further configured to:

[0033] In the case that the offset position last occupied by the data stored this time and the offset position last occupied by the data stored last time correspond to the same sub-strip, the corresponding sub-strip is divided into two new sub-strips; wherein, one new sub-strip is located between the offset position last occupied by the data stored this time and the minimum offset position of the corresponding sub-strip, and the other new sub-strip is located between the offset position last occupied by the data stored last time and the maximum offset position of the corresponding sub-strip.

[0034] Preferably, the sending module is specifically used to:

[0035] The storage location of the currently stored data in each sub-strip is sent to the encoding manager.

[0036] The efficient encoding and storage method and apparatus provided by the present invention horizontally group the stripes to be encoded to obtain multiple sub-strips. Information about at least one target sub-strip for current data storage is sent to an encoding manager. The encoding manager then allocates an idle target CPU core from multiple CPU cores to each target sub-strip, which then encodes and stores the target sub-strip. This invention enables simultaneous encoding and storage across multiple CPU cores, maximizing CPU utilization and significantly improving encoding and storage efficiency, reducing user wait time, and enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0038] Figure 1 A flow chart of a method for efficient encoding and storage provided by an embodiment of the present invention;

[0039] Figure 2 is a schematic diagram of the strips;

[0040] Figure 3 Schematic diagram of horizontal grouping of stripes;

[0041] Figure 4 This is a schematic diagram of data storage;

[0042] Figure 5 A partial method flow chart of the efficient encoding and storage method provided by an embodiment of the present invention;

[0043] Figure 6 (a)-(d) is another schematic diagram of data storage;

[0044] Figure 7 This is another data storage schematic diagram;

[0045] Figure 8 This is another data storage schematic diagram;

[0046] Figure 9 Schematic diagram of stripe division results;

[0047] Figure 10 A schematic diagram of the structure of a high-efficiency coding storage device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] To facilitate understanding, first introduce the concepts related to encoding storage:

[0050] Disk striping: Disk contention occurs when multiple processes access a disk simultaneously. Most disk systems have limits on the number of I / O operations per second and the data transfer rate (the amount of data transferred per second). When these limits are reached, subsequent processes needing to access the disk must wait, a phenomenon known as disk contention. Disk striping involves dividing a continuous block of data into smaller portions and storing them across different disks.

[0051] Data block: A data block is a set of records arranged sequentially and contiguously. It is a unit of data transferred between main memory and input / output devices or external storage. There are three ways to correspond between physical data records and logical data records (logically connected data units occupying a group of adjacent cells in memory): ① A block is a record; ② A block contains several logical records; ③ A logical record occupies several blocks. Data block size can be fixed or variable, with gaps between blocks. The design of data block size is influenced by many factors, including input and output efficiency, storage space cost, and computer application characteristics.

[0052] Each data block has a corresponding index node, which stores the index number of the data block. The file system reads the data of the data block based on the index number.

[0053] Coding system: One of the data storage methods. The data processing method of the coding system is to reduce the space occupancy of the backup storage of 1 to n (n>=1, that is, one piece of metadata has at least one corresponding mirror) and reduce hardware costs. Therefore, the coding system generally uses 2:1, that is, the metadata is 2 and the encoded data is 1.

[0054] The embodiment of the present invention provides an efficient coding storage method, the method flow chart of which is as follows: Figure 1 As shown, the following steps are included:

[0055] S10, obtaining a stripe to be encoded; wherein the stripe to be encoded includes a first data block for storing original data and a second data block for storing encoded data, and the storage space size of the first data block and the second data block is the same.

[0056] For ease of understanding, this embodiment continues to take the 4+2 mode as an example. Figure 2 This is an example of a stripe in this mode, where blocks 1 to 4 represent first data blocks for storing original data, blocks 5 to 6 represent second data blocks for storing encoded data, and the storage space size of each data block from blocks 1 to 6 is the same.

[0057] S20, horizontally grouping the first data block and the second data block to obtain multiple sub-strips; wherein each sub-strip includes a first sub-data block for storing original data and a second sub-data block for storing encoded data, and the storage space size of the first sub-data block and the second sub-data block of each sub-strip is the same.

[0058] In this embodiment, each first data block is horizontally grouped into multiple first sub-data blocks according to a preset space size, and each second data block is horizontally grouped into multiple second sub-data blocks. The first sub-data blocks and the second sub-data blocks in the same position interval form a sub-strip.

[0059] For ease of understanding, this embodiment continues to take the 4+2 mode as an example. Figure 2 The storage space size of each data block in the stripe shown is 4M. In this embodiment, the preset space size is described using 32K as an example:

[0060] like Figure 3 The horizontal grouping diagram shown in FIG. The four first sub-data blocks in the [0, 32K) interval of blocks 1 to 4 and the two second sub-data blocks in the [0, 32K) interval of blocks 5 to 6 form sub-stripe 1. The four first sub-data blocks in the [32K, 64K) interval of blocks 1 to 4 and the two second sub-data blocks in the [32K, 64K) interval of blocks 5 to 6 form sub-stripe 2. Ultimately, 128 sub-stripes are obtained.

[0061] It should be noted that, in order to improve the processing capability of the computer, the preset space size is preferably an integer multiple of the computer's one-time addressing space.

[0062] S30: Determine at least one target sub-strip for storing data this time from the multiple sub-strips.

[0063] In this embodiment, the first sub-data block and offset position last occupied by the last stored data can be determined based on the data position of the last stored data, and then all the first sub-data blocks occupied by the current stored data can be determined in combination with the size of the space occupied by the current stored data. Finally, the sub-strip corresponding to each first sub-data block occupied by the current data is used as a target sub-strip.

[0064] For ease of understanding, this embodiment continues to take the 4+2 mode as an example. Figure 4 In the data storage diagram shown, assuming the data location of the last stored data was 40K, it can be determined that the first sub-data block last occupied by the last stored data is sub-block 2 within block 1, and the offset of sub-block 2 is 40K-32K=8K. Assuming the space occupied by the currently stored data is 121K, the data location of the currently stored data can be determined to be 40K+121K=161K. Accordingly, the first sub-data blocks occupied by the currently stored data are sub-block 2 corresponding to the interval position [32K, 64K), sub-block 3 corresponding to the interval position [64K, 96K), sub-block 4 corresponding to the interval position [96K, 128K), sub-block 5 corresponding to the interval position [128K, 160K), and sub-block 6 corresponding to the interval position [160K, 192K]. Therefore, sub-stripes 2 to 6 corresponding to sub-blocks 2 to 6 within block 1 can be used as the five target sub-stripes.

[0065] In some other embodiments, in order to improve the computational efficiency of determining the target sub-strip, step S30 "determining at least one target sub-strip for storing data this time from multiple sub-strips" can adopt the following steps. A partial method flow chart is shown in FIG. Figure 5 As shown:

[0066] S301, determine whether the space occupied by the currently stored data is smaller than the storage space of the first data block; if so, execute step S302.

[0067] For ease of understanding, this embodiment continues to use the 4+2 mode as an example. As shown in the data storage diagram of Figure 6, the space occupied by the currently stored data increases sequentially based on the previously stored data. In Figure 6(a), the space occupied by the currently stored data is less than 4M, in Figure 6(b), the space occupied by the currently stored data is also less than 4M, in Figure 6(c), the space occupied by the currently stored data is equal to 4M, and in Figure 6(d), the space occupied by the currently stored data is greater than 4M.

[0068] Depend on Figure 6(a) to Figure 6(d) It can be seen that when the space occupied by the currently stored data is not less than (greater than or equal to) the storage space size of the first data block, each sub-strip is a target sub-strip; and when the space occupied by the currently stored data is less than the storage space size of the first data block, only some sub-strips can serve as target sub-strips.

[0069] S302, determining the first sub-data block and offset position last occupied by the last stored data, and determining the first sub-data block and offset position last occupied by the current stored data according to the size of the space occupied by the current stored data.

[0070] During step S302, all first sub-data blocks occupied by the currently stored data can be determined based on the first sub-data block and offset position last occupied by the previously stored data, and in combination with the space occupied by the currently stored data. This process can be found in the description of step S30 above, and will not be repeated here in this embodiment.

[0071] Furthermore, the last occupied first sub-data block and the offset position are determined based on all first sub-data blocks occupied by the currently stored data.

[0072] For ease of understanding, this embodiment continues to take the 4+2 mode as an example. Figure 7 In the data storage diagram shown, assuming that the data position of the last stored data is 40K, it can be determined that the first sub-data block last occupied by the last stored data is sub-block 2 in block 1, and the offset position of sub-block 2 is 40K-32K=8K. Assuming that the space occupied by the currently stored data is 4060K, it can be determined that the data position of the currently stored data is 40K+4060K=4100K. Accordingly, the first sub-data blocks occupied by the currently stored data are sub-block 2 corresponding to the interval position [32K, 64K) in block 1, sub-block 3 corresponding to the interval position [64K, 96K), ..., sub-block 128 corresponding to the interval position [4064K, 4096K), and sub-block 1 corresponding to the interval position [0K, 32K) in block 2, and the offset position of sub-block 1 in block 2 is 4K.

[0073] S303, determine whether the first data block corresponding to the first sub-data block last occupied by the last stored data is the same as the first data block corresponding to the first sub-data block last occupied by the current stored data; if so, execute step S304; if not, execute step S305.

[0074] In this embodiment, whether the currently stored data spans data blocks can be determined by determining whether the first data block corresponding to the first sub-data block last occupied by the last stored data is the same as the first data block corresponding to the first sub-data block last occupied by the currently stored data. Figure 4 As shown in the data storage diagram, the first data block corresponding to the last sub-block 2 occupied by the last stored data is block 1, and the first data block corresponding to the last sub-block 6 occupied by the current stored data is block 1, so it can be determined that the current stored data does not cross data blocks. Figure 7 The data storage diagram shown in the figure shows that the first data block corresponding to the sub-block 2 last occupied by the last stored data is block 1, and the first data block corresponding to the sub-block 1 last occupied by the current stored data is block 2. Therefore, it can be determined that the current stored data spans data blocks.

[0075] S304 : Use a sub-strip located between the offset position last occupied by data stored last time and the offset position last occupied by data stored this time among the multiple sub-strips as at least one target sub-strip for storing data this time.

[0076] Continue to see Figure 4 In the data storage diagram shown, the stored data does not span data blocks. Therefore, sub-stripes between 40KB and 161KB, namely sub-strip 2, sub-strip 3, sub-strip 4, sub-strip 5, and sub-strip 6, can be used as the five target sub-stripes.

[0077] S305 : Use the remaining sub-strips of the multiple sub-strips except the sub-strip located between the offset position last occupied by the currently stored data and the offset position last occupied by the last stored data as at least one target sub-strip for currently storing data.

[0078] Continue to see Figure 7 In the data storage diagram shown, the data being stored spans data blocks. Therefore, the substripe between subblock 1 offset 4K and subblock 2 offset 8K can be removed, and all remaining substripes can be used as target substripes. Since there are no substripes between subblock 1 offset 4K and subblock 2 offset 8K, substripes 1 through 128 can be used as the 128 target substripes.

[0079] For ease of understanding, this embodiment is described by taking another example: Figure 8 In the data storage diagram shown, assuming the data location of the last stored data was 72K, it can be determined that the first sub-data block last occupied by the last stored data was sub-block 3 within block 1, and the offset position of sub-block 3 is 72K-64K=8K. Assuming the space occupied by the currently stored data is 4054K, it can be determined that the data location of the currently stored data is 72K+4054K=4126K. Accordingly, the first sub-data blocks occupied by the currently stored data are sub-block 3 corresponding to the interval position [64K, 96K) within block 1, ..., sub-block 128 corresponding to the interval position [4064K, 4096K], and sub-block 1 corresponding to the interval position [0K, 32K) within block 2, and the offset position of sub-block 1 within block 2 is 30K. Therefore, the sub-stripes between the offset position 30K of sub-block 1 and the offset position 8K of sub-block 3 can be eliminated, and all remaining sub-stripes are used as target sub-stripes. Since the sub-strip 2 is located between the offset position 30K of the sub-block 1 and the offset position 8K of the sub-block 3, the sub-strip 1 and the sub-strip 3 to the sub-strip 128 can be used as 127 target sub-strips.

[0080] In some other embodiments, in order to remove non-currently stored data from the target sub-strip, before using the remaining sub-strips of the multiple sub-strips except the sub-strips located between the offset position last occupied by the currently stored data and the offset position last occupied by the last stored data as the at least one target sub-strip for the currently stored data, the following steps may be further included:

[0081] When the offset position last occupied by the currently stored data and the offset position last occupied by the last stored data correspond to the same sub-stripe, the corresponding sub-stripe is divided into two new sub-stripes; one new sub-stripe is located between the offset position last occupied by the currently stored data and the minimum offset position of the corresponding sub-stripe, and the other new sub-stripe is located between the offset position last occupied by the last stored data and the maximum offset position of the corresponding sub-stripe.

[0082] For ease of understanding, this embodiment continues to use the 4+2 mode as an example. Referring to FIG6(b), stripe 2 can be divided into two new sub-strips a and b. The division result is shown in FIG6(b). Figure 9 .

[0083] S40: Send information about at least one target sub-stripe to a coding manager, so that the coding manager allocates an idle target CPU core from multiple CPU cores to each target sub-stripe. The target CPU core encodes the data in the first sub-data block of the corresponding target sub-stripe and stores the encoding result in the second sub-data block of the corresponding target sub-stripe.

[0084] In this embodiment, at least the stripe identifier of the target sub-stripe can be sent to the encoding manager, and the encoding manager allocates a corresponding target CPU core to the target sub-stripe. The target CPU core encodes the data in the first sub-data block of the corresponding target sub-stripe and stores the encoding result in the second sub-data block of the corresponding target sub-stripe.

[0085] Preferably, the storage location of the currently stored data in the target sub-strip may also be sent to the encoding manager, so that the target CPU core only encodes and stores the currently stored data based on the storage location, thereby reducing the data processing load of the target CPU core.

[0086] It should be noted that if the amount of data stored this time is not an integer multiple of the computer's addressable space, the storage location of the data stored this time can be padded to an integer multiple of the addressable space, and the data in the padded location can be left empty.

[0087] The efficient encoding and storage method provided by embodiments of the present invention horizontally groups the stripes to be encoded to obtain multiple sub-strips. Information about at least one target sub-strip for current data storage is sent to an encoding manager. The encoding manager then allocates an idle target CPU core from multiple CPU cores to each target sub-strip, which then encodes and stores the target sub-strip. This method enables simultaneous encoding and storage across multiple CPU cores, maximizing CPU utilization and significantly improving encoding and storage efficiency, reducing user wait time, and enhancing the user experience.

[0088] Based on the efficient coding and storage method provided in the above embodiment, an embodiment of the present invention provides a device for executing the above efficient coding and storage method. The structural diagram of the device is shown in FIG. Figure 10 As shown, including:

[0089] An acquisition module 10 is configured to acquire a stripe to be encoded, wherein the stripe to be encoded includes a first data block for storing original data and a second data block for storing encoded data, and the storage space size of the first data block and the second data block is the same;

[0090] a grouping module 20 configured to horizontally group the first data block and the second data block to obtain a plurality of sub-strips; wherein each sub-strip includes a first sub-data block for storing original data and a second sub-data block for storing encoded data, and the storage space of the first sub-data block and the second sub-data block of each sub-strip is the same in size;

[0091] A determination module 30, configured to determine at least one target sub-strip for storing data this time from a plurality of sub-strips;

[0092] The sending module 40 is configured to send information about at least one target sub-strip to the encoding manager, so that the encoding manager allocates an idle target CPU core from the multiple CPU cores to each target sub-strip. The target CPU core encodes the data in the first sub-data block of the corresponding target sub-strip and stores the encoding result in the second sub-data block of the corresponding target sub-strip.

[0093] Optionally, the determination module 30 is specifically configured to:

[0094] Determine whether the size of the space occupied by the currently stored data is smaller than the size of the storage space of the first data block; if the size of the space occupied by the currently stored data is smaller than the size of the storage space of the first data block, determine the first sub-data block and offset position last occupied by the last stored data, and determine the first sub-data block and offset position last occupied by the currently stored data according to the size of the space occupied by the currently stored data; determine whether the first data block corresponding to the first sub-data block last occupied by the last stored data is the same as the first data block corresponding to the first sub-data block last occupied by the currently stored data; if the first data block corresponding to the first sub-data block last occupied by the last stored data is the same as the first data block corresponding to the first sub-data block last occupied by the currently stored data If the first data block corresponding to the first sub-data block last occupied by the data is the same, the sub-strip located between the offset position last occupied by the last stored data and the offset position last occupied by the current stored data among the multiple sub-strips is used as at least one target sub-strip for the current stored data; if the first data block corresponding to the first sub-data block last occupied by the last stored data is different from the first data block corresponding to the first sub-data block last occupied by the current stored data, the remaining sub-strips among the multiple sub-strips except the sub-strip located between the offset position last occupied by the current stored data and the offset position last occupied by the last stored data are used as at least one target sub-strip for the current stored data.

[0095] Optionally, the determination module 30 is further configured to:

[0096] If the space occupied by the currently stored data is not less than the storage space of the first data block, each of the multiple sub-strips is used as a target sub-strip.

[0097] Optionally, the determination module 30 is further configured to:

[0098] When the offset position last occupied by the currently stored data and the offset position last occupied by the last stored data correspond to the same sub-stripe, the corresponding sub-stripe is divided into two new sub-stripes; one new sub-stripe is located between the offset position last occupied by the currently stored data and the minimum offset position of the corresponding sub-stripe, and the other new sub-stripe is located between the offset position last occupied by the last stored data and the maximum offset position of the corresponding sub-stripe.

[0099] Optionally, the sending module 40 is specifically configured to:

[0100] The storage location of the currently stored data in each sub-strip is sent to the encoding manager.

[0101] The efficient encoding and storage device provided by an embodiment of the present invention horizontally groups the stripes to be encoded to obtain multiple sub-strips. Information about at least one target sub-strip for current data storage is sent to an encoding manager. The encoding manager then allocates an idle target CPU core from multiple CPU cores to each target sub-strip, which then encodes and stores the target sub-strip. This invention enables simultaneous encoding and storage across multiple CPU cores, maximizing CPU capacity and significantly improving encoding and storage efficiency, reducing user wait time, and enhancing the user experience.

[0102] The above is a detailed introduction to an efficient coding storage method and device provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

[0103] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0104] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that the process, method, article, or apparatus comprising a series of elements inherent to the elements, or also including elements inherent to these processes, methods, articles, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0105] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An efficient coding storage method, characterized in that: The method comprises: Obtaining a stripe to be encoded; wherein the stripe to be encoded includes a first data block for storing original data and a second data block for storing encoded data, and the storage space size of the first data block and the second data block is the same; The first data block and the second data block are horizontally grouped to obtain a plurality of sub-strips; wherein each of the sub-strips includes a first sub-data block for storing original data and a second sub-data block for storing encoded data, and the storage space size of the first sub-data block and the second sub-data block of each sub-strip is the same; Determining whether the space occupied by the currently stored data is smaller than the storage space of the first data block; If the space occupied by the currently stored data is smaller than the storage space of the first data block, determining the first sub-data block and offset position last occupied by the previously stored data, and determining the first sub-data block and offset position last occupied by the currently stored data based on the space occupied by the currently stored data; Determining whether the first data block corresponding to the first sub-data block last occupied by the last stored data is the same as the first data block corresponding to the first sub-data block last occupied by the current stored data; If the first data block corresponding to the first sub-data block last occupied by the data stored last time is the same as the first data block corresponding to the first sub-data block last occupied by the data stored this time, a sub-strip between the offset position last occupied by the data stored last time and the offset position last occupied by the data stored this time among the multiple sub-strips is used as at least one target sub-strip for the data stored this time; If the first data block corresponding to the first sub-data block last occupied by the data stored last time is different from the first data block corresponding to the first sub-data block last occupied by the data stored this time, the remaining sub-strips of the multiple sub-strips except the sub-strips located between the offset position last occupied by the data stored this time and the offset position last occupied by the data stored last time are used as at least one target sub-strip for the data stored this time; The information of the at least one target sub-stripe is sent to the encoding manager, so that the encoding manager allocates an idle target CPU core from multiple CPU cores to each target sub-stripe, and the target CPU core encodes the data in the first sub-data block of the corresponding target sub-stripe and stores the encoding result in the second sub-data block of the corresponding target sub-stripe.

2. The method according to claim 1, characterized in that The method further comprises: If the space occupied by the currently stored data is not less than the storage space of the first data block, each of the multiple sub-strips is used as a target sub-strip.

3. The method according to claim 1, characterized in that If the first sub-data block last occupied by the last stored data is different from the first sub-data block last occupied by the current stored data, the method further includes: In the case that the offset position last occupied by the data stored this time and the offset position last occupied by the data stored last time correspond to the same sub-strip, the corresponding sub-strip is divided into two new sub-strips; wherein, one new sub-strip is located between the offset position last occupied by the data stored this time and the minimum offset position of the corresponding sub-strip, and the other new sub-strip is located between the offset position last occupied by the data stored last time and the maximum offset position of the corresponding sub-strip.

4. The method according to claim 1, wherein The sending the information of the at least one target sub-strip to the encoding manager includes: The storage location of the currently stored data in each sub-strip is sent to the encoding manager.

5. An efficient coding storage device, characterized in that: The device comprises: An acquisition module, configured to acquire a stripe to be encoded; wherein the stripe to be encoded includes a first data block for storing original data and a second data block for storing encoded data, and the storage space size of the first data block and the second data block is the same; a grouping module, configured to horizontally group the first data block and the second data block to obtain a plurality of sub-strips; wherein each of the sub-strips includes a first sub-data block for storing original data and a second sub-data block for storing encoded data, and the storage space size of the first sub-data block and the second sub-data block of each sub-strip is the same; a determination module for determining whether the size of the space occupied by the currently stored data is smaller than the size of the storage space of the first data block; if the size of the space occupied by the currently stored data is smaller than the size of the storage space of the first data block, determining the first sub-data block and offset position last occupied by the previously stored data, and determining the first sub-data block and offset position last occupied by the currently stored data according to the size of the space occupied by the currently stored data; determining whether the first data block corresponding to the last sub-data block of the previously stored data is the same; if the first data block corresponding to the first sub-data block last occupied by the previously stored data is the same as the first data block corresponding to the first sub-data block last occupied by the currently stored data If the first data block last occupied by the last sub-data block of the last stored data is the same as the first corresponding first data block last occupied by the current stored data, the sub-strip located between the offset position last occupied by the last stored data and the offset position last occupied by the current stored data among the multiple sub-strips is used as at least one target sub-strip for the current stored data; if the first data block corresponding to the first sub-data block last occupied by the last stored data is different from the first data block corresponding to the first sub-data block last occupied by the current stored data, the remaining sub-strips among the multiple sub-strips except the sub-strip located between the offset position last occupied by the current stored data and the offset position last occupied by the last stored data are used as at least one target sub-strip for the current stored data; and a sending module, configured to send information about the at least one target sub-strip to an encoding manager, so that the encoding manager allocates an idle target CPU core from a plurality of CPU cores to each target sub-strip, and the target CPU core encodes data in a first sub-data block corresponding to the target sub-strip and stores the encoding result in a second sub-data block corresponding to the target sub-strip.

6. The device according to claim 5, characterized in that The determining module is further configured to: If the space occupied by the currently stored data is not less than the storage space of the first data block, each of the multiple sub-strips is used as a target sub-strip.

7. The device according to claim 5, characterized in that The determining module is further configured to: In the case that the offset position last occupied by the data stored this time and the offset position last occupied by the data stored last time correspond to the same sub-strip, the corresponding sub-strip is divided into two new sub-strips; wherein, one new sub-strip is located between the offset position last occupied by the data stored this time and the minimum offset position of the corresponding sub-strip, and the other new sub-strip is located between the offset position last occupied by the data stored last time and the maximum offset position of the corresponding sub-strip.

8. The device according to claim 5, characterized in that The sending module is specifically used to: The storage location of the currently stored data in each sub-strip is sent to the encoding manager.

Images