Data storage method and device based on distributed object storage and electronic equipment

Abstract

The invention relates to a data storage method and device based on distributed object storage and electronic equipment. The method comprises the following steps: under the condition that a target storage request of a target user is received, obtaining target data to be stored according to the target storage request; the target data is stored in a target storage block in a target storage bank of a data node, the data node comprises a plurality of storage banks, and each storage bank comprises a plurality of storage blocks; a unique object identifier of the target data in the target memory bank is generated, and a corresponding relation between the unique object identifier and a memory block is recorded in the target memory bank; and storing the storage bank unique identifier of the target storage bank and the corresponding relationship between the unique object identifier and the unique user identifier of the target user. The technical problem of storage space resource waste in distributed object storage is solved.

Description

Technical Field

[0001] The present application relates to the field of data storage, and in particular to a data storage method, device and electronic device based on distributed object storage. Background Art

[0002] In the prior art, for distributed object storage, a data node usually stores data in a data block, then generates a unique identifier block id of the data block and a unique object identifier object id of the data object, and stores the block id and object id in a metadata node so that a user can search for data by the block id and object id.

[0003] However, with the above method, as more and more blocks are created, the logical space occupied becomes larger and larger, but the data stored in the block may not be much. Moreover, since both the block ID and the object ID are stored in the metadata node, the blocks cannot be merged, resulting in a waste of storage space resources. Summary of the invention

[0004] The present application provides a data storage method, device, storage medium and electronic device based on distributed object storage to solve the technical problem of storage space resource waste in distributed object storage.

[0005] In a first aspect, the present application provides a data storage method based on distributed object storage, comprising: upon receiving a target storage request from a target user, obtaining the target data to be stored according to the target storage request; storing the target data in a target storage block in a target storage body of a data node, wherein the data node includes multiple storage bodies, and each of the storage bodies includes multiple storage blocks; generating a unique object identifier for the target data in the target storage body, wherein the target storage body records the correspondence between the unique object identifier and the storage block; storing the correspondence between the storage body unique identifier of the target storage body, the unique object identifier and the unique user identifier of the target user.

[0006] In a second aspect, the present application provides a data storage device based on distributed object storage, comprising: an acquisition module, used to acquire the target data to be stored according to the target storage request received from the target user; a processing module, used to store the target data in a target storage block in a target storage body of a data node, wherein the data node includes multiple storage bodies, and each of the storage bodies includes multiple storage blocks; a generation module, used to generate a unique object identifier of the target data in the target storage body, wherein the target storage body records the correspondence between the unique object identifier and the storage block; a storage module, used to store the correspondence between the storage body unique identifier of the target storage body, the unique object identifier and the unique user identifier of the target user.

[0007] As an optional example, the acquisition module includes: a determination unit for splitting the first data into a plurality of target data when the data size of the first data requested to be stored by the target storage request is greater than a predetermined threshold, wherein the data size of each target data is less than or equal to the predetermined threshold; and determining the first data as the target data when the data size of the first data is less than or equal to the predetermined threshold.

[0008] As an optional example, the above-mentioned processing module includes: an allocation unit, which is used to obtain the parameter value of the identification parameter as the unique object identifier of the target data after each target data is stored in any storage block in the target storage body, and increase the parameter value of the identification parameter by 1, wherein the identification parameter is a parameter for allocating the object identifier and has an initial value; and store the binding relationship between the unique storage block identifier of the storage block where the target data is stored and the unique object identifier in the target storage body.

[0009] As an optional example, the above-mentioned processing module also includes: a merging unit, which is used to merge the above-mentioned multiple storage blocks into one storage block after storing the binding relationship between the unique storage block identifier of the storage block where the above-mentioned target data is stored and the above-mentioned unique object identifier in the above-mentioned target storage body, if the multiple storage blocks in the above-mentioned target storage body meet the preset merging conditions; and adjust the binding relationship of the above-mentioned unique object identifiers in the above-mentioned multiple storage blocks to be bound to the above-mentioned one storage block after the merger.

[0010] As an optional example, the above-mentioned merging unit includes: a merging sub-unit, which is used to merge the above-mentioned multiple storage blocks into one storage block when the total size of data stored in the multiple storage blocks in the above-mentioned target storage body is less than the total storage space size of one storage block; or merge the above-mentioned multiple storage blocks into one storage block when the number of access times of the multiple storage blocks in the above-mentioned target storage body within a predetermined time length is less than the access threshold and the total size of data stored in the above-mentioned multiple storage blocks is less than the total storage space size of one storage block.

[0011] As an optional example, the above-mentioned storage module includes: a storage unit, used to determine the above-mentioned unique user identifier as the key in the key-value pair, and determine the storage body unique identifier of the above-mentioned target storage body and the above-mentioned unique object identifier as the value of the key-value pair; and store the above-mentioned key-value pair in the metadata node.

[0012] As an optional example, the above-mentioned device also includes: a search module, which is used to obtain the above-mentioned unique user identifier from the above-mentioned data access request after storing the above-mentioned key-value pair in the metadata node, when receiving the above-mentioned target user's data access request; query the above-mentioned storage body unique identifier and the above-mentioned unique object identifier from the above-mentioned metadata node according to the above-mentioned unique user identifier; determine the above-mentioned target storage body according to the above-mentioned storage body unique identifier; determine the above-mentioned target storage block from the above-mentioned target storage body according to the above-mentioned unique object identifier; and send the above-mentioned target data in the above-mentioned target storage block to the above-mentioned target user.

[0013] In a third aspect, the present application provides an electronic device comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus, wherein the memory stores a computer program, and the processor is configured to implement any of the above-mentioned data storage methods based on distributed object storage when executing the computer program.

[0014] In a fourth aspect, the present application further provides a computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute any of the above-mentioned data storage methods based on distributed object storage in the present application.

[0015] Compared with the prior art, the above technical solution provided by the embodiment of the present application has the following advantages: the method provided by the embodiment of the present application, in distributed object storage, adds a storage volume layer on the upper layer of the block, and a volume includes multiple blocks. After the data is stored in the block, the relationship between the unique object identifier and the block is stored in the volume, and the correspondence between the unique user identifier, the unique object identifier and the storage body unique identifier is stored, so that the user does not need to care about which block the data is stored in, and can find the storage body unique identifier and the unique object identifier according to the unique user identifier, and then find the volume according to the storage body unique identifier, and find the specific block in the volume according to the unique object identifier to obtain the data. Therefore, the framework using this method does not need to record the relationship between the block and the unique user identifier, and provides support for merging blocks, which can reduce the waste of storage space resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0019] Figure 1 A flowchart of a data storage method based on distributed object storage provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of data nodes of a data storage method based on distributed object storage provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of storage target data of a data storage method based on distributed object storage provided in an embodiment of the present application;

[0022] Figure 4 A flowchart of another data storage method based on distributed object storage provided in an embodiment of the present application;

[0023] Figure 5 A flowchart of another data storage method based on distributed object storage provided in an embodiment of the present application;

[0024] Figure 6 A flowchart of another data storage method based on distributed object storage provided in an embodiment of the present application;

[0025] Figure 7 A schematic diagram of the structure of a data storage device based on distributed object storage provided in an embodiment of the present application;

[0026] Figure 8 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0028] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0029] Figure 1 A flowchart of a data storage method based on distributed object storage provided in an embodiment of the present application. Figure 1 As shown, the above-mentioned data storage method based on distributed object storage includes:

[0030] S102, when receiving a target storage request from a target user, obtaining target data to be stored according to the target storage request;

[0031] S104, storing the target data in a target storage block of a target storage body of the data node, wherein the data node includes multiple storage bodies, and each storage body includes multiple storage blocks;

[0032] S106, generating a unique object identifier of the target data in the target storage body, wherein the target storage body records a correspondence between the unique object identifier and the storage block;

[0033] S108, storing the corresponding relationship between the storage body unique identifier, the unique object identifier and the unique user identifier of the target user.

[0034] Distributed object storage is a storage technology that can store data in different places through multiple servers. Distributed object storage technology can have multiple data nodes, which are used to store data, and metadata nodes, which are used to record data nodes for data storage. This solution deploys the volume layer on the data node. Each volume includes multiple storage blocks, and each storage block can store data. Figure 2 As shown, Figure 2 In the example, the data nodes include volume1 and volume2. Volume1 includes block1 and block2, and volume1 has an object list. Volume2 includes block3 and block4, and volume2 has an object list.

[0035] A file or a piece of data can be split into multiple data blocks, each of which is stored in a block. It can be stored in the same block or in different blocks. Figure 3 As shown, a piece of data is split into data block 1, data block 2 and data block 3. Each data block can be a target data. Data block 1 can be stored in block1, data block 2 can be stored in block2, and data block 3 can be stored in block4. Figure 3 This is an example. Multiple database blocks can also be stored in one block. The object list is used to record the data blocks in the volume.

[0036] Multiple blocks under a volume share an object system, and data blocks stored in different blocks have unique object IDs under the volume.

[0037] After storing the data block in the data node, for each data block, the unique user ID of the user who stores the data block can be recorded in the metadata node in the form of a key-value pair, with the unique user ID as the key, and the unique identifier of the volume, volume id, and the unique object ID generated after storing the data block as the value. When the user wants to access the data, he can bring his own unique user ID KEY, query the volume id and objectid in the metadata node through the KEY, query the volume where the data block is located through the volume id, and then find the block where the data block is located from the corresponding relationship between the unique identifier of the data block stored on the volume and the block id, and finally get the data from the corresponding block.

[0038] The method provided in the embodiment of the present application adds a storage volume layer on top of the block in distributed object storage. A volume includes multiple blocks. After the data is stored in the block, the relationship between the unique object identifier and the block is stored in the volume, and the corresponding relationship between the unique user identifier, the unique object identifier and the storage body unique identifier is stored. Therefore, the user does not need to care about which block the data is stored in. The storage body unique identifier and the unique object identifier can be found according to the unique user identifier, and then the volume can be found according to the storage body unique identifier. In the volume, the specific block can be found according to the unique object identifier to obtain the data. Therefore, the framework using this method does not need to record the relationship between the block and the unique user identifier, and provides support for merging blocks, which can reduce the waste of storage space resources.

[0039] As an alternative example, Figure 4 As shown, in the case of receiving a target storage request from a target user, obtaining target data to be stored according to the target storage request includes:

[0040] S402, when the data size of the first data requested to be stored by the target storage request is greater than a predetermined threshold, split the first data into multiple target data, wherein the data size of each target data is less than or equal to the predetermined threshold;

[0041] S404: When the data size of the first data is less than or equal to a predetermined threshold, determine the first data as target data.

[0042] In this embodiment, the first data requested to be stored by the user and the target data can be the same data. If the size of the first data requested to be stored by the user is less than or equal to a predetermined threshold, it is allowed to be stored in the block as the target data. If the data size of the first data requested to be stored by the user is greater than the predetermined threshold, it needs to be split into multiple target data, and the size of each target data is less than or equal to the predetermined threshold. For example, if the predetermined threshold is 5 megabytes and the file size requested to be stored by the user is 100 megabytes, it can be divided into 20 small files of 5 megabytes, or divided into more than 20 files less than 5 megabytes. In short, the size of each small file cannot exceed 5 megabytes. Each small file is stored in a block as a target data.

[0043] As an optional example, generating a unique object identifier for target data in a target storage body includes: after each target data is stored in any storage block in the target storage body, obtaining a parameter value of an identification parameter as the unique object identifier of the target data, and incrementing the parameter value of the identification parameter by 1, wherein the identification parameter is a parameter with an initial value for assigning an object identifier; and storing a binding relationship between the unique storage block identifier of the storage block where the target data is stored and the unique object identifier in the target storage body.

[0044] In this embodiment, different data blocks stored in the same volume share a set of object systems. For example, the data that the user wishes to store is divided into 5 target data, which are stored in volume 1 and volume 2 respectively. Target data 1, target data 2 and target data 3 are stored in volume 1, and target data 4 and target data 5 are stored in volume 2. Then, when target data 1, target data 2 and target data 3 are stored in volume 1, volume 1 allocates a unique object ID to target data 1, target data 2 and target data 3 respectively. If target data 1 is stored first, then target data 3 is stored, and finally target data 2 is stored, then target data 1 can be allocated object ID 001 (for example only), target data 3 can be allocated object ID 002, and target data 2 can be allocated object ID 003, that is, allocated in the order of storage. The unique object IDs of the three target data are different. In volume 2, target data 4 can be allocated object ID 001, and target data 5 can be allocated object ID 002. It can be seen that multiple target data in a volume are distinguished by unique identifiers, and the object IDs of different volumes do not affect each other.

[0045] As an optional example, after storing the binding relationship between the unique storage block identifier and the unique object identifier of the storage block where the target data is stored in the target storage body, the above method also includes: if multiple storage blocks in the target storage body meet preset merging conditions, merging multiple storage blocks into one storage block; and adjusting the binding relationship of the unique object identifiers in the multiple storage blocks to be bound to the merged storage block.

[0046] In this embodiment, a volume may correspond to a maximum number of blocks. For example, a volume may include a maximum of 10 blocks. Then, if the 10 blocks meet the conditions, block merging may be performed. The merging may be the merging of two blocks or the merging of multiple blocks. After the merging, the number of blocks in a volume is less than ten.

[0047] Since the volume records the target data stored in all blocks on the volume (recording the relationship between the block and the unique object ID of the stored target data), when the block is merged, the corresponding relationship between the object ID and the block also needs to be adjusted. Since the corresponding relationship is stored in the volume, it can be adjusted. It will not affect the key-value data stored in the metadata node.

[0048] As an optional example, when multiple storage blocks in the target storage body meet preset merging conditions, merging the multiple storage blocks into one storage block includes: when the total size of data stored in the multiple storage blocks in the target storage body is less than the total storage space size of one storage block, merging the multiple storage blocks into one storage block; or when the number of accesses to the multiple storage blocks in the target storage body within a predetermined time period is less than an access threshold and the total size of data stored in the multiple storage blocks is less than the total storage space size of one storage block, merging the multiple storage blocks into one storage block.

[0049] In this embodiment, whether to merge multiple blocks can be determined based on a merge condition. The merge condition can be the total size of the data stored in the block. For example, a block can store up to 8G of data, but multiple blocks only store 1G of data (1G is just an example, less than 8G of data), then multiple blocks can be merged. In other words, if the data that can be stored in one block is actually stored in multiple blocks, then multiple blocks can be merged. In addition, it can also be determined based on the access frequency. If the access frequency of the data in multiple blocks is very low, and the total size of the data stored in multiple blocks is less than 8G, then multiple blocks can be merged. After the merger, the number of blocks in the volume is less than the maximum number. For example, if the maximum number is 10, then the number of blocks is less than 10. If there is a demand for blocks, the number of blocks can be increased to 10.

[0050] As an alternative example, Figure 5 As shown, the correspondence between the storage body unique identifier of the storage target storage body, the unique object identifier and the unique user identifier of the target user includes:

[0051] S502, determining the unique user identifier as the key key in the key-value pair, and determining the storage body unique identifier and the unique object identifier of the target storage body as the value value of the key-value pair;

[0052] S504, storing the key-value pair in the metadata node.

[0053] In this embodiment, the target data is stored in the block, and the block belongs to the volume. Therefore, the volume in which the target data is stored can be found by recording the volumeid, and the block in which the target data is stored can be found by recording the object id (the volume record has the relationship between the object id and the block id). The user's unique user ID is used to distinguish different users. Therefore, the unique user ID is used as the key of the key-value pair, and the object id and volume id are used as the values ​​of the key-value pair and stored on the metadata node.

[0054] As an alternative example, Figure 6 As shown, after storing the key-value pair in the metadata node, the method further includes:

[0055] S602, when receiving a data access request from a target user, obtaining a unique user identifier from the data access request;

[0056] S604, querying the storage body unique identifier and the unique object identifier from the metadata node according to the unique user identifier;

[0057] S606, determining a target storage body according to the storage body unique identifier;

[0058] S608, determining a target storage block from the target storage body according to the unique object identifier;

[0059] S610: Send target data in the target storage block to a target user.

[0060] When a user wants to access data stored in a data node, he can send a data access request. After receiving the user's data access request, the distributed object storage system obtains the user's unique user ID (i.e., the key used for querying in the key-value pair) from the data access request. Using this unique user ID, the volume id and object id are searched from the metadata node. Then, the volume where the target data is located is determined based on the volume id, and the block where the target data is located is determined from the volume based on the object id, and finally the target data is obtained from the block.

[0061] Let's use an example to illustrate. For example, the maximum data that can be stored in each block is 8G, a volume includes 10 blocks, and the first data that the user requests to store is 100M. The user initiates a request to store the first data of 100M. First, because the 100M first data exceeds the limit, such as the limit is 5M, the 100M first data is divided into 20 5M target data. Among the 20 5M target data, each target data is stored in a block. For example, 5 target data are stored in block 1 of volume 1, and blocks 2-10 of volume 1 store one target data each, and the remaining 6 target data are stored in block 1 of volume 2. Then, first of all, each volume needs to record the stored target data and generate an object directory. Volume 1 stores a total of 14 target data, of which block 1 stores 6, and the remaining blocks 2-10 store 1 each. The object directory of volume 1 includes 14 object ids, corresponding to the 14 target data, and which block the 14 target data are recorded on. The 14 object ids are all different. For example, they are 001-014. The target data saved first generates the object id earlier. Volume 2 also generates an object directory, which contains 6 object ids, for example, 001-006. It can be seen that the object id in the same volume is unique, and the object ids in different volumes can be repeated. After the object id is generated, the user's unique user ID, the volume id of the volume, and the object id can be stored as a key-value pair in the metadata node.

[0062] When a user wants to access data, he can carry the key of the key-value pair in the access request. The metadata node queries volume 1 and volume 2, as well as 20 object IDs based on the key. Among them, volume 1 includes 14 object IDs, and volume 2 includes 6 object IDs. Then, in volume 1 and volume 2, the object is queried respectively to determine the block where each target data is located, and the target data is returned to the user.

[0063] In the process of storing data, such as block 1 to block 10 of volume 1 and block 1 to block 10 of volume 2, if the total amount of data stored in multiple blocks is small, you can merge the blocks, such as merging block 2 and block 5 of volume 1 into a new block, which contains the data of block 2 and block 5. Then the volume contains 9 blocks. The merging condition of the blocks can be that the total amount of data in multiple blocks is too small, such as less than the preset value, or less than the maximum storage data value of the block. The preset value is less than the maximum storage data value.

[0064] Figure 7 The following is a schematic diagram of a data storage device based on distributed object storage provided in an embodiment of the present application. Figure 7 As shown, the above-mentioned data storage device based on distributed object storage includes:

[0065] An acquisition module 702 is used to acquire target data to be stored according to the target storage request when receiving a target storage request from a target user;

[0066] Processing module 704, configured to store the target data in a target storage block in a target storage body of the data node, wherein the data node includes multiple storage bodies, and each storage body includes multiple storage blocks;

[0067] A generating module 706, configured to generate a unique object identifier of the target data in a target storage body, wherein the target storage body records a correspondence between the unique object identifier and the storage block;

[0068] The storage module 708 is used to store the correspondence between the storage body unique identifier of the target storage body, the unique object identifier and the unique user identifier of the target user.

[0069] Distributed object storage is a storage technology that can store data in different places through multiple servers. Distributed object storage technology can have multiple data nodes, which are used to store data, and metadata nodes, which are used to record data nodes for data storage. This solution deploys the volume layer on the data node. Each volume includes multiple storage blocks, and each storage block can store data. Figure 2 As shown, Figure 2In the example, the data nodes include volume1 and volume2. Volume1 includes block1 and block2, and volume1 has an object list. Volume2 includes block3 and block4, and volume2 has an object list.

[0070] A file or a piece of data can be split into multiple data blocks, each of which is stored in a block. It can be stored in the same block or in different blocks. Figure 3 As shown, a piece of data is split into data block 1, data block 2 and data block 3. Each data block can be a target data. Data block 1 can be stored in block1, data block 2 can be stored in block2, and data block 3 can be stored in block4. Figure 3 This is an example. Multiple database blocks can also be stored in one block. The object list is used to record the data blocks in the volume.

[0071] Multiple blocks under a volume share an object system, and data blocks stored in different blocks have unique object IDs under the volume.

[0072] After storing the data block in the data node, for each data block, the unique user ID of the user who stores the data block can be recorded in the metadata node in the form of a key-value pair, with the unique user ID as the key, and the unique identifier of the volume, volume id, and the unique object ID generated after storing the data block as the value. When the user wants to access the data, he can bring his own unique user ID KEY, query the volume id and objectid in the metadata node through the KEY, query the volume where the data block is located through the volume id, and then find the block where the data block is located from the corresponding relationship between the unique identifier of the data block stored on the volume and the block id, and finally get the data from the corresponding block.

[0073] The method provided in the embodiment of the present application adds a storage volume layer on top of the block in distributed object storage. A volume includes multiple blocks. After the data is stored in the block, the relationship between the unique object identifier and the block is stored in the volume, and the corresponding relationship between the unique user identifier, the unique object identifier and the storage body unique identifier is stored. Therefore, the user does not need to care about which block the data is stored in. The storage body unique identifier and the unique object identifier can be found according to the unique user identifier, and then the volume can be found according to the storage body unique identifier. In the volume, the specific block can be found according to the unique object identifier to obtain the data. Therefore, the framework using this method does not need to record the relationship between the block and the unique user identifier, and provides support for merging blocks, which can reduce the waste of storage space resources.

[0074] For other examples of this embodiment, please refer to the above examples and will not be described in detail here.

[0075] like Figure 8 As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0076] Memory 113, used for storing computer programs;

[0077] In one embodiment of the present application, the processor 111 is used to implement the data storage method based on distributed object storage provided by any one of the aforementioned method embodiments when executing the program stored in the memory 113.

[0078] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of providing a data storage method based on distributed object storage as described in any of the aforementioned method embodiments are implemented.

[0079] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0080] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0081] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0082] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A data storage method based on distributed object storage, characterized in that: include: When receiving a target storage request from a target user, acquiring target data to be stored according to the target storage request; Storing the target data in a target storage block of a target storage body of a data node, wherein the data node includes a plurality of storage bodies, and each of the storage bodies includes a plurality of storage blocks; Generate a unique object identifier of the target data in the target storage body, wherein the target storage body records a corresponding relationship between the unique object identifier and the storage block; The corresponding relationship among the storage body unique identifier of the target storage body, the unique object identifier and the unique user identifier of the target user is stored.

2. The method according to claim 1, characterized in that In the case of receiving a target storage request from a target user, acquiring target data to be stored according to the target storage request includes: In a case where the data size of the first data requested to be stored by the target storage request is greater than a predetermined threshold, splitting the first data into a plurality of target data, wherein the data size of each of the target data is less than or equal to the predetermined threshold; In a case where the data size of the first data is smaller than or equal to the predetermined threshold, the first data is determined as the target data.

3. The method according to claim 1, characterized in that Generating a unique object identifier of the target data in the target storage body includes: After each target data is stored in any storage block in the target storage body, a parameter value of an identification parameter is obtained as the unique object identifier of the target data, and the parameter value of the identification parameter is incremented by 1, wherein the identification parameter is a parameter with an initial value for allocating an object identifier; The binding relationship between the unique storage block identifier of the storage block where the target data is stored and the unique object identifier is stored in the target storage body.

4. The method according to claim 3, characterized in that After storing the binding relationship between the unique storage block identifier of the storage block where the target data is stored and the unique object identifier in the target storage body, the method further includes: When multiple storage blocks in the target storage body meet a preset merging condition, merging the multiple storage blocks into one storage block; The binding relationship of the unique object identifiers in the multiple storage blocks is adjusted to be bound to the merged one storage block.

5. The method according to claim 4, characterized in that When the plurality of storage blocks in the target storage body meet the preset merging condition, merging the plurality of storage blocks into one storage block comprises: In a case where the total size of data stored in the multiple storage blocks in the target storage body is smaller than the total storage space size of one storage block, merging the multiple storage blocks into one storage block; or When the number of accesses to multiple storage blocks in the target storage body within a predetermined time period is less than an access threshold and the total size of data stored in the multiple storage blocks is less than the total storage space size of one storage block, the multiple storage blocks are merged into one storage block.

6. The method according to claim 1, characterized in that The correspondence between the storage body unique identifier of the target storage body, the unique object identifier and the unique user identifier of the target user includes: Determine the unique user identifier as a key in a key-value pair, and determine the storage body unique identifier and the unique object identifier of the target storage body as values ​​of the key-value pair; The key-value pair is stored in a metadata node.

7. The method according to claim 6, characterized in that After storing the key-value pair in the metadata node, the method further includes: Upon receiving a data access request from the target user, obtaining the unique user identifier from the data access request; Query the storage body unique identifier and the unique object identifier from the metadata node according to the unique user identifier; Determine the target storage body according to the storage body unique identifier; Determining the target storage block from the target storage body according to the unique object identifier; The target data in the target storage block is sent to the target user.

8. A data storage device based on distributed object storage, characterized in that: include: An acquisition module, configured to, upon receiving a target storage request from a target user, acquire the target data to be stored according to the target storage request; A processing module, configured to store the target data in a target storage block of a target storage body of a data node, wherein the data node includes a plurality of storage bodies, and each of the storage bodies includes a plurality of storage blocks; A generating module, configured to generate a unique object identifier of the target data in the target storage body, wherein the target storage body records a corresponding relationship between the unique object identifier and a storage block; The storage module is used to store the corresponding relationship between the storage body unique identifier of the target storage body, the unique object identifier and the unique user identifier of the target user.

9. An electronic device, characterized in that: include: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor coupled to the at least one bus; At least one memory connected to the at least one bus, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the method described in any one of claims 1 to 7 of the present application.

Images