Data storage method, device, equipment, medium and program product

Abstract

The present invention provides a data storage method, device, equipment, medium and program product, the data storage method comprising: in response to a received new data request, obtaining a data table identifier, data information to be stored, a user identifier and a mapping relationship table of a distributed database in the new data request; determining a user level based on the user identifier and a mapping relationship between the user identifier and the user level; determining multiple storage nodes corresponding to the data table based on the data table identifier and a mapping relationship between the data table identifier and the storage node; determining at least one candidate node from the multiple storage nodes based on the user level; obtaining a storage data shard value based on the data information to be stored using a preset sharding algorithm; determining a target node from at least one candidate node based on the storage data shard value and a mapping relationship between the storage node and the shard value; storing the data to be stored on the target node, the target node being used to provide storage services for the user corresponding to the user identifier.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing, and in particular to a data storage method, device, equipment, medium and program product. Background Art

[0002] With the development of technology, more and more users choose to store data in cloud databases. Cloud databases have high availability and high security, and can provide users with convenient and efficient data storage solutions.

[0003] In related technologies, memory, processor and other resources are generally divided through resource pools, and users are then bound to resource pools to isolate multi-user node group resources. However, this method evenly distributes a user's data throughout the distributed database when storing data, resulting in that if a node has a problem, the data stored by all users will be affected. Summary of the invention

[0004] In view of the above problems, the present invention provides a data storage method, apparatus, device, medium and program product.

[0005] According to a first aspect of the present invention, a data storage method is provided, the method is applied to a distributed database, the distributed database includes multiple storage nodes, the method includes: in response to a received new data request, obtaining a mapping relationship table between a data table identifier, data information to be stored, a user identifier and the distributed database in the new data request, the mapping relationship table including a mapping relationship between a user identifier and a user level, a mapping relationship between a data table identifier and a storage node, and a mapping relationship between a storage node and a hash value; determining a user level based on the user identifier and the mapping relationship between the user identifier and the user level; determining multiple storage nodes corresponding to the data table based on the data table identifier and the mapping relationship between the data table identifier and the storage node; determining at least one candidate node from the multiple storage nodes based on the user level; obtaining a storage data shard value based on the data information to be stored using a preset sharding algorithm; determining a target node from the at least one candidate node based on the storage data shard value and the mapping relationship between the storage node and the shard value; storing the data to be stored on the target node, the target node being used to provide storage services for the user corresponding to the user identifier.

[0006] According to an embodiment of the present invention, the above-mentioned mapping relationship table also includes: a mapping relationship between a user identifier and the above-mentioned storage node. Before the above-mentioned response to the received new data request obtains the data table identifier, the data information to be stored, the user identifier in the above-mentioned new data request and the mapping relationship table of the above-mentioned distributed database, it also includes: in response to the received table creation request, obtains the user identifier in the above-mentioned table creation request, the mapping relationship table of the data table information to be processed and the above-mentioned distributed database; based on the above-mentioned user identifier and the mapping relationship between the above-mentioned user identifier and the above-mentioned storage node, determines a plurality of storage nodes corresponding to the above-mentioned user identifier; generates a data table based on the above-mentioned data table information to be processed and the plurality of storage nodes corresponding to the above-mentioned user identifier.

[0007] According to an embodiment of the present invention, before the above-mentioned response to the received table creation request, obtaining the user identifier in the above-mentioned table creation request, the data table information to be processed and the mapping relationship table of the above-mentioned distributed database, it includes: responding to the received new user request, obtaining the mapping relationship between the user identifier in the above-mentioned new user request and the above-mentioned storage node; storing the mapping relationship between the above-mentioned user identifier and the above-mentioned storage node in the above-mentioned mapping relationship table.

[0008] According to an embodiment of the present invention, the above method also includes: in response to a scheduled backup task being triggered, obtaining multiple binding nodes of a preset target user; physically backing up the above multiple binding nodes to obtain backup information of the above preset target user, and the backup information is used to restore the data of the preset target user when the binding storage node fails.

[0009] According to an embodiment of the present invention, the above-mentioned preset sharding algorithm includes a hash algorithm, and the above-mentioned use of the preset sharding algorithm to obtain the storage data sharding value based on the above-mentioned data information to be stored includes: generating a data key value based on the above-mentioned data information to be stored; using a hash function to map the above-mentioned data key value to a storage data sharding value, the above-mentioned storage data sharding value is a hash value, and the above-mentioned sharding value is used to determine the target node corresponding to the storage space from the storage node having a mapping relationship with the user identifier.

[0010] According to an embodiment of the present invention, the above-mentioned mapping relationship table also includes a mapping relationship between storage nodes and node levels, and the above-mentioned determining at least one candidate node from the above-mentioned multiple storage nodes based on the above-mentioned user level includes: determining the node level corresponding to the storage node based on the above-mentioned multiple storage nodes and the mapping relationship between the above-mentioned storage nodes and node levels; determining at least one candidate node from the above-mentioned multiple storage nodes based on the above-mentioned node level and the above-mentioned user level, the node level of the above-mentioned storage node being less than or equal to the storage node level corresponding to the above-mentioned user level.

[0011] The second aspect of the present invention provides a data storage device, which is configured in a distributed database, and the distributed database includes multiple nodes, including: an acquisition module, which is used to respond to a received new data request, obtain a data table identifier in the new data request, data information to be stored, a user identifier and a mapping relationship table of the above-mentioned distributed database, the above-mentioned mapping relationship table including a mapping relationship between a user identifier and a user level, a mapping relationship between a data table identifier and a storage node, and a mapping relationship between a storage node and a hash value; a user level determination module, which is used to determine a user level based on the above-mentioned user identifier and the mapping relationship between the above-mentioned user identifier and the user level; a storage node determination module, which is used to determine a user level based on the above-mentioned user identifier and the mapping relationship between the above-mentioned user identifier and the user level; The above-mentioned data table identifier and the mapping relationship between the above-mentioned data table identifier and the storage node determine multiple storage nodes corresponding to the above-mentioned data table; a candidate node determination module is used to determine at least one candidate node from the above-mentioned multiple storage nodes based on the above-mentioned user level; a mapping module is used to obtain a storage data shard value based on the above-mentioned data information to be stored using a preset sharding algorithm; a target node determination module is used to determine a target node from the above-mentioned at least one candidate node based on the above-mentioned storage data shard value and the mapping relationship between the above-mentioned storage node and the shard value; a storage module is used to store the above-mentioned data to be stored on the above-mentioned target node, and the target node is used to provide storage services for the user corresponding to the user identifier.

[0012] According to an embodiment of the present invention, the above-mentioned mapping relationship table also includes: a mapping relationship between the user identifier and the above-mentioned storage node, and the above-mentioned device also includes: a table creation acquisition module, which is used to respond to the received table creation request and obtain the user identifier, the data table information to be processed and the mapping relationship table of the above-mentioned distributed database in the above-mentioned table creation request; a storage node determination module, which is used to determine the multiple storage nodes corresponding to the above-mentioned user identifier based on the above-mentioned user identifier and the mapping relationship between the above-mentioned user identifier and the above-mentioned storage node; a data table generation module, which is used to generate a data table based on the above-mentioned data table information to be processed and the multiple storage nodes corresponding to the above-mentioned user identifier.

[0013] According to an embodiment of the present invention, the above-mentioned device also includes: a new user acquisition module, which is used to respond to the received new user request and obtain the mapping relationship between the user identifier in the above-mentioned new user request and the above-mentioned storage node; a mapping relationship storage module, which is used to store the mapping relationship between the above-mentioned user identifier and the above-mentioned storage node in the above-mentioned mapping relationship table.

[0014] According to an embodiment of the present invention, the above-mentioned device also includes: a backup acquisition module, which is used to obtain multiple binding nodes of the preset target user in response to the triggering of a scheduled backup task; a backup module, which is used to physically back up the above-mentioned multiple binding nodes to obtain the backup information of the above-mentioned preset target user, and the above-mentioned backup information is used to restore the data of the preset target user when the binding storage node fails.

[0015] According to an embodiment of the present invention, the above-mentioned preset sharding algorithm includes a hash algorithm, and the above-mentioned mapping module includes: a data key value generation submodule, used to generate a data key value based on the above-mentioned data information to be stored; a shard value submodule, using a hash function, mapping the above-mentioned data key value to a storage data shard value, the above-mentioned storage data shard value is a hash value, and the above-mentioned shard value is used to determine the target node corresponding to the storage space from the storage node having a mapping relationship with the user identifier.

[0016] According to an embodiment of the present invention, the above-mentioned mapping relationship table also includes a mapping relationship between storage nodes and node levels, and the above-mentioned candidate node determination module includes: a node level determination submodule, used to determine the node level corresponding to the storage node based on the above-mentioned multiple storage nodes and the mapping relationship between the above-mentioned storage nodes and node levels; a level determination submodule, used to determine at least one candidate node from the above-mentioned multiple storage nodes based on the above-mentioned node level and the above-mentioned user level, and the node level of the above-mentioned storage node is less than or equal to the storage node level corresponding to the above-mentioned user level.

[0017] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0018] The fourth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0019] The fifth aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0020] According to an embodiment of the present invention, when new data is added, a mapping relationship table stored in a distributed database is queried to determine at least one candidate node from multiple storage nodes based on the user level, and then a target node is determined by a preset sharding algorithm, and the data to be stored is stored on the target node. The user can be bound to the storage node, so that when a failure occurs, only the corresponding user will be affected, reducing the scope of the failure impact, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0022] Figure 1 An application scenario diagram of a data storage method and device according to an embodiment of the present invention is shown;

[0023] Figure 2 A flow chart of a data storage method according to an embodiment of the present invention is shown;

[0024] Figure 3 A schematic diagram of data storage in the related art is shown;

[0025] Figure 4 A schematic diagram of data storage according to an embodiment of the present invention is shown;

[0026] Figure 5 A structural block diagram of a data storage device according to an embodiment of the present invention is shown; and

[0027] Figure 6 A block diagram of an electronic device suitable for implementing a data storage method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0028] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0029] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0030] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0031] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0032] In the current related technologies, database systems generally manage memory, processors, data input and output through resource pools. Users are bound to corresponding resource pools to achieve on-demand allocation and isolation of system resources according to users. However, this method has a coarse granularity and cannot completely isolate user resources. When a storage node fails, all users will be affected, and the user experience will be poor.

[0033] In view of this, an embodiment of the present invention provides a data storage method, apparatus, device, medium and program product, the above method is applied to a distributed database, the distributed database includes multiple storage nodes, the above method includes: in response to a received new data request, obtaining a data table identifier, data information to be stored, a user identifier and a mapping relationship table of the distributed database in the new data request, the mapping relationship table including a mapping relationship between the user identifier and the user level, a mapping relationship between the data table identifier and the storage node, and a mapping relationship between the storage node and the hash value; determining the user level based on the user identifier and the mapping relationship between the user identifier and the user level; determining multiple storage nodes corresponding to the data table based on the data table identifier and the mapping relationship between the data table identifier and the storage node; determining at least one candidate node from the multiple storage nodes based on the user level; using a hash algorithm to obtain a hash value of the stored data based on the data information to be stored; determining a target node from at least one candidate node based on the hash value of the stored data and the mapping relationship between the storage node and the hash value; storing the data to be stored on the target node.

[0034] Figure 1 An application scenario diagram of a data storage method and device according to an embodiment of the present invention is shown.

[0035] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0036] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0037] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0038] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0039] It should be noted that the data storage method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the data storage device provided in the embodiment of the present invention can generally be set in the server 105. The data storage method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the data storage device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0040] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0041] The following will be based on Figure 1 The scene described by Figure 2~Figure 4 The data storage method according to the embodiment of the present invention is described in detail.

[0042] Figure 2 A flow chart of a data storage method according to an embodiment of the present invention is shown.

[0043] like Figure 2 As shown, the data storage method of this embodiment includes operations S210 to S270.

[0044] In operation S210, in response to the received new data request, a mapping relationship table of a data table identifier, data information to be stored, a user identifier and a distributed database in the new data request is obtained.

[0045] The mapping relationship table includes a mapping relationship between a user identifier and a user level, a mapping relationship between a data table identifier and a storage node, and a mapping relationship between a storage node and a hash value.

[0046] According to an embodiment of the present invention, the mapping relationship between the data table identifier and the storage node may be stored in the data table as a distribution key attribute of the table, for example.

[0047] In operation S220, the user level is determined based on the user identification and the mapping relationship between the user identification and the user level.

[0048] Exemplarily, the mapping relationship table stores a mapping relationship between user identifiers and user levels, which can be expressed as: userid: 1, userlevel: 2. That is, the user whose user identifier is 1 has a user level of 2.

[0049] In operation S230 , a plurality of storage nodes corresponding to the data table are determined based on the data table identifier and a mapping relationship between the data table identifier and the storage node.

[0050] Optionally, the user level may be included in the new data request to determine a plurality of storage nodes corresponding to the data table; the determined plurality of storage nodes may also be directly included in the message of the new request, which is not limited in the present invention.

[0051] In operation S240, at least one candidate node is determined from among a plurality of storage nodes based on the user level.

[0052] According to an embodiment of the present invention, the mapping relationship table may include a mapping relationship between a user level and a storage node, and a user may query the mapping relationship table of the distributed database based on the user level to obtain a candidate node.

[0053] Exemplarily, the mapping relationship between user level and storage node can be expressed as: user level: 1, available nodes: node 1, node 2; user level: 2, available nodes: node 3, node 4. The mapping relationship between user level and storage node is determined based on the physical storage space, network resources and other attributes of the storage node. For storage nodes with larger physical storage space, they can be set to have a mapping relationship with higher-level users.

[0054] In operation S250, a preset sharding algorithm is used to obtain a storage data sharding value based on the data information to be stored.

[0055] According to an embodiment of the present invention, the preset sharding algorithm may be, for example, a hash algorithm, a range algorithm or a list algorithm, and the present invention does not impose any limitation on this.

[0056] In operation S260, a target node is determined from at least one candidate node based on the storage data shard value and a mapping relationship between the storage node and the shard value.

[0057] In operation S270, the data to be stored is stored in a target node, where the target node is used to provide storage services for the user corresponding to the user identifier.

[0058] Figure 3 A schematic diagram of data storage in the related art is shown.

[0059] like Figure 3 As shown, in the related art, for tables created by different users, the distributed database is generally distributed to the default node group by default when storing data, that is, user 1 stores data 1-6 (all in diagonal patterns) in nodes 1-6, and user 2 stores data 1-9 and data A-C (all in checkered patterns) in nodes 1-6. As a result, any node stores part of the data of all users. Taking node 1 as an example, it stores data 1 of user 1, data 1 of user 2, and data 7 of user 2.

[0060] Figure 4 A schematic diagram of data storage according to an embodiment of the present invention is shown.

[0061] like Figure 4 As shown, the data stored according to the embodiment of the present invention binds the user to the node group, that is, user 1 stores data 1 to 6 (all in diagonal patterns) on node 1 and node 2; user 2 stores data 1 to 9 and data A to C (all in checkered patterns) on nodes 3 to 6. According to the data storage method of the embodiment of the present invention, the user is bound to the storage node, so that when data is stored, it is fixedly stored on the node to which the user is bound.

[0062] According to an embodiment of the present invention, when new data is added, a mapping relationship table stored in a distributed database is queried to determine at least one candidate node from multiple storage nodes based on the user level, and then a target node is determined by a hash algorithm, and the data to be stored is stored on the target node. The user can be bound to the storage node, so that when a failure occurs, only the corresponding user will be affected, reducing the scope of the failure impact, thereby improving the user experience.

[0063] According to an embodiment of the present invention, the above-mentioned mapping relationship table also includes a mapping relationship between a user identifier and a storage node. Before obtaining the data table identifier, the data information to be stored, the user identifier and the mapping relationship table of the distributed database in the new data request in response to a received new data request, it also includes: in response to a received table creation request, obtaining the user identifier, the data table information to be processed and the mapping relationship table of the distributed database in the table creation request; determining multiple storage nodes corresponding to the user identifier based on the user identifier and the mapping relationship between the user identifier and the storage node; and generating a data table based on the data table information to be processed and the multiple storage nodes corresponding to the user identifier.

[0064] According to an embodiment of the present invention, a user-specific table can be created after logging in. When creating the table, the database will search for the corresponding storage node group in the mapping relationship table in the distributed database based on the currently logged in user, and use it as the distribution key attribute of the data table.

[0065] According to an embodiment of the present invention, before the above-mentioned response to the received table creation request, obtaining the user identifier in the table creation request, the information of the data table to be processed and the mapping relationship table of the distributed database, includes: responding to the received new user request, obtaining the mapping relationship between the user identifier in the new user request and the storage node; storing the mapping relationship between the user identifier and the storage node in the mapping relationship table.

[0066] For example, a new user request of "CREATE USER [user_name] PASSWORD '[pwd]' NODE GROUP [logic_cluster_name]" is used as an example, where user_name may represent a user ID, and logic_cluster_name may represent a storage node group, which may include multiple storage nodes. The mapping relationship between the user ID and the storage node may be stored in a mapping relationship table.

[0067] According to an embodiment of the present invention, the above storage node group can be combined and divided by a database administrator according to business requirements, and the node group can be created using "CREATE NODE GROUP".

[0068] According to an embodiment of the present invention, by binding users and storage nodes and storing them in a mapping relationship table, the storage node corresponding to the current user is automatically searched when a new data table is created, and a data table is generated based on the data table information to be processed and the storage node, thereby reducing operations, saving time and improving table building efficiency.

[0069] Optionally, the above method may also include: in response to the scheduled backup task being triggered, obtaining multiple binding nodes of the preset target user; physically backing up the multiple binding nodes to obtain backup information of the preset target user, and the backup information is used to restore the data of the preset target user when the binding storage node fails.

[0070] According to an embodiment of the present invention, for a user bound to a node group, physical backup of the user's corresponding node can be performed regularly, so that when a storage node corresponding to the user fails, the user's data can be quickly restored through the physical backup data, thereby ensuring the integrity and security of the user's data. Compared with the prior art of performing logical backup through a coordinator, physical backup is more flexible and faster, and the speed of restoring data is also faster, further improving the user experience.

[0071] According to an embodiment of the present invention, the above-mentioned preset sharding algorithm includes a hash algorithm. The above-mentioned use of the preset sharding algorithm to obtain the storage data sharding value based on the data information to be stored includes: generating a data key value based on the data information to be stored; using a hash function to map the data key value to a hash value, and the sharding value is used to determine the target node corresponding to the storage space from the storage node having a mapping relationship with the user identifier.

[0072] Exemplarily, the data information to be stored may be {data: 123abc}, and a unique data key value wasd123 may be generated according to the information to be processed, and the unique data key value may be processed using a hash algorithm to obtain a mapped hash value.

[0073] According to an embodiment of the present invention, the above-mentioned mapping relationship table also includes a mapping relationship between storage nodes and node levels, and determining at least one candidate node from multiple storage nodes based on the user level, including: determining the node level corresponding to the storage node based on multiple storage nodes and the mapping relationship between the storage nodes and the node level; determining at least one candidate node from multiple storage nodes based on the node level and the user level, the node level of the storage node being less than or equal to the storage node level corresponding to the user level.

[0074] According to an embodiment of the present invention, storage nodes can be rated based on their network resources, physical storage space, storage efficiency, storage stability and other attributes, and stored in a mapping relationship table. Node groups with different numbers and levels can also be assigned according to user levels to provide services to users. For users with higher levels, node groups with higher levels and more nodes can be assigned to meet user needs. Furthermore, user data security can be ensured through spare nodes in the node group. According to an embodiment of the present invention, users can be rated according to their needs. For example, when a user needs a larger storage space, the user can be rated as a higher-level user.

[0075] According to an embodiment of the present invention, by rating and matching users and storage nodes respectively to determine the user's candidate nodes, a corresponding node group can be assigned to each user more accurately, thereby reducing resource waste and improving user experience. Furthermore, since the ratings are performed separately, the level corresponding to the storage node can be directly modified after the storage node device is changed, without having to modify the overall mapping relationship, thereby reducing the number of operation steps and improving efficiency.

[0076] Based on the above data storage method, the present invention also provides a data storage device. Figure 5 The device is described in detail.

[0077] Figure 5 A structural block diagram of a data storage device according to an embodiment of the present invention is shown.

[0078] like Figure 5 As shown, the data storage device 500 of this embodiment includes an acquisition module 510, a user level determination module 520, a storage node determination module 530, a candidate node determination module 540, a mapping module 550, a target node determination module 560, and a storage module 570.

[0079] The acquisition module 510 is used to respond to the received new data request and obtain the data table identifier, the data information to be stored, the user identifier and the mapping relationship table of the distributed database in the new data request, the mapping relationship table including the mapping relationship between the user identifier and the user level, the mapping relationship between the data table identifier and the storage node, and the mapping relationship between the storage node and the hash value. In one embodiment, the acquisition module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0080] The user level determination module 520 is used to determine the user level based on the user identifier and the mapping relationship between the user identifier and the user level. In one embodiment, the user level determination module 520 can be used to perform the operation S220 described above, which will not be described in detail here.

[0081] The storage node determination module 530 is used to determine a plurality of storage nodes corresponding to the data table based on the data table identifier and the mapping relationship between the data table identifier and the storage node. In one embodiment, the storage node determination module 530 can be used to perform the operation S230 described above, which will not be described in detail here.

[0082] The candidate node determination module 540 is used to determine at least one candidate node from a plurality of storage nodes based on the user level. In one embodiment, the candidate node determination module 540 may be used to perform the operation S240 described above, which will not be described in detail herein.

[0083] The mapping module 550 is used to obtain the storage data shard value based on the to-be-stored data information by using a preset sharding algorithm. In one embodiment, the mapping module 550 can be used to perform the operation S250 described above, which will not be described in detail here.

[0084] The target node determination module 560 is used to determine the target node from at least one candidate node based on the storage data shard value and the mapping relationship between the storage node and the shard value, and the target node is used to provide storage services for the user corresponding to the user identifier. In one embodiment, the target node determination module 560 can be used to perform the operation S260 described above, which will not be repeated here.

[0085] The storage module 570 is used to store the data to be stored on the target node. In one embodiment, the storage module 570 can be used to perform the operation S270 described above, which will not be described in detail here.

[0086] According to an embodiment of the present invention, the above-mentioned mapping relationship table also includes: a mapping relationship between the user identifier and the above-mentioned storage node, and the above-mentioned device also includes: a table creation acquisition module, which is used to respond to the received table creation request and obtain the user identifier, the data table information to be processed and the mapping relationship table of the above-mentioned distributed database in the above-mentioned table creation request; a storage node determination module, which is used to determine the multiple storage nodes corresponding to the above-mentioned user identifier based on the above-mentioned user identifier and the mapping relationship between the above-mentioned user identifier and the above-mentioned storage node; a data table generation module, which is used to generate a data table based on the above-mentioned data table information to be processed and the multiple storage nodes corresponding to the above-mentioned user identifier.

[0087] According to an embodiment of the present invention, the above-mentioned device also includes: a new user acquisition module, which is used to respond to the received new user request and obtain the mapping relationship between the user identifier in the above-mentioned new user request and the above-mentioned storage node; a mapping relationship storage module, which is used to store the mapping relationship between the above-mentioned user identifier and the above-mentioned storage node in the above-mentioned mapping relationship table.

[0088] According to an embodiment of the present invention, the above-mentioned device also includes: a backup acquisition module, which is used to obtain multiple binding nodes of the preset target user in response to the triggering of a scheduled backup task; a backup module, which is used to physically back up the above-mentioned multiple binding nodes to obtain the backup information of the above-mentioned preset target user, and the above-mentioned backup information is used to restore the data of the preset target user when the binding storage node fails.

[0089] According to an embodiment of the present invention, the above-mentioned preset sharding algorithm includes a hash algorithm, and the above-mentioned mapping module includes: a data key value generation submodule, used to generate a data key value based on the above-mentioned data information to be stored; a shard value submodule, using a hash function, mapping the above-mentioned data key value to a storage data shard value, the above-mentioned storage data shard value is a hash value, and the above-mentioned shard value is used to determine the target node corresponding to the storage space from the storage node having a mapping relationship with the user identifier.

[0090] According to an embodiment of the present invention, the above-mentioned mapping relationship table also includes a mapping relationship between storage nodes and node levels, and the above-mentioned candidate node determination module includes: a node level determination submodule, used to determine the node level corresponding to the storage node based on the above-mentioned multiple storage nodes and the mapping relationship between the above-mentioned storage nodes and node levels; a level determination submodule, used to determine at least one candidate node from the above-mentioned multiple storage nodes based on the above-mentioned node level and the above-mentioned user level, and the node level of the above-mentioned storage node is less than or equal to the storage node level corresponding to the above-mentioned user level.

[0091] According to an embodiment of the present invention, any multiple modules among the acquisition module 510, the user level determination module 520, the storage node determination module 530, the candidate node determination module 540, the mapping module 550, the target node determination module 560, and the storage module 570 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 510, the user level determination module 520, the storage node determination module 530, the candidate node determination module 540, the mapping module 550, the target node determination module 560, and the storage module 570 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or can be implemented in any one of the three implementation modes of software, hardware and firmware or in a proper combination of any of them. Alternatively, at least one of the acquisition module 510, the user level determination module 520, the storage node determination module 530, the candidate node determination module 540, the mapping module 550, the target node determination module 560, and the storage module 570 can be at least partially implemented as a computer program module, which can perform the corresponding function when the computer program module is executed.

[0092] Figure 6 A block diagram of an electronic device suitable for implementing a data storage method according to an embodiment of the present invention is shown.

[0093] like Figure 6 As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0094] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0095] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0096] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0097] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0098] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the data storage method provided by the embodiment of the present invention.

[0099] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0100] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0101] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0102] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0103] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0104] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All of these combinations and / or combinations fall within the scope of the present invention.

[0105] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination. The scope of the present invention is defined by the attached claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A data storage method, characterized in that: The method is applied to a distributed database, the distributed database includes a plurality of storage nodes, and the method includes: In response to the received new data request, obtain a mapping relationship table of a data table identifier, data information to be stored, a user identifier and the distributed database in the new data request, wherein the mapping relationship table includes a mapping relationship between a user identifier and a user level, a mapping relationship between a data table identifier and a storage node, and a mapping relationship between a storage node and a hash value; Determining the user level based on the user identifier and the mapping relationship between the user identifier and the user level; Determine a plurality of storage nodes corresponding to the data table based on the data table identifier and a mapping relationship between the data table identifier and the storage node; determining at least one candidate node from the plurality of storage nodes based on the user level; Using a preset sharding algorithm, a storage data sharding value is obtained based on the data information to be stored; Determine a target node from the at least one candidate node based on the storage data shard value and the mapping relationship between the storage node and the shard value; Storing the data to be stored on the target node; The preset sharding algorithm includes a hash algorithm, and the using the preset sharding algorithm to obtain a storage data sharding value based on the to-be-stored data information includes: Generate a data key value based on the data information to be stored; The data key value is mapped to a storage data shard value using a hash function, and the storage data shard value is a hash value.

2. The method according to claim 1, characterized in that The mapping relationship table further includes: a mapping relationship between a user identifier and the storage node; before the step of obtaining the mapping relationship table between the data table identifier, the data information to be stored, the user identifier and the distributed database in the new data request in response to the received new data request, further includes: In response to the received table creation request, obtaining the user identifier, the to-be-processed data table information and the mapping relationship table of the distributed database in the table creation request; Determine a plurality of storage nodes corresponding to the user identifier based on the user identifier and a mapping relationship between the user identifier and the storage node; A data table is generated based on the to-be-processed data table information and a plurality of storage nodes corresponding to the user identifier.

3. The method according to claim 2, characterized in that Before obtaining the user identifier, the to-be-processed data table information and the mapping relationship table of the distributed database in the table creation request in response to the received table creation request, the method includes: In response to the received new user request, obtaining a mapping relationship between a user identifier in the new user request and the storage node; The mapping relationship between the user identifier and the storage node is stored in the mapping relationship table.

4. The method according to claim 1, characterized in that: The method further comprises: In response to the scheduled backup task being triggered, multiple binding nodes of a preset target user are obtained; Physically back up the multiple binding nodes to obtain backup information of the preset target user, and the backup information is used to restore data of the preset target user when a storage node fails.

5. The method according to claim 1, characterized in that The mapping relationship table further includes a mapping relationship between storage nodes and node levels, and determining at least one candidate node from the plurality of storage nodes based on the user level includes: Determine a node level corresponding to the storage node based on the plurality of storage nodes and a mapping relationship between the storage node and the node level; At least one candidate node is determined from among the plurality of storage nodes based on the node level and the user level.

6. A data storage device, characterized in that: The device is configured in a distributed database, the distributed database includes a plurality of nodes, and the device includes: an acquisition module, configured to, in response to a received new data request, acquire a data table identifier, data information to be stored, a user identifier and a mapping relationship table of the distributed database in the new data request, wherein the mapping relationship table includes a mapping relationship between a user identifier and a user level, a mapping relationship between a data table identifier and a storage node, and a mapping relationship between a storage node and a hash value; A user level determination module, used to determine the user level based on the user identifier and the mapping relationship between the user identifier and the user level; A storage node determination module, used to determine a plurality of storage nodes corresponding to the data table based on the data table identifier and a mapping relationship between the data table identifier and the storage node; a candidate node determination module, configured to determine at least one candidate node from the plurality of storage nodes based on the user level; A mapping module, used to obtain a storage data shard value based on the data information to be stored by using a preset sharding algorithm; A target node determination module, configured to determine a target node from the at least one candidate node based on the storage data shard value and the mapping relationship between the storage node and the shard value; A storage module, used for storing the data to be stored on the target node; The preset sharding algorithm includes a hash algorithm, and the mapping module includes: A data key value generation submodule, used to generate a data key value based on the data information to be stored; The shard value submodule uses a hash function to map the data key value to a storage data shard value, where the storage data shard value is a hash value.

7. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images