Data processing method, electronic equipment, readable storage medium and program product

Abstract

The invention relates to the technical field of data processing, and discloses a data processing method, electronic equipment, a readable storage medium and a program product. In the method, a key-value database can store data corresponding to different businesses in different data tables, and different database identifiers are configured for each data table. When a client of a service needs to access target data, an access request including a database identifier of a target database corresponding to the service can be sent to a key-value database cluster. After receiving an access request, the key-value database cluster can determine a target database corresponding to the access request according to a database identifier in the access request, and respond to the access request based on the target database. Thus, even if different businesses adopt the same key to access data, the different businesses also access the data in the respective corresponding target databases, and the data conflict problem caused by the uniqueness of keys in a traditional key-value database can be avoided.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a data processing method, an electronic device, a readable storage medium, and a program product. Background Art

[0002] In a key-value (KV) database cluster, the data (or data shards) to be stored is called a value, and the unique identifier of each value within the scope of the KV database cluster is called a key. The key and the value are stored in the form of a key-value pair. Generally, the key-value pairs in the KV database cluster are stored in a data table, and the keys and values in this data table are in one-to-one correspondence. A service that needs to access the KV database cluster can access the data in the KV database cluster by sending an access request (such as a read request, a write request, a delete request, etc.) including the key to the node in the KV database cluster for storing the data to be accessed.

[0003] Generally, a KV database cluster provides services for multiple services (which can be services of the same tenant (customer or user) of the KV database cluster or services of different tenants of the KV database cluster). When different services access the database cluster through the same key, it may cause data conflicts, resulting in data loss or data errors in the KV database cluster. For example, when service 0 sends a write request to write the key-value pair "name=jim" (the part before "=" is the key, and the part after "=" is the value) to the KV database cluster, and service 1 sends a write request to write the key-value pair "name=lily" to the KV database cluster, the KV database cluster will select one of the key-value pairs "name=jim" and "name=lily" to write into the KV database cluster or not write these two key-value pairs, resulting in data loss. Summary of the Invention

[0004] In view of this, this application provides a data processing method, an electronic device, a readable storage medium, and a program product.

[0005] In a first aspect, this application provides a data processing method, which includes: a client detects an access instruction to access first data corresponding to a first service in a key-value database system; the client sends an access request to access the first data to the key-value database system, where the access request is used to instruct the key-value database system to respond to the access request based on a first data table, and the first data table is one of at least one data table stored in the key-value database system.

[0006] In this method, when the client needs to access the first data corresponding to the first service in the key-value database system, the client can send an access request to the key-value database system, instructing the key-value database system to respond to the access request based on the first data table in at least one data table stored in the key-value database system. In this way, the key-value database system can respond to the access request based on the first data table indicated by the access request, rather than responding to the access request based on the data tables corresponding to other services, which can avoid data conflicts caused by accessing data with the same key for different services.

[0007] In a possible implementation of the first aspect above, the first data table is the data table corresponding to the first service in at least one data table.

[0008] In this implementation, the access request instruction sent by the client enables the key-value database system to respond to the access request through the data table corresponding to the first service, which can avoid data conflicts caused by accessing data with the same key for different services.

[0009] In a possible implementation of the first aspect above, the key-value database system is a key-value database cluster.

[0010] In a possible implementation of the first aspect above, the client sends an access request to the key-value database system to access the first data, including: the client determines the first node for storing the first data among multiple nodes of the key-value database cluster based on the first data; the client sends an access request to the first node.

[0011] In this implementation, the client can determine the slot (or partition) for storing the key based on the first data (such as the key of the first data), and then determine the first node for storing the first data according to the corresponding relationship between the slots and partitions in the key-value database cluster, and send an access request to the first node.

[0012] In a possible implementation of the first aspect above, the access request includes a first identifier corresponding to the first data table, and the first identifier is used to instruct the first node to respond to the access request based on the first sub-data table of the first data table on the first node.

[0013] That is to say, there are corresponding sub-data tables of the first data table on multiple nodes of the key-value database cluster. The node that receives the access request (such as the first node) can determine the first sub-data table of the first data table on the first node based on the first identifier in the access request, and respond to the access request based on the first sub-data table.

[0014] In a possible implementation of the first aspect above, the first sub-data table may include a hash index corresponding to the data of the first data table on the first node.

[0015] In this implementation, the access request sent by the client to the first node includes the first identifier corresponding to the first data table. That is, for different service ends, an access request carrying the identifier of the first data table corresponding to each service (such as the data table identifier or the database identifier of the logical database in the following text, etc.) can be sent to the key-value database cluster (or a node in the key-value database cluster) so that the key-value database cluster can respond to the access request based on the data table corresponding to the identifier. For example, the first node can respond to the access request based on the first sub-data table of the first data table on the first node. In this way, since the clients of different services will only access the data tables corresponding to their respective services (i.e., the logical databases in the following text), even if different services access data through the same key, there will be no key conflict.

[0016] In a possible implementation of the above first aspect, the client sending an access request to the first node includes: obtaining a first connection from the connection pool between the client and the first node; and sending an access request to the first node based on the first connection.

[0017] In a possible implementation of the above first aspect, the first connection is a connection established before detecting an access instruction for accessing the first data corresponding to the first service in the key-value database system and is used to access the first sub-data table of the first data table on the first node.

[0018] In this implementation manner, the client can first establish a first connection with the key-value database system for accessing the first sub-data table of the first data table on the first node, and each time it accesses the data on the first node, it sends an access request to the first node through the first connection. After receiving the access request, the key-value database system can determine based on the source of the access request (the first connection) that the access request is used to access the first sub-data table of the first data table on the first node and respond to the access request based on the first sub-data table.

[0019] In a possible implementation of the above first aspect, the access request for instructing the key-value database system to respond to the access request based on the first data table includes: corresponding to the access instruction being a write instruction, the access request is used to instruct the first node to write the first data into the first sub-data table.

[0020] In a possible implementation of the above first aspect, the access request for instructing the key-value database system to respond to the access request based on the first data table further includes: corresponding to the access instruction being a read instruction, the access request is used to instruct the first node to read the first data from the first sub-data table and send the first data to the client.

[0021] In a possible implementation of the first aspect described above, the access request is used to instruct the key-value database system to respond to the access request based on the first data table, and further includes: corresponding to the access instruction being a deletion instruction, the access request is used to instruct the first node to delete the first data from the first sub-data table.

[0022] In a possible implementation of the first aspect described above, the access request further includes authentication information, and the authentication information is used to instruct the key-value database system to respond to the access request based on the first data table when the authentication based on the authentication information passes.

[0023] In a possible implementation of the first aspect described above, the client includes a first process corresponding to the first service and a software development kit for the key-value database cluster; and the client sends an access request for accessing the first data to the key-value database system, including: the first service sends an access instruction to the software development kit; the software development kit sends an access request including the first identifier and the access instruction to the client.

[0024] In a possible implementation of the first aspect described above, the first identifier is sent by the first process to the software development kit.

[0025] In a second aspect, the present application provides a data processing method, and the method includes: the client detects an access instruction for accessing the first data corresponding to the first service in the key-value database system; the client sends an access request for accessing the first data to the key-value database system; the key-value database system responds to the access request based on the first data table, where the first data table is one of at least one data table stored in the key-value database system.

[0026] In this method, when the client needs to access the first data corresponding to the first service in the key-value database system, it can send an access request to the key-value database system instructing the key-value database system to respond to the access instruction based on the first data table among at least one data table stored in the key-value database system. After receiving the access request, the key-value database system can respond to the access request based on the first data table indicated by the access request, rather than responding to the access request based on the data tables corresponding to other services, which can avoid data conflicts caused when different services access data using the same key.

[0027] In a possible implementation of the second aspect described above, the first data table is a data table corresponding to the first service among at least one data table.

[0028] In this implementation, the key-value database system responds to the access requests sent by the client of the first service through the data table corresponding to the first service. Correspondingly, for the access requests sent by the clients of other services, the key-value database system can respond to the access requests based on the data tables corresponding to other services, which can avoid data conflicts caused when different services access data using the same key.

[0029] In a possible implementation of the second aspect above, the key-value database system is a key-value database cluster.

[0030] In a possible implementation of the second aspect above, the client sends an access request to the key-value database system to access the first data, including: the client determines the first node for storing the first data among multiple nodes of the key-value database cluster based on the first data; the client sends an access request to the first node.

[0031] In this implementation, the client can determine the slot (or partition) for storing the key based on the first data (such as the key of the first data), and then determine the first node for storing the first data according to the correspondence between the slots and partitions in the key-value database cluster, and send an access request to the first node.

[0032] In a possible implementation of the second aspect above, the client sends an access request to the first node, including: the client obtains the first connection from the connection pool between the client and the first node; the client sends an access request to the first node based on the first connection.

[0033] In a possible implementation of the second aspect above, the first connection is a connection established by the client before detecting the access instruction to access the first data corresponding to the first service in the key-value database system, and is used to access the first sub-data table of the first data table on the first node.

[0034] In this implementation manner, the client can first establish the first connection with the key-value database system to access the first sub-data table of the first data table on the first node, and each time it accesses the data on the first node, it sends an access request to the first node through the first connection. After receiving the access request, the key-value database system can determine that the access request is used to access the first sub-data table of the first data table on the first node based on the source of the access request (the first connection), and respond to the access request based on the first sub-data table.

[0035] In a possible implementation of the second aspect described above, the access request includes a first identifier corresponding to the first data table, and the first node includes sub-data tables respectively corresponding to each data table in at least one data table; moreover, the key-value database system responds to the access request based on the first data table, including: the first node determines, based on the first identifier, the first sub-data table corresponding to the first data table from the sub-data tables respectively corresponding to each data table in at least one data table; and the first node responds to the access request based on the first sub-data table.

[0036] In this implementation, each data table in at least one data table in the key-value database system has a corresponding sub-data table on each node. After receiving the access request including the first identifier, the first node can determine, from the sub-data tables corresponding to each data table, the first sub-data table in the first node that corresponds to the first data table identified by the first identifier, and respond to the access request based on the first sub-data table.

[0037] In a possible implementation of the second aspect described above, the first sub-data table may include a hash index corresponding to the data of the first data table in the first node.

[0038] In a possible implementation of the second aspect described above, among the sub-data tables stored in any node of the key-value database system, there may be included hash indexes corresponding to the data of the data tables corresponding to the respective sub-data tables in this node.

[0039] In a possible implementation of the second aspect described above, the key-value database system responds to the access request based on the first data table, including: corresponding to the access instruction being a write instruction, the first node writes the first data to the first sub-data table.

[0040] In a possible implementation of the second aspect described above, the key-value database system responding to the access request based on the first data table further includes: corresponding to the access instruction being a read instruction, the first node reads the first data from the first sub-data table and sends the first data to the client.

[0041] In a possible implementation of the second aspect described above, the key-value database system responding to the access request based on the first data table further includes: corresponding to the access instruction being a delete instruction, the first node deletes the first data from the first sub-data table.

[0042] In a possible implementation of the second aspect described above, the method further includes: before the client detects an access instruction for accessing the first data corresponding to the first service in the key-value database system, the key-value database system initializes the correspondence between the first service and the first data table.

[0043] In this implementation manner, the key-value database system can pre-establish the corresponding relationships between different services and different data tables, that is, allocate different logical databases (i.e., different data tables) in the key-value database system to different services, so that when the client of a service sends an access request to the key-value database system, it can carry the database identifier of its corresponding logical database, so that the key-value database system can respond to the access request based on the logical database (i.e., data table) corresponding to each service.

[0044] In a possible implementation of the second aspect above, the access request further includes authentication information, and the key-value database system responding to the access request based on the first data table includes: the key-value database system responding to the access request based on the first data table when the authentication based on the authentication information passes.

[0045] That is to say, the key-value database system responds to the access request based on the first data table only when the authentication based on the authentication information passes (that is, when it is determined that the client has the permission to access the first data table). In this way, it can be avoided that one service accesses the data of another service, which is beneficial to improving the security of the data in the key-value database system.

[0046] In a third aspect, the present application provides a data processing method, which includes: the key-value database system receives an access request sent by a client, where the access is used to request to access the data corresponding to the first service in the key-value database system; the key-value database system responds to the access request based on the first data table, where the first data table is one of at least one data table stored in the key-value database system.

[0047] In a possible implementation of the third aspect above, the key-value database system is a key-value database cluster.

[0048] In a possible implementation of the third aspect above, the first data table is the data table corresponding to the first service among at least one data table.

[0049] In a possible implementation of the third aspect above, the access request includes a first identifier corresponding to the first data table; and the key-value database system responding to the access request based on the first data table includes: the key-value database system determines the first data table from at least one data table based on the first identifier; the key-value database system responds to the access request based on the first data table.

[0050] In a possible implementation of the third aspect above, the key-value database system responding to the access request based on the first data table includes: the key-value database system determines the first data table to be accessed by the access request based on the first connection that sends the access request; the key-value database system responds to the access request based on the first data table.

[0051] In a possible implementation of the third aspect above, the access request further includes authentication information, and the key-value database system responds to the access request based on the first data table, including: when the key-value database system passes the authentication based on the authentication information, it responds to the access request based on the first data table.

[0052] That is to say, the key-value database system responds to the access request based on the first data table only when the authentication based on the authentication information passes (that is, when it is determined that the client has the permission to access the first data table). In this way, it is possible to avoid one service accessing the data of another service, which is beneficial to improving the security of the data in the key-value database system.

[0053] Fourth aspect, the present application provides an electronic device, which includes: a memory for storing one or more programs; a processor for executing one or more programs to enable the electronic device to implement the data processing method provided in the first aspect and any possible implementation of the first aspect above, or the data processing method provided in the second aspect and any possible implementation of the first aspect above, or the data processing method provided in the third aspect and any possible implementation of the third aspect above.

[0054] Fifth aspect provides a readable storage medium, which includes one or more programs. When the one or more programs are executed on an electronic device, they enable the electronic device to implement the data processing method provided in the first aspect and any possible implementation of the first aspect above, or the data processing method provided in the second aspect and any possible implementation of the first aspect above, or the data processing method provided in the third aspect and any possible implementation of the third aspect above.

[0055] Sixth aspect provides a program product, which when running on an electronic device enables the electronic device to implement the data processing method provided in the first aspect and any possible implementation of the first aspect above, or the data processing method provided in the second aspect and any possible implementation of the first aspect above, or the data processing method provided in the third aspect and any possible implementation of the third aspect above.

[0056] It should be understood that the beneficial effects of the fourth aspect to the sixth aspect above can refer to the descriptions of the first aspect to the third aspect, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 According to some embodiments of the present application, a schematic structural diagram of a database system 10 is shown;

[0058] Figure 2AAccording to some embodiments of the present application, a schematic diagram of a scenario for configuring a logical database for service 0 is shown;

[0059] Figure 2B According to some embodiments of the present application, a schematic diagram of an interaction process for configuring a logical database is shown;

[0060] Figure 2C According to some embodiments of the present application, a schematic diagram of a management interface U1 of a database cluster management console 30 is shown;

[0061] Figure 2D According to some embodiments of the present application, a schematic diagram of a logical database selection interface U2 of a database cluster management console 30 is shown;

[0062] Figure 2E According to some embodiments of the present application, a schematic diagram of a user information input interface U3 of a database cluster management console 30 is shown;

[0063] Figure 2F According to some embodiments of the present application, a schematic diagram of a user creation success interface U4 of a database cluster management console 30 is shown;

[0064] Figure 3 According to some embodiments of the present application, a schematic diagram of a scenario for storing data corresponding to different services in a KV database 21 is shown;

[0065] Figure 4 According to some embodiments of the present application, a schematic diagram of the structure of a database system 20 is shown;

[0066] Figure 5 According to some embodiments of the present application, a schematic diagram of an interaction process of a data processing method is shown;

[0067] Figure 6 According to some embodiments of the present application, a schematic diagram of an interaction process of another data processing method is shown;

[0068] Figure 7 According to some embodiments of the present application, a schematic diagram of the structure of an electronic device 100 is shown. Detailed Embodiments

[0069] Exemplary embodiments of the present application include, but are not limited to, data processing methods, electronic devices, readable storage media, and program products.

[0070] It should be understood that the KV database cluster can be any database cluster that stores data in the form of key-value pairs, including but not limited to Redis databases, Memcached database clusters, Cassandra database clusters, LevelDB database clusters, etc.

[0071] The technical solution of the present application will be introduced below with reference to the accompanying drawings.

[0072] Since a KV database cluster generally stores data through a data table, and any key in the data table is uniquely present. When a KV database cluster usually provides services for multiple services, if different services access the database cluster through the same key, it may lead to data conflicts, resulting in data loss or data errors in the KV database cluster.

[0073] For example, Figure 1 According to some embodiments of the present application, a schematic structural diagram of a database system 10 is shown.

[0074] As Figure 1 shown, the database system 10 includes a KV database cluster 11, a client 12 corresponding to service 0, and a client 13 corresponding to service 1. Among them, the KV database cluster 11 includes nodes N1, N2, and N3. Node N1 corresponds to slot S1 to slot S 5041 , node N2 corresponds to slot S 5042 to slot S 10922 , and node N3 corresponds to slot S 10923 to slot S 16383 . The key-value pairs in the KV database cluster 11 are all stored in the data table T. The hash index in the data table T records the correspondence between keys and values (which can also be the correspondence between keys and the addresses of storage areas where values are stored). For example, K1 = V1, K2 = V2,..., K J = V J etc. Among them, before "=", it is the key, and after "=", it is the value or the address of the storage area where the value is stored.

[0075] It should be understood that a slot is also called a partition, which is used to determine the node in the KV database cluster for storing key-value pairs. For example, in some embodiments, a KV database cluster can include at most 16,384 slots, which are respectively set in multiple nodes of the KV database cluster. The number of slots set in each node can be the same or different.

[0076] In some embodiments, slot S i can be the i-th slot in the KV database cluster, and the slot identifier of slot S i can be i.

[0077] In some embodiments, for any key-value pair, the client 12 or the client 13 can determine the slot corresponding to the key-value pair through a hashing algorithm (such as the consistent hashing algorithm), and then determine the node where the slot is located according to the correspondence between the slots and the nodes in the KV database cluster 11, so as to determine the node storing the key-value pair. Secondly, the client 12 or the client 13 can send a corresponding access request to the determined node, so that the nodes in the KV database cluster can perform corresponding operations in response to the access request.

[0078] Exemplarily, referring to Figure 1 , when the client 12 needs to write the key-value pair "name=jim" to the KV database cluster 11, it can first perform a consistent hash on the 16-bit cyclic redundancy check (CRC16) code of the key "name" (for example, hexadecimal 5092, corresponding to decimal 20626) to obtain the slot identifier corresponding to the key "name" as 4242 (corresponding to slot S 4242 ). Then, the client 12 can determine the corresponding node N1 of slot S 4242 according to the correspondence between the slots and the nodes of the KV database cluster 11. Secondly, the client 12 can send a write request to write the key-value pair "name=jim" to the node N1, so that the node N1 can write the key-value pair "name=lily" to the data table T.

[0079] If both the client 12 and the client 13 access the key-value pair with the key "name", a key conflict will occur, resulting in data errors or data loss in the KV database cluster 11.

[0080] For example, if the client 12 sends a write request to write the key-value pair "name=jim" to the node N1 while the client 13 sends a write request to write the key-value pair "name=lily" to the node N1, the KV database cluster 11 can select one of the key-value pairs "name=jim" and "name=lily" to write into the database cluster, or even not write the key-value pair with the key "name", resulting in data loss of service 0 and / or service 1.

[0081] For another example, after the node N1 writes the key-value pair "name=jim" to the data table T, if the client 13 sends a write request to write the key-value pair "name=lily" to the node N1, the key-value pair "name=lily" will overwrite the key-value pair "name=jim" corresponding to service 0, resulting in data loss of service 0.

[0082] For another example, after the node N1 writes the key-value pair "name=jim" into the data table T, if the client 13 sends a deletion request to the node N1 to delete the key-value pair with the key "name", the node N1 will directly delete the key-value pair "name=jim" corresponding to service 0 from the data table T, resulting in the loss of data for service 0.

[0083] To solve the data conflict problem caused by the uniqueness of keys in the KV database cluster, an embodiment of the present application provides a data processing method. In this method, the KV database cluster can store the data corresponding to different services in different logical databases, and configure different database identifiers for each logical database.

[0084] In some embodiments, the data corresponding to a logical database can be stored in each node of the KV database cluster respectively. The table used to store the data of a certain logical database in each node can be called the sub-data table corresponding to this logical database. The set of sub-data tables corresponding to a certain logical database in each node of the KV database cluster can be called the data table corresponding to this logical database.

[0085] In some embodiments, the database identifier corresponding to each sub-data table of a logical database is the same as the database identifier of this logical database.

[0086] In some embodiments, the logical database corresponding to a service can be assigned to each service by the KV database cluster or the operator of the KV database cluster, and integrated into the client, application program or instruction corresponding to the service.

[0087] For the sake of description, hereinafter, the data that the client of the service needs to access is called the target data, the logical database used by the KV database cluster to store the data of this service is called the target database, and the node where the slot for storing the target data in the target database is located is called the target node.

[0088] When the client of the service needs to access the target data of the KV database cluster, it can first determine the target node according to the key of the target data, and then send an access request including the database identifier of the target database to the target node.

[0089] After receiving the access request, the target node can determine the target database corresponding to the access request according to the database identifier in the access request, and respond to the access request based on the target database. For example, the target node can write the target data into the sub-data table of the target database in the target node according to the type of the access request (such as write request, delete request, read request, etc.), delete the target data from the sub-data table of the target database in the target node, read the target data from the sub-data table of the target database in the target node and send it to the client of the service, etc.

[0090] In some embodiments, when the client of a service is initialized, it can establish connections with each node in the KV database cluster, and the KV database cluster can record the corresponding relationships between different connections and logical databases (or the database identifiers of logical databases, or the data tables of logical databases, or the sub-data tables of logical databases). Thus, after receiving an access request, the target node in the KV database cluster can, based on the connection sending the access request and this corresponding relationship, determine the target database to be accessed by the access request, and respond to the access request based on the target database. For example, the target node can write target data into the sub-data table of the target database in the target node according to the type of the access request (such as a write request, a delete request, a read request, etc.), delete target data from the sub-data table of the target database in the target node, read target data from the sub-data table of the target database in the target node and send it to the client of the service, etc.

[0091] In some embodiments, the logical database for storing a certain service in the KV database cluster can be allocated to this service by the KV database cluster when this service is registered with the KV database cluster.

[0092] In some embodiments, the logical database for storing a certain service in the KV database cluster can be allocated to this service by the operator of the KV database cluster, and the database identifier of this logical database can be integrated into the client of this service by the developer or user of this service.

[0093] Based on the above method, since the data of different services in the KV database cluster is stored in different logical databases, the keys of different logical databases can be repeated. In this way, even if different services use the same key to access data, different services will only access the data in the target databases corresponding to their respective services, and data conflict problems caused by the uniqueness of keys in the KV database cluster can be avoided.

[0094] In addition, in a KV database cluster that stores the data of multiple services through one data table, since there is only one data table in the KV database cluster and multiple services all have the permission to access this data table, one service can access the data of another service, which affects the security of the data in the KV database cluster. However, in the method provided by the embodiments of the present application, since the data of different services in the KV database cluster is stored in different data tables, the nodes in the KV database cluster can also authenticate the access request after receiving the access request. If the client sending the access request does not have the permission to access the target database, the KV database cluster can not respond to the access request. In this way, one service can be prevented from accessing the data of another service, and the security of the data in the KV database cluster can be improved.

[0095] In some embodiments, the KV database cluster can serve multiple services, and the KV database cluster or the operator of the KV database cluster can assign different logical databases to each service. Each node in the KV database cluster can include sub-data tables established for different services, and associate each sub-data table with the logical database corresponding to each service, so that after each node receives an access request including a database identifier, it can respond to the access request based on the sub-data table of the logical database corresponding to the database identifier.

[0096] First, a technical solution for configuring different logical databases for different services in the KV database cluster is introduced.

[0097] Exemplarily, Figure 2A According to some embodiments of the present application, a schematic diagram of a scenario for configuring the logical database of Service 0 is shown.

[0098] As Figure 2A shown, the database system 200 includes a database cluster management console 30 and a database cluster 40. Among them, the database cluster management console 30 is used for the maintenance and management of the KV database cluster running in the database cluster 40. The database cluster 40 can include each KV database cluster running in the database system 200, such as the KV database cluster 21, the KV database cluster 31, etc. A KV database cluster can be an actual running database instance.

[0099] The client 50 is a client of the database cluster management console 30, and is used to obtain user information (such as username, password, etc.) input by the user of the service, and send a service request including the user information to the database cluster management console 30. For example, when a user of Service 0 needs to store data using the KV database cluster, the user can send a service request to the database cluster management console 30 through the client 50, and the request can include the user information corresponding to Service 0.

[0100] After receiving the service request, the database cluster management console 30 can determine whether there is a KV database cluster with an idle logical database among the KV database clusters running in the database cluster 40. If there is a KV database cluster with an idle logical database among the KV database clusters running in the database cluster 40, the database cluster management console 30 can configure the logical database corresponding to Service 0 in the KV database cluster with the idle logical database; if there is no KV database cluster with an idle logical database among the KV database clusters running in the database cluster 40, the database cluster management console 30 can create a new KV database cluster in the database cluster 40 and configure the logical database corresponding to Service 0 in the newly created KV database cluster.

[0101] For example, the database cluster management console 30 can negotiate with the KV database cluster with idle logical databases based on the user information (such as username, password, etc.) in the service request sent by the client 50 to establish a user with the permission to access a certain logical database. Then, the database cluster management console 30 can send the user information (such as username, password, etc.) of the established user and the logical database configuration information of the logical database corresponding to service 0 (database identifier, IP addresses of each node corresponding to the logical database, authentication information (such as username, password, key, certificate), etc.) to the client 50. Thus, the user of service 0 or the developer of service 0 can add the logical database configuration information to the running logic of service 0 or the data relied on when service 0 runs.

[0102] In some embodiments, the maintenance personnel of the database cluster management console 30 can also input the user information corresponding to service 0 and select the logical database corresponding to service 0 through the user interface of the database cluster management console 30, so that the database cluster management console 3 can interact with the KV database cluster corresponding to the logical database based on the user information input by the maintenance personnel of the database cluster management console 30 and the selected logical database to obtain the logical database configuration information corresponding to service 0, and feedback the logical database configuration information to the user of service 0.

[0103] Taking the way that the maintenance personnel of the database cluster management console 30 input the user information corresponding to service 0 through the user interface of the database cluster management console 30 as an example, combined with Figures 2B to 2F , the process of configuring the logical database of service 0 is introduced.

[0104] Exemplarily, Figure 2B According to some embodiments of the present application, a schematic diagram of the interaction process for configuring a logical database is shown. As Figure 7 shown, the process includes the following steps:

[0105] S201, the database cluster management console 30 creates and runs the KV database cluster 21.

[0106] The database cluster management console 30 can establish and run the KV database cluster 21 according to the operations of the maintenance personnel.

[0107] S202, the database cluster management console 30 determines the logical database corresponding to service 0.

[0108] When maintenance personnel need to configure the logical database corresponding to Service 0, they can view the idle logical databases in the database cluster 40 through the user interface of the database cluster management console 30, and select one from the idle logical databases as the logical database corresponding to Service 0, that is, the logical database used to store the data corresponding to Service 0. That is to say, the database cluster management console 30 can use the logical database selected by the maintenance personnel as the logical database corresponding to Service 0.

[0109] For example, Figure 2C According to some embodiments of the present application, a schematic diagram of a management interface U1 of a database cluster management console 30 is shown; Figure 2D According to some embodiments of the present application, a schematic diagram of a select logical database interface U2 of a database cluster management console 30 is shown.

[0110] As Figure 2C shown, the management interface U1 may include information about the database instances (i.e., KV database clusters) running on the database cluster management console 30, such as the names of the database instances (KV database clusters), the used logical databases, the available logical databases, etc., as well as a new instance control U11 and a new service control U12. For example, for the KV database cluster 21 shown in U1, the number of used logical databases is 1, and the number of available logical databases is P; for the KV database cluster 31 shown in U1, the number of used logical databases is A, and the number of available logical databases is B.

[0111] When maintenance personnel need to add a new service, they can select the new service control U12 in the management interface U1. After the database cluster management console 30 detects the operation of the user selecting the new service control U12, it can query the database instances in the database clusters managed by the database cluster management console 30 that have idle logical databases and display them in the select logical database interface. Exemplarily, referring to Figure 2D , the database cluster management console 30 can display the select logical database interface U2, and the select logical database interface U2 may include the idle logical databases in the database clusters managed by the database cluster management console 30, such as the logical databases DB0, DB3, DB4, DB5, etc. in the KV database cluster 21. The user can select the logical database corresponding to Service 0 in the select logical database interface U2, such as the logical database DB0 in the KV database cluster 21.

[0112] S203. The database cluster management console 30 sends a create user request to the KV database cluster 21.

[0113] After the database cluster management console 30 determines the logical database corresponding to Service 0, it can obtain the username and password entered by the maintenance personnel and send a user creation request to the KV database cluster 21. The user creation request may include the username, password, and the database identifier of the logical database corresponding to Service 0.

[0114] For example, after the database cluster management console 30 detects that the user operates on the logical database interface U2 for the logical database DB0 and clicks the next control U21, it can display Figure 2E the user information input interface U3. The user can enter the username and password corresponding to Service 0 on the user information input interface U3. After the database cluster management console 30 detects that the user enters the username (e.g., user0) and password (e.g., password1) and selects the next control U31, it can send a user creation request to the KV database cluster 21. The user creation request may include the username user0, password password1, and database identifier DB0.

[0115] S204. The KV database cluster 21 creates a user and sends a message indicating that the user creation is successful to the database cluster management console 30.

[0116] After receiving the user creation request, the KV database cluster 21 can create a corresponding user according to the username, password, and database identifier in the user creation request, and send a message indicating that the user creation is successful to the database cluster management console 30 after the creation is successful.

[0117] S205. The database cluster management console 30 sends an authorization request to the KV database cluster 21.

[0118] After receiving the message indicating that the user creation is successful, the database cluster management console 30 can send an authorization request to the KV database cluster 21. The authorization request may include the username (e.g., user0) and database identifier (e.g., DB0).

[0119] In some embodiments, the authorization request may further include authentication information, password, etc.

[0120] S206. The KV database cluster 21 authorizes the user and sends a message indicating that the authorization is successful to the database cluster management console 30.

[0121] After receiving the authorization request, the KV database cluster 21 can grant the permission to access the logical database corresponding to the database identifier in the authorization request to the username in the authorization request, and send a message indicating that the authorization is successful to the database cluster management console 30 after the authorization is successful.

[0122] S207. The database cluster management console 30 displays the logical database configuration information.

[0123] After receiving the message of successful authorization, the database cluster management console 30 can display the logical database configuration information, such as the IP addresses of each node in the KV database cluster 21, the user name and password corresponding to service 0, the database identifier of the logical database corresponding to service 0, etc.

[0124] For example, after receiving the message of successful authorization, the database cluster management console 30 can display Figure 2F the user creation success interface U4 as shown. The user creation success interface U4 may include the logical database configuration information corresponding to service 0, such as the IP addresses of each node in the KV database cluster 21, the database identifier DB0 of the logical database DB0, and the aforementioned user name user0, password password0, etc. In this way, the maintenance personnel can send the above logical database configuration information to the users of service 0, so that the users or developers of service 0 can add the logical database configuration information to the operation logic of the client of service 0 or the service process 221.

[0125] It should be understood that Figure 2B the execution process of each step shown is only an example. In some other embodiments, some steps may be combined or split, and the order of some steps may also be adjusted, which is not limited herein. For example, the database cluster management console 30 may first obtain the user information corresponding to service 0, and then obtain the logical database selected by the user. For another example, step S203 and step S205 may be combined into the same step, and correspondingly, step S204 and step S206 may also be combined into the same step.

[0126] It should be understood that Figures 2C to 2F the interface shown is only an example. In some other embodiments, the user interface of the database cluster management console 30 may include more or less content, or may be another form of interface for interaction through the command line, which is not limited herein.

[0127] It should be understood that corresponding to the KV database cluster 21, the configuration process of the logical database corresponding to each service can all adopt Figure 2B the manner shown, which is not limited herein.

[0128] Hereinafter, taking the KV database cluster 21 as an example, the technical solution of the present application will be introduced.

[0129] For example, Figure 3 According to some embodiments of the present application, a schematic diagram of a scenario for storing data corresponding to different services in the KV database cluster 21 is shown.

[0130] As Figure 3As shown in the figure, the KV database cluster 21 includes nodes N1, N2, and N3, and the KV database cluster 21 stores data corresponding to P + 1 (P is an integer greater than 0) services (such as service 0, service 1, ……, service P). The logical database that stores the key-value pairs of service t (t = 0, 1, ……, P) in the KV database cluster 21 is the logical database DB t (the database identifier is DB t ), and the sub-data table used to store the data of service t in node N S (s = 1, 2, 3) is the sub-data table T t-s , the sub-data table T t-1 , the sub-data table T t-2 , the sub-data table T t-3 data table T t . That is to say, the KV database cluster 21 stores the data of P + 1 services through P + 1 logical databases. For example, the database identifier of the logical database DB0 that stores service 0 (i.e., t = 0) is DB0, and the sub-data tables used to store the data of the logical database DB0 in nodes N1, N2, and N3 are the sub-data tables T 0-1 , the sub-data table T 0-2 and the sub-data table T 0-3 respectively, and the data table T0 corresponding to the logical database DB0 is the set of the sub-data table T 0-1 , the sub-data table T 0-2 and the sub-data table T 0-3 . Another example, the database identifier of the logical database DB1 that stores service 1 (i.e., t = 1) is DB1, and the sub-data tables used to store the data of the logical database DB0 in nodes N1, N2, and N3 are the sub-data tables T 1-1 , the sub-data table T 1-2 and the sub-data table T 1-3 respectively, and the data table T1 corresponding to the logical database DB1 is the set of the sub-data table T 1-1 , the sub-data table T 1-2 and the sub-data table T 1-3 . Another example, the database identifier of the logical database DB P that stores service P (i.e., t = P) is DB P , and the sub-data tables used to store the data of the logical database DB P in nodes N1, N2, and N3 are the sub-data tables T P-1 , the sub-data table T P-2 and the sub-data table T P-3 respectively, and the data table T2 corresponding to the logical database DB P is the set of the sub-data table T P-1 , the sub-data table T P-2 and the sub-data table TP-3 set

[0131] When the client of service t needs to access the target data in the KV database cluster 21, it can first determine the target node in the KV database cluster 21 for storing the target data according to the key of the target data, and send an access request including the database identifier DB t to the target node. When the target node (such as node N1) receives the access request, it can be based on the logical database DB t corresponding to the database identifier DB t (as the target database) corresponding sub-data table T t-1 to respond to the access request, for example, read the target data from the sub-data table T t-1 and send it to the client of service t, delete the target data from the sub-data table T t-1 , write the target data into the sub-data table T t-1 and so on.

[0132] In some embodiments, when storing key-value pairs, the KV database cluster 21 can directly store the corresponding relationship between the key and the value itself, or store the corresponding relationship between the key and the address of the storage area of the stored value in a hash table. The embodiments of the present application do not limit the way the KV database cluster stores key-value pairs.

[0133] In some embodiments, the data structure of the data table stored in the KV database cluster 21 can be a hash table, a hash tree, or other data structures that can store key-value pairs, which are not limited here.

[0134] In some embodiments, each logical database in the KV database cluster 21 can correspond to a hash index (equivalent to each data table corresponding to a hash index). After each node in the KV database cluster receives an access request for the target data in a certain logical database, it can respond to the access request based on the hash index corresponding to the logical database, for example, obtain the value corresponding to the target data from the hash index, write the key-value pair corresponding to the target data into the hash index, delete the key-value pair corresponding to the target data, etc.

[0135] In some embodiments, services 0 to P may be different services that can access the KV database cluster 21. For example, they can be services of different tenants of the KV database cluster or different services of the same tenant of the KV database cluster. For instance, assume that different tenants operate different applications, and each tenant stores the service data (such as user data, etc.) of the applications they operate in the KV database cluster 21. Then, services 0 to P may respectively correspond to different tenants. Another example is that a certain tenant develops multiple types of devices and stores the user data of different types of devices through the KV database cluster 21. Services 0 to P may correspond to different types of devices of this tenant.

[0136] Based on the above method, since the data of different services in the KV database cluster 21 is stored in different logical databases (that is, the data of different services is stored in different data tables), when the same key is used to access data in different services, only the logical database corresponding to each service will be accessed, and the logical databases of other services will not be accessed, which can avoid data conflict problems caused by the uniqueness of keys in the KV database cluster 21.

[0137] In some embodiments, after receiving an access request, the nodes in the KV database cluster 21 can also authenticate the access request. If the client sending the access request does not have the permission to access the target database, the KV database cluster 21 may not respond to the access request. In this way, it can be avoided that one service accesses the data of another service, improving the security of the data in the KV database cluster 21.

[0138] It should be understood that Figure 3 The structure of the KV database cluster 21 shown is only an example. In some other embodiments, the KV database cluster 21 may include more or fewer nodes (for example, the KV database cluster 21 may include 2 or more than 3 nodes, which is not limited here), and more or fewer service data may also be stored in the KV database cluster 21, which is not limited here.

[0139] Next, taking the KV database cluster as Figure 3 the KV database cluster 21 shown, where the KV database cluster 21 stores the data of service 0 and service 1 as an example, the technical solution of the present application will be introduced.

[0140] Exemplarily, Figure 4 According to some embodiments of the present application, a schematic structural diagram of a database system 20 is shown.

[0141] As Figure 4As shown in the figure, the database system 20 includes a KV database cluster 21, clients 22 and 23. Among them, client 22 includes a service process 221 corresponding to service 0 and a software development kit (SDK) 222; client 23 includes a service process 231 corresponding to service 1 and an SDK 232.

[0142] The service process 221 can be used to pass the database identification library (such as DB0) of the logical database to be accessed by the service process 221 to the SDK 222.

[0143] The service process 221 can also be used to send an access instruction for accessing target data to the SDK 222 according to the user's operation or the running logic of service 0, and receive the target data sent by the KV database cluster 21 from the SDK 222. In some embodiments, the access instruction may include the key of the target data. For example, when service 0 needs to read the target data with the key "name" from the database cluster 21, it can send a read instruction "get name" to the SDK 222 (this is just an example, and the form of the read instruction may be different for different KV database clusters).

[0144] The SDK 222 is used to proxy the communication process between the service process 221 and the KV database cluster 21. Exemplarily, the SDK 222 may include an external interface 223, a configuration module 224, a connection pool 225, and a connection module 226.

[0145] The external interface 223 is used to provide an access interface to the service process 221, receive the database identification (such as DB0) passed by the service process 221, receive the access instruction sent by the process of service 0 and pass the access instruction to the connection module 226, and receive the access result corresponding to the access instruction from the connection module 226 and send the access result to the service process 221.

[0146] The configuration module 224 is used to configure the logical database to be connected by the client 22.

[0147] The configuration module 224 can obtain the database identification (such as DB0) of the logical database corresponding to service 0 from the service process 221, and pass the database identification to the connection pool 225 and the connection module 226.

[0148] The connection pool 225 is used to establish and store the connection between the service process 221 and the KV database cluster 21.

[0149] In some embodiments, the connection pool 225 may include connections between the service process 221 and each node in the KV database cluster 21. For example, assume that the Internet Protocol (IP) address of node N1 in the KV database cluster 21 is 10.40.130.230, the IP address of node N2 is 10.40.130.111, and the IP address of node N3 is 10.40.130.48. Then the connection pool 225 establishes connections with the IP address 10.40.130.230, the IP address 10.40.130.111, and the IP address 10.40.130.48 respectively for the connection module 226 to use.

[0150] In some embodiments, one connection pool 225 may correspond to one node. That is to say, the number of connection pools 225 included in the SDK 222 is the same as the number of nodes in the KV database cluster 21.

[0151] It should be understood that the connections between the service process 221 and each node in the KV database cluster 21 can be connections based on any communication protocol, including but not limited to Transmission Control Protocol / Internet Protocol (TCP / IP) connections, Hypertext Transfer Protocol (HTTP) connections, Java Database Connectivity (JDBC), etc.

[0152] In some embodiments, the connection (which can also be called a connection object) corresponding to one node (or one IP address) in the connection pool 225 may include connection information for the service process 221 to communicate with this node, including but not limited to authentication information (such as username, password, verification code, key, etc.), the IP address of the node, port number, etc. For example, the connection information recorded in the connection object established by the connection pool 225 with the IP address 10.40.130.230 (i.e., node N1) may include "10.40.130.230:2881", where 10.40.130.230 is the IP address of node N1 and 2881 is the port number.

[0153] In some embodiments, when the connection pool 225 establishes connections with each node in the KV database cluster 21, it may also obtain the database identifier (such as DB0) of the logical database (such as logical database DB0) used to store service 0 in the KV database cluster 21 from the configuration module 224, and add the database identifier of this logical database to the connection pools between the service process 221 and each node.

[0154] For example, the connection pool corresponding to node N1 may include connection information "10.40.130.230:2881[DB0]" to indicate that the database identifier of the database storing service 0 in node N1 is DB0. The connection pool corresponding to node N2 may include connection information "10.40.130.111:2881[DB0]" to indicate that the database identifier of the database storing service 0 in node N2 is DB0. The connection pool corresponding to node N3 may include connection information "10.40.130.48:2881[DB0]" to indicate that the database identifier of the database storing service 0 in node N3 is DB0.

[0155] In some embodiments, the connection pool 225 may have a maximum number of connections and a minimum number of connections. When the connection pool 225 is initialized, connections with the minimum number of connections can be established. When the number of connections in the connection pool 225 reaches the maximum number of connections, the connection pool 225 needs to wait for a connection in the connection pool to be released before a new connection can be established.

[0156] In some embodiments, after each node in the KV database cluster 21 establishes a connection with the client 22, it can record the correspondence between different connections / connection pools and the corresponding logical databases (or the database identifiers of the logical databases, or the sub-data tables of the logical databases on this node), so that after receiving an access request, the logical database or data table or sub-data table to be accessed by the access request can be determined based on the connection sending the access request.

[0157] The connection module 226 can be used to receive the access instruction sent by the external interface 223, determine the target node storing the target data according to the key of the target data included in the access instruction, and obtain the target connection of the target node from the connection pool 225. The connection module 226 is also used to, after obtaining the target connection, send an access request corresponding to the access instruction to the target node based on the target connection, and the access request may include the access instruction and the database identifier of the target database (such as DB0).

[0158] Exemplarily, when the connection module 226 receives the foregoing read instruction "get name", it may perform consistent hashing on the CRC16 code of the key "name", and the slot identifier for storing the key-value pair with the key "name" may be the foregoing 4242. If the slot setting method in the KV database cluster 21 is the same as that in the foregoing KV database cluster 11, the node (target node) for storing the key-value pair with the key "name" is node N1. Secondly, the connection module 226 may obtain a target connection corresponding to the target node (such as node N1) from the connection pool 225, and obtain the database identifier of the target database from the target connection; then, the connection module 226 may send an access request carrying the database identifier of the target database and an access instruction to the target node based on the target connection. For example, in some embodiments, the connection module 226 may send an access request including the database identifier "DB0" and the access instruction "get name" to node N1 through the socket "10.40.130.230:2881" that communicates with node N1 based on the process 221.

[0159] In some embodiments, if the database identifier of the target database corresponding to service 0 is not stored in the connection pool 225, after receiving the access instruction sent by the service process, the connection module 226 may also obtain the database identifier of the target database from the configuration module 224, and send an access request carrying the database identifier of the target database and the access instruction to the target node.

[0160] In some embodiments, the connection module 226 may also obtain and store the database identifier (such as DB0) of the target database from the configuration module 224, and after receiving the access instruction sent by the service process, read the database identifier of the target database from itself, and send an access request carrying the database identifier of the target database and the access instruction to the target node.

[0161] In some embodiments, the access request sent by the connection module 226 to the target node may further include authentication information (such as a key, username and password, verification code, etc.).

[0162] It should be understood that the form of the foregoing access request is only an example. In some other embodiments, the access request may be in any form that can indicate the target database to be accessed by the access request, and is not limited herein.

[0163] It should be understood that using the SDK 222 to proxy the communication between the service process 221 and the KV database cluster 21 is just an example. In some other embodiments, the functions of the SDK 222 can also be implemented by other forms of modules, such as applications, services, application programming interfaces (APIs), etc., which are not limited herein.

[0164] In some embodiments, the SDK 222 and the service 0 process can be integrated into the same application or can be independent modules, which are not limited herein.

[0165] The service process 231 can be used to pass the database identification library (such as DB1) of the logical database to be accessed by the service process 231 to the SDK 232.

[0166] The service process 231 is also used to send an access instruction for accessing the target data to the SDK 232 according to the user's operation or the running logic of the service 1, and receive the target data sent by the KV database cluster 21 from the SDK 232. In some embodiments, the access instruction may include the key of the target data. For example, when the service 1 needs to read the target data with the key "name" from the database cluster 21, it can send a read instruction "get name" to the SDK 232 (just an example, and the form of the read instruction may be different for different KV database clusters).

[0167] The SDK 232 is used to proxy the communication process between the service process 231 and the KV database cluster 21. Exemplarily, the SDK 232 may include an external interface 233, a configuration module 234, a connection pool 235, and a connection module 236. The functions of each module in the SDK 232 can refer to the functions of each module in the foregoing SDK 222, which will not be elaborated herein.

[0168] Exemplarily, when Service 1 needs to read data with the key "name" (as the target data) from the database cluster 21, it can send a read instruction "get name" to the connection module 236 through the external interface 232. When the connection module 236 receives the read instruction "get name", it can perform consistent hashing on the CRC16 code of the key "name" to obtain the slot identifier for storing the key-value pair with the key "name" as the aforementioned 4242. If the slot setting method in the KV database cluster 21 is the same as that in the aforementioned KV database cluster 11, the node for storing the key-value pair with the key "name" is node N1 (as the target node). Secondly, the connection module 236 can obtain the target connection corresponding to the target node (such as node N1) from the connection pool 235 and obtain the database identifier (such as DB1) of the target database corresponding to Service 1 from the target connection. Then, the connection module 236 can send an access request with the read instruction and the database identifier "DB1" to the target node based on the target connection. For example, in some embodiments, the connection module 236 can send an access request including the database identifier DB1 and the access instruction "get name" to node N1 based on the socket "10.40.130.230:2881" through which the service process 231 communicates with node N1.

[0169] In some embodiments, the access request sent by the connection module 236 to the target node may further include authentication information (such as keys, usernames and passwords, verification codes, etc.).

[0170] After receiving the access request sent by the client 22 or the client 23, the target node in the KV database cluster 21 can respond to the access request through the target database corresponding to the database identifier based on the database identifier included in the access request.

[0171] For example, when node N1 receives the aforementioned access request including the database identifier DB0 and the access instruction "get name" sent by the connection module 226, it can determine the target database as the logical database DB0 based on the database identifier "DB0", and then obtain the value "jim" corresponding to the key "name" from the sub-data table T 0-1 corresponding to node N1 in the logical database DB0 according to the access instruction "get name", and send the value "jim" to the client 22.

[0172] For another example, when node N1 receives the foregoing access request including database identifier DB1 and access instruction "get name" sent by connection module 236, it can determine that the target database is logical database DB1 based on the database identifier "DB1", and then obtain the value "lily" corresponding to the key "name" from the sub-data table T 1-1 corresponding to logical database DB1 in node N1 based on the access instruction "get name", and send the value "lily" to client 23.

[0173] In some embodiments, after receiving an access request, a node of KV database cluster 21 can also determine the sub-data table of the logical database to be accessed by the access request on this node based on the connection sending the access request.

[0174] In some embodiments, after receiving an access request, a node of KV database cluster 21 can also authenticate the access request based on the authentication information in the access request to determine whether the access request has the permission to access the target database. If the node determines that the access request does not have the permission to access the target database, it does not respond to the access request. In this way, the security of the data in the KV database cluster can be improved. For example, assume that the database identifier included in the access request sent by client 23 to a certain node in KV database cluster 21 is "DB0", but the authentication information included in this access request is the authentication information corresponding to service 1. KV database cluster 21 can determine that this access request does not have the permission to access logical database DB0 and does not respond to this access request.

[0175] In the foregoing database system 20, since KV database cluster 21 stores the data of different services in different logical databases respectively and configures different database identifiers for different logical databases. The clients of each service can add the database identifiers of the logical databases they need to access to the access requests sent to the nodes in KV database cluster 21 based on the SDK corresponding to KV database cluster 21, so that KV database cluster 21 can respond to the access requests through the logical databases corresponding to each service based on the database identifiers in the access requests. In this way, even if each service accesses KV database cluster 21 through the same key, there will be no key collision. And since the logical databases of each service are independent of each other, the security of the data of each service can be improved, and it is avoided that one service accesses the data of another service.

[0176] It should be understood that Figure 4 the structure of the illustrated database system 20 is only an example. In some other embodiments, the database system 20 may include more clients, and KV database cluster 21 may also include more or fewer nodes, which is not limited herein.

[0177] Next, in combination with Figure 4 the structure of the database system 20 shown in Figure 5 and

[0178] Exemplarily, Figure 5 According to some embodiments of the present application, a schematic diagram of the interaction process of a data processing method is shown. As Figure 5 shown, the interaction process includes the following steps:

[0179] S501, the service process 221 / 231 initializes the service.

[0180] When the service process 221 or the service process 231 starts, it can initialize the corresponding service.

[0181] For example, the service process 221 / 231 can obtain the IP addresses of each node of the KV database cluster 21 (for example, the IP address 10.40.130.230 of the aforementioned node N1, the IP address of node N2 is 10.40.130.111, and the IP address of node N3 is 10.40.130.48), the database identifier of the logical database in the KV database cluster 21 for storing the service corresponding to each of them (for example, the database identifier of the logical database DB0 corresponding to service 0 is DB0, and the database identifier of the logical database DB1 corresponding to service 1 is DB1), etc.

[0182] In some embodiments, the IP addresses of each node of the KV database cluster 21, the database identifiers of the logical databases in the KV database cluster 21 for storing each service, etc. can be preset in the running logic of the service process of each service, or can be obtained in real time by the service process of each service from the nodes of the KV database cluster 21 or other devices, which is not limited herein.

[0183] S502, the service process 221 / 231 sends the IP addresses of each node in the KV database cluster 21 and the database identifiers of the services corresponding to each of them to the connection module 226 / 236.

[0184] After the service process 221 / 231 initializes the corresponding service, it can send the IP addresses of each node in the KV database cluster 21 and the database cluster identifiers corresponding to each service to the corresponding connection module.

[0185] For example, the service process 221 may send the IP address 10.40.130.230, the IP address 10.40.130.111, the IP address 10.40.130.48, and the database identifier DB0 corresponding to service 0 to the connection module 226 through the foregoing external interface 222. For another example, the service process 231 may send the IP address 10.40.130.230, the IP address 10.40.130.111, the IP address 10.40.130.48, and the database identifier DB1 corresponding to service 1 to the connection module 236 through the foregoing external interface 232.

[0186] S503, the connection module 226 / 236 establishes a connection pool 225 / 235.

[0187] After receiving the IP addresses of the nodes in the KV database cluster 21 and the database identifiers of their respective corresponding services sent by their respective corresponding service processes, the connection module 226 / 236 may establish a connection pool.

[0188] For example, after receiving the node IP addresses and database identifiers sent by the service process 221, the connection module 226 may establish the foregoing connection pool 225. The connection pool 225 may include a connection pool between the service process 221 and the IP address 10.40.130.230 (corresponding to node N1), a connection pool between the service process 221 and the IP address 10.40.130.111 (corresponding to node N2), and a connection pool between the service process 221 and the IP address 10.40.130.48 (corresponding to node N3).

[0189] For another example, after receiving the node IP addresses and database identifiers sent by the service process 231, the connection module 236 may establish the foregoing connection pool 235. The connection pool 235 may include a connection pool between the service process 231 and the IP address 10.40.130.230 (corresponding to node N1), a connection pool between the service process 221 and the IP address 10.40.130.111 (corresponding to node N2), and a connection pool between the service process 231 and the IP address 10.40.130.48 (corresponding to node N3).

[0190] In some embodiments, the connection module 226 or the connection module 236 may establish a connection pool for each node in the KV database cluster 21, or may establish a unified connection pool for all nodes in the KV database cluster 21, which is not limited herein.

[0191] In some embodiments, before, after, or during the execution of step S503, the connection module 226 / 236 may also obtain and store the correspondence between each node of the KV database cluster 21 and the slots in the KV database cluster 21 through control instructions, such as cluster nodes instructions, cluster slots instructions, etc., so that when receiving an access instruction sent by the service process 221 / 231, the node (target node) storing the target data to be accessed by the access instruction can be determined based on this correspondence.

[0192] Exemplarily, the information of the KV database cluster 21 obtained by the connection module 226 / 236 based on the cluster nodes instruction may include the information of node N1 "0_2 10.40.130.230:2881 master, normal, 0-5461", where 0_2 indicates that node N1 is the 0th node among 3 nodes, "10.40.130.230:2881" represents the IP address and port number of node N1, "master" indicates that node N1 is the master node, "normal" indicates that the running state of N1 is normal, and "0-5461" indicates that node N1 includes 5462 slots with slot identifiers from 0 to 5461 (i.e., the foregoing slots S0 - slot S 5461 ); the information of node N2 "1_2 10.40.130.111:2881 master, normal, 5462-10922" (the specific meaning refers to the relevant description of node N1 and will not be elaborated here); the information of node N3 "2_2 10.40.130.48:2881 master, normal, 10923-16383" (the specific meaning refers to the relevant description of node N1 and will not be elaborated here). The connection module 226 / 236 may store the correspondence between node N1 and slots S0 - slot S 5461 ; the correspondence between node N2 and slot S 5462 - slot S 10922 ; the correspondence between node N3 and slot S 10923 - slot S 16383 .

[0193] S504, the connection pool 225 / 235 sends connection requests to the input / output (IO) threads of each node.

[0194] After the connection pool 225 / 235 is established, connection requests can be sent to the corresponding nodes through the IO threads of the corresponding nodes. In some embodiments, the connection requests may include authentication information and the database identifier of the logical database to which the connection pool is to connect. For example, the connection request sent by the connection pool 225 may include the authentication information for the corresponding service 0 and the database identifier DB0, and the connection request sent by the connection pool 235 may include the authentication information for the corresponding service 1 and the database identifier DB1.

[0195] For example, assume that the connection module 226 establishes a unified connection pool 225 for all nodes in the KV database cluster 21. After the connection pool 225 is established, connection requests can be sent to the IO threads of node N1, node N2, and node N3 respectively. As another example, assume that the connection module 226 establishes a connection pool 225 for each node of the KV database cluster. If the node corresponding to a certain connection pool is N1, then this connection pool can send a connection request to the IO thread of node N1.

[0196] It should be understood that in some other embodiments, the connection information recorded in the connection object may also include more or less information, which is not limited herein.

[0197] It should be understood that the IO thread is a thread running on each node in the KV database cluster 21 and is used to communicate with the client. In some other embodiments, the functions of the IO thread may also be implemented by other threads, which are not limited herein.

[0198] S505, The IO thread sends the connection result to the connection pool 225 / 235.

[0199] After receiving the connection request, the IO threads of each node can authenticate the connection request based on the database identifier and authentication information in the connection request. If it is determined that the connection request has the permission to access the logical database corresponding to the database identifier (i.e., the authentication is successful), the IO thread can send a connection success connection result to the connection pool 225 / 235. If it is determined that the connection request does not have the permission to access the logical database corresponding to the database identifier (i.e., the authentication fails), the IO thread can send a connection failure connection result to the connection pool 225 / 235.

[0200] In some embodiments, the IO thread can also record the correspondence between different connections and the logical database (or the sub-data tables of the logical database on the node).

[0201] S506, The connection pool 225 / 235 configures the connection pool.

[0202] After receiving the connection result sent by the IO thread of a certain node, if the connection result indicates a successful connection, the connection pool 225 / 235 can configure the connection pool, for example, configure the connection information with that node. In some implementations, the connection pool 225 / 235 can add the database identifier of the logical database it is connected to to the connection information.

[0203] For example, for the aforementioned service 0, after successfully establishing a connection with node N1, the connection pool 225 can add the database identifier "DB0" to the connection information "10.40.130.230:2881" corresponding to node N1 to obtain the connection information "10.40.130.230:2881[DB0]" to indicate that the database identifier of the database storing service 0 in node N1 is DB0. After successfully establishing a connection with node N2, the connection pool 225 can add the database identifier "DB0" to the connection information "10.40.130.111:2881" corresponding to node N2 to obtain the connection information "10.40.130.111:2881[DB0]" to indicate that the database identifier of the database storing service 0 in node N2 is DB0. After successfully establishing a connection with node N3, the connection pool 225 can add the database identifier "DB0" to the connection information "10.40.130.48:2881" corresponding to node N3 to obtain the connection information "10.40.130.48:2881[DB0]" to indicate that the database identifier of the database storing service 0 in node N3 is DB0.

[0204] Another example, for the aforementioned service 1, after successfully establishing a connection with node N1, the connection pool 235 can add the database identifier "DB1" to the connection information "10.40.130.230:2881" corresponding to node N1 to obtain the connection information "10.40.130.230:2881[DB1]" to indicate that the database identifier of the database storing service 1 in node N1 is DB1. After successfully establishing a connection with node N2, the connection pool 235 can add the database identifier "DB1" to the connection information "10.40.130.111:2881" corresponding to node N2 to obtain the connection information "10.40.130.111:2881[DB1]" to indicate that the database identifier of the database storing service 1 in node N2 is DB1. After successfully establishing a connection with node N3, the connection pool 235 can add the database identifier "DB1" to the connection information "10.40.130.48:2881" corresponding to node N3 to obtain the connection information "10.40.130.48:2881[DB1]" to indicate that the database identifier of the database storing service 1 in node N3 is DB1.

[0205] It should be understood that the database identifier can be added to any position in the connection information between the service process and each node, or added to the connection information in other forms, which is not limited here.

[0206] S507, the connection pool 225 / 235 sends a connection pool establishment success message to the connection module 226 / 236.

[0207] After adding the database identifier to the connection pool corresponding to their respective services, the connection pools 225 / 235 can send a connection pool establishment success message to their corresponding connection modules (the connection pool 225 corresponds to the connection module 226, and the connection pool 235 corresponds to the connection module 236).

[0208] S508, the connection module 226 / 236 sends an SDK initialization success message to the service process 221 / 231.

[0209] After receiving the connection pool establishment success message, the connection modules 226 / 236 can send an SDK initialization success message to their corresponding service processes (the connection module 226 corresponds to the service process 221, and the connection module 236 corresponds to the service process 231).

[0210] It should be understood that after receiving the SDK initialization success message, the service processes 221 / 231 can access the KV database cluster 21 through access instructions.

[0211] In some embodiments, the access instructions may include read instructions for reading data in the KV database cluster 21, write instructions for writing data to the KV database cluster, delete instructions for deleting data in the KV database cluster 21, etc.

[0212] In some embodiments, the connection modules 226 / 236 can also send control instructions for obtaining the status of the KV database cluster 21 (including but not limited to the number of key-value pairs stored in each node, the space occupied by different data tables in each node, the routing information of each node in the KV database cluster, etc.) to each node of the KV database cluster 21. The control instructions may also include the corresponding database identifier, so that the KV database cluster can respond to the control instructions based on the logical database corresponding to the database identifier in the control instructions.

[0213] For example, assume sub-data table T 0-1 、sub-data table T 0-2 、sub-data table T 0-3The numbers of stored key-value pairs are 0, 1, and 0 respectively, and the occupied spaces are 0 bytes (byte, B), 156B, and 0B respectively: If service process 221 sends control instructions "info keyspace DB0" (used to obtain the number of key-value pairs stored in the data table with database identifier DB0) to node N1, node N2, and node N3 respectively, the information received from node N1, node N2, and node N3 can include "DB0: keys = 0" (indicating that the number of key-value pairs stored in the data table with database identifier DB0 in node N1 is 0), "DB0: keys = 1" (indicating that the number of key-value pairs stored in the data table with database identifier DB0 in node N2 is 1), "DB0: keys = 0" (indicating that the number of key-value pairs stored in the data table with database identifier DB0 in node N3 is 0); If service process 221 sends control instructions "info memory DB0" (used to obtain the size of the occupied space in the data table with database identifier DB0) to node N1, node N2, and node N3 respectively, the information received from node N1, node N2, and node N3 can include "DB0_used_memory: 0B" (indicating that the occupied space of the data table with database identifier DB0 stored in node N1 is 0B), "DB0_used_memory: 156B" (indicating that the occupied space of the data table with database identifier DB0 stored in node N2 is 156B), "DB0_used_memory: 0B" (indicating that the occupied space of the data table with database identifier DB0 stored in node N3 is 0B).

[0214] For another example, assume sub-data table T 1-1 , sub-data table T 1-2 , sub-data table T 1-3The numbers of stored key-value pairs are 608399, 608458, and 608421 respectively, and the occupied spaces are 681 megabytes (MB), 681MB, and 681MB respectively: If service process 231 sends control instructions "info keyspace DB1" (for obtaining the number of key-value pairs stored in the data table with database identifier DB1) to node N1, node N2, and node N3 respectively, the information received from node N1, node N2, and node N3 may include "DB1: keys = 608399" (indicating that the number of key-value pairs stored in the data table with database identifier DB1 stored in node N1 is 608399), "DB1: keys = 608458" (indicating that the number of key-value pairs stored in the data table with database identifier DB1 stored in node N2 is 608458), "DB1: keys = 608421" (indicating that the number of key-value pairs stored in the data table with database identifier DB1 stored in node N3 is 608421); If service process 231 sends control instructions "info memory DB1" (for the size of the occupied space in the data table with database identifier DB1) to node N1, node N2, and node N3 respectively, the information received from node N1, node N2, and node N3 may include "DB1_used_memory: 681MB" (indicating that the occupied space of the data table with database identifier DB1 stored in node N1 is 681MB), "DB1_used_memory: 681MB" (indicating that the occupied space of the data table with database identifier DB1 stored in node N2 is 681MB), "DB1_used_memory: 681MB" (indicating that the occupied space of the data table with database identifier DB1 stored in node N3 is 681MB).

[0215] Taking the writing of data by service process 221 / 231 to the KV database cluster 21 as an example, the technical solution of the embodiment of the present application will be introduced below.

[0216] S509, service process 221 / 231 sends a writing instruction to connection module 226 / 236.

[0217] When service process 221 / 231 needs to write data to the KV database cluster 21, it can send a writing instruction to its corresponding connection module (service process 221 corresponds to connection module 226, and service process 231 corresponds to connection module 236).

[0218] For example, if the service process 221 needs to write the key-value pair "name=jim" to the KV database cluster 21, the service process 221 can send the write instruction "set name=jim" to the connection module 226 through the external interface 223.

[0219] For another example, if the service process 231 needs to write the key-value pair "name=lily" to the KV database cluster 21, the service process 231 can send the write instruction "set name=lily" to the connection module 236 through the external interface 233.

[0220] It should be understood that the form of the above write instruction is only an example. In some other embodiments, the write instruction can also be in other forms, and the write instruction can also include other information (such as authentication information, etc.), which is not limited herein.

[0221] S510, the connection module 226 / 236 determines the target node and obtains the target connection from the connection pool 225 / 235.

[0222] After receiving the write instruction, the connection module 226 / 236 can perform consistent hashing on the key of the target data in the write instruction (for example, taking the remainder of the CRC16 code of the key of the target data, or it can also be other algorithms defined by the KV database cluster for determining the corresponding relationship between the key and the slot, which is not limited herein) to determine the slot in the KV database cluster 21 for storing the target data. Then, the connection module 226 / 236 can determine the target node where the slot for storing the target data is located according to the corresponding relationship between the nodes and the slots in the KV database cluster 21, and obtain the target connection of the target node from the corresponding connection pool.

[0223] For example, after the connection module 226 receives the foregoing write instruction "set name=jim", it can perform consistent hashing on the CRC16 code of the key "name" (for example, hexadecimal 5092, corresponding to decimal 20626) to obtain that the identifier of the slot for storing the key-value pair "name=jim" is 4242 (corresponding to slot S5 242 ). Then, since slots S0 to slot S 5041 are set on node N1, the connection module 226 can determine that the target node for storing the target data is node N1, and obtain the connection between the service process 221 and node N1 from the connection pool 225 (as the target connection).

[0224] For another example, after receiving the aforementioned write instruction "set name=lily", the connection module 236 can perform consistent hashing on the CRC16 code of the key "name" (e.g., 5092 in hexadecimal, corresponding to 20626 in decimal), and obtain that the identifier of the slot for storing the key-value pair "name=lily" is 4242 (corresponding to slot S5 242 ). Then, since slots S0 to S 5041 are set on node N1, the connection module 226 can determine that the target node for storing the target data is node N1, and obtain the connection between the service process 231 and node N1 from the connection pool 235 (as the target connection).

[0225] S511, the connection module 226 / 236 sends a write request including the write instruction and the database identifier to the IO thread of the target node.

[0226] After obtaining the target connection with the target node, the connection module 226 / 236 can obtain the corresponding database identifier from the target connection, and send a write request including the write instruction and the database identifier to the IO thread of the target node.

[0227] For example, after obtaining the target connection between the service process 221 and node N1, the connection module 226 can obtain the database identifier DB0 from the target connection, and send a write request including the database identifier DB0 and the write instruction "set name=jim" to the IO thread of node N1.

[0228] For another example, after obtaining the target connection between the service process 231 and node N1, the connection module 236 can obtain the database identifier DB1 from the target connection, and send a write request including the database identifier DB1 and the write instruction "set name=lily" to the IO thread of node N1.

[0229] It should be understood that in some embodiments, the write request may also include other information, such as authentication information, etc., which is not limited herein.

[0230] In some embodiments, the connection module 226 / 236 can also obtain the database identifier of the logical database to be accessed by the corresponding service from itself or the configuration module 224 / 234, which is not limited herein.

[0231] In some embodiments, the write request may also not include the database identifier, which is not limited herein.

[0232] S512, the IO thread of the target node sends the write request to the working thread of the target node.

[0233] After receiving a write request, the IO thread of the target node can send the write request to the working thread of the target node.

[0234] In some embodiments, the IO thread of the target node can also authenticate the write request based on the authentication information in the write request to determine whether the sender of the write request has the permission to access the logical database corresponding to the database identifier included in the write request. If the IO thread of the target node determines that the write request is illegal (for example, the sender of the write request does not have the permission to access the logical database corresponding to the database identifier included in the write request), it does not send the write request to the working thread, but sends a message indicating access failure to the sender of the write request.

[0235] It should be understood that the working thread is a thread running on each node in the KV database cluster 21 and is used to execute operations corresponding to access requests (including but not limited to writing data to a sub-data table, deleting data from a sub-data table, and reading data from a sub-data table). In other embodiments, the functions of the working thread can also be implemented by other threads, which are not limited herein.

[0236] In some embodiments, if there is no database identifier in the received write request, the IO thread of the target node can determine the database identifier of the logical database to be accessed by the write request based on the recorded corresponding relationships between the aforementioned different connections and the logical databases (or sub-data tables of the logical databases on the nodes) and the target connection for sending the write request.

[0237] S513, the working thread of the target node writes the target data to the target database corresponding to the database identifier in the write request.

[0238] After receiving a write request, the working thread of the target node can write the target data to the sub-data table of the target database corresponding to the database identifier in the write request in the target node.

[0239] For example, after the working thread of node N1 receives a write request including the database identifier DB0 and the write instruction "set name=jim", it can determine that the sub-data table of the target database corresponding to the database identifier DB0 on node N1 is sub-data table T 0-1 , and write the key-value pair "name=jim" to sub-data table T 0-1 .

[0240] For another example, after the working thread of node N1 receives a write request including the database identifier DB1 and the write instruction "set name=lily", it can determine that the sub-data table of the target database corresponding to the database identifier DB1 on node N1 is sub-data table T 1-1 , and write to sub-data table T1-1 Write the key-value pair "name=lily".

[0241] S514. The worker thread of the target node sends the write result to the IO thread of the target node.

[0242] After writing the target data to the sub-data table of the target database in the target node, the worker thread of the target node can send the write result to the IO thread of the target node.

[0243] After successfully writing the target data to the sub-data table of the target database in the target node, the worker thread of the target node can send a write result indicating successful writing to the IO thread of the target node. If the worker thread of the target node fails to write the target data to the sub-data table of the target database in the target node (for example, the storage medium storing the target database does not have enough free space, the storage medium for storing the sub-data table of the target database in the target node fails, etc.), it can send a write result indicating a write failure to the IO thread of the target node.

[0244] For example, after the worker thread of node N1 writes the key-value pair "name=jim" to the sub-data table T 0-1 or after writing the key-value pair "name=lily" to the sub-data table T 1-1 it can send the write result to the IO thread of node N1.

[0245] S515. The IO thread of the target node sends the write result to the connection module 226 / 236.

[0246] After receiving the write result sent by the worker thread of the target node, the IO thread of the target node can send the write result to the connection module 226 / 236 corresponding to the write request.

[0247] For example, if the write result received by the IO thread of node N1 is the write result corresponding to successfully writing the key-value pair "name=jim", the IO thread of node N1 can send the write result to the connection module 226; if the write result received by the IO thread of node N1 is the write result corresponding to successfully writing the key-value pair "name=lily", the IO thread of node N1 can send the write result to the connection module 236.

[0248] S516. The connection module 226 / 236 sends the write result to the service process 221 / 231.

[0249] After receiving the write result sent by the IO thread of the target node, the connection module 226 / 236 can send the write result to their respective corresponding service processes.

[0250] For example, after receiving the write result sent by node N1, the connection module 226 can send the write result to the service process 221 through the external interface 223. Another example is that after receiving the write result sent by node N1, the connection module 236 can send the write result to the service process 231 through the external interface 233.

[0251] The following takes the service processes 221 / 231 reading data from the KV database cluster as an example to introduce the technical solution of the embodiment of the present application.

[0252] S517, the service processes 221 / 231 send read instructions to the connection modules 226 / 236.

[0253] When the service processes 221 / 231 need to read data from the KV database cluster 21, they can send read instructions to their respective corresponding connection modules (the service process 221 corresponds to the connection module 226, and the service process 231 corresponds to the connection module 236).

[0254] For example, corresponding to the Figure 4 scenario shown above, if the service process 221 needs to read the key-value pair with the key "name" from the KV database cluster 21, the service process 221 can send the read instruction "get name" to the connection module 226 through the external interface 223.

[0255] Another example is that for the Figure 4 scenario shown above, if the service process 231 needs to read the key-value pair with the key "name" from the KV database cluster 21, the service process 231 can send the read instruction "get name" to the connection module 236 through the external interface 233.

[0256] It should be understood that the form of the above read instruction is only an example. In other embodiments, the read instruction can also be in other forms, and the read instruction can also include other information (such as authentication information, etc.), which is not limited here.

[0257] S518, the connection modules 226 / 236 determine the target node and obtain the target connection from the connection pools 225 / 235.

[0258] After receiving the read instruction, the connection module 226 / 236 can perform consistent hashing on the key of the target data in the write instruction (for example, taking the remainder of the CRC16 code of the key of the target data, or other algorithms defined by the KV database cluster 21 for determining the correspondence between the key and the slot, which is not limited here) to determine the slot in the KV database cluster 21 for storing the target data. Then, the connection module 226 / 236 can determine the target node where the slot storing the target data is located according to the correspondence between the nodes and the slots in the KV database cluster 21, and obtain the target connection of the target node from the corresponding connection pool.

[0259] For example, after the connection module 226 receives the aforementioned read instruction "get name", it can perform consistent hashing on the CRC16 code of the key "name" (for example, hexadecimal 5092, corresponding to decimal 20626), and obtain that the identifier of the slot for storing the key-value pair "name=jim" is 4242 (corresponding to slot S5 242 ). Then, since slots S0 to S 5041 are set on node N1, the connection module 226 can determine that the target node storing the target data is node N1, and obtain the connection between the service process 221 and node N1 from the connection pool 225.

[0260] Again, for example, after the connection module 236 receives the aforementioned read instruction "get name", it can perform consistent hashing on the CRC16 code of the key "name" (for example, hexadecimal 5092, corresponding to decimal 20626), and obtain that the identifier of the slot for storing the key-value pair "name=lily" is 4242 (corresponding to slot S5 242 ). Then, since slots S0 to S 5041 are set on node N1, the connection module 226 can determine that the target node storing the target data is node N1, and obtain the connection between the service process 231 and node N1 from the connection pool 235.

[0261] S519, the connection module 226 / 236 sends a read request including the read instruction and the database identifier to the IO thread of the target node.

[0262] After obtaining the target connection with the target node, the connection module 226 / 236 can obtain the corresponding database identifier from the target connection, and send a read request including the read instruction and the database identifier to the IO thread of the target node.

[0263] For example, for Figure 4In the situation shown, after the connection module 226 obtains the target connection with the node N1, it can obtain the database identifier DB0 from the target connection and send a read request including the database identifier DB0 and the read instruction "get name" to the IO thread of the node N1.

[0264] For another example, for Figure 4 In the situation shown, after the connection module 236 obtains the target connection with the node N1, it can obtain the database identifier DB1 from the target connection and send a read request including the database identifier DB1 and the read instruction "get name" to the IO thread of the node N1.

[0265] It should be understood that in some embodiments, the read request may further include other information, such as authentication information, etc., which is not limited herein.

[0266] In some embodiments, the read request may also not include the database identifier, which is not limited herein.

[0267] S520. The IO thread of the target node sends the read request to the working thread of the target node.

[0268] After receiving the read request, the IO thread of the target node can send the read request to the working thread of the target node.

[0269] In some embodiments, the IO thread of the target node can also authenticate the read request based on the authentication information in the read request to determine whether the sender of the read request has the permission to access the logical database corresponding to the database identifier included in the read request. If the IO thread of the target node determines that the read request is illegal (for example, the sender of the read request does not have the permission to access the logical database corresponding to the database identifier included in the read request), it does not send the write request to the working thread, but sends a message of access failure to the sender of the read request.

[0270] In some embodiments, if there is no database identifier in the received read request, the IO thread of the target node can determine the database identifier of the logical database to be accessed by the read request based on the recorded corresponding relationship between the foregoing different connections and the logical databases (or the sub-data tables of the logical databases on the node) and the target connection for sending the read request.

[0271] S521. The working thread of the target node reads the target data from the target database corresponding to the database identifier in the read request.

[0272] After receiving the write request, the working thread of the target node can read the target data from the target database corresponding to the database identifier in the read request.

[0273] For example, corresponding toFigure 4 In the shown scenario, after the worker thread of node N1 receives a read request including database identifier DB0 and read instruction "get name", it can determine that the sub-data table of the target database corresponding to database identifier DB0 on node N1 is sub-data table T 0-1 , and reads the value "jim" corresponding to the key "name" from sub-data table T 0-1 .

[0274] For another example, corresponding to Figure 4 the shown scenario, after the worker thread of node N1 receives a read request including database identifier DB1 and read instruction "get name", it can determine that the sub-data table of the target database corresponding to database identifier DB1 on node N1 is sub-data table T 1-1 , and reads the value "lily" corresponding to the key "name" from sub-data table T 1-1 .

[0275] S522, the worker thread of the target node sends the read result to the IO thread of the target node.

[0276] After the worker thread of the target node reads the target data from the sub-data table of the target database on the target node, it can send the read result to the IO thread of the target node.

[0277] After the worker thread of the target node successfully reads the target data from the sub-data table of the target database in the target node, it can send the read result including the target data to the IO thread of the target node. If the worker thread of the target node fails to read the target data from the sub-data table of the target database in the target node (for example, there is no target data in the sub-data table of the target database in the target node, the storage medium for storing the sub-data table of the target database in the target node fails, etc.), it can send the read result indicating the read failure to the IO thread of the target node.

[0278] For example, corresponding to Figure 4 the shown scenario, after the worker thread of node N1 reads the value "jim" corresponding to the key "name" from sub-data table T 0-1 , it can send the read result including the value "jim" to the IO thread of node N1. For another example, corresponding to Figure 4 the shown scenario, after the worker thread of node N1 reads the value "lily" corresponding to the key "name" from sub-data table T 1-1 , it can send the read result including the value "lily" to the IO thread of node N1.

[0279] S523, the IO thread of the target node sends the read result to connection module 226 / 236.

[0280] After the IO thread of the target node receives the read result sent by the working thread of the target node, it can send the read result to the connection module corresponding to the read request.

[0281] For example, corresponding to Figure 4 the situation shown, the IO thread of node N1 can send the read result including the value "jim" to connection module 226, and send the read result including the value "lily" to connection module 236.

[0282] S524, connection module 226 / 236 sends the read result to service process 221 / 231.

[0283] After connection module 226 / 236 receives the read result sent by the IO thread of the target node, it can send the read result to their respective corresponding service processes.

[0284] For example, after connection module 226 receives the read result including the value "jim" sent by node N1, it can send this read result to service process 221 through external interface 223. Also for example, after connection module 236 receives the read result including the value "lily" sent by node N1, it can send this read result to service process 231 through external interface 233. That is to say, although the read instructions sent by both service process 221 and service process 231 are "get name", the read results sent by node N1 to service process 221 and service process 231 are different.

[0285] In some embodiments, when service process 221 / 231 needs to delete target data from KV database cluster 21, it can correspondingly send a deletion request including their respective corresponding database identifiers and the keys of the target data to the IO thread of the target node through their respective corresponding connection modules, so that the working thread of the target node deletes the target data from the corresponding target database based on the database identifier in the deletion request. The specific process can refer to the content of writing data and reading data described above and will not be elaborated here.

[0286] Based on the above data processing method, since in KV database cluster 21, the data of service 0 and service 1 are stored in logical databases DB0 and DB1 respectively, service 0 and service 1 can store different values with the same key, which can avoid key conflicts. And KV database cluster 21 can also authenticate the access requests sent by service 0 and service 1 to avoid service 0 accessing the data of service 1 or service 1 accessing the data of service 0, which is beneficial to improving the security of the data stored in KV database cluster 21.

[0287] The present application also provides a data processing method, which is applied to a client and a KV database cluster.

[0288] In some embodiments, the client may be a device that runs a business application (such as the aforementioned business process), or may be a program code or application that integrates the aforementioned business process and SDK functions. The form of the client is not limited herein.

[0289] Exemplarily, Figure 6 According to some embodiments of the present application, a schematic diagram of the interaction process of another data processing method is shown. As Figure 6 shown, the interaction process includes the following steps:

[0290] S601, the client of the first service detects an access instruction to access target data, and sends an access request including a database identifier to the KV database cluster.

[0291] When the client of the first service detects an access instruction to access target data, it may send an access request including the database identifier corresponding to the first service to the KV database cluster.

[0292] Exemplarily, in some embodiments, the KV database cluster may include multiple nodes. When the client of the first service detects an instruction to access target data, it may determine a target node in the KV database cluster for storing the target data based on the key of the target data, and send an access request including the database identifier corresponding to the first service to the target node. The specific manner for the client of the first service to determine the target node may refer to the description of step S509 or step S517 above, which will not be elaborated herein.

[0293] Exemplarily, when the client of the first service detects an operation by a user to access target data, or when the logic of the business process of the first service confirms that it is necessary to access target data, it may detect an access instruction to access target data. In some embodiments, the client of the first service may also detect an access instruction to access target data in other situations, which is not limited herein.

[0294] It should be understood that the access instruction to access target data may include, but is not limited to, a read instruction, a delete instruction, and a write instruction.

[0295] In some embodiments, in addition to the database identifier, the access request may further include other contents such as authentication information and access instructions, which is not limited herein.

[0296] In some embodiments, the access request may also not include a database identifier, which is not limited herein.

[0297] In some embodiments, when the client of the first service detects an access instruction to access target data, it may send an access request to the KV database cluster based on a pre-established connection for accessing the sub-data table corresponding to the target database in the target node.

[0298] S602. The KV database cluster determines the target database to be accessed according to the database identifier in the access request.

[0299] After receiving the access request, the KV database cluster (such as the target node in the KV database cluster) may determine the sub-data table of the target database to be accessed by the access request on the target node according to the database identifier in the access request.

[0300] For example, the KV database cluster may use the logical database in the database identifiers of each logical database in the KV database cluster that is the same as the database identifier in the access request as the target database.

[0301] In some embodiments, the target node in the KV database cluster may also authenticate the access request according to the authentication information in the access request. If the KV database cluster determines that the access request does not have the permission to access the target database, the KV database cluster may send a message indicating access failure to the first service client; if the KV database cluster determines that the access request has the permission to access the target database, it proceeds to step S603. In this way, the security of the data stored in the KV database cluster can be improved.

[0302] In some embodiments, the KV database cluster may also determine the target database to be accessed according to the connection for sending the access request and the database identifier of the logical database corresponding to the connection when it is established.

[0303] S603. The KV database cluster responds to the access request based on the target database.

[0304] After determining the target database, or after determining the target database and determining that the access request has the permission to access the target database, the target node in the KV database cluster may respond to the access request based on the sub-data table of the target database in the target node. Specifically, reference may be made to the foregoing steps S513 and S521, which will not be elaborated here.

[0305] For example, when the access instruction included in the access request is a deletion instruction, the target node in the KV database cluster can delete the target data from the sub - data table of the target database on the target node; when the access instruction included in the access request is a write instruction, the target node in the KV database cluster can write the target data to the sub - data table of the target database on the target node; when the access instruction included in the access request is a read instruction, the target node in the KV database cluster can read the value corresponding to the key of the target data from the corresponding sub - data table of the target database on the target node.

[0306] It should be understood that corresponding to different access instructions, the KV database cluster can also respond to the access request based on the target database in other corresponding forms, which is not limited herein.

[0307] S604, the KV database cluster sends the access result to the client of the first service.

[0308] After the KV database cluster (such as the target node in the KV database cluster) responds to the access request based on the target database, it can send the access result to the client of the first service. The access result can be a successful access result or a failed access result.

[0309] For example, in the case where the access result is a successful access result: if the access instruction included in the access request is a deletion instruction, the access result can include a message of successfully deleting the target data; if the access instruction included in the access request is a write instruction, the access result can include a message of successfully writing the target data; if the access instruction included in the access request is a read instruction, the access result can include the value corresponding to the key of the target data.

[0310] Based on the above method, since the KV database cluster stores different services in different logical databases, and the keys of different services are independent of each other and can be repeated (that is, different services can store different values with the same key), key conflicts can be avoided. The KV database cluster can also authenticate the access requests sent by the clients of different services to prevent the clients of one service from accessing the data of another service, which is beneficial to improving the security of the data stored in the KV database cluster.

[0311] This application also provides an electronic device 100, which can be used to deploy the aforementioned client, the nodes in the KV database cluster, and the aforementioned database cluster management console.

[0312] Exemplarily, Figure 7 According to some embodiments of the present application, a schematic structural diagram of an electronic device 100 is shown. As Figure 7As shown, the electronic device 100 may include one or more processors 101, a memory 102, a communication interface 103, and a bus 104 for coupling the processor 101, the memory 102, and the communication interface 103.

[0313] The processor 101 may include one or more processing units, such as a central processing unit (CPU), a graphics processing unit (GPU), an image signal processor (ISP), a controller, a digital signal processor (DSP), a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0314] In some embodiments, the processor 101 may be used to execute relevant instructions to implement the data processing method provided in the embodiments of the present application.

[0315] For example, when the electronic device 100 is a client, the processor 101 may be used to execute instructions for configuring the database cluster identifier of the KV database cluster, sending an access request including the database identifier to the KV database cluster when an access instruction for accessing target data is detected, and obtaining an access result from the KV database cluster and sending the access result to the service process, etc.

[0316] For another example, when the electronic device 100 is a node in the KV database cluster, the processor 101 may be used to execute instructions for storing data of different services in sub-data tables of their respective corresponding logical databases, configuring different database identifiers for different sub-data tables, receiving an access request, and responding to the access request based on the target database corresponding to the database identifier in the access request, etc.

[0317] The memory 102 may include non-volatile memory and volatile memory for storing instructions or data.

[0318] For example, when the electronic device 100 is a client, the memory 102 may be used to store instructions corresponding to the foregoing service process and the SDK, and store target data, etc.

[0319] For another example, when the electronic device 100 is a node in the KV database cluster, the memory 102 may be used to store instructions related to the foregoing IO process and work process functions, and may also be used to store sub-data tables corresponding to each service.

[0320] The communication interface 103 may include a wired or wireless communication interface for the interaction between the electronic device 100 and other devices.

[0321] For example, when the electronic device 100 is a client, the electronic device 100 may configure the database cluster identifier with the KV database cluster through the communication interface 103, send the access request to the KV database cluster, and receive the access result from the KV database cluster, etc.

[0322] Again, for example, when the electronic device 100 is a KV database cluster, the electronic device 100 may receive the access request sent by the client through the communication interface 103, configure the database cluster identifier with the client, and send the access result to the client, etc.

[0323] The bus 104 may be used to couple each module in the electronic device 100.

[0324] It should be understood that Figure 7 The structure of the electronic device 100 shown is only an example. In some other embodiments, the electronic device 100 may include more or fewer modules, and some modules may also be combined or split, which is not limited herein.

[0325] It should be understood that the electronic device 100 may be any electronic device, including but not limited to laptop computers, desktop computers, tablet computers, mobile phones, wearable devices, head-mounted displays, servers, mobile email devices, reader devices, smart home devices, etc., which is not limited herein.

[0326] The embodiments of the present application also provide a program product. When the program product is executed on an electronic device, it can enable the electronic device to implement the data access methods provided in the foregoing embodiments.

[0327] The embodiments of the present application also provide a readable storage medium. Instructions are stored in the readable storage medium. When the instructions are executed by the electronic device, the electronic device is enabled to implement the data access methods provided in the foregoing embodiments.

[0328] It should be understood that the technical solutions of the present application are introduced by taking a KV database cluster including multiple nodes as an example in the foregoing embodiments. The solutions provided in the foregoing embodiments are also applicable to a non-clustered KV database (equivalent to a KV database cluster including only one node), which is not limited herein. The specific implementation process of the solutions provided in the foregoing embodiments in a non-clustered KV database (equivalent to a KV database cluster including only one node) may refer to the implementation process of the KV database cluster including multiple nodes in the foregoing, which will not be elaborated herein.

[0329] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or ordering may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Additionally, the inclusion of a structural or method feature in a particular figure does not imply that such a feature is required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0330] It should be noted that in the examples and the description of this patent, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or device that comprises a series of elements includes not only those elements but also other elements not expressly listed, or elements that are inherent to such a process, method, article, or device. Without further limitation, an element defined by the statement "comprising one" does not exclude the presence of additional identical elements in the process, method, article, or device that comprises the element.

Claims

1. A data processing method, characterized in that, Including: The client detects an access instruction for the first data corresponding to the first service in the key-value database system; The client sends an access request for accessing the first data to the key-value database system, where the access request is used to instruct the key-value database system to respond to the access request based on the first data table, and the first data table is one of at least one data table stored in the key-value database system.

2. The method according to claim 1, characterized in that, The first data table is the data table corresponding to the first service among the at least one data table.

3. The method according to claim 1, characterized in that, The key-value database system is a key-value database cluster.

4. The method according to claim 3, characterized in that, The client sending an access request for accessing the first data to the key-value database system includes: The client determines a first node for storing the first data among multiple nodes of the key-value database cluster based on the first data; The client sends the access request to the first node.

5. The method according to claim 4, characterized in that, The client sending the access request to the first node includes: Obtaining a first connection from the connection pool between the client and the first node; Sending the access request to the first node based on the first connection.

6. The method according to claim 5, characterized in that, The first connection is a connection established before detecting the access instruction for the first data corresponding to the first service in the key-value database system and is used to access the first sub-data table of the first data table on the first node.

7. The method according to claim 4, characterized in that, The access request includes a first identifier corresponding to the first data table, and the first identifier is used to instruct the first node to respond to the access request based on the first sub-data table of the first data table on the first node.

8. The method according to claim 6 or 7, characterized in that, The access request being used to instruct the key-value database system to respond to the access request based on the first data table includes: Corresponding to the access instruction being a write instruction, the access request is used to instruct the first node to write the first data to the first sub-data table.

9. The method according to claim 8, characterized in that, The access request being used to instruct the key-value database system to respond to the access request based on the first data table further includes: Corresponding to the access instruction being a read instruction, the access request is used to instruct the first node to read the first data from the first sub-data table and send the first data to the client.

10. The method according to claim 9, characterized in that, The access request being used to instruct the key-value database system to respond to the access request based on the first data table further includes: Corresponding to the access instruction being a delete instruction, the access request is used to instruct the first node to delete the first data from the first sub-data table.

11. A data processing method, characterized in that, Including: The client detects an access instruction for the first data corresponding to the first service in the key-value database system; The client sends an access request for accessing the first data to the key-value database system; The key-value database system responds to the access request based on the first data table, where the first data table is one of at least one data table stored in the key-value database system.

12. The method according to claim 11, characterized in that, The first data table is the data table corresponding to the first service among the at least one data table.

13. The method according to claim 11, characterized in that, The key-value database system is a key-value database cluster.

14. The method according to claim 13, characterized in that, The client sends an access request to the key - value database system to access the first data, including: Based on the first data, the client determines a first node among multiple nodes of the key - value database cluster for storing the first data; The client sends the access request to the first node.

15. The method according to claim 14, wherein, The client sending the access request to the first node includes: The client obtains a first connection from the connection pool between the client and the first node; The client sends the access request to the first node based on the first connection.

16. The method according to claim 15, wherein, The first connection is a connection established by the client before detecting an access instruction for accessing the first data corresponding to the first service in the key - value database system, and is used to access the first sub - data table of the first data table on the first node.

17. The method according to claim 14, wherein, The access request includes a first identifier corresponding to the first data table, and the first node includes sub - data tables corresponding to each data table in the at least one data table; And, the key - value database system responds to the access request based on the first data table, including: Based on the first identifier, the first node determines the first sub - data table corresponding to the first data table from the sub - data tables corresponding to each data table in the at least one data table; The first node responds to the access request based on the first sub - data table.

18. The method according to claim 16 or 17, wherein, The key - value database system responds to the access request based on the first data table, including: Corresponding to the access instruction being a write instruction, the first node writes the first data to the first sub - data table.

19. The method according to claim 18, wherein, The key - value database system responding to the access request based on the first data table further includes: Corresponding to the access instruction being a read instruction, the first node reads the first data from the first sub - data table and sends the first data to the client.

20. The method according to claim 19, wherein, The key - value database system responding to the access request based on the first data table further includes: Corresponding to the access instruction being a delete instruction, the first node deletes the first data from the first sub - data table.

21. The method according to claim 11, wherein, The method further includes: Before the client detects an access instruction for accessing the first data corresponding to the first service in the key - value database system, the key - value database system initializes the correspondence between the first service and the first data table.

22. A data processing method, wherein, Including: The key - value database system receives an access request sent by the client, where the access request is used to request access to data corresponding to the first service in the key - value database system; The key - value database system responds to the access request based on the first data table, where the first data table is one of at least one data tables stored in the key - value database system.

23. The method according to claim 22, wherein, The key - value database system is a key - value database cluster.

24. The method according to claim 22, wherein, The first data table is a data table corresponding to the first service in the at least one data tables.

25. The method according to claim 22, wherein, The access request includes a first identifier corresponding to the first data table; and, the key - value database system responds to the access request based on the first data table, including: The key-value database system determines the first data table from the at least one data table based on the first identifier; The key-value database system responds to the access request based on the first data table.

26. An electronic device, wherein, Comprising: A memory for storing one or more programs; A processor for executing the one or more programs to cause the electronic device to implement the data processing method according to any one of claims 1 to 25.

27. A readable storage medium, characterized in that, The readable storage medium includes one or more programs, and when the one or more programs are executed on an electronic device, the electronic device is caused to implement the data processing method according to any one of claims 1 to 25.

28. A program product, characterized in that, When the program product runs on an electronic device, the electronic device is caused to implement the data processing method according to any one of claims 1 to 25.

Images