Data query system, method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention discloses a data query system, which is characterized in that a database server serving as a master node stores a master copy, at least one other database server serving as a slave node stores corresponding slave copies respectively, and when a load balancer receives a data query request, the data query request is sent to the slave node. The data query request is preferentially routed to the master node, then the master node selects a target slave node from the slave nodes, the data query request is routed to the target slave node, and the target slave node returns a slave copy stored in the target slave node. According to the system, the master / slave nodes storing the master / slave copies can cope with the concurrent query requests of the copies corresponding to the same partition, and the master node can directly route the data query requests to any slave node, so that the query efficiency of the distributed database system can be effectively improved.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of computer technology, and in particular, to a data query system, method, device, storage medium and electronic equipment. BACKGROUND

[0002] With the development of the times, the demand for data storage has increased dramatically, and currently, a distributed database system is often used to meet the storage needs of super large-scale data.

[0003] Generally, in order to improve the query efficiency and reliability of the distributed database system, the data table is usually partitioned, such as partitioning the data whose key first letter is a~m into one partition and the data whose key first letter is n~z into another partition, and storing the replicas of different partitions on different database servers in the distributed database system to share the query pressure of the same data table.

[0004] Then, how to manage these replicas to further improve the query efficiency of the distributed database system and cope with more and more high-concurrency query requirements is a problem to be solved. SUMMARY

[0005] The embodiments of the present specification provide a data query system, method, device, storage medium and electronic equipment to partially solve the problems existing in the prior art.

[0006] The embodiments of the present specification adopt the following technical solutions:

[0007] The data query system provided by the present specification comprises a load balancer and a plurality of database servers; wherein:

[0008] The load balancer is configured to receive a data query request sent by a requestor, determine, among the database servers, a database server storing a master replica of to-be-queried data corresponding to the data query request as a master node, and send the data query request to the master node.

[0009] The master node is configured to, in response to the data query request sent by the load balancer, determine, among the database servers, all database servers storing a slave replica corresponding to the master replica as slave nodes, select a target slave node among the slave nodes, and send the data query request to the target slave node.

[0010] The target slave node is configured to, in response to receiving the data query request sent by the master node, query the slave replica and return the queried slave replica to the requestor through the load balancer.

[0011] The data query method provided in the specification is applied to a database server in which a primary copy is stored in a distributed database; the method comprises:

[0012] The data query request sent by the load balancer is received, and the data query request is sent by a requestor to the load balancer.

[0013] In each database server, all database servers storing the slave copy corresponding to the primary copy are determined as each slave node; wherein the primary copy contains to-be-queried data corresponding to the data query request.

[0014] A target slave node is selected from each slave node.

[0015] The data query request is sent to the target slave node, so that the target slave node queries the slave copy, and the queried slave copy is returned to the requestor through the load balancer.

[0016] The data query device provided in the specification is applied to a database server in which a primary copy is stored in a distributed database, and the device comprises:

[0017] The receiving module is configured to receive a data query request sent by a load balancer, and the data query request is sent by a requestor to the load balancer.

[0018] The determining module is configured to determine, in each database server, all database servers storing the slave copy corresponding to the primary copy as each slave node; wherein the primary copy contains to-be-queried data corresponding to the data query request.

[0019] The selecting module is configured to select a target slave node from each slave node.

[0020] The sending module is configured to send the data query request to the target slave node, so that the target slave node queries the slave copy, and the queried slave copy is returned to the requestor through the load balancer.

[0021] The computer-readable storage medium provided in the specification stores a computer program, and the computer program is executed by a processor to implement the above data query method.

[0022] The electronic device provided in the specification comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the above data query method when executing the program.

[0023] The above at least one technical solution adopted by the embodiment of the specification can achieve the following beneficial effects:

[0024] The embodiment of the present specification discloses a data query system, which stores a master copy by a database server as a master node, respectively stores corresponding slave copies by at least one other database server as a slave node, and a load balancer routes a data query request to the master node in priority when receiving the data query request, and the master node selects a target slave node from each slave node and routes the data query request to the target slave node, and the target slave node returns a slave copy stored by itself. The above system can cope with concurrent query requests for the same partition corresponding copy by the master / slave node storing the master / slave copy, and the data query request can be routed to any slave node by the master node, which can effectively improve the query efficiency of the distributed database system. BRIEF DESCRIPTION OF DRAWINGS

[0025] The drawings described herein are used to provide further understanding of the present specification, constitute a part of the present specification, the illustrative embodiments of the present specification and the description thereof are used to explain the present specification, and do not constitute improper limitation on the present specification. In the drawings:

[0026] Figure 1 A data query system schematic diagram provided by the embodiment of the present specification;

[0027] Figure 2 A data query method flowchart provided by the embodiment of the present specification;

[0028] Figure 3 A deployment schematic diagram of different partition corresponding copies in a distributed database system provided by the embodiment of the present specification;

[0029] Figure 4 A data query device schematic diagram provided by the embodiment of the present specification;

[0030] Figure 5 A structure schematic diagram of an electronic device provided by the embodiment of the present specification. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical scheme and advantages of the present specification clearer, the technical scheme of the present specification will be described clearly and completely in combination with the embodiment of the present specification and corresponding drawings. Obviously, the described embodiment is only a part of the embodiment of the present specification, not all the embodiments. Based on the embodiment in the present specification, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present specification.

[0032] The technical scheme provided by the embodiment of the present specification will be described in detail in combination with the drawings as follows.

[0033] Figure 1A data query system schematic diagram provided in an embodiment of the present specification, the system comprising a load balancer and a plurality of database servers. Figure 1 The data query system shown is essentially a distributed database system, the load balancer is used to distribute a user's data query request to a certain database server, the data query request is processed by the database server to query the to-be-queried data corresponding to the data query request, and the to-be-queried data is returned to the load balancer, and the load balancer returns the to-be-queried data.

[0034] In an embodiment of the present specification, the data table stored in the distributed database system is also partitioned, each partition corresponds to at least one copy, the copies include one master copy and a plurality of slave copies, whether the master copy or the slave copy contains all the data in the partition, but the different copies corresponding to the partition are respectively stored in different database servers, then Figure 1 The distributed database system shown can use the method shown to query data. Figure 2 The method shown can be used to query data.

[0035] Figure 2 A data query method flowchart provided in an embodiment of the present specification, specifically comprising the following steps:

[0036] S200: The load balancer receives the data query request sent by the requester.

[0037] In an embodiment of the present specification, the load balancer in the distributed database system directly faces the requester, and the database server does not directly provide services to the requester, therefore, the data query request sent by the requester will be sent to the load balancer first.

[0038] Among them, the requester in the embodiment of the present specification can be any user terminal or application program.

[0039] S202: In each database server, determine the database server storing the master copy of the to-be-queried data corresponding to the data query request as the master node.

[0040] The load balancer in the embodiment of the present specification is a global load balancer in the distributed database system, which pre-stores the metadata of each partition, the metadata of each partition can be used as the index of each partition, and also stores the correspondence between the master copy corresponding to each partition and the database server where each master copy is located.

[0041] After receiving the query request, the load balancer can first determine the partition where the data requested by the data query request (i.e., the to-be-queried data) is located according to the data query request. Specifically, the load balancer can determine the partition where the to-be-queried data is located according to the key of the to-be-queried data contained in the data query request and the metadata of each partition.

[0042] After determining the partition where the to-be-queried data is located, the load balancer can determine the database server where the primary replica corresponding to the partition where the to-be-queried data is located is located as the master node based on the correspondence between the saved primary replicas corresponding to each partition and the database servers where the primary replicas are located.

[0043] It should be noted that the master node described in the present specification and the slave node appearing hereinafter are logical concepts relative to the replicas corresponding to the same partition. For a partition, if a database server stores a primary replica corresponding to the partition, the database server is a master node, and if a database server stores a slave replica corresponding to the partition, the database server is a slave node.

[0044] S204: Send the data query request to the master node.

[0045] In the embodiments of the present specification, after the load balancer determines the master node storing the primary replica where the to-be-queried data is located, the load balancer preferentially sends the data query request to the master node, and the master node first attempts to process the data query request.

[0046] The steps S200-S204 shown in the present specification Figure 2 The steps S200-S204 shown in the present specification are executed by the load balancer, which can be an independent server for globally balancing the distributed database system in the distributed database system, or a software module for globally balancing the distributed database system in a certain database server. Whether the load balancer is an independent server or a software module in a certain database server, the above-mentioned steps S200-S204 can be realized by executing a computer program in a storage medium (including any storage medium such as memory, disk, etc.) of the server where the load balancer is located or in a processing chip (such as FPGA, ASIC, etc.) connected to the server. The embodiments of the present specification do not limit the hardware structure of the above-mentioned load balancer.

[0047] S206: The master node determines whether the current load state of the master node satisfies a preset condition. If yes, step S216 is executed, otherwise step S208 is executed.

[0048] The master node receives the data query request, and first determines whether a current load state of the master node satisfies a preset condition. The preset condition includes, but is not limited to, a CPU occupancy rate being lower than a preset CPU occupancy rate, a memory occupancy rate being lower than a preset memory occupancy rate, a number of data query requests waiting for processing being no more than a preset number, and the like. That is, the master node needs to first determine whether the current load of the master node is too high.

[0049] In S208, the master node determines, in the database servers, all database servers storing the slave replicas corresponding to the master replica as the slave nodes.

[0050] In an embodiment of the present specification, the master node can pre-store a routing table corresponding to the master replica, and the routing table records the master replica and all database servers where the slave replicas corresponding to the master replica are located.

[0051] If the master node determines that the current load of the master node is too high, the master node can determine, according to the routing table, all database servers storing the slave replicas corresponding to the master replica as the slave nodes.

[0052] In S210, a target slave node is selected from the slave nodes.

[0053] After the master node determines all the slave nodes according to the routing table, the master node can send a load query request to each slave node to query a current load state of each slave node. After the master node receives the current load state of each slave node returned by each slave node, the master node can select a target slave node from the slave nodes according to the current load state of each slave node.

[0054] Specifically, the master node can select, from the slave nodes, a slave node with the smallest current load as the target slave node according to the current load state of each slave node.

[0055] In S212, the data query request is sent to the target slave node.

[0056] After the target slave node is determined, the master node directly sends the data query request to the target slave node, so that the target slave node processes the data query request. That is, in the embodiment of the present specification, the data query request is not sequentially transmitted downward level by level. Sequentially transmitting downward level by level means that the data query request is sent to a lower-level slave node, and the lower-level slave node continues to attempt to process the data query request. If the load of the lower-level slave node is light, the lower-level slave node directly processes the data query request, otherwise the lower-level slave node continues to send the data query request to a slave node at a next lower level, and thus the data query request is sequentially transmitted downward from the master node until the data query request is processed by a slave node with a light load.

[0057] The step-by-step downward transmission method can seriously reduce the query efficiency in a high concurrency scenario, and the master node in the embodiment of the present specification can route the data query request to a slave node with lighter load according to the routing table, which can effectively improve the query efficiency in a high concurrency query scenario.

[0058] S214: The target slave node queries the slave copy and returns the queried slave copy to the load balancer.

[0059] After receiving the data query request, the target slave node can query the slave copy corresponding to the master copy in the data stored in itself, and return the queried slave copy to the load balancer, and the load balancer returns the slave copy to the requester, completing the query.

[0060] S216: The master node queries the master copy and returns the queried master copy to the load balancer.

[0061] In step S206, if the master node determines that its current load is low and meets the preset condition, the master node can directly query the master copy, and return the master copy to the load balancer, and the load balancer returns the master copy to the requester, completing the query.

[0062] The steps S206-S212 and S216 shown in the present specification Figure 2 The steps S206-S212 and S216 shown in the present specification

[0063] The method can cope with concurrent query requests of the replicas corresponding to the same partition by the master / slave node storing the master / slave replicas, and the data query request can be directly routed to any slave node by the master node, so that the query efficiency of the distributed database system can be effectively improved.

[0064] Further, since the load balancer in the embodiment of the present specification preferentially routes the data query request to the master node, and preferentially attempts to query the master replica by the master node, the master node can also monitor the query frequency of the master replica, that is, the query frequency of the partition corresponding to the master replica. When the query frequency reaches a preset threshold, the master node can replicate the master replica to obtain a slave replica corresponding to the master replica, and select a new slave node from the database servers other than the slave nodes, and send the newly replicated slave replica to the new slave node for storage.

[0065] Specifically, the master node can determine whether the load states of the master node and all current slave nodes do not satisfy the above-mentioned preset condition when the query frequency of the partition reaches the preset threshold. If so, it means that a slave replica should be added at this time, so that the master replica is replicated to obtain a slave replica, and a new slave node is selected to store the new slave replica. That is, the distribution of the replica in the embodiment of the present specification is not performed from the master node to the slave nodes in stages, but is directly performed from the master node to the slave nodes, so that the distribution efficiency of the replica can be improved.

[0066] It should be noted that from the distribution method of the replicas corresponding to one partition, it can be seen that the master replica and the slave replica in the embodiment of the present specification are also logical concepts, and in essence, all slave replicas are replicated from the master replica, and the slave replica is not different from the master replica.

[0067] In addition, after sending the newly replicated slave replica to the newly selected slave node for storage, the master node also saves the routing pointer pointing to the newly selected slave node in the routing table corresponding to the master replica stored in the master node itself. That is, Figure 2 In step S208, the master node determines that the routing table based on each slave node records the routing pointers of the database servers where all replicas (including the master replica and the slave replica) corresponding to the partition are located. The master node can determine the database servers pointed to by each routing pointer as the slave nodes storing the slave replicas corresponding to the master replica by querying the pre-stored routing pointers in the routing table corresponding to the master replica.

[0068] The master node can also synchronize the above-mentioned routing table to each slave node storage, so that each slave node can replace the master node to meet disaster recovery needs. Specifically, the load balancer can select a target node from each slave node when monitoring that the master node fails, and re-determine the slave copy stored in the selected target node as a master copy, and re-determine the target node as a master node. Subsequently, when a data query request for the partition is received, the data query request is preferentially routed to the re-determined master node.

[0069] Furthermore, in order to balance the load of each database server in the distributed database system as much as possible, for different partitions, the load balancer can store the master copy corresponding to different partitions in different database servers, so that when the data query request is preferentially routed to the master node where the master copy is located, the data query requests of different partitions will be preferentially processed by different database servers, as shown in Figure 3

[0070] In Figure 3 , the distributed database system includes four database servers Server0-Server3, and the copies corresponding to different partitions to be stored are P1-P4, wherein the unshaded copies are master copies and the shaded copies are slave copies. As Figure 3 can be seen, the master copies of P1-P4 are respectively on Server0-Server3, and the database servers are different.

[0071] The above is a data query system and a data query method provided by an embodiment of the present specification, based on the same idea, the present specification also provides corresponding devices, storage media and electronic equipment.

[0072] Figure 4 A data query device provided by an embodiment of the present specification is shown in the figure, the device is applied to a database server in a distributed database that stores a master copy, and the device includes:

[0073] The receiving module 401 is configured to receive a data query request sent by a load balancer, wherein the data query request is sent by a requester to the load balancer.

[0074] The determining module 402 is configured to determine all database servers that store the slave copy corresponding to the master copy in each database server as each slave node, wherein the master copy contains to-be-queried data corresponding to the data query request.

[0075] The selecting module 403 is configured to select a target slave node from each slave node.

[0076] ​The sending module 404 is configured to send the data query request to the target slave node, so that the target slave node queries the slave replica, and returns the queried slave replica to the requester through the load balancer.

[0077] Optionally, the determining module 402 is further configured to determine that its current load state does not meet a preset condition before the sending module 404 sends the data query request to the target slave node;

[0078] The device further comprises:

[0079] The query module 405 is used to query the primary copy when its current load status meets the preset condition, and return the queried primary copy to the requester through the load balancer.

[0080] Optionally, the selection module 403 is specifically configured to send a load query request to each slave node; receive a current load status returned by each slave node; and select a target slave node from each slave node according to the current load status of each slave node.

[0081] Optionally, the device further comprises:

[0082] The distribution module 406 is used to determine the query frequency of the master copy; when the query frequency reaches a preset threshold, obtain the slave copy corresponding to the master copy, and select a slave node from each database server except the slave node; send the obtained slave copy to the selected slave node for storage, and save the routing pointer pointing to the selected slave node in the routing table corresponding to the master copy stored in itself.

[0083] Optionally, the determination module 402 is specifically used to query the routing pointers pre-saved in the routing table corresponding to the master copy; and determine the database server pointed to by the routing pointers pre-saved in the routing table as the database server storing the slave copy corresponding to the master copy.

[0084] This specification also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can be used to execute the data query method provided above.

[0085] based on Figure 2 The data query method shown in this specification also provides Figure 5 The structural diagram of the electronic device shown in FIG. Figure 5At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course can also include other hardware required by the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to implement the above-mentioned data query method.

[0086] The above only describes the embodiments of the present specification and is not intended to limit the present specification. The present specification can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present specification shall be included in the scope of claims of the present specification.

Claims

1. A data query system, comprising a load balancer and several database servers; wherein: The load balancer is configured to receive a data query request sent by a requester, determine, among the database servers, a database server storing a primary copy of the data to be queried corresponding to the data query request as a primary node, and send the data query request to the primary node; The master node, in response to the data query request sent by the load balancer, determines, among the database servers, all database servers storing slave replicas corresponding to the master replica as slave nodes, selects a target slave node from among the slave nodes, and sends the data query request to the target slave node; The target slave node, in response to receiving the data query request sent by the master node, queries the slave replica and returns the queried slave replica to the requester through the load balancer.

2. The system according to claim 1, wherein the master node is specifically configured to send a load query request to each slave node; Any slave node, in response to the load query request, sends the current load status of the slave node to the master node; The master node is specifically configured to select a target slave node from each slave node according to a current load status of each slave node.

3. In the system as described in claim 1, the master node is also used to determine the query frequency of the master copy. When the query frequency reaches a preset threshold, the slave copy corresponding to the master copy is obtained, and a slave node is selected from each database server other than the slave node. The obtained slave copy is sent to the selected slave node for storage, and a routing pointer pointing to the selected slave node is saved in the routing table corresponding to the master copy stored in the master node itself.

4. The system according to claim 3, wherein the master node is further configured to synchronize the routing table to each slave node; The load balancer is further configured to, when monitoring that the master node fails, select a target node from each slave node, and redefine the slave replica stored in the selected target node as the master replica, thereby redefining the target node as the master node.

5. A data query method, comprising: receiving a data query request sent by a load balancer, where the data query request is sent by a requester to the load balancer; In each database server, all database servers storing slave copies corresponding to the master copy are determined as slave nodes; wherein the master copy contains the data to be queried corresponding to the data query request; Select a target slave node from among the slave nodes; The data query request is sent to the target slave node, so that the target slave node queries the slave replica, and returns the queried slave replica to the requester through the load balancer.

6. The method according to claim 5, before sending the data query request to the target slave node, the method further comprises: Determine that its current load state does not meet the preset conditions; The method further comprises: When its current load status meets the preset condition, the primary copy is queried, and the queried primary copy is returned to the requester through the load balancer.

7. The method according to claim 5, wherein selecting a target slave node from each slave node comprises: Send load query requests to each slave node; Receive the current load status returned by each slave node; A target slave node is selected from each slave node according to the current load status of each slave node.

8. The method of claim 5, further comprising: Determining the query frequency of the primary replica; When the query frequency reaches a preset threshold, obtaining a slave copy corresponding to the master copy; and, selecting a slave node from each database server except the slave node; The obtained slave copy is sent to the selected slave node for storage, and the routing pointer pointing to the selected slave node is saved in the routing table corresponding to the master copy stored in itself.

9. The method according to claim 8, wherein determining, in each database server, all database servers storing slave copies corresponding to the master copy comprises: Query each routing pointer pre-stored in the routing table corresponding to the master copy; The database server pointed to by each routing pointer pre-stored in the routing table is determined as the database server storing the slave copy corresponding to the master copy.

10. A data query device, comprising: A receiving module, configured to receive a data query request sent by a load balancer, wherein the data query request is sent by a requester to the load balancer; A determination module is configured to determine, in each database server, all database servers storing slave copies corresponding to the master copy as slave nodes; wherein the master copy contains the data to be queried corresponding to the data query request; A selection module is used to select a target slave node from among the slave nodes; The sending module is used to send the data query request to the target slave node, so that the target slave node queries the slave replica, and returns the queried slave replica to the requester through the load balancer.

11. A computer-readable storage medium storing a computer program, wherein the computer program implements the method according to any one of claims 5 to 9 when executed by a processor.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 5 to 9 when executing the program.

Images