Data query method, device, storage medium and electronic device

Abstract

The embodiment of the present application discloses a data query method, device, storage medium and electronic device, which belongs to the field of online education. Multiple clusters are set, each cluster includes at least one node for storing data, and each cluster is configured with a Bloom filter. When the target data needs to be queried, the object name of the target data is obtained, and the object name of the target data is hashed using multiple different hash functions to obtain hash values ​​respectively. The Bloom filters of each cluster are used to filter the obtained multiple hash values ​​to determine the candidate cluster, and then the object name is used to query in the candidate cluster in turn. In this way, when the cluster is expanded, the target data can be quickly and accurately found without migrating the data in the existing cluster, which has the characteristics of low cost and high query efficiency.

Description

Technical Field

[0001] The present application relates to the field of online education, and in particular to a data query method, device, storage medium and electronic device. Background Art

[0002] The storage of massive data generally adopts distributed technology, and massive data is stored in multiple nodes. When the data to be stored is received, a hash function is used to store the data to be stored in a node among multiple nodes. When querying data, the target node is first determined according to the hash function, and then the target data is queried on the target node. However, when the storage space of multiple nodes is insufficient, the nodes need to be expanded. After adding new nodes, how to use the existing hash function to query in the multiple nodes after expansion will not be able to determine the correct target node. At this time, the data in the node needs to be migrated to ensure that the user can accurately query the target data on the expanded node. However, the migration of massive data is time-consuming and costly. Summary of the invention

[0003] The data query method, device, storage medium and electronic device provided in the embodiments of the present application can solve the problem of long time consumption and high cost caused by the need to migrate data of the expanded nodes. The technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides a data query method, the method comprising:

[0005] Receiving a data query request from a terminal device; wherein the data query request carries a first object name;

[0006] Performing a hash operation on the first object name according to k different hash functions to obtain k hash values; wherein k is an integer greater than 1;

[0007] Filtering the k hash values ​​according to the Bloom filter of each cluster in the n clusters to determine j candidate clusters in the n clusters; wherein j is less than n and j is an integer greater than or equal to 1;

[0008] The corresponding target data is searched in the j candidate clusters according to the first object name.

[0009] In a possible design, searching for corresponding target data in the j candidate clusters according to the first object name includes:

[0010] Performing a hash operation on the first object name to obtain a first hash value;

[0011] Performing a modulo operation on the first hash value according to the number of nodes in the candidate cluster to obtain a first modulus value;

[0012] determining a target node in the candidate cluster according to the first modulus value;

[0013] The target node is queried according to the first object name.

[0014] In one possible design, it also includes:

[0015] In response to a data storage request of the terminal device, obtaining a second object name; wherein the second object name is an object name of the data to be stored;

[0016] Performing a hash operation on the second object name to obtain a second hash value;

[0017] Performing a modulo operation on the second hash value according to the total number of nodes in the n clusters to obtain a second modulo value;

[0018] A target node is determined in the n clusters according to the second modulus value.

[0019] In one possible design, it also includes:

[0020] receiving a capacity expansion request, wherein the capacity expansion request carries a node identifier;

[0021] Generating a new cluster with at least one node indicated by the node identifier;

[0022] A Bloom filter is configured for the new cluster.

[0023] In one possible design, the lengths of the Bloom filters of each cluster are equal.

[0024] In a second aspect, an embodiment of the present application provides a data query device, the data query device comprising:

[0025] A transceiver unit, configured to receive a data query request from a terminal device; wherein the data query request carries a first object name;

[0026] a computing unit, configured to perform a hash operation on the first object name according to k different hash functions to obtain k hash values; wherein k is an integer greater than 1;

[0027] A filtering unit, configured to filter the k hash values ​​according to the Bloom filter of each cluster in the n clusters to determine j candidate clusters in the n clusters; wherein j is less than n and j is an integer greater than or equal to 1;

[0028] A query unit is used to query corresponding target data in the j candidate clusters according to the first object name.

[0029] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0030] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0031] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:

[0032] Multiple clusters are set up, each cluster includes at least one node for storing data, and each cluster is configured with a Bloom filter. When the target data needs to be queried, the object name of the target data is obtained, and a plurality of different hash functions are used to perform hash operations on the object name of the target data to obtain hash values ​​respectively. The Bloom filters of each cluster are used to filter the obtained multiple hash values ​​to determine the candidate cluster, and then the object name is used to query in the candidate cluster in turn. In this way, when the cluster is expanded, the target data can be quickly and accurately found without migrating the data in the existing cluster, which has the characteristics of low cost and high query efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 This is a network architecture diagram provided by an embodiment of the present application;

[0035] Figure 2 It is a flowchart of a data query method provided in an embodiment of the present application;

[0036] Figure 3 is another flow chart of the data query method provided in the embodiment of the present application;

[0037] Figure 4 is a schematic diagram of the structure of a device provided in an embodiment of the present application;

[0038] Figure 5 It is a structural schematic diagram of a device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0040] Figure 1 An exemplary system architecture that can be applied to the data query method or data query device of the present application is shown.

[0041] like Figure 1 As shown, the system architecture may include: a terminal device 11, a proxy server 12 and n clusters, n is an integer greater than 1, and each cluster is provided with a Bloom filter. The Bloom filter is used to perform Bloom filtering, and the Bloom filter includes an array of length m, m is an integer greater than 1, and in the initial state, the values ​​of all bits of the array are zero. When storing data in a cluster associated with the Bloom filter, k hash functions are used to perform hash operations on the storage addresses of the data to be stored to obtain k hash values, k is an integer greater than 1, and then m is modulo-operated according to the k hash values ​​to obtain k modulo values, and k corresponding bits are determined in the array by the k modulo values, and the values ​​of the determined k bits are set to 1. When querying data, k hash functions are used to perform query operations on the storage addresses to obtain k hash values, and then m is modulo-operated according to the k hash values ​​to obtain k modulo values, and k bits are determined in the array of each Bloom filter according to the k modulo values. If the values ​​of all bits of the k bits are 1, it indicates that the target data is located in the cluster associated with the Bloom filter.

[0042] Among them, wireless communication links or wired communication links can be used between the various devices in the system architecture. For example, the wired communication link includes optical fiber, twisted pair or coaxial cable, and the wireless communication link includes Bluetooth communication link, wireless fidelity (WIreless-FIdelity, Wi-Fi) communication link or microwave communication link, etc.

[0043] The network architecture of the present application can be used in online teaching scenarios, and the teaching mode can be one-to-one or one-to-many, that is, one teacher corresponds to one student or one teacher corresponds to multiple students. Accordingly, in a one-to-one teaching mode, a terminal device used for the teacher to teach and a terminal device used for the student to teach communicate with each other; in a one-to-many teaching mode, a terminal device used for the teacher to teach and multiple terminal devices used for the student to teach communicate with each other.

[0044] The terminal device can be installed with various communication client applications, such as: video recording applications, video playback applications, voice interaction applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0045] The terminal device can be hardware or software. When the terminal device is hardware, it can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers. When the terminal device is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, to provide distributed services), or it can be implemented as a single software or software module, which is not specifically limited here.

[0046] When the terminal device is hardware, a display device and a camera may be installed thereon. The display device may be any device capable of realizing a display function, and the camera is used to collect video streams; for example, the display device may be a cathode ray tube display (CR), a light-emitting diode display (LED), an electronic ink screen, a liquid crystal display (LCD), a plasma display panel (PDP), etc. The user may use the display device on the terminal device to view displayed text, pictures, videos and other information.

[0047] It should be noted that the data query method provided in the embodiment of the present application is generally executed by the proxy server 12 , and accordingly, the data query device is generally arranged in the proxy server 12 .

[0048] The proxy server may be a server that provides various services, and the proxy server may be hardware or software. When the proxy server is hardware, it may be implemented as a distributed server cluster consisting of multiple servers, or it may be implemented as a single server. When the proxy server is software, it may be implemented as multiple software or software modules (for example, for providing distributed services), or it may be implemented as a single software or software module, which is not specifically limited here.

[0049] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be used as required.

[0050] The following will be combined with the attached Figure 2 -Attached Figure 3 , the data query method provided by the embodiment of the present application is introduced in detail. Among them, the data query device in the embodiment of the present application can be any proxy server in the task distribution cluster.

[0051] See also Figure 2 , which is a flow chart of a data query method provided in an embodiment of the present application. Figure 2 As shown, the method of the embodiment of the present application may include the following steps:

[0052] S201. Receive a data query request from a terminal device.

[0053] Among them, when the terminal device needs to query data, based on the user's query instruction, it sends a data query request to the proxy server, and the data query request carries a first object name. The data query request can be an HTTP protocol message or an HTTPS protocol message, or other communication protocol messages, which are not limited in the embodiment of the present application. The first object name includes, but is not limited to, a file name, a storage address, or other types of strings, and different data have different object names.

[0054] For example, the data query request sent by the terminal device is: "http: / / xxx.com / shanghai / table / myimg.jpg", and the first object name obtained by the proxy server after parsing the data query request is "myimg.jpg".

[0055] S202: Perform a hash operation on the first object name according to k different hash functions to obtain k hash values.

[0056] The proxy server uses different hash functions to perform hash processing on the first object name to obtain k hash values. Obviously, the k hash values ​​are also different, and k is an integer greater than 1.

[0057] For example: k=3, the first object name is "x1", the three hash functions are hash1(), hash2() and hash3(), and the hash values ​​obtained by hashing the first object name using the three hash functions are f1=hash1(x1), f2=hash2(x1), and f2=hash3(x1).

[0058] S203 : Filter the k hash values ​​according to the Bloom filter of each cluster among the n clusters to determine j candidate clusters among the n clusters.

[0059] Each cluster includes multiple nodes, and databases are deployed on the multiple nodes. Each node is used to store data, and the data types include but are not limited to pictures, videos, audio or other types of data. The database can be a relational database or a non-relational database. Each cluster is provided with a Bloom filter, and the length of the array of each Bloom filter is equal or unequal. The process of filtering k hash values ​​according to the Bloom filter of each cluster to determine j candidate clusters includes:

[0060] For each of the n clusters, obtain the length m of the Bloom filter array of the cluster, use m to calculate the modulus of k hash values ​​to obtain k modulus values, determine k positions in the array according to the k modulus values, and determine whether the values ​​corresponding to the k positions are all specified values. If so, use the cluster as a candidate cluster.

[0061] For example, k=3, n=8, m=10, and the eight clusters are clusters 0 to 7, respectively. The three hash values ​​calculated in S202 are f1, f2, and f3, and the initial value of the Bloom filter array of all clusters is 0. Traverse the eight clusters, and for any one of the eight clusters, obtain the Bloom filter of the cluster, and calculate the modulus of the length 10 of the Bloom filter according to the three hash values, and obtain the modulus values ​​of 1, 3, and 8, respectively. Then, query the values ​​at the positions of the values ​​of the Bloom filter with serial numbers 1, 3, and 8. If the values ​​corresponding to the positions with serial numbers 1, 3, and 8 are all 1, it indicates that the target data may be located in the cluster, and the cluster is used as a candidate cluster. If at least one of the positions with serial numbers 1, 3, and 8 is not 0, it indicates that the target data cannot exist in the cluster, and continue to query the next cluster until the eight clusters are traversed and at least one candidate cluster is obtained.

[0062] S204 : Query corresponding target data in j candidate clusters according to the first object name.

[0063] Among them, the proxy server queries the corresponding target data in j candidate clusters respectively. For each candidate cluster, the query method can be: perform a hash operation on the first object name according to a preset hash function to obtain a first hash value. The preset hash function can be any one of the k hash functions, then directly obtain the first hash value from S202 without recalculation, saving calculation time; if the preset hash function is different from the k hash functions, then the first hash value needs to be recalculated. Determine the number of nodes in the candidate cluster, perform a modulus operation on the first hash value according to the number of nodes to obtain a first modulus value, query on the node corresponding to the first modulus value according to the first object name, if the target data is queried, end the process; if the target data is not queried, continue to use the above method to query in the next candidate cluster, and directly complete the query of the j candidate clusters. The target data must be on one of the j candidate clusters.

[0064] When the scheme of the embodiment of the present application is executed, multiple clusters are set up, each cluster includes at least one node for storing data, and each cluster is configured with a Bloom filter. When it is necessary to query target data, the object name of the target data is obtained, and a plurality of different hash functions are used to perform hash operations on the object name of the target data to obtain hash values ​​respectively. The Bloom filters of each cluster are used to filter the obtained multiple hash values ​​to determine a candidate cluster, and then the object name is used to query in the candidate clusters in turn. In this way, when the cluster is expanded, the target data can be quickly and accurately found without migrating the data in the existing cluster, which has the characteristics of low cost and high query efficiency.

[0065] See also Figure 3 , which is a flow chart of a data query method according to an embodiment of the present application, wherein the execution subject of the present embodiment is a proxy server. The data query method may include the following steps:

[0066] S301: Receive a capacity expansion request.

[0067] When the capacity of the existing cluster is insufficient, the existing cluster is expanded to add at least one node to the existing cluster. The terminal device sends an expansion request to the proxy server, and the expansion request carries a node identifier, the number of which is one or more. The node identifier is used to indicate the identity of the node. For example, the node identifier is one or more of an IP address, a domain name, and a MAC address.

[0068] S302: Generate a new cluster using at least one node indicated by the node identifier.

[0069] Among them, at least one of the expanded nodes will form a new cluster, and the number of current clusters will increase by 1. For example: the current number of clusters is 4, and this expansion adds 3 nodes, and the 3 nodes form a new cluster, so the current number of clusters changes to 5.

[0070] S303: Configure a Bloom filter for the new cluster.

[0071] The Bloom filter array includes an array of m bits. In the initial state, the array of each bit of the array is 0, m is an integer greater than 1, and the length of the Bloom filter array configured in the new cluster can be equal to that of the existing cluster.

[0072] S304: In response to a data storage request from the terminal device, obtain a second object name.

[0073] The second object name is the object name of the data to be stored, and the type of the data to be stored is not limited to pictures, videos, audio or text, etc. The proxy server receives a data storage request from the terminal device, and obtains the second object name of the data to be stored after parsing the data storage request.

[0074] S305: Determine a target node in n clusters according to the second object name.

[0075] The n clusters are the number of all current clusters of the storage system, each cluster includes at least one node, and the proxy server determines the target node for storing the data to be stored in the n clusters according to the second object name.

[0076] Method 1: Perform a hash operation on the second object name to obtain a second hash value, and determine the p nodes included in the n clusters; perform a modulus operation on the second hash value according to the total number of nodes p to obtain a second modulus value, and determine the target node among the p nodes according to the second modulus value.

[0077] For example, n=4, the four clusters are cluster 0, cluster 1, cluster 2, and cluster 3, and the four clusters contain 16 nodes in total. The 16 nodes are numbered from 0 to node 15. The proxy server performs a hash operation on the second object name of the data to be stored to obtain a second hash value, performs a modulus operation on the second hash value according to 15, and the range of the obtained second modulus value is 0 to 15. The second modulus value is used as the node serial number, that is, the node corresponding to the second modulus value is the target node.

[0078] Method 2: Perform a hash operation on the second object name to obtain a second hash value, perform a modulo operation on the second hash value according to the number of clusters n to obtain a second modulus value, determine a target cluster among the n clusters according to the second modulus value, perform a modulo operation on the second hash value according to the number of nodes in the target cluster to obtain a third modulus value, and determine a target node in the target cluster according to the third modulus value.

[0079] For example: n=4, the 4 clusters are numbered 0 to 3, recorded as cluster 0, cluster 1, cluster 2 and cluster 3, the proxy server performs a hash operation on the second object name to obtain a second hash value, performs a modulo operation on the second hash value according to 4 to obtain a second modulus value, the value range of the second modulus value is 0 to 3, the second modulus value is used as the sequence number, the cluster corresponding to the second modulus value is the target cluster, assuming that the second modulus value is 2, cluster 2 is the target cluster. The number of nodes contained in cluster 2 is obtained as 8, the 8 nodes are numbered 0 to 7, recorded as node 0 to node 7, the second hash value is performed modulo operation according to 7 to obtain a third modulus value, the value range of the third modulus value is 0 to 7, the third modulus value is used as the sequence number of the target node, assuming that the third modulus value is 5, node 5 is the target node.

[0080] S306: Write the data to be stored to the target node.

[0081] The second hash value is used as the storage address, and the address to be stored is written into the storage address of the target node.

[0082] S307 . Perform a hash operation on the second object name according to k different hash functions to obtain k hash values.

[0083] Among them, after the data to be stored is written to the target node, the array of the Bloom filter of the cluster where the target node is located is updated. First, the second object is hashed according to k different hash functions to obtain k hash values, where k is an integer greater than 1. Preferably, the values ​​of k and m are equal. Obviously, the k hash values ​​are not equal.

[0084] S308 . Update according to the k hash values ​​and the Bloom filter of the cluster where the target node is located.

[0085] Among them, the length of the Bloom filter array is determined to be m, and the k hash values ​​are calculated modulo m to obtain k modulus values. The value range of the k modulus values ​​is 0 to m-1, and the bits of the array with a length of m are numbered 0 to m-1. The corresponding bits are determined in the m arrays according to the k modulus values, and then the value of the bit is set to 1.

[0086] For example: the length of the array is 10, the bits of the array are numbered 0 to 9, k=3, three hash functions are used to perform hash operations on the second object name to obtain three hash values, and the modulo values ​​obtained by performing modulo operations on 10 using the three hash values ​​are 1, 3, and 9, respectively, then the bits numbered 1, 3, and 9 in the array are set to 1.

[0087] S309: Receive a data query request from a terminal device.

[0088] Wherein, the data query request carries the first object name. When the terminal device needs to query data, based on the user's query instruction, it sends a data query request to the proxy server. The data query request carries the first object name. The data query request can be an HTTP protocol message or an HTTPS protocol message, or other communication protocol messages, which is not limited in the embodiment of the present application. The first object name includes but is not limited to a file name, a storage address, or other types of strings. Different data have different object names.

[0089] For example, the data query request sent by the terminal device is: "http: / / xxx.com / shanghai / table / myimg.jpg", and the first object name obtained by the proxy server after parsing the data query request is "myimg.jpg".

[0090] S310. Perform a hash operation on the first object name according to k different hash functions to obtain k hash values; wherein k is an integer greater than 1.

[0091] The proxy server uses different hash functions to perform hash processing on the first object name to obtain k hash values. Obviously, the k hash values ​​are also different, and k is an integer greater than 1.

[0092] For example: k=3, the first object name is "x1", the three hash functions are hash1(), hash2() and hash3(), and the hash values ​​obtained by hashing the first object name using the three hash functions are f1=hash1(x1), f2=hash2(x1), and f2=hash3(x1).

[0093] S311 , filtering the k hash values ​​according to the Bloom filter of each cluster among the n clusters, and determining j candidate clusters among the n clusters.

[0094] Wherein, j is less than n, and j is an integer greater than or equal to 1; each cluster includes multiple nodes, and databases are deployed on multiple nodes. Each node is used to store data, and the data types include but are not limited to pictures, videos, audios or other types of data. The database can be a relational database or a non-relational database. Each cluster is provided with a Bloom filter, and the lengths of the arrays of each Bloom filter are equal or unequal. The process of filtering k hash values ​​according to the Bloom filters of each cluster to determine j candidate clusters includes:

[0095] For each of the n clusters, obtain the length m of the Bloom filter array of the cluster, use m to calculate the modulus of k hash values ​​to obtain k modulus values, determine k positions in the array according to the k modulus values, and determine whether the values ​​corresponding to the k positions are all specified values. If so, use the cluster as a candidate cluster.

[0096] For example, k=3, n=8, m=10, and the eight clusters are clusters 0 to 7, respectively. The three hash values ​​calculated in S202 are f1, f2, and f3, and the initial value of the Bloom filter array of all clusters is 0. Traverse the eight clusters, and for any one of the eight clusters, obtain the Bloom filter of the cluster, and calculate the modulus of the length 10 of the Bloom filter according to the three hash values, and obtain the modulus values ​​of 1, 3, and 8, respectively. Then, query the values ​​at the positions of the values ​​of the Bloom filter with serial numbers 1, 3, and 8. If the values ​​corresponding to the positions with serial numbers 1, 3, and 8 are all 1, it indicates that the target data may be located in the cluster, and the cluster is used as a candidate cluster. If at least one of the positions with serial numbers 1, 3, and 8 is not 0, it indicates that the target data cannot exist in the cluster, and continue to query the next cluster until the eight clusters are traversed and at least one candidate cluster is obtained.

[0097] S312: Perform a hash operation on the first object name to obtain a first hash value.

[0098] Among them, for each of the j candidate clusters, a hash operation is performed on the first object name according to a preset hash function to obtain a first hash value. The preset hash function can be any one of the k hash functions. In this way, the first hash value is directly obtained from S202 without the need for recalculation, thus saving computing time.

[0099] S313: Perform a modulo operation on the first Hash value according to the number of nodes in the candidate cluster to obtain a first modulus value.

[0100] S314: Determine a target node in the candidate cluster according to the first module value.

[0101] S315: Query on the target node according to the first object name.

[0102] A first modulus value is obtained by performing a modulo operation on the first hash value according to the number of nodes, and a query is performed on the node corresponding to the first modulus value according to the first object name. If the target data is found, the process ends; if the target data is not found, the above method is continued to be used to query in the next candidate cluster until the query of the j candidate clusters is completed. The target data must be in one of the j candidate clusters.

[0103] In an embodiment of the present application, multiple clusters are set up, each cluster includes at least one node for storing data, and each cluster is configured with a Bloom filter. When it is necessary to query target data, the object name of the target data is obtained, and a plurality of different hash functions are used to perform hash operations on the object name of the target data to obtain hash values ​​respectively. The Bloom filters of each cluster are used to filter the obtained multiple hash values ​​to determine a candidate cluster, and then the object name is used to query in the candidate clusters in turn. In this way, when the cluster is expanded, the target data can be quickly and accurately found without migrating the data in the existing cluster, which has the characteristics of low cost and high query efficiency.

[0104] The following are device embodiments of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0105] See also Figure 4 , which shows a schematic diagram of the structure of a data query device provided by an exemplary embodiment of the present application. Hereinafter referred to as device 4, device 4 can be implemented as all or part of a proxy server through software, hardware or a combination of both. Device 4 includes: a receiving unit 401, a computing unit 402, a filtering unit 403 and a query unit 404.

[0106] The receiving unit 401 is configured to receive a data query request from a terminal device; wherein the data query request carries a first object name;

[0107] A computing unit 402, configured to perform a hash operation on the first object name according to k different hash functions to obtain k hash values; wherein k is an integer greater than 1;

[0108] A filtering unit 403 is used to filter the k hash values ​​according to the Bloom filter of each cluster in the n clusters to determine j candidate clusters in the n clusters; wherein j is less than n and j is an integer greater than or equal to 1;

[0109] The query unit 404 is configured to query corresponding target data in the j candidate clusters according to the first object name.

[0110] In one or more embodiments, querying the j candidate clusters for corresponding target data according to the first object name includes:

[0111] Performing a hash operation on the first object name to obtain a first hash value;

[0112] Performing a modulo operation on the first hash value according to the number of nodes in the candidate cluster to obtain a first modulus value;

[0113] determining a target node in the candidate cluster according to the first modulus value;

[0114] The target node is queried according to the first object name.

[0115] In one or more embodiments, the invention further includes: a writing unit, configured to:

[0116] In response to a data storage request of the terminal device, obtaining a second object name; wherein the second object name is an object name of the data to be stored;

[0117] Determine a target node in the n clusters according to the second object name;

[0118] Writing the data to be stored to the target node;

[0119] Performing a hash operation on the second object name according to the k different hash functions to obtain k hash values;

[0120] The update is performed according to the k hash values ​​and the Bloom filter of the cluster where the target node is located.

[0121] In one or more embodiments, determining the target node in the n clusters according to the second object name includes:

[0122] Performing a hash operation on the second object name to obtain a second hash value;

[0123] Determine p nodes included in the n clusters;

[0124] Performing a modulo operation on the second hash value according to the total number of nodes p to obtain a second modulo value;

[0125] A target node is determined among the p nodes according to the second modulus value.

[0126] In one or more embodiments, determining the target node in the n clusters according to the second object name includes:

[0127] Performing a hash operation on the second object name to obtain a second hash value;

[0128] Performing a modulo operation on the second Hash value according to the number of clusters n to obtain a second modulus value;

[0129] Determine a target cluster among the n clusters according to the second modulus value;

[0130] Performing a modulo operation on the second hash value according to the number of nodes in the target cluster to obtain a third modulo value;

[0131] A target node in the target cluster is determined according to the third modulus value.

[0132] In one or more embodiments, further comprising:

[0133] A capacity expansion unit, configured to receive a capacity expansion request, wherein the capacity expansion request carries a node identifier;

[0134] Generating a new cluster with at least one node indicated by the node identifier;

[0135] A Bloom filter is configured for the new cluster.

[0136] In one or more embodiments, the lengths of the Bloom filters of each cluster are equal.

[0137] It should be noted that, when the device 4 provided in the above embodiment executes the data query method, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the data query method embodiments provided in the above embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0138] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0139] The present application also provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded and executed by a processor as described above. Figure 2-Figure 3 The method steps of the embodiment shown in the figure can be found in the specific implementation process. Figure 2-Figure 3 The specific description of the illustrated embodiment will not be repeated here.

[0140] The present application also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the data query method described in the above embodiments.

[0141] Figure 5 A schematic diagram of the structure of a data query device provided in an embodiment of the present application, hereinafter referred to as device 5, which can be integrated into the aforementioned proxy server, such as Figure 5 As shown, the device includes: a memory 502, a processor 501, an input device 503, an output device 504 and a communication interface.

[0142] The memory 502 may be an independent physical unit, and may be connected to the processor 501, the input device 503, and the output device 504 via a bus. The memory 502, the processor 501, the input device 503, and the output device 504 may also be integrated together and implemented by hardware.

[0143] The memory 502 is used to store programs for implementing the above method embodiments or various modules of the device embodiments, and the processor 501 calls the program to execute the operations of the above method embodiments.

[0144] The input device 502 includes but is not limited to a keyboard, a mouse, a touch panel, a camera and a microphone; the output device includes but is not limited to a display screen.

[0145] The communication interface is used to send and receive various types of messages, and the communication interface includes but is not limited to a wireless interface or a wired interface.

[0146] Optionally, when part or all of the distributed task scheduling method in the above embodiment is implemented by software, the device may also include only a processor. The memory for storing the program is located outside the device, and the processor is connected to the memory through a circuit / wire to read and execute the program stored in the memory.

[0147] The processor may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.

[0148] The processor may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0149] The memory may include volatile memory, such as random-access memory (RAM); the memory may also include non-volatile memory, such as flash memory, hard disk drive (HDD) or solid-state drive (SSD); the memory may also include a combination of the above types of memory.

[0150] The processor 501 calls the program code in the memory 502 to perform the following steps:

[0151] Receiving a data query request from a terminal device; wherein the data query request carries a first object name;

[0152] Performing a hash operation on the first object name according to k different hash functions to obtain k hash values; wherein k is an integer greater than 1;

[0153] Filtering the k hash values ​​according to the Bloom filter of each cluster in the n clusters to determine j candidate clusters in the n clusters; wherein j is less than n and j is an integer greater than or equal to 1;

[0154] The corresponding target data is searched in the j candidate clusters according to the first object name.

[0155] In one or more embodiments, the processor 501 executes the querying of the corresponding target data in the j candidate clusters according to the first object name, including:

[0156] Performing a hash operation on the first object name to obtain a first hash value;

[0157] Performing a modulo operation on the first hash value according to the number of nodes in the candidate cluster to obtain a first modulus value;

[0158] determining a target node in the candidate cluster according to the first modulus value;

[0159] The target node is queried according to the first object name.

[0160] In one or more embodiments, the processor 501 is further configured to execute:

[0161] In response to a data storage request of the terminal device, obtaining a second object name; wherein the second object name is an object name of the data to be stored;

[0162] Determine a target node in the n clusters according to the second object name;

[0163] Writing the data to be stored to the target node;

[0164] Performing a hash operation on the second object name according to the k different hash functions to obtain k hash values;

[0165] The update is performed according to the k hash values ​​and the Bloom filter of the cluster where the target node is located.

[0166] In one or more embodiments, the processor 501 performs the step of determining the target node in the n clusters according to the second object name, including:

[0167] Performing a hash operation on the second object name to obtain a second hash value;

[0168] Determine p nodes included in the n clusters;

[0169] Performing a modulo operation on the second hash value according to the total number of nodes p to obtain a second modulo value;

[0170] A target node is determined among the p nodes according to the second modulus value.

[0171] In one or more embodiments, the processor 501 performs the step of determining the target node in the n clusters according to the second object name, including:

[0172] Performing a hash operation on the second object name to obtain a second hash value;

[0173] Performing a modulo operation on the second Hash value according to the number of clusters n to obtain a second modulus value;

[0174] Determine a target cluster among the n clusters according to the second modulus value;

[0175] Performing a modulo operation on the second hash value according to the number of nodes in the target cluster to obtain a third modulo value;

[0176] A target node in the target cluster is determined according to the third modulus value.

[0177] In one or more embodiments, the processor 501 is further configured to execute:

[0178] receiving a capacity expansion request, wherein the capacity expansion request carries a node identifier;

[0179] Generating a new cluster with at least one node indicated by the node identifier;

[0180] A Bloom filter is configured for the new cluster.

[0181] In one or more embodiments, the lengths of the Bloom filters of each cluster are equal.

[0182] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the data query method provided in the above embodiment.

[0183] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0184] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0185] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0186] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

Claims

1. A data query method, characterized in that: include: Multiple clusters; Receiving a data query request from a terminal device; wherein the data query request carries a first object name; Performing a hash operation on the first object name according to k different hash functions to obtain k hash values; wherein k is an integer greater than 1; The k hash values ​​are filtered according to the Bloom filter of each cluster in the n clusters to delete the cluster that is unlikely to include the target data corresponding to the first object name in the n clusters, and j candidate clusters are determined, where the j candidate clusters are clusters that may include the target data in the n clusters; wherein n is an integer greater than 1, j is less than n, and j is an integer greater than or equal to 1; Performing a hash operation on the first object name to obtain a first hash value; For each candidate cluster, the number of nodes of the candidate cluster is obtained, and a modulo operation is performed on the first Hash value according to the number of nodes of the candidate cluster to obtain a first modulo value corresponding to the candidate cluster, so as to obtain first modulo values ​​corresponding to each of the j candidate clusters; For each candidate cluster, determine a node whose sequence number is the first module value corresponding to the candidate cluster among the multiple nodes included in the candidate cluster as the target node corresponding to the candidate cluster, so as to obtain the target nodes corresponding to each of the j candidate clusters; The target data is queried in the first node. If the target data is found in the first node, the query process is terminated; if the target data is not found in the first node, the second node is used as the first node to perform the step of querying the target data in the first node; wherein the first node is any one of j target nodes, and the second node is one of the other target nodes among the j target nodes except the first node; In response to a data storage request of the terminal device, obtaining a second object name; wherein the second object name is an object name of the data to be stored; Determine a target node in the n clusters according to the second object name; Determining the target node in the n clusters according to the second object name includes: Performing a hash operation on the second object name to obtain a second hash value; Determine p nodes included in the n clusters; Performing a modulo operation on the second hash value according to the total number of nodes p to obtain a second modulo value; Determine a target node among the p nodes according to the second modulus value; or; Performing a hash operation on the second object name to obtain a second hash value; Performing a modulo operation on the second Hash value according to the number of clusters n to obtain a second modulus value; Determine a target cluster among the n clusters according to the second modulus value; Performing a modulo operation on the second hash value according to the number of nodes in the target cluster to obtain a third modulo value; Determine a target node in the target cluster according to the third modulus value; Writing the data to be stored to the target node; Performing a hash operation on the second object name according to the k different hash functions to obtain k hash values; The update is performed according to the k hash values ​​and the Bloom filter of the cluster where the target node is located.

2. The method according to claim 1, characterized in that Also includes: receiving a capacity expansion request, wherein the capacity expansion request carries a node identifier; Generating a new cluster with at least one node indicated by the node identifier; A Bloom filter is configured for the new cluster.

3. The method according to claim 1 or 2, characterized in that: The length of the Bloom filters of each cluster is equal.

4. A data query device, characterized in that: include: A receiving unit, configured to receive a data query request from a terminal device; wherein the data query request carries a first object name; a computing unit, configured to perform a hash operation on the first object name according to k different hash functions to obtain k hash values; wherein k is an integer greater than 1; a filtering unit, configured to filter the k hash values ​​according to the Bloom filter of each cluster in the n clusters, so as to delete the cluster in the n clusters that is unlikely to include the target data corresponding to the first object name, and determine j candidate clusters, where the j candidate clusters are clusters in the n clusters that may include the target data; wherein n is an integer greater than 1, j is less than n, and j is an integer greater than or equal to 1; A query unit, configured to perform a hash operation on the first object name to obtain a first hash value; for each candidate cluster, obtain the number of nodes of the candidate cluster, perform a modulus operation on the first hash value according to the number of nodes of the candidate cluster to obtain a first modulus value corresponding to the candidate cluster, so as to obtain first modulus values ​​corresponding to each of the j candidate clusters; for each candidate cluster, determine a node whose serial number is the first modulus value corresponding to the candidate cluster among multiple nodes included in the candidate cluster as a target node corresponding to the candidate cluster, so as to obtain target nodes corresponding to each of the j candidate clusters; query the target data in the first node, and terminate the query process if the target data is found in the first node; if the target data is not found in the first node, use the second node as the first node and execute the step of querying the target data in the first node; wherein the first node is any one of the j target nodes, and the second node is one of the other target nodes among the j target nodes except the first node; An acquiring unit, configured to acquire a second object name in response to a data storage request of the terminal device; wherein the second object name is an object name of data to be stored; Determine a target node in the n clusters according to the second object name; Determining the target node in the n clusters according to the second object name includes: Performing a hash operation on the second object name to obtain a second hash value; Determine p nodes included in the n clusters; Performing a modulo operation on the second hash value according to the total number of nodes p to obtain a second modulo value; Determine a target node among the p nodes according to the second modulus value; or; Performing a hash operation on the second object name to obtain a second hash value; Performing a modulo operation on the second Hash value according to the number of clusters n to obtain a second modulus value; Determine a target cluster among the n clusters according to the second modulus value; Performing a modulo operation on the second hash value according to the number of nodes in the target cluster to obtain a third modulo value; Determine a target node in the target cluster according to the third modulus value; A writing unit, used for writing the data to be stored into the target node; The operation unit is further configured to perform a hash operation on the second object name according to the k different hash functions to obtain k hash values; An updating unit is used to update according to the k hash values ​​and the Bloom filter of the cluster where the target node is located.

5. A computer storage medium, characterized in that: The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 3.

6. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 3.

Images