Data query method, device, apparatus, medium and product

Abstract

This application discloses a data query method, apparatus, device, medium, and product, applicable to various scenarios such as big data processing, distributed computing, and real-time data processing. The method includes: receiving batch query requests; generating a key list based on the batch query requests; generating the key list based on the batch query requests, and then determining the key type of each key in the key list based on the key list and a hash table in an LRU cache; obtaining the type data of each key according to the key type, and combining the type data of each key to generate target data as the query result of the batch query request. This application uses a hash table in an LRU cache to determine the type of keys in the key list, thereby obtaining the type data of each key for different types of keys and combining the type data of each key, realizing batch querying of the LRU cache and improving data query efficiency.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a data query method, apparatus, device, medium, and product. Background Technology

[0002] In modern applications, query caching is typically used to retrieve frequently accessed data in order to improve system performance and user experience.

[0003] When the cache space is insufficient, the LRU (Latest Recent Used) cache will evict the least recently used data to ensure that the cache always stores the latest and most frequently used data.

[0004] However, since the LRU cache only provides the insertion or return of a single key, it is not suitable for batch query scenarios. Summary of the Invention

[0005] The main purpose of this application is to provide a data query method, apparatus, device, medium and product that can realize batch query of LRU cache to improve data query efficiency.

[0006] To achieve the above objectives, firstly, this application provides a data query method, including:

[0007] Receive batch query requests;

[0008] Generate a key list based on batch query requests;

[0009] Based on the key list and the hash table in the LRU cache, determine the key type of each key in the key list;

[0010] Retrieve the type data for each key based on the key type, and combine the type data of each key to generate the target data as the query result for the batch query request.

[0011] In one implementation, generating a key list based on batch query requests includes:

[0012] For each query request in the batch query request, obtain the key corresponding to each query request;

[0013] Sort and summarize the keys corresponding to each query request in chronological order to generate a key list.

[0014] In one implementation, the key type of each key in the key list is determined based on the key list and the hash table in the LRU cache, including:

[0015] Iterate through the keys in the key list and check if the key exists in the hash table in the LRU cache;

[0016] If the key does not exist in the hash table of the LRU cache, insert a new key and use the new key as a second type key;

[0017] If a key exists in the hash table of the LRU cache, the type of the key is determined based on the value corresponding to the key and the update queue corresponding to the key.

[0018] In one implementation, querying whether a key exists in the hash table of the LRU cache includes:

[0019] Calculate the hash value of the key;

[0020] Search for the key in the doubly linked list within the bucket of the hash table using the key's hash value.

[0021] In one embodiment, the method further includes:

[0022] If the key does not exist in the hash table in the LRU cache, register an update event in the update queue for the new key.

[0023] In one implementation, determining the type of a key based on the value corresponding to the key and the update queue corresponding to the key includes:

[0024] Determine if the value corresponding to the key is valid;

[0025] If the value corresponding to the key is valid, the key is a first-type key;

[0026] If the value corresponding to the key is invalid, check if the update queue corresponding to the key is empty;

[0027] If the update queue corresponding to the key is empty, the key is a second type key;

[0028] If the update queue corresponding to the key is not empty, the key is a third type key.

[0029] In one embodiment, the method further includes:

[0030] If the update queue corresponding to the key is empty, register the update event in the key's update queue;

[0031] If the update queue corresponding to the key is not empty, register a wait event in the key's update queue.

[0032] In one implementation, obtaining type data for each key based on its key type includes:

[0033] If the key in the key list is a first type key, retrieve the first type data of the first type key, where the first type data is the value corresponding to the key retrieved from the hash table in the LRU cache;

[0034] If the key in the key list is a second type key, retrieve the second type data of the second type key, where the second type data is the data retrieved from memory using a callback function;

[0035] If the key in the key list is a third type key, retrieve the third type data of the third type key, where the third type data is the cached data updated by the update event in the key's update queue.

[0036] In one embodiment, the keys in the key list are of at least one of a first type of key, a second type of key, and a third type of key;

[0037] The type data for each key is combined to generate target data as the query result for the batch query request, including:

[0038] Combine at least one of the first type of data, the second type of data, and the third type of data to generate target data as the query result of the batch query request.

[0039] Secondly, embodiments of this application provide a data query device, including:

[0040] The receiving module is used to receive batch query requests;

[0041] The list generation module is used to generate a list of keys based on batch query requests;

[0042] The type determination module is used to determine the key type of each key in the key list based on the key list and the hash table in the LRU cache;

[0043] The data return module is used to obtain the type data of each key according to the key type, and combine the type data of each key to generate the target data as the query result of the batch query request.

[0044] In one embodiment, the list generation module is further configured to obtain the key corresponding to each query request for each query request in the batch query request;

[0045] Sort and summarize the keys corresponding to each query request in chronological order to generate a key list.

[0046] In one implementation, the type determination module is also used to traverse the keys in the key list and query whether the key exists in the hash table in the LRU cache;

[0047] If the key does not exist in the hash table of the LRU cache, insert a new key and use the new key as a second type key;

[0048] If a key exists in the hash table of the LRU cache, the type of the key is determined based on the value corresponding to the key and the update queue corresponding to the key.

[0049] In one implementation, the type determination module is also used to calculate the hash value of the key;

[0050] Search for the key in the doubly linked list within the bucket of the hash table using the key's hash value.

[0051] In one embodiment, the apparatus further includes a node insertion module, which is used to register an update event in the update queue of a new key if the key does not exist in the hash table in the LRU cache.

[0052] In one embodiment, the type determination module is also used to determine whether the value corresponding to the key is valid;

[0053] If the value corresponding to the key is valid, the key is a first-type key;

[0054] If the value corresponding to the key is invalid, check if the update queue corresponding to the key is empty;

[0055] If the update queue corresponding to the key is empty, the key is a second type key;

[0056] If the update queue corresponding to the key is not empty, the key is a third type key.

[0057] In one embodiment, the apparatus further includes an event registration module, which is used to register an update event in the update queue of the key if the update queue corresponding to the key is empty;

[0058] If the update queue corresponding to the key is not empty, register a wait event in the key's update queue.

[0059] In one embodiment, the data return module is further configured to obtain first type data of the first type key if the key in the key list is a first type key, wherein the first type data is the value corresponding to the key obtained from the hash table in the LRU cache;

[0060] If the key in the key list is a second type key, retrieve the second type data of the second type key, where the second type data is the data retrieved from memory using a callback function;

[0061] If the key in the key list is a third type key, retrieve the third type data of the third type key, where the third type data is the cached data updated by the update event in the key's update queue.

[0062] In one embodiment, the keys in the key list are of at least one of a first type of key, a second type of key, and a third type of key;

[0063] The data return module is also used to combine at least one of the first type of data, the second type of data, and the third type of data to generate target data as the query result of the batch query request.

[0064] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0065] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0066] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0067] This application provides a data query method, apparatus, device, medium, and product, comprising: first receiving a batch query request; then generating a key list based on the batch query request; then determining the key type of each key in the key list based on the key list and a hash table in an LRU cache; finally obtaining the type data of each key according to the key type; and combining the type data of each key to generate target data as the query result of the batch query request. This application uses a hash table in an LRU cache to determine the type of keys in the key list, thereby obtaining the type data of each key for different types of keys and combining the type data of each key, realizing batch querying of the LRU cache and improving data query efficiency. Furthermore, for keys that have expired in the cache, this application calls a callback function to concurrently read data from the backend storage in batches to update the cache, avoiding redundant access to the backend storage when the cache expires, and reducing the request volume and load on the backend storage. Attached Figure Description

[0068] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings:

[0069] Figure 1 This is a schematic diagram of the structure of an LRU cache device provided in an embodiment of this application;

[0070] Figure 2 This is an application scenario diagram of a data query method provided in an embodiment of this application;

[0071] Figure 3 This is a flowchart illustrating a data query method provided in an embodiment of this application;

[0072] Figure 4 This is a schematic diagram of the structure of a data query device provided in an embodiment of this application;

[0073] Figure 5 This is a schematic diagram of the computer device provided in the embodiments of this application. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0075] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein.

[0076] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0077] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0078] It should be understood that in this application, "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product or device.

[0079] It should be understood that in this application, "multiple" refers to two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, "and / or B" can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "Contains A, B, and C", "Contains A, B, and C" means that all three A, B, and C are contained; "Contains A, B, or C" means that one of A, B, and C is contained; "Contains A, B, and / or C" means that any one, two, or three of A, B, and C are contained.

[0080] It should be understood that in this application, "B corresponding to A", "B corresponding to A", "A corresponds to B", or "B corresponds to A" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Matching A and B is defined as a similarity between A and B that is greater than or equal to a preset threshold.

[0081] Depending on the context, "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection."

[0082] The data involved in this application may be data authorized by the tester or fully authorized by all parties. The collection, dissemination, and use of the data shall comply with the relevant laws, regulations and standards of the relevant countries and regions. The implementation methods / executives of this application may be combined with each other.

[0083] The embodiments of this application can be applied to various scenarios such as big data processing, distributed computing, and real-time data processing.

[0084] The technical solutions of this application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0085] The present application will now be described in conjunction with the accompanying drawings and specific embodiments.

[0086] Please see Figure 1 , Figure 1 This is a schematic diagram of an LRU cache device provided in an embodiment of this application.

[0087] The LRU cache device consists of a hash table array structure, which contains multiple buckets, such as bucket1, bucket2, bucket3, etc. Each bucket stores N key nodes, and the data structure of each key node contains a key, a value, and an update queue, where N is an integer greater than or equal to 1.

[0088] All key nodes in multiple buckets are managed through a doubly linked list, such as an LRU list.

[0089] LRU (Least Recently Used) linked lists (also known as LRU chains) are a commonly used cache eviction algorithm used to manage data in the cache, ensuring that the least recently used data item is evicted first. LRU linked lists are implemented by maintaining a doubly linked list, where the head of the list represents the most recently used data item and the tail represents the least recently used data item.

[0090] Specifically, the working principle of the LRU linked list is as follows:

[0091] Insertion operation: When a new data item is accessed, if the cache is not full, it is directly inserted into the head of the linked list; if the cache is full, the data item at the tail of the linked list is removed, and the new data item is inserted into the head of the linked list.

[0092] Access operation: When a data item is accessed, it is removed from its current position and inserted at the head of the list. This ensures that the data item at the head of the list is always the most recently accessed.

[0093] Elimination operation: When a data item needs to be eliminated, simply remove the data item at the end of the linked list.

[0094] The LRU caching mechanism can be implemented using a hash table supplemented by a doubly linked list. A hash table and a doubly linked list are used to maintain all key-value pairs in the cache. The doubly linked list stores these key-value pairs in the order they are used, with the most recently used pairs near the head and the least recently used pairs near the tail. The hash table is essentially a regular hash map (HashMap), mapping the keys of cached data to their positions in the doubly linked list.

[0095] First, a hash table is used to locate the cache item in the doubly linked list. Then, it is moved to the head of the doubly linked list, thus completing the data retrieval or placement operation. Specifically, the hash value of the key is calculated first, the bucket is located by the hash value, and then the keys in the corresponding doubly linked list of the bucket are traversed to find the required key node.

[0096] Please see Figure 2 , Figure 2This diagram illustrates an application scenario of a data query method provided in an embodiment of this application. The server 204 receives a batch query request from the terminal device 202. Based on the batch query request, the server 204 generates a key list. Then, based on the key list and a hash table in the LRU cache, it determines the key type of each key in the key list. Next, it retrieves the type data for each key according to its key type and combines the type data of each key to generate target data as the query result of the batch query request.

[0097] The server 204 and the terminal device 202 can communicate through any communication method, including but not limited to network communication. The network can include, but is not limited to, wired networks and wireless networks. Wired networks include local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs). Wireless networks include Bluetooth, Wi-Fi, and other networks that enable wireless communication. The terminal device 202 includes, but is not limited to, at least one of the following: mobile phones (such as Android phones, iOS phones, etc.), laptops, tablets, handheld computers, MIDs (Mobile Internet Devices), PADs, desktop computers, smart TVs, etc. The server 204 can be a field server or a remote server. Both field servers and remote servers can be independent servers or service clusters composed of multiple servers. The above is merely an example, and no limitations are imposed in this embodiment.

[0098] Please see Figure 3 , Figure 3 This is a flowchart illustrating a data query method provided in an embodiment of this application. Figure 3 As shown, applied to Figure 2 The server returned a 204 error, including the following steps:

[0099] Step S301: Receive batch query requests.

[0100] The LRU cache receives requests in two main forms: get(key) and put(key, value).

[0101] get(key): This request retrieves the value associated with the specified key. If the key exists in the cache, the value is returned; otherwise, -1 is returned.

[0102] `put(key, value)`: This request adds or updates a key-value pair to the cache. If the key already exists, its value is updated; if the key does not exist, a new key-value pair is added. When the cache reaches its capacity limit, the least recently used data needs to be evicted to make room for new data.

[0103] For example, if the received batch requests are used to retrieve the value corresponding to a specified key, then the batch query requests include query request 1 (get(key1)), query request 2 (get(key2)), and query request 3 (get(key3)), etc.

[0104] It should be noted that the query requests in the batch query request are not limited to different keys; they can also be multiple query requests from different users for the same key.

[0105] Step S302: Generate a key list based on the batch query request.

[0106] In one embodiment, to generate a key list based on batch query requests, it is necessary to obtain the key corresponding to each query request in the batch query requests, and then sort and summarize the keys corresponding to each query request according to time to generate a key list.

[0107] Once the individual query requests in the batch query request are obtained, the corresponding key can be retrieved for each query request. Then, the keys corresponding to all query requests can be arranged in chronological order to form a list of keys corresponding to the batch query requests.

[0108] For example, a batch query request includes query request 1 (get(key1)), query request 2 (get(key2)) and query request 3 (get(key3)). The key corresponding to each query request can be determined through each query request. For example, the key corresponding to query request 1 (get(key1)) is key1, the key corresponding to query request 2 (get(key2)) is key2, and the key corresponding to query request 3 (get(key3)) is key3.

[0109] If query request 1 (get(key1)), query request 2 (get(key2)), and query request 3 (get(key3)) are received in chronological order, then the keys corresponding to each query request can be sorted and summarized in chronological order to generate a key list, where key list = {key1, key2, key3}.

[0110] In another embodiment, for generating a key list based on batch query requests, a hash function is first obtained. The URL parameters in each query request in the batch query requests are processed by the hash function to generate a string corresponding to each query request. Then, the string corresponding to each query request is processed to generate a key corresponding to each query request. Finally, the keys corresponding to each query request are sorted and summarized according to time to generate a key list.

[0111] The hash algorithms include, but are not limited to, MD5 (Message-Digest Algorithm 5), SHA-1 (Secure Hash Algorithm 1), SHA-256 (Secure Hash Algorithm 256-bit), etc.

[0112] The steps for generating a string corresponding to each query request using the SHA-1 (Secure Hash Algorithm 1) algorithm include:

[0113] First, hash the URL parameters in each query request within the batch query request to obtain hash strings for the attribute values. Then, among the hash strings of multiple attribute values ​​in the same tuple, the first hash string is selected as the one with the largest value. Specifically, the hash strings of multiple attribute values ​​in the same tuple are sorted, and the hash string of the attribute value with the largest value is used as the first hash string.

[0114] Then, the hash string of the attribute value and the first hash string are merged by bit cross-merging to generate a merged hash string, and a random starting bit is determined in the merged hash string. Specifically, determining the random starting bit in the merged hash string is done by using a preset random bit finding model to determine the random starting bit in the merged hash string.

[0115] Then, based on a random starting bit, a hash string is extracted from the merged hash string to obtain the final attribute string. A preset extraction model is used, based on the random starting bit and the extraction model, to extract the hash string from the merged hash string to obtain the final attribute string. The extracted hash string from the merged hash string is then used as the final attribute string.

[0116] Furthermore, based on a preset matching method, a binary matching algorithm is executed on the final attribute string and proportion string. Specifically:

[0117] Following the preset bit matching direction, obtain the current bit of the final attribute string and the current bit of the ratio string; perform binary matching on the current bit of the final attribute string and the current bit of the ratio string; if the current bit of the final attribute string is 1 and the current bit of the ratio string is 0, the match fails; if the current bit of the final attribute string is 1 and the current bit of the ratio string is 1, the match succeeds; if the current bit of the final attribute string is 0, determine whether the current bit of the ratio string is the last bit of the ratio string. If the current bit of the ratio string is not the last bit of the ratio string, perform binary matching on the next bit of the current bit of the final attribute string and the next bit of the current bit of the ratio string according to the preset matching method. If the current bit of the ratio string is the last bit of the ratio string, the matching ends and the matching fails.

[0118] For ease of explanation, the following example is provided:

[0119] If the current bit of the ratio string is not the last bit of the ratio string, it is necessary to continue to observe the subsequent bits according to the preset matching method.

[0120] Assuming the final attribute string is A and the ratio string is B, the matching process starts from the rightmost bit of A and the leftmost bit of B. If the matching process does not end, it will continue to observe the next bit in the corresponding direction, rather than just observing the situation of one bit.

[0121] That is, the matching begins with the bit following the rightmost bit of A and the bit following the leftmost bit of B, and continues until the current bit of B is the last bit. If a match cannot be found, it is considered a failure, and the matching ends. For B, the bit matching direction is from left to right; for A, the bit matching direction is from right to left.

[0122] For ease of explanation, the matching process between A and B can end in three ways, as detailed below:

[0123] One approach is to end the matching process once a successful match is confirmed. Another approach is to end the matching process once a failed match is confirmed. The last approach is to continuously observe the next bit position until the last bit of the proportional string B is reached. If the match is still unconfirmed, it is considered a failed match, and the matching process ends.

[0124] Specifically, according to the preset bit matching direction, the current bit of the final attribute string and the current bit of the ratio string are obtained, as follows:

[0125] Starting from right to left, obtain the i-th bit of the final attribute string as the current bit of the final attribute string; starting from left to right, obtain the i-th bit of the ratio string as the current bit of the ratio string; where i is a positive integer.

[0126] Finally, when the attribute values ​​are used to construct the key, based on the preset assembly rules, multiple successfully matched final attribute strings are assembled and concatenated into the key, that is, the string corresponding to each query request is generated.

[0127] After generating the string corresponding to each query request, each string can be processed to generate a key for each query request. The processing method primarily involves retaining specific strings from the generated strings corresponding to each query request, while deleting unnecessary strings, based on requirements.

[0128] Then, the keys corresponding to each query request are sorted and summarized according to time to generate a key list.

[0129] If query request 1 (get(key1)), query request 2 (get(key2)), and query request 3 (get(key3)) are received in chronological order, then the keys corresponding to each query request can be sorted and summarized in chronological order to generate a key list, where key list = {key1, key2, key3}.

[0130] This application generates a key list by sorting the keys corresponding to each query request in chronological order, which makes it easier for users to obtain the data they need in chronological order. At the same time, it avoids the confusion of multiple keys corresponding to batch query requests, which could lead to users obtaining incorrect data and affect the user experience.

[0131] Step S303: Determine the key type of each key in the key list based on the key list and the hash table in the LRU cache.

[0132] To determine the type of a key in the key list based on the key list and the hash table in the LRU cache, we first need to traverse the keys in the key list and check if the key exists in the hash table in the LRU cache. If the key does not exist in the hash table in the LRU cache, the key is a second type key. If the key exists in the hash table in the LRU cache, the type of the key is determined based on the value corresponding to the key and the update queue corresponding to the key.

[0133] The process of querying whether a key exists in the hash table of the LRU cache includes: calculating the hash value of the key; and searching for the doubly linked list in the bucket of the hash table to see if the key exists.

[0134] The key type is determined based on the value corresponding to the key and the update queue corresponding to the key, including: determining whether the value corresponding to the key is valid; if the value corresponding to the key is valid, the key is a first type key; if the value corresponding to the key is invalid, determining whether the update queue corresponding to the key is empty; if the update queue corresponding to the key is empty, the key is a second type key; if the update queue corresponding to the key is not empty, the key is a third type key.

[0135] Once the list of keys is obtained, you can directly query whether the key exists in the HashMap in the LRU cache by using the key in the list.

[0136] Specifically, taking key1 as an example, the hash value of key1 needs to be calculated first.

[0137] The algorithms for calculating the hash value of the key include, but are not limited to, MD5 (Message-Digest Algorithm 5), SHA-1 (Secure Hash Algorithm 1), and SHA-256 (Secure Hash Algorithm 256-bit).

[0138] The hash value of key1 is calculated using the SHA-1 (Secure Hash Algorithm 1) algorithm. The steps include:

[0139] Step 1: Import the hashlib library

[0140] First, you need to import Python's hashlib library, which provides support for various hash algorithms, including SHA-1.

[0141] Step 2: Create a SHA-1 hash object

[0142] Use the hashlib.sha1() function to create a SHA-1 hash object.

[0143] Step 3: Update the hash object

[0144] Use the update() method to pass the data (the string "KEY1") to which you want to calculate the hash value to the hash object. Note that the update() method requires a byte object, so you need to encode the string as bytes.

[0145] Step 4: Obtain the hash value

[0146] Use the hexdigest() method to get the hexadecimal representation of the hash value.

[0147] After calculating the hash value of key1 using the above algorithm, the hash value of key1 can be used to search for whether key exists in the doubly linked list in the bucket of the HashMap.

[0148] Suppose that the doubly linked list in the bucket of the HashMap includes key1, key2, key3, key4, etc., and each key is configured with a corresponding hash value, such as the hash value of key1 is xx, the hash value of key2 is yy, the hash value of key3 is zz, and the hash value of key4 is tt.

[0149] If the calculated hash value of key1 is xx, then key1 exists in the doubly linked list of the bucket in the HashMap; if the calculated hash value of key1 is not xx, then key1 does not exist in the doubly linked list of the bucket in the HashMap.

[0150] If the calculated hash value of key1 is not xx, then key1 does not exist in the doubly linked list of the bucket in the HashMap, and key1 is determined to be a second type key. At this time, a new key (i.e., key1) needs to be inserted into the HashMap in the LRU cache, and an update event needs to be registered in the update queue of the new key.

[0151] If key1 exists in the HashMap in the LRU cache, then the type of key1 needs to be determined based on the value corresponding to key1 and the update queue corresponding to key1.

[0152] Specifically, it is necessary to first determine whether the value corresponding to key1 is valid. If the value corresponding to key1 is valid, then key1 is a first type key.

[0153] If the value corresponding to key1 is invalid, it is also necessary to check whether the update queue corresponding to key1 is empty. If the update queue corresponding to key1 is empty, then key1 is a second type key, and an update event is registered in the update queue of key1; if the update queue corresponding to key1 is not empty, then key1 is a third type key, and a wait event is registered in the update queue of key1.

[0154] It should be noted that the method for determining the type of other keys is similar to that for key1 in the example, and will not be repeated here.

[0155] This application uses a hash table in the LRU cache to determine the type of keys in the key list, thereby retrieving data according to the data retrieval method corresponding to different types of keys, realizing batch query of LRU cache and improving data query efficiency.

[0156] Step S304: Obtain the type data of each key according to the key type, and combine the type data of each key to generate target data as the query result of the batch query request.

[0157] In one embodiment, obtaining the type data of each key according to the key type includes: if the key in the key list is a first type key, obtaining the first type data of the first type key, wherein the first type data is the value corresponding to the key obtained from the hash table in the LRU cache; if the key in the key list is a second type key, obtaining the second type data of the second type key, wherein the second type data is data obtained from memory using a callback function; if the key in the key list is a third type key, obtaining the third type data of the third type key, wherein the third type data is cached data updated by update events in the key's update queue.

[0158] If a first-type key exists in the hash table of the LRU cache and its value is valid, then the first-type key can be found by directly searching the doubly linked list in the bucket of the hash table. If the first-type key data structure contains its corresponding value, then the value can be retrieved directly.

[0159] The second type of key includes two cases: keys that do not exist in the hash table of the LRU cache and keys that exist in the hash table of the LRU cache but whose corresponding values ​​are invalid and whose update queue is empty. In either case, the corresponding value cannot be directly obtained from the hash table of the LRU cache. Therefore, it is necessary to retrieve the data from the memory through a callback function.

[0160] Callback functions are a common technique in computer programming, especially in asynchronous and event-driven programming. A callback function is a function passed as an argument to another function and invoked when a specific condition or event occurs. This mechanism allows programs to handle tasks more flexibly and efficiently during execution. It is primarily used for asynchronous operations, event monitoring, and delayed execution.

[0161] In this application, each query request in the batch query request needs to call a callback function to retrieve data from memory. However, in order to avoid high concurrency in the batch query request data retrieval, each query request in the batch query request of this application is an asynchronous request (i.e., single concurrency). That is to say, each query request in the batch query request calls the callback function one by one to retrieve data from memory.

[0162] Specifically, the execution steps for each query request to call the callback function to retrieve data from memory are as follows:

[0163] 1. Define the asynchronous request function:

[0164] Create a function to initiate asynchronous requests, such as query requests.

[0165] This function accepts a callback function as an argument, which is invoked after the request is completed.

[0166] 2. Initiate an asynchronous request:

[0167] Inside the asynchronous request function, use the appropriate library or method to make the request (e.g., using XMLHttpRequest, fetch, or axios).

[0168] 3. Request processing results:

[0169] When the request succeeds or fails, the passed-in callback function is invoked, and the corresponding data or error information is passed.

[0170] 4. Define the callback function:

[0171] Define a callback function to handle the data or errors returned by the request.

[0172] 5. Call the asynchronous request function:

[0173] Call the asynchronous request function and pass in the predefined callback function.

[0174] If the third type key is a key that exists in the hash table of the LRU cache but its corresponding value is invalid and the update queue is not empty, it means that there are other events in the update queue corresponding to the third type key. If there are update events in the other events in the update queue, it means that the wait event registered with the third type key only needs to retrieve the cached data updated by the previous update event. If all other events in the update queue are wait events, then after the previous wait events have ended, a callback function needs to be used to retrieve the data from memory.

[0175] Once the type data of the above three types of keys is determined, the type of the key corresponding to the request in the batch query request can be obtained and combined to generate target data as the query result of the batch query request.

[0176] When the key in the key list is of at least one of a first type key, a second type key, and a third type key, the step of combining the type data of each key to generate target data as the query result of the batch query request includes: combining at least one of the first type data, the second type data, and the third type data to generate target data as the query result of the batch query request.

[0177] Specifically, if the key corresponding to the batch query request is only one of the three types of keys mentioned above, then at least one of the following will be used as the target data: the value corresponding to the key obtained from the hash table in the LRU cache, the data obtained from the memory using the callback function, and the cached data updated by the update event in the update queue of the key. The target data will then be returned as the query result.

[0178] For example, suppose the batch query requests include query request 1, query request 2 and query request 3, and the key1 corresponding to query request 1, the key2 corresponding to query request 2 and the key3 corresponding to query request 3 are all keys of the first type. Then, the doubly linked list in the bucket of the HashMap can be searched directly to find key1, key2 and key3, and the corresponding value1, value2 and value3 can be obtained from key1, key2 and key3 respectively.

[0179] After obtaining value1, value2, and value3, we need to combine value1, value2, and value3 into the target data and return the target data as the query result.

[0180] If the key corresponding to the batch query request is only two or more of the above three types of keys, the target data will be obtained from two or more of the following: the value corresponding to the key obtained from the hash table in the LRU cache, the data obtained from the memory using the callback function, and the cached data updated by the update event in the update queue of the key. The target data will then be returned as the query result.

[0181] For example, suppose the batch query requests include query request 1, query request 2 and query request 3, and the key1 corresponding to query request 1 is a first type key, the key2 corresponding to query request 2 is a second type key and the key3 corresponding to query request 3 is a first type key. Then, key1 and key3 can be found by directly searching the doubly linked list in the bucket of the HashMap, and the corresponding value1 and value3 can be obtained from key1 and key3 respectively.

[0182] For key2 corresponding to query request 2, value2 needs to be retrieved from memory using a callback function.

[0183] After obtaining value1, value2, and value3, we need to combine value1, value2, and value3 into the target data and return the target data as the query result.

[0184] In another embodiment, to return the query results corresponding to the batch query request based on the type of the key in the key list, it is necessary to first obtain the type of the key in the key list; then obtain the user ID, determine the data acquisition method of the key type in the key list based on the user ID and the type of the key in the key list, then obtain the target data through the data acquisition method, and return the target data as the query result.

[0185] The data retrieval method for determining the key type in the key list based on the user ID and the key type in the key list includes: matching the user ID with the key type in the key list to obtain the key type corresponding to the user ID; if the key corresponding to the user ID is a first type key, retrieving the value corresponding to the key from the hash table in the LRU cache; if the key corresponding to the user ID is a second type key, retrieving the data from the memory using a callback function; and if the key corresponding to the user ID is a third type key, retrieving the cached data updated by the update event in the key's update queue.

[0186] If a key of type 1 exists in a HashMap in an LRU cache and its value is valid, then the key of type 1 can be found by directly searching the doubly linked list in the bucket of the HashMap. If the key of type 1 exists in the data structure of its corresponding value, then the value can be retrieved directly.

[0187] The second type of key includes two cases: keys that do not exist in the Hashmap in the LRU cache and keys that exist in the Hashmap in the LRU cache but whose corresponding values ​​are invalid and whose update queues are empty. In either case, the corresponding value cannot be directly obtained from the Hashmap in the LRU cache. Therefore, it is necessary to retrieve the data from the memory through a callback function.

[0188] Callback functions are a common technique in computer programming, especially in asynchronous and event-driven programming. A callback function is a function passed as an argument to another function and invoked when a specific condition or event occurs. This mechanism allows programs to handle tasks more flexibly and efficiently during execution. It is primarily used for asynchronous operations, event monitoring, and delayed execution.

[0189] In this application, each query request in the batch query request needs to call a callback function to retrieve data from memory. However, in order to avoid high concurrency in the batch query request data retrieval, each query request in the batch query request of this application is an asynchronous request. That is, each query request in the batch query request calls the callback function one by one to retrieve data from memory.

[0190] Specifically, the execution steps for each query request to call the callback function to retrieve data from memory are as follows:

[0191] 1. AJAX requests are initiated using the XMLHttpRequest object, such as query requests.

[0192] 2. Set up a function to respond to changes in the status of HTTP requests.

[0193] 3. Initialize an HTTP request for the XMLhttpRequest object, specifying the URL and request method, etc.

[0194] Define an asynchronous request function: Create a function to initiate an asynchronous request. Inside the asynchronous request function, use an appropriate library or method to initiate the request (e.g., using XMLHttpRequest, fetch, or axios).

[0195] 4. Make a request.

[0196] 5. Obtain the data returned by the server.

[0197] If the third type key is a key that exists in the hash table of the LRU cache but its corresponding value is invalid and the update queue is not empty, it means that there are other events in the update queue corresponding to the third type key. If there are update events in the other events in the update queue, it means that the wait event registered with the third type key only needs to retrieve the cached data updated by the previous update event. If all other events in the update queue are wait events, then after the previous wait events have ended, a callback function needs to be used to retrieve the data from memory.

[0198] Once the data retrieval methods for the three types of keys mentioned above are determined, the target data can be obtained by retrieving the type of the key corresponding to the request in the batch query request, and the target data can be returned as the query result.

[0199] If the key in the key list is of at least one of the following types: first type key, second type key, and third type key, the target data is obtained through a data retrieval method and returned as a query result. This includes: retrieving the value corresponding to the key from the hash table in the LRU cache, retrieving data from the memory using a callback function, and retrieving cached data updated by update events in the update queue of the key as the target data, and returning the target data as a query result.

[0200] Specifically, if the key corresponding to the batch query request is only one of the three types of keys mentioned above, then at least one of the following will be used as the target data: the value corresponding to the key obtained from the HashMap in the LRU cache, the data obtained from the memory using the callback function, and the cached data updated by the update event in the update queue of the key. The target data will then be returned as the query result.

[0201] For example, suppose the batch query requests include query request 1, query request 2 and query request 3, and the key1 corresponding to query request 1, the key2 corresponding to query request 2 and the key3 corresponding to query request 3 are all keys of the first type. Then, the doubly linked list in the bucket of the HashMap can be searched directly to find key1, key2 and key3, and the corresponding value1, value2 and value3 can be obtained from key1, key2 and key3 respectively.

[0202] After obtaining value1, value2, and value3, we need to combine value1, value2, and value3 into the target data and return the target data as the query result.

[0203] If the key corresponding to the batch query request is only two or more of the above three types of keys, the target data will be obtained from two or more of the following: the value corresponding to the key obtained from the hash table in the LRU cache, the data obtained from the memory using the callback function, and the cached data updated by the update event in the update queue of the key. The target data will then be returned as the query result.

[0204] For example, suppose the batch query requests include query request 1, query request 2 and query request 3, and the key1 corresponding to query request 1 is a first type key, the key2 corresponding to query request 2 is a second type key and the key3 corresponding to query request 3 is a first type key. Then, key1 and key3 can be found by directly searching the doubly linked list in the bucket of the HashMap, and the corresponding value1 and value3 can be obtained from key1 and key3 respectively.

[0205] For key2 corresponding to query request 2, value2 needs to be retrieved from memory using a callback function.

[0206] After obtaining value1, value2, and value3, we need to combine value1, value2, and value3 into the target data and return the target data as the query result.

[0207] This application provides a data query method, comprising: first receiving a batch query request; then generating a key list based on the batch query request; then determining the key type of each key in the key list based on the key list and a hash table in an LRU cache; finally obtaining the type data of each key according to the key type; and combining the type data of each key to generate target data as the query result of the batch query request. This application uses a hash table in an LRU cache to determine the key type in the key list, thereby obtaining the type data of each key for different key types and combining the type data of each key, realizing batch querying of the LRU cache and improving data query efficiency. Furthermore, for keys that have expired in the cache, this application calls a callback function to concurrently read data from the backend storage in batches to update the cache, avoiding redundant access to the backend storage when the cache expires, and reducing the request volume and load on the backend storage.

[0208] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0209] The following are device embodiments of this application. For details not described in detail, please refer to the corresponding method embodiments described above.

[0210] Figure 4 The diagram illustrates the structure of a data query device according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. The data query device includes a receiving module 401, a list generation module 402, a type determination module 403, and a data return module 404, as detailed below:

[0211] Receiver module 401 is used to receive batch query requests;

[0212] List generation module 402 is used to generate a list of keys based on batch query requests;

[0213] Type determination module 403 is used to determine the key type of each key in the key list based on the key list and the hash table in the LRU cache;

[0214] The data return module 404 is used to obtain the type data of each key according to the key type, and combine the type data of each key to generate target data as the query result of the batch query request.

[0215] In one embodiment, the list generation module 402 is further configured to obtain the key corresponding to each query request for each query request in the batch query request;

[0216] Sort and summarize the keys corresponding to each query request in chronological order to generate a key list.

[0217] In one embodiment, the type determination module 403 is also used to traverse the keys in the key list and query whether the key exists in the hash table in the LRU cache;

[0218] If the key does not exist in the hash table of the LRU cache, insert a new key and use the new key as a second type key;

[0219] If a key exists in the hash table of the LRU cache, the type of the key is determined based on the value corresponding to the key and the update queue corresponding to the key.

[0220] In one embodiment, the type determination module 403 is also used to calculate the hash value of the key;

[0221] Search for the key in the doubly linked list within the bucket of the hash table using the key's hash value.

[0222] In one embodiment, the apparatus further includes a node insertion module, which is used to register an update event in the update queue of a new key if the key does not exist in the hash table in the LRU cache.

[0223] In one embodiment, the type determination module 403 is further used to determine whether the value corresponding to the key is valid;

[0224] If the value corresponding to the key is valid, the key is a first-type key;

[0225] If the value corresponding to the key is invalid, check if the update queue corresponding to the key is empty;

[0226] If the update queue corresponding to the key is empty, the key is a second type key;

[0227] If the update queue corresponding to the key is not empty, the key is a third type key.

[0228] In one embodiment, the apparatus further includes an event registration module, which is used to register an update event in the update queue of the key if the update queue corresponding to the key is empty;

[0229] If the update queue corresponding to the key is not empty, register a wait event in the key's update queue.

[0230] In one embodiment, the data return module 404 is further configured to obtain first type data of the first type key if the key in the key list is a first type key, wherein the first type data is the value corresponding to the key obtained from the hash table in the LRU cache;

[0231] If the key in the key list is a second type key, retrieve the second type data of the second type key, where the second type data is the data retrieved from memory using a callback function;

[0232] If the key in the key list is a third type key, retrieve the third type data of the third type key, where the third type data is the cached data updated by the update event in the key's update queue.

[0233] In one embodiment, the keys in the key list are of at least one of a first type of key, a second type of key, and a third type of key;

[0234] The data return module 404 is also used to combine at least one of the first type of data, the second type of data, and the third type of data to generate target data as the query result of the batch query request.

[0235] This application provides a data query apparatus, specifically used for: first receiving batch query requests; then generating a key list based on the batch query requests; then determining the key type of each key in the key list based on the key list and a hash table in the LRU cache; finally obtaining the type data of each key according to the key type; and combining the type data of each key to generate target data as the query result of the batch query requests. This application uses a hash table in the LRU cache to determine the type of keys in the key list, thereby obtaining the type data of each key for different types of keys and combining the type data of each key, realizing batch querying of the LRU cache and improving data query efficiency. Furthermore, for keys that have expired in the cache, this application calls a callback function to concurrently read data from the backend storage in batches to update the cache, avoiding redundant access to the backend storage when the cache expires, and reducing the request volume and load on the backend storage.

[0236] This application Figure 5 A schematic diagram of a computer device is provided. (Example) Figure 5 As shown, the computer device 5 in this embodiment includes a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501. When the processor 501 executes the computer program 503, it implements the steps in the various data query method embodiments described above, for example... Figure 3 Steps 301 to 304 are shown. Alternatively, when processor 501 executes computer program 503, it implements the functions of each module / unit in the above-described data query device embodiments, for example... Figure 4 The functions of modules / units 401 to 404 are shown.

[0237] This application also provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the data query methods provided in the various embodiments described above.

[0238] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application-Specific Integrated Circuit (ASIC). Alternatively, the ASIC can be located in a user device. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0239] This application also provides a program product including executable instructions stored in a readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the execution of the executable instructions by the at least one processor causes the device to implement the data query methods provided in the various embodiments described above.

[0240] In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0241] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data query method, characterized by, include: Receive batch query requests; Generate a key list based on the batch query request; Based on the key list and the hash table in the LRU cache, determine the key type of each key in the key list; The type data of each key is obtained according to the key type, and the type data of each key is combined to generate target data as the query result of the batch query request.

2. The data query method as described in claim 1, characterized in that, The generation of the key list based on the batch query request includes: For each query request in the batch query request, obtain the key corresponding to each query request; The keys corresponding to each query request are sorted and summarized in chronological order to generate the key list.

3. The data query method as described in claim 1, characterized in that, The step of determining the key type of each key in the key list based on the key list and the hash table in the LRU cache includes: Iterate through the keys in the key list and check if the key exists in the hash table of the LRU cache; If the key does not exist in the hash table of the LRU cache, insert a new key and use the new key as a second type key; If a key exists in the hash table of the LRU cache, the type of the key is determined based on the value corresponding to the key and the update queue corresponding to the key.

4. The data query method as described in claim 3, characterized in that, The query to determine if a key exists in the hash table of the LRU cache includes: Calculate the hash value of the key; The hash value of the key is used to search for whether the key exists in the doubly linked list within the bucket of the hash table.

5. The data query method as described in claim 3, characterized in that, The method further includes: If the key does not exist in the hash table of the LRU cache, an update event is registered in the update queue of the new key.

6. The data query method as described in claim 3, characterized in that, Determining the type of the key based on the value corresponding to the key and the update queue corresponding to the key includes: Determine whether the value corresponding to the key is valid; If the value corresponding to the key is valid, the key is a first type key; If the value corresponding to the key is invalid, determine whether the update queue corresponding to the key is empty; If the update queue corresponding to the key is empty, the key is a second type key; If the update queue corresponding to the key is not empty, the key is a third type key.

7. The data query method as described in claim 6, characterized in that, The method further includes: If the update queue corresponding to the key is empty, register an update event in the update queue of the key; If the update queue corresponding to the key is not empty, register a wait event in the update queue of the key.

8. The data query method as described in claim 1, characterized in that, The step of obtaining the type data of each key according to the key type includes: If the key in the key list is a first type key, obtain the first type data of the first type key, wherein the first type data is the value corresponding to the key obtained from the hash table in the LRU cache; If the key in the key list is a second type key, obtain the second type data of the second type key, wherein the second type data is data obtained from memory using a callback function; If a key in the key list is a third-type key, obtain the third-type data of the third-type key, wherein the third-type data is cached data updated by update events in the key's update queue.

9. The data query method as described in claim 8, characterized in that, The key in the key list is of at least one of the first type of key, the second type of key, and the third type of key; The step of combining the type data of each key to generate target data as the query result of the batch query request includes: The target data is generated by combining at least one of the first type of data, the second type of data, and the third type of data to serve as the query result of the batch query request.

10. A data query device, characterized in that, include: The receiving module is used to receive batch query requests; The list generation module is used to generate a list of keys based on the batch query request; A type determination module is used to determine the key type of each key in the key list based on the key list and the hash table in the LRU cache; The data return module is used to obtain the type data of each key according to the key type, and combine the type data of each key to generate target data as the query result of the batch query request.

11. A computer device, characterized in that, Includes a memory, and one or more processors communicatively connected to the memory; The memory stores instructions that can be executed by the one or more processors to cause the one or more processors to implement the data query method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, Includes a program or instructions that, when run on a computer, implement the data query method according to any one of claims 1 to 9.

13. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the data query method according to any one of claims 1 to 9.

Images