Data query methods, devices, electronic equipment and storage media

Abstract

This application relates to a data query method, apparatus, electronic device, and storage medium. The method includes: obtaining a set of parameters to be queried; searching for corresponding data in a first cache to obtain first data corresponding to a first query parameter; the first cache being a cache in an accelerator processor; if the first query parameter is a subset of query parameters in the set of parameters to be queried, performing data query processing in a second cache to obtain second data corresponding to a second query parameter, wherein the second query parameter is a query parameter other than the first query parameter in the set of parameters to be queried; the second cache being a cache in a CPU; and obtaining target data corresponding to the set of query parameters based on the first data and the second data. According to the technical solution of this application, data query efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of computer processing technology, and in particular to a data query method, apparatus, electronic device and storage medium. Background Technology

[0002] With the development of internet applications and computer technology, data processing speeds are increasing rapidly. In related technologies, such as model training, to address the issues of large model parameters and low training throughput, a common approach is to store the embedded feature table in the CPU (central processing unit) and combine GPU (graphics processing unit) cache lookups with queries from the CPU's feature table to improve query efficiency; alternatively, the feature table can be split and distributed for distributed data query processing. However, the former method, using GPU memory as a cache, has limited applicability; the latter introduces communication overhead between distributed nodes, increasing processing latency. These methods are not effective for scenarios with high timeliness requirements and suffer from high latency when dealing with large-scale data queries. Summary of the Invention

[0003] In view of the aforementioned technical problems, this application proposes a data query method, apparatus, electronic device, and storage medium.

[0004] According to one aspect of this application, a data query method is provided, the method comprising:

[0005] Once the set of parameters to be queried is obtained, the corresponding data is searched in the first cache to obtain the first data corresponding to the first query parameter; the first cache is a cache associated with the accelerator processor;

[0006] If the first query parameter is a subset of the query parameters in the set of parameters to be queried, data query processing is performed in the second cache to obtain the second data corresponding to the second query parameter. The second query parameter belongs to a query parameter other than the first query parameter in the set of parameters to be queried. The second cache is a CPU-associated cache.

[0007] Based on the first data and the second data, obtain the target data corresponding to the query parameter set.

[0008] According to another aspect of this application, a data query apparatus is provided, comprising:

[0009] The first query module is used to obtain the set of parameters to be queried, search for the corresponding data in the first cache, and obtain the first data corresponding to the first query parameter; the first cache is a cache associated with the accelerator processor;

[0010] The second query module is used to perform data query processing in the second cache if the first query parameter is a part of the query parameters in the set of parameters to be queried, and obtain the second data corresponding to the second query parameter, wherein the second query parameter belongs to the query parameters other than the first query parameter in the set of parameters to be queried; the second cache is a cache associated with the CPU;

[0011] The target data acquisition module is used to acquire the target data corresponding to the query parameter set based on the first data and the second data.

[0012] According to another aspect of this application, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the above-described method.

[0013] According to another aspect of this application, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, implement the above-described method.

[0014] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described above.

[0015] By setting up caches associated with both the accelerator processor and the CPU, the query speed for both the GPU and CPU can be improved, thereby reducing data query latency and increasing data query efficiency. This expands the applicability of data queries, effectively addressing large-scale data query scenarios, such as the real-time requirements of large-scale embedded feature queries in recommendation model applications. Furthermore, this multi-level caching approach in the accelerator processor and CPU allows for independent cache management, making the cache more flexible and scalable. While improving data query efficiency, it also reduces the consumption of communication resources.

[0016] Other features and aspects of this application will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0017] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this application together with the specification and serve to explain the principles of this application.

[0018] Figure 1 This diagram illustrates an application system provided according to an embodiment of the present application.

[0019] Figure 2 This diagram illustrates a cache configuration according to an embodiment of the present application.

[0020] Figure 3a This illustration shows a standalone caching architecture according to an embodiment of the present application.

[0021] Figure 3b This illustration shows a multi-machine caching architecture provided according to an embodiment of this application.

[0022] Figure 4 A flowchart illustrating a data query provided according to an embodiment of this application is shown.

[0023] Figure 5 This illustration shows a storage diagram of a GPU cache according to an embodiment of this application.

[0024] Figure 6 This diagram illustrates a data query process according to an embodiment of the present application.

[0025] Figure 7 This diagram illustrates a feature merging scheme according to an embodiment of the present application.

[0026] Figure 8a and Figure 8b This diagram illustrates a storage schematic of a CPU cache according to an embodiment of this application.

[0027] Figure 9 This diagram illustrates a data query device according to an embodiment of the present application.

[0028] Figure 10 This diagram illustrates an electronic device for data retrieval according to an embodiment of the present application. Detailed Implementation

[0029] Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0030] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0031] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented without certain specific details. In some instances, methods, means, components, and circuits well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0032] Please see Figure 1 , Figure 1 This diagram illustrates an application system according to an embodiment of this application. The application system can be used in the data query method of this application. Figure 1 As shown, the application system may include at least server 01 and terminal 02.

[0033] In this embodiment of the application, the server 01 can be used for data query. The server 01 may include an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0034] In this embodiment, the terminal 02 can be used to send object recommendation requests, search requests, etc., and can also be used to receive and display target data. The terminal 02 may include physical devices such as smartphones, desktop computers, tablets, laptops, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, and smart wearable devices. Physical devices may also include software running on the physical device, such as applications. In this embodiment, the operating system running on the terminal 02 may include, but is not limited to, Android, iOS, Linux, and Windows.

[0035] In the embodiments described in this specification, the terminal 02 and the server 01 can be directly or indirectly connected through wired or wireless communication, and this application does not limit this connection.

[0036] In a specific embodiment, when server 02 is a distributed system, this distributed system can be a blockchain system. When the distributed system is a blockchain system, it can be formed by multiple nodes (any form of computing device connected to the network, such as servers or user terminals). These nodes form a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP). In a distributed system, any machine, such as a server or terminal, can join and become a node. A node includes a hardware layer, a middleware layer, an operating system layer, and an application layer. Specifically, the functions of each node in the blockchain system may include:

[0037] Routing is a basic function of nodes, used to support communication between nodes.

[0038] In addition to routing capabilities, nodes can also have the following functions:

[0039] 2) Applications are deployed in the blockchain to implement specific business needs. They record data related to the implementation of functions to form record data, carry digital signatures in the record data to indicate the source of the task data, and send the record data to other nodes in the blockchain system. When other nodes successfully verify the source and integrity of the record data, they add the record data to a temporary block.

[0040] It should be noted that in the specific implementation of this application, user-related data is involved. When the following embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0041] In the embodiments of this specification, local storage and caching can be combined, and the cache from the GPU and CPU to the target server can be dynamically set based on the storage resources required by the target query object. The target query object can represent the correspondence between all preset query parameters and data, serving as the basis for data querying. The preset query parameters can be any parameters that can be used for querying, and the data in the correspondence can be content links, multimedia, images, text, items, etc., which are not limited in this disclosure. As an example, the form of the correspondence can be table entries, matrix, etc., which are also not limited in this disclosure.

[0042] For example, the target query object can be the correspondence between all preset search keywords and content links; or the target query object can be the feature table corresponding to the object recommendation model, such as an embedding matrix, which can include a large number of key-value pairs. In one example, the key can be a query feature, and the value can be an embedding vector, referred to in this application as the first embedding vector.

[0043] In the embodiments of this specification, the cache configuration method can be as follows: Obtain the target query object, such as the feature table corresponding to the object recommendation model. If the storage resources associated with the accelerator (e.g., GPU, NPU, etc.) (e.g., GPU memory storage resources) are greater than or equal to the storage resources required by the target query object, a first cache (e.g., GPU cache) can be configured in the storage resources associated with the accelerator, and the target query object can be stored in the first cache. Taking GPU as an example, this means that if the target query object can be entirely stored in the video memory, the GPU cache is equivalent to local storage in the video memory; that is, the target query object is stored entirely in the video memory, and can be directly queried without continuously updating the cache. If the storage resources associated with the accelerator are less than the storage resources required by the target query object, the storage resources of the CPU memory can be obtained; and if the storage resources of the CPU memory are greater than or equal to the storage resources required by the target query object, a second cache (or CPU cache) can be configured in the CPU memory, and the target query object can be stored in the second cache. In other words, when the target query object cannot fit in the GPU storage but can be entirely stored in memory, the CPU cache can be considered equivalent to partial storage in memory. In this case, the GPU cache can use cache updates to cache only a portion of the target query object. This allows for a combination of GPU and CPU caching to improve query speed.

[0044] Optionally, if the storage resources of the CPU memory are less than the storage resources required by the target query object, that is, the target query object cannot be stored entirely in the CPU memory, a second cache can be configured in the CPU memory; and the target query object can be stored in the target server (e.g., in the parameter server, corresponding to multi-machine mode) or the target query object can be stored in the CPU external storage (corresponding to single-machine mode).

[0045] Specifically, the following example uses an object recommendation model and a GPU as the accelerating processor. (See [link to relevant documentation]). Figure 2 , Figure 2 This diagram illustrates a cache configuration according to an embodiment of the present application. Figure 2As shown, in one possible implementation, the tiered cache configuration may include: obtaining the feature table corresponding to the object recommendation model; the object recommendation model can be used to process multiple object recommendation requests; the feature table represents the correspondence between all query features and embedding vectors, such as an embedding matrix. If the storage resources of GPU memory are greater than or equal to the storage resources required for the feature table, a first cache is configured in GPU memory, and the feature table is stored in the first cache; if the storage resources of GPU memory are less than the storage resources required for the feature table, the storage resources of CPU memory are obtained; if the storage resources of CPU memory are greater than or equal to the storage resources required for the feature table, a second cache is configured in CPU memory, and the feature table is stored in the second cache; if the storage resources of CPU memory are less than the storage resources required for the feature table, a second cache can be configured in CPU memory, and the feature table can be stored in the parameter server or CPU external storage. From the above configuration, the CPU cache can be the next level after the GPU cache, and the parameter server or CPU external storage can be the next level after the CPU cache. Specifically, the GPU cache and CPU cache (i.e., GPU and CPU embedding caches) can be configured separately according to the model size (e.g., the size of the feature table) and the hardware memory size. The cache ratio r (0 <= r <= 1) describes the process. When the cache does not use the next level, r equals 1, which is equivalent to the cache fully loading the target query object. When the cache is not used, r equals 0, and the corresponding cache will not be used. In other cases, r is between 0 and 1, and the cache will be updated adaptively, such as updating the storage hotspot features to accelerate feature queries by combining multi-level caches. Here, hotspot features can refer to the number of corresponding accounts that meet the quantity threshold in a single data query, or it can refer to the number of times that are called in multiple data queries that meet the frequency threshold. This disclosure does not limit this.

[0046] As an example, the architecture after configuring caching can be as follows: Figure 3a The architecture shown is for a single-machine scenario, and as... Figure 3b The architecture shown is for a multi-machine scenario. Taking the target query object as an embedding matrix as an example, see [link to example]. Figure 3a In standalone mode, this corresponds to the scenario where the embedding matrix is ​​stored in either CPU external storage or CPU memory. Since the embedding matrix is ​​very large (hundreds of gigabytes), it can be stored in the CPU's external storage or memory, while the CPU cache can reside in memory. The GPU cache can reside in the GPU's local storage (video memory). Optionally, modularity is supported. When the hardware has a GPU, deployment on the GPU is supported; if the hardware does not have a GPU, it can be considered that the GPU's video memory storage resources are zero, in which case only the CPU cache can be deployed. It should be noted that standalone mode can be viewed as a worker node mode, meaning that configuration is done on a single worker node. Figure 3aThe diagram shows the cache and local storage of the embedded matrix. Standalone mode can improve the reliability and efficiency of data processing.

[0047] See Figure 3b In a multi-machine architecture, the difference from a single-machine architecture lies in the fact that the embedding matrix can reside in the external storage or memory of the parameter server. Each Worker (Worker node) corresponds to one server, and the Workers and parameter servers are interconnected via a network. Here, the Worker can act as a node responding to search requests or object recommendation requests. GPU and CPU caches are configured within the Worker.

[0048] By configuring the cache sequentially from GPU memory, CPU memory, and parameter server, a multi-level caching mechanism for different processors is implemented, enabling heterogeneous management of multi-level caches and improving the speed of subsequent data queries. Furthermore, by determining the size of storage resources and the resources required by the target query object, the overall storage of the target query object is achieved sequentially from GPU memory, CPU memory, and parameter server, which can reduce the overhead of cache updates.

[0049] The aforementioned object recommendation model can include an input module for constructing the model's input, which is typically high-dimensional and sparse. A feature embedding module transforms the high-dimensional, sparse features into low-dimensional, dense vectors. This module can maintain one or more embedding matrices, each containing several key-value pairs, where the key is the input feature and the value is an embedding vector. The feature embedding module can iterate through all input features, find the corresponding embedding vectors in the embedding matrix, and then use these embedding vectors as input to the feature extraction module. The feature extraction module can extract features from the input to obtain the extracted features. These extracted features can then be input to the output module to obtain the final output. The query operation of the feature embedding module is particularly time-consuming due to the large size of the embedding matrix (ranging from 100GB to 1000GB) and the large number of features to be queried in the input (ranging from tens of thousands to millions). Therefore, the query operation of the feature embedding module is very time-consuming. To address this, the data query function of this application can be used to implement the query operation of the feature embedding module, reducing latency.

[0050] Figure 4 A flowchart illustrating a data query method according to an embodiment of this application is shown. Figure 4 As shown, the method may include:

[0051] S401: Obtain the set of parameters to be queried, search for the corresponding data in the first cache, and obtain the first data corresponding to the first query parameter.

[0052] The first cache can be a cache associated with the accelerator processor, which can refer to a cache configured in the memory corresponding to the accelerator processor. For example, when the accelerator processor is a GPU, the cache associated with the accelerator processor can be a cache configured in the GPU's video memory.

[0053] In the embodiments of this specification, the query parameters in the parameter set to be queried can be the parameters that need to be queried. In one possible implementation, query parameters of multiple target accounts can be obtained, and these query parameters can be combined into a parameter set to be queried. The target account can be the account corresponding to the terminal request, or the account for which the server wants to push content. Optionally, considering that the number of target accounts is generally large, and different target accounts may have the same query parameters, in order to save query resources, the query parameters of multiple target accounts can be deduplicated to obtain a parameter set to be queried, which includes the deduplicated query parameters. This reduces the number of query parameters faced by the query processing, improving query efficiency and saving query processing resources.

[0054] In one example, such as a search scenario, we can retrieve search requests for a target account within a preset time period, such as search requests within one second of the current time. We can extract search keywords from these requests, and then use these keywords to form a set of query parameters; that is, the query parameters can be search keywords. Correspondingly, the data queried can be data corresponding to the search keywords, and the data format can be multimedia, images, text, items, etc.

[0055] In another example, such as in an object recommendation scenario, personalized object recommendations can be made based on target accounts. The recommended objects can include multimedia, images, text, items, etc., and this disclosure does not limit this. Based on this, the query parameters can be query features. Accordingly, multiple object recommendation requests from target accounts can be received; multiple query features can be extracted from these multiple object recommendation requests; and a query feature set is obtained based on these multiple query features. Optionally, the multiple query features can be deduplicated to obtain multiple target query features; thereby constructing a query feature set from the multiple target query features. Deduplicating the multiple query features further improves data processing efficiency. This query feature set can be used as input to the object recommendation model.

[0056] As an example, deduplication of multiple query features can be performed as follows:

[0057] u, umap = Unique(x),

[0058] Where x represents multiple query features, number m; u represents multiple target query features, number n (n <= m); umap is the coordinate mapping between the same query features in x and u, which can be used for the location and recovery of subsequent features.

[0059] The above are examples of application scenarios and do not limit this disclosure. Accordingly, the query parameters in the parameter set to be queried can be any parameters used for indexing or matching in data querying. In the embodiments of this specification, after obtaining the parameter set to be queried, the corresponding data can be found in the first cache to obtain the first data corresponding to the first query parameter. The first query parameter can be the query parameter corresponding to the data found in the first cache within the parameter set to be queried; that is, the first query parameter can be the query parameter for which the query result is found in the first cache within the parameter set to be queried.

[0060] S403, if the first query parameter is a subset of the query parameters in the set of parameters to be queried, data query processing is performed in the second cache to obtain the second data corresponding to the second query parameter. This second query parameter belongs to all or part of the query parameters outside the first query parameter in the set of parameters to be queried; that is, the second query parameter is all or part of the query parameters outside the first query parameter in the set of parameters to be queried. This second cache can be a CPU-associated cache. A CPU-associated cache can refer to a cache configured in the CPU memory, which can be called a CPU cache.

[0061] In the embodiments of this specification, if the first query parameter is only a portion of the query parameters in the set of parameters to be queried, that is, if no data corresponding to all query parameters in the set of parameters to be queried is found in the first cache, data query processing can be performed in the second cache to obtain the second data corresponding to the second query parameter. If the first query parameter is all the query parameters in the set of parameters to be queried, the query can be terminated, and the first data can be used as the target data.

[0062] Optionally, the method may also include updating the GPU cache, such as... Figure 6As shown, this could include, for example, determining a first ratio between the number of first query parameters and the number of fourth query parameters; the fourth query parameters being query parameters in the set of parameters to be queried, excluding the first query parameters; if the first ratio is lower than a ratio threshold, it indicates that the GPU cache is still being built, and the data found in the CPU can be used to update the GPU cache. For example, the second query parameters, the third query parameters, and the corresponding second and third data can be updated in the first cache. As an example, the second query parameters and the third query parameters, and their corresponding first and second hash values ​​can be determined respectively; the first storage group corresponding to the first hash value in the first cache, and the second storage group corresponding to the second hash value in the first cache can be obtained; the first target data that has not been called within a preset time period in the first storage group, and the second target data that has not been called within a preset time period in the second storage group can be obtained; the first target data can be replaced with the second data, and the second target data can be replaced with the third data. For the processing method of not being called within the preset time period, a counter can be maintained for each embedded feature in the GPU cache, and the counter can be set to the latest value whenever the embedded feature is searched. When updating the GPU cache, the hash value corresponding to the query parameter to be updated can be determined. Then, the embedded feature whose counter value has not been updated for the longest time under the cache_set corresponding to the hash value can be selected as the replacement object to replace it, thereby realizing GPU cache update.

[0063] Optionally, if the first ratio is not lower than the ratio threshold, it indicates that the GPU cache is stabilizing, and the GPU cache update step can be skipped to avoid the overhead of frequent updates. By setting the first ratio to perform GPU cache updates, both adaptive updates of the GPU cache can be achieved, resources can be saved, and the timeliness of queries based on embedded features can be applied to improve the data hit rate in the GPU cache.

[0064] like Figure 5 As shown, to improve cache hit rate and fully utilize the high concurrency capabilities of the GPU, the GPU cache can be stored using a three-dimensional array, and a set-associative mapping method can be used for key-value mapping. For example... Figure 5As shown, the three-dimensional array includes a storage set (cache_set), a storage path (cache_lane), and data (such as an embedding vector). Each cache_set can contain multiple cache_lanes, and each cache_lane can store one embedding vector. Direct mapping is used between cache_sets, and fully associative mapping is used between cache_lanes within the same cache_set. Linear probing is used to resolve collisions. Fully associative mapping means that cache_lanes within the same cache_set correspond to the same hash value, i.e., hash values ​​belonging to the same cache_set. This application does not limit the hash processing method. Linear probing resolves collisions when cache_lanes within the same cache_set correspond to the same hash value. In this case, collisions may occur, and linear probing can be a lookup-by-lookup method to avoid collisions. Based on this, the high concurrency of the GPU can be used for parallel probing along the cache_lane dimension; once an embedding vector is selected, the GPU can be used to copy or update the vector in parallel.

[0065] It's important to note that since there are no dependencies between the parameters in the set of parameters to be queried, parallel lookup and update can be performed using the GPU. The lookup process only involves read operations on the GPU cache, eliminating data contention and the need for cache locking. However, during GPU cache updates, since the first or second target data being replaced may reside in the same cache_set, mutual exclusion locking can be implemented across the GPU cores to ensure the reliability of the update and prevent data contention.

[0066] S405, based on the first data and the second data, obtain the target data corresponding to the query parameter set.

[0067] In the embodiments described in this specification, the first data and the second data can be merged to obtain the merged data as the target data corresponding to the query parameter set.

[0068] Furthermore, based on the query parameters of each target account, the query data corresponding to each target account can be filtered out from the target data, that is, the data corresponding to the query parameters of each target account.

[0069] By setting up caches associated with both the accelerator processor and the CPU, the query speed for both the GPU and CPU can be improved, thereby reducing data query latency and increasing data query efficiency. This expands the applicability of data queries, effectively addressing large-scale data query scenarios, such as the real-time requirements of large-scale embedded feature queries in recommendation model applications. Furthermore, this multi-level caching approach in the accelerator processor and CPU allows for independent cache management, making the cache more flexible and scalable. While improving data query efficiency, it also reduces the consumption of communication resources.

[0070] Optionally, the method may further include: if a third query parameter is not found in the second cache, the third data corresponding to the third query parameter can be searched in the target query object; the target query object represents the correspondence between all preset query parameters and data. The query parameters in the above-mentioned set of parameters to be queried may belong to all preset query parameters. Accordingly, the above-mentioned S205 can be replaced by: obtaining the target data based on the first data, the second data, and the third data. By storing all preset query parameters and data, a combination of caching and full storage is achieved, ensuring the comprehensiveness and accuracy of the query.

[0071] In practical applications, the target query object can be stored in CPU memory or external storage, such as... Figure 3a As shown; or it can be stored in a parameter server, such as Figure 3b As shown. Accordingly, for Figure 3a In the standalone mode shown, the query process with both GPU and CPU caches configured is as follows: The GPU queries its local storage GPU cache, saving the first data corresponding to the first query parameter that can be found in the GPU cache. The fourth query parameter, which cannot be found, is sent to the next level, the CPU cache. The CPU receives the fourth query parameter that cannot be found from the GPU, queries these keys in its memory CPU cache, and returns the second data corresponding to the second query parameter that can be found in the CPU cache to the GPU for GPU cache updates. For the third query parameter that cannot be found, the CPU queries its embedding matrix to obtain the third data. The third query parameter and the third data are then updated in the CPU cache before being returned to the GPU for GPU cache updates.

[0072] Optionally, in the absence of a GPU (i.e., GPU storage resources are 0), the query process under a single CPU cache can be as follows: The CPU queries the CPU cache in memory, saving the values ​​corresponding to the keys that can be found in the cache. For keys that are not found, the CPU queries the embedding matrix to obtain the values ​​corresponding to these keys, updates the CPU cache with these keys and values, and finally returns the query result. Here, the query result can refer to the target data. In a single-machine scenario, the embedding matrix is ​​located in local memory or external storage, avoiding communication overhead and improving query efficiency.

[0073] See Figure 3b In a multi-machine mode, worker nodes can first query their own GPU and CPU caches. If the key is not found, it is sent to the parameter server. The parameter server receives these keys and queries the GPU and CPU caches sequentially. If the key is not found there, it queries the embedding matrix. Then, it returns the retrieved values ​​level by level to update the caches accordingly. In a multi-machine scenario, the performance improvement from caching is more significant. Without caching, all worker queries must first be sent to the parameter server over the network, and the parameter server must then return the results to the worker over the network. Compared to a single-machine scenario where the embedding matrix resides in local memory or external storage, a multi-machine scenario incurs additional network transmission overhead. With caching, if the queried key can be found in the cache, the communication overhead can be eliminated, resulting in a significant performance improvement.

[0074] In the embodiments described in this specification, Figure 6 The data merging shown can include the merging of the first data, second data, and third data mentioned above. When the query parameter is a query feature and the target data is a first embedding vector, feature merging can also include merging the first embedding vectors corresponding to the same target account. For example, preset feature groups and the first embedding vectors corresponding to the query features of each target account can be obtained; statistical processing within each preset feature group can be performed on the first embedding vectors corresponding to each target account to obtain the second embedding vectors of each target account under each preset feature group; and inter-group concatenation processing can be performed on the second embedding vectors of each target account under each preset feature group to obtain the query data corresponding to each target account. The number of preset feature groups can be multiple, and each preset feature group can include at least one query feature. For example, a preset feature group is an account feature group, and the included query features can be account identifier features, age features, etc. Or a preset feature group is an object feature group, and the included query features can be object identifier features, object category features, etc. This disclosure does not limit these.

[0075] Taking the query of an embedded matrix as an example, Uq =CacheLookUp(u), where u is the query feature after deduplication; U q This can be the first embedded vector found in the query, with a size of n*p, where n is a variable of U. q The quantity, p is U q The dimension. Furthermore, it can be seen from U q The first embedding vector corresponding to each target account is selected from the data.

[0076] Since the query features come from different preset feature groups, the embedding vectors within each preset feature group can be merged for each target account. Specifically, for example, the position of each query feature in the deduplication list can be obtained using the aforementioned UMAP, thereby determining the preset feature group to which the query feature belongs and the corresponding target account; and statistical processing can be performed on features within the same preset feature group, as follows:

[0077] S = Combine(U, umap),

[0078] Where S can be a second embedding vector of size k*p, k is the number of S (the number of preset feature groups), and p is the dimension of S; U can be the first embedding vector corresponding to each target account. Statistical processing can include averaging, summing, etc., which are not limited in this application. Thus, feature concatenation between preset groups can be performed to obtain the concatenated features corresponding to each target account, i.e. Figure 6 The output embedding features are shown.

[0079] For example, such as Figure 7 As shown, the query features (input features) of target account 1 are query features 1 to 5. Through data querying, the first embedding vectors corresponding to query features 1 to 5 are obtained as E1 to E5. If query features 1 to 3 belong to preset feature group 1, and query features 4 to 5 belong to preset feature group 2, then the second embedding vector of preset feature group 1 is V1 = (E1 + E2 + E3) / 3; and the second embedding vector of preset feature group 2 is V2 = (E4 + E5) / 2. Therefore, the query data for target account 1 can be (V1, V2).

[0080] Furthermore, such as Figure 6As shown, the method may further include updating the CPU cache, for example, it may include: determining a second ratio between the number of second query parameters and the number of third query parameters; if the second ratio is lower than a ratio threshold, updating the third query parameters and the corresponding third data to the second cache. For example, the third hash value corresponding to the third query parameter can be obtained; the third storage group corresponding to the third hash value in the second cache can be determined; if there is a target storage path with no stored data among the multiple storage paths included in the third storage group, the third data can be stored in the target storage path. By setting the second ratio to perform CPU cache updates, dynamic updates of the CPU cache can be achieved, while frequent cache updates can be avoided; and data updates in any storage path can be achieved.

[0081] As an example, a CPU cache can be an embedded CPU cache with query, update, and eviction capabilities, similar to a hardware cache in computer architecture. A query retrieves the corresponding value using a key. An update uses the key and value to be updated, updating the cache for subsequent query operations. The cache has a limited capacity; if an update finds the cache full, some data in the cache needs to be evicted before the new data is added.

[0082] like Figure 8a and Figure 8b As shown, the CPU cache can use a multi-way set-associative design, where the memory way and memory set are flexibly configurable, supporting the LRU (Least Recently Used) eviction policy (eviction of data that has not been used for the longest time, based on time) or other custom policies. The CPU cache can consist of M data sets, and each data set can be composed of N ways. As an example, such as... Figure 8b As shown, the CPU cache can be implemented using hash tables and doubly linked lists. The specific implementation of the CPU cache in this application is not limited. Using a hash table (HashMap) and a doubly linked list, access to any key can be achieved, supporting fast lookups and cache updates. In the HashMap, Addr0 to Addr2 can refer to hash values.

[0083] For a key to be queried, firstly, the hash value corresponding to the key is calculated based on the key's value, thus determining that the hash value belongs to the Xth set, for example, X = Key % M. Further, the key is checked in N paths under the Xth set to see if it exists. If it does, the corresponding value, i.e., the embedding vector, is returned; otherwise, the next level of query is performed. When the next level query result is returned, the key and its corresponding value are updated in the N paths.

[0084] When updating a key, a third hash value can be calculated first. This allows you to determine if the multiple storage paths in the third storage group corresponding to the third hash value are full. If not, the key and value are directly added to the last path. If the path is full, the data in one path is cleared according to the eviction policy, and then the new key and value are filled into that path.

[0085] Figure 9 This diagram illustrates a data query apparatus according to an embodiment of the present application. Figure 9 As shown, the device may include:

[0086] The first query module 901 is used to obtain the set of parameters to be queried, search for the corresponding data in the first cache, and obtain the first data corresponding to the first query parameter; the first cache is a cache associated with the accelerator processor;

[0087] The second query module 903 is used to perform data query processing in the second cache if the first query parameter is a part of the query parameters in the set of parameters to be queried, and to obtain the second data corresponding to the second query parameter, wherein the second query parameter belongs to the query parameters other than the first query parameter in the set of parameters to be queried; the second cache is a cache associated with the CPU;

[0088] The target data acquisition module 905 is used to acquire the target data corresponding to the query parameter set based on the first data and the second data.

[0089] In one possible implementation, the above-mentioned apparatus may further include:

[0090] The third query module is used to search for the third data corresponding to the third query parameter in the target query object if a third query parameter is not found in the second cache; the target query object represents the correspondence between all preset query parameters and data.

[0091] Accordingly, the target data acquisition module 905 can also be used to acquire the target data based on the first data, the second data, and the third data.

[0092] In one possible implementation, the above-mentioned apparatus may further include:

[0093] The first ratio determination module is used to determine a first ratio between the number of the first query parameters and the number of the fourth query parameters; the fourth query parameters are query parameters in the set of parameters to be queried, excluding the first query parameters.

[0094] The first update module is used to update the second query parameter, the third query parameter, and the corresponding second data and the third data to the first cache if the first ratio is lower than the ratio threshold.

[0095] In one possible implementation, the first update module mentioned above may include:

[0096] A hash value determination unit is used to determine the first hash value and the second hash value corresponding to the second query parameter and the third query parameter, respectively.

[0097] The first storage group determination unit is used to obtain the first storage group corresponding to the first hash value in the first cache, and the second storage group corresponding to the second hash value in the first cache;

[0098] The data to be replaced acquisition unit is used to acquire first target data in the first storage group that has not been called within a preset time period, and second target data in the second storage group that has not been called within a preset time period;

[0099] The first update unit is configured to replace the first target data with the second data, and to replace the second target data with the third data.

[0100] In one possible implementation, the above-mentioned apparatus may further include:

[0101] The second ratio determination module is used to determine a second ratio between the number of the second query parameters and the number of the third query parameters;

[0102] The second update module is used to update the third query parameter and the corresponding third data to the second cache if the second ratio is lower than the ratio threshold.

[0103] In one possible implementation, the second update module described above may include:

[0104] A hash value acquisition unit is used to acquire the third hash value corresponding to the third query parameter;

[0105] The second storage group determination unit is used to determine the third storage group corresponding to the third hash value in the second cache.

[0106] The second update unit is used to store the third data in the target storage path if there is a target storage path that does not store data among the multiple storage paths included in the third storage group.

[0107] In one possible implementation, the first query module described above may include:

[0108] The query parameter acquisition unit is used to acquire the query parameters for each of the multiple target accounts.

[0109] The deduplication unit is used to deduplicate the query parameters of the multiple target accounts to obtain the query parameter set, wherein the query parameters included in the query parameter set are the deduplicated query parameters.

[0110] In one possible implementation, the above-mentioned apparatus may further include:

[0111] The query data filtering module is used to filter the query data corresponding to each target account from the target data based on the query parameters of each target account.

[0112] In one possible implementation, the query parameter is a query feature, and the target data is a first embedding vector; the query data filtering module may include:

[0113] The first embedding vector acquisition unit is used to acquire the first embedding vector corresponding to the preset feature group and the query features of each target account;

[0114] The intra-group feature merging unit is used to perform statistical processing on the first embedding vector corresponding to each target account within each preset feature group to obtain the second embedding vector of each target account under each preset feature group.

[0115] The feature concatenation unit is used to perform inter-group concatenation processing on the second embedding vector of each target account under each preset feature group to obtain the query data corresponding to each target account.

[0116] In one possible implementation, the above-mentioned apparatus may further include:

[0117] The target query object acquisition module is used to acquire the target query object; the target query object represents the correspondence between all preset query parameters and data;

[0118] The first configuration module is configured to configure the first cache in the storage resources associated with the accelerator processor and store the target query object in the first cache if the storage resources associated with the accelerator processor are greater than or equal to the storage resources required by the target query object.

[0119] The CPU memory storage resource acquisition module is used to acquire the CPU memory storage resources if the storage resources associated with the accelerated processor are less than the storage resources required by the target query object.

[0120] The second configuration module is used to configure the second cache in the CPU memory and store the target query object in the second cache if the storage resources of the CPU memory are greater than or equal to the storage resources required by the target query object.

[0121] The third configuration module is used to configure the second cache in the CPU memory and store the target query object in the CPU external memory if the storage resources of the CPU are less than the storage resources required by the target query object, or to store the target query object in the target server.

[0122] Regarding the apparatus in the above embodiments, the specific manner in which each module and unit performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0123] Figure 10 A block diagram of an electronic device for data querying according to an embodiment of this application is shown. The electronic device may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, this electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a data query method.

[0124] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0125] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the data query method as described in the embodiments of this application.

[0126] In an exemplary embodiment, a storage medium is also provided, which, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to execute the data query method of the present application embodiments.

[0127] In an exemplary embodiment, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the data query method of the embodiments of this application.

[0128] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0129] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0130] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A data query method, characterized in that, The method includes: Once the set of parameters to be queried is obtained, the corresponding data is searched in the first cache to obtain the first data corresponding to the first query parameter; the first cache is a cache associated with the accelerator processor; If the first query parameter is a subset of the query parameters in the set of parameters to be queried, data query processing is performed in the second cache to obtain the second data corresponding to the second query parameter. The second query parameter belongs to a query parameter other than the first query parameter in the set of parameters to be queried. The second cache is a CPU-associated cache. Based on the first data and the second data, obtain the target data corresponding to the query parameter set.

2. The method according to claim 1, characterized in that, The method further includes: If a third query parameter is not found in the second cache, the third data corresponding to the third query parameter is searched in the target query object; the target query object represents the correspondence between all preset query parameters and data. The step of obtaining the target data corresponding to the query parameter set based on the first data and the second data includes: The target data is obtained based on the first data, the second data, and the third data.

3. The method according to claim 2, characterized in that, The method further includes: Determine a first ratio between the number of the first query parameters and the number of the fourth query parameters; the fourth query parameters are query parameters in the set of parameters to be queried, excluding the first query parameters. If the first ratio is lower than the ratio threshold, the second query parameter, the third query parameter, and the corresponding second data and the third data are updated in the first cache.

4. The method according to claim 3, characterized in that, The step of updating the second query parameter, the third query parameter, and the corresponding second data and third data to the first cache includes: Determine the first hash value and the second hash value corresponding to the second query parameter and the third query parameter, respectively; Obtain the first storage group corresponding to the first hash value in the first cache, and the second storage group corresponding to the second hash value in the first cache; Obtain first target data from the first storage group that has not been accessed within a preset time period, and second target data from the second storage group that has not been accessed within a preset time period; The first target data is replaced with the second data, and the second target data is replaced with the third data.

5. The method according to claim 2, characterized in that, The method further includes: Determine a second ratio between the number of the second query parameters and the number of the third query parameters; If the second ratio is lower than the ratio threshold, the third query parameter and the corresponding third data are updated in the second cache.

6. The method according to claim 5, characterized in that, The step of updating the third query parameter and the corresponding third data to the second cache includes: Obtain the third hash value corresponding to the third query parameter; Determine the third storage group corresponding to the third hash value in the second cache; If there is a target storage path that does not store data among the multiple storage paths included in the third storage group, the third data is stored in the target storage path.

7. The method according to claim 1 or 2, characterized in that, The process of obtaining the parameter set to be queried includes: Retrieve query parameters for each of the multiple target accounts; The query parameters of each of the multiple target accounts are deduplicated to obtain the set of query parameters, which includes the deduplicated query parameters.

8. The method according to claim 7, characterized in that, The method further includes: Based on the query parameters of each target account, the query data corresponding to each target account is filtered out from the target data.

9. The method according to claim 8, characterized in that, The query parameters are query features, and the target data is a first embedding vector; the step of filtering query data corresponding to each target account from the target data based on the query parameters of each target account includes: Obtain the first embedding vector corresponding to the preset feature groups and the query features of each target account; Perform statistical processing on the first embedding vector corresponding to each target account within each preset feature group to obtain the second embedding vector of each target account under each preset feature group. The second embedding vector of each target account under each preset feature group is concatenated between groups to obtain the query data corresponding to each target account.

10. The method according to claim 1 or 2, characterized in that, The method further includes: Obtain the target query object; the target query object represents the correspondence between all preset query parameters and data; If the storage resources associated with the accelerator are greater than or equal to the storage resources required by the target query object, the first cache is configured in the storage resources associated with the accelerator, and the target query object is stored in the first cache. If the storage resources associated with the accelerator are less than the storage resources required by the target query object, obtain the storage resources of the CPU memory; If the storage resources of the CPU memory are greater than or equal to the storage resources required by the target query object, the second cache is configured in the CPU memory, and the target query object is stored in the second cache; If the storage resources of the CPU are less than the storage resources required by the target query object, the second cache is configured in the CPU memory, and the target query object is stored in the CPU external memory, or the target query object is stored in the target server.

11. A data query device, characterized in that, include: The first query module is used to obtain the set of parameters to be queried, search for the corresponding data in the first cache, and obtain the first data corresponding to the first query parameter. The first cache is a cache associated with the acceleration processor; The second query module is used to perform data query processing in the second cache if the first query parameter is a part of the query parameters in the set of parameters to be queried, and to obtain the second data corresponding to the second query parameter, wherein the second query parameter belongs to the query parameters other than the first query parameter in the set of parameters to be queried; The second cache is a CPU-associated cache; The target data acquisition module is used to acquire the target data corresponding to the query parameter set based on the first data and the second data.

12. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions to implement the method according to any one of claims 1 to 10.

13. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 10.

Images