Search method, device, equipment and system

Abstract

A retrieval method, device, equipment and system are disclosed, and relate to the field of artificial intelligence. The retrieval system comprises multiple computing nodes configured to perform retrieval on multiple types of databases. The multiple computing nodes are connected with a control node. The control node determines at least one database from the multiple types of databases according to a feature relationship between a retrieval request and the multiple types of databases, instructs the computing node deploying the at least one database to perform retrieval, and obtains a retrieval result. The method provided in the present application calculates the feature relationship between the retrieval request and the multiple types of databases, and selects a part of the databases with similar feature relationships to the retrieval request to perform retrieval. Similar data does not need to be retrieved in all types of databases, thereby reducing the waste of computing resources and improving the utilization rate of computing resources. Similar data is retrieved in the databases with similar feature relationships to the retrieval request, thereby improving the retrieval accuracy.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a retrieval method, apparatus, device, and system. Background Technology

[0002] Currently, with the development of big data applications, massive amounts of data are emerging, such as unstructured data like images, text, video, and audio. Converting unstructured data into high-dimensional vectors and using these vectors to represent the semantics of the unstructured data is a process called embedding. Search results are obtained by retrieving data from databases based on search requests; for example, retrieving vectors similar to the query content from the database, thus enabling the analysis and retrieval of unstructured data. As the scale and complexity of data increase, databases are deployed across multiple computing devices, and search results are obtained by retrieving data from these devices based on search requests. However, when vectors similar to the query content exist in only some databases, if computing nodes search each database, it will waste computing resources and reduce the utilization rate of computing resources. Searching each database involves a large amount of data, which can lead to low search accuracy. Summary of the Invention

[0003] This application provides a retrieval method, apparatus, device, and system that improves retrieval accuracy and resource utilization.

[0004] Firstly, a retrieval method is provided, applied to a computing system comprising multiple computing nodes for performing retrievals on various types of databases. The computing nodes are connected to a control node, and the retrieval method is executed by the control node. The method includes determining at least one database from the various database types based on the characteristic relationship between a retrieval request and the database types, instructing the computing nodes deploying the at least one database to perform the retrieval, and obtaining retrieval results. The retrieval results include data from the at least one database that is similar to the retrieval request.

[0005] The method provided in this application calculates the feature relationships between a retrieval request and various types of databases, and selects databases with similar feature relationships to the retrieval request for retrieval. This eliminates the need to search for similar data in all types of databases, reducing wasted computational resources and improving resource utilization. Retrieving similar data from databases with similar feature relationships to the retrieval request improves retrieval accuracy.

[0006] In one possible implementation, determining at least one database from multiple types of databases based on the characteristic relationship between the retrieval request and multiple types of databases includes: determining at least one database from multiple types of databases based on the degree of similarity between the retrieval request and multiple types of databases.

[0007] Based on the similarity between the search request and various types of databases, the database with a high degree of similarity to the search request is selected for retrieval, thereby improving the utilization of computing resources and the accuracy of retrieval.

[0008] In another possible implementation, determining at least one database from multiple types of databases based on the similarity between the retrieval request and multiple types of databases includes: determining at least one database from multiple types of databases based on the probability that the data to be retrieved indicated by the retrieval request belongs to multiple types of databases.

[0009] By using probability to quantify the similarity between a search request and a database, a clear basis for measuring similarity is provided, improving the accuracy of selecting databases containing data similar to the search request and thus increasing search precision.

[0010] In another possible implementation, instructing a computing node that has deployed at least one database to perform a retrieval and obtain retrieval results includes: sending an instruction message, which instructs at least one computing node to perform a retrieval, wherein at least one computing node has deployed at least one database; and receiving retrieval results from at least one computing node.

[0011] Instruction messages are sent to compute nodes that have deployed databases highly similar to the search request, instructing them to retrieve similar data. This eliminates the need for every compute node with a database to perform the search, reducing unnecessary waste of computing resources and improving resource utilization.

[0012] In another possible implementation, at least one operation, either deduplication or sorting, is performed on the retrieval results fed back by at least one computing node to obtain the retrieval results.

[0013] By receiving search results from different types of databases, the search results are deduplicated to eliminate redundant information. The search results are then sorted to obtain data with higher similarity to the search request, thus improving search accuracy.

[0014] In another possible implementation, the data in the dataset is divided into multiple types of databases based on various data types.

[0015] Dividing datasets by data type allows similar data within the dataset to be grouped into the same type of database, thereby increasing the accuracy of retrieving similar data from the database and improving retrieval precision.

[0016] In another possible implementation, the data in the dataset is divided into multiple types of databases based on multiple data types, including: dividing the dataset into multiple types of databases based on the probability that the dataset belongs to multiple data types.

[0017] Using probability to quantify the similarity between data in a dataset and different types of databases provides a clear basis for measuring similarity.

[0018] In another possible implementation, multiple types of databases are deployed across multiple compute nodes based on load balancing.

[0019] By rationally allocating the databases, databases with high retrieval frequency are deployed on computing nodes with sufficient computing resources, while databases with low retrieval frequency are deployed on computing nodes with insufficient computing resources, thus improving the utilization rate of computing resources.

[0020] In a second aspect, a retrieval apparatus is provided, comprising modules for performing the methods of the first aspect or any possible design of the first aspect. For example, the retrieval apparatus includes a communication module and a processing module.

[0021] A communication module is used to acquire a search request. A processing module is used to determine at least one database from the multiple types of databases based on the characteristic relationship between the search request and the multiple types of databases. The processing module is also used to instruct computing nodes that deploy at least one database to perform a search and obtain search results, which include data in at least one database that is similar to the search request.

[0022] In one possible implementation, when the processing module determines at least one database from the multiple types of databases based on the characteristic relationship between the retrieval request and the multiple types of databases, it is specifically used to: determine at least one database from the multiple types of databases based on the similarity between the retrieval request and the multiple types of databases.

[0023] In another possible implementation, when the processing module determines at least one database from multiple types of databases based on the similarity between the retrieval request and multiple types of databases, it is specifically used to: determine at least one database from multiple types of databases based on the probability that the data to be retrieved indicated by the retrieval request belongs to multiple types of databases.

[0024] In another possible implementation, the communication module is also used to send an instruction message instructing at least one computing node to perform a retrieval, wherein the at least one computing node has deployed at least one database. The communication module is also used to receive retrieval results from at least one computing node.

[0025] In another possible implementation, the processing module is also used to perform at least one operation, either deduplication or sorting, on the retrieval results fed back by at least one computing node to obtain the retrieval results.

[0026] In another possible implementation, the processing module is also used to divide the data in the dataset into multiple types of databases based on various data types.

[0027] In another possible implementation, when the processing module divides the data in the dataset into multiple types of databases based on multiple data types, it is also used to divide the dataset into multiple types of databases based on the probability that the dataset belongs to multiple data types.

[0028] In another possible implementation, the processing module is also used to deploy multiple types of databases across multiple compute nodes based on load balancing.

[0029] Thirdly, a computer device is provided, the computer device including a memory and a plurality of processors, the memory being used to store a set of computer instructions; when the processors execute the set of computer instructions, the plurality of processors jointly execute the operation steps of the method as described in the first aspect or any possible implementation of the first aspect.

[0030] Fourthly, a retrieval system is provided, which includes multiple computing nodes and at least one control node. The control node is used to execute the operation steps of the method in the first aspect or any possible implementation of the first aspect to realize database retrieval.

[0031] Fifthly, a computer-readable storage medium is provided, comprising: computer software instructions; when the computer software instructions are executed in a processor, causing the processor to perform operational steps of the method as described in the first aspect or any possible implementation thereof.

[0032] In a sixth aspect, a computer program product is provided that, when run on a computer, causes the computer to perform operational steps of the method as described in the first aspect or any possible implementation thereof.

[0033] The technical effects of any of the design methods in aspects two through six can be found in aspect one or in different design methods in aspect one, and will not be repeated here.

[0034] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0035] Figure 1 A schematic diagram of a retrieval system provided for the prior art;

[0036] Figure 2 A flowchart illustrating a retrieval method provided by existing technology;

[0037] Figure 3 A schematic diagram of another retrieval system provided by existing technology;

[0038] Figure 4 A schematic diagram of the architecture of a retrieval system provided in this application;

[0039] Figure 5 This application provides a flowchart illustrating a method for deploying multiple types of databases.

[0040] Figure 6 A flowchart illustrating a retrieval method provided in this application;

[0041] Figure 7 A flowchart illustrating another retrieval method provided in this application;

[0042] Figure 8 A schematic diagram of the structure of a retrieval device provided in this application;

[0043] Figure 9 This is a schematic diagram of the structure of a computer device provided in this application. Detailed Implementation

[0044] To better understand the embodiments of this application, some terms or technologies involved in the embodiments of this application will be explained below.

[0045] A vector, also known as a Euclidean vector or a geometric vector, is a quantity in mathematics that has both magnitude and direction. A vector is represented by a line segment with an arrowhead; the arrow points to the direction of the vector, and the length of the line segment represents the magnitude of the vector.

[0046] Since computer devices can only recognize numbers, they use a set of numbers to represent or identify things; this set of numbers can be a vector. A vector containing n numbers is called an n-dimensional vector. For example, when a computer device recognizes an image, it converts the image into an n-dimensional or higher-dimensional vector.

[0047] Vector retrieval is a vector-based search technique. Its core idea is to calculate the similarity between the vector to be retrieved and vectors in a database, thereby retrieving vectors that are similar or nearly identical to the vector to be retrieved. For example, methods for calculating vector similarity include Euclidean distance, inner product distance, and cosine similarity.

[0048] Vector retrieval is widely used in recommendation systems, image search, video fingerprinting, speech processing, natural language processing, and other fields. Examples include ad recommendations, search engine keyword suggestions, image search, video search, product search, and file retrieval.

[0049] An index is a data structure used to accelerate vector retrieval. By indexing vectors, the search scope within a database can be narrowed, eliminating the need to calculate the similarity between the retrieved vector and every vector in the database. Examples of indexes include, but are not limited to, graph indexes, tree indexes, and inverted indexes.

[0050] Distributed retrieval refers to the process of retrieving useful information from a large amount of information resources in a distributed environment using distributed computing technology. Here, the distributed environment refers to information resources that are physically located in different places, but logically form a whole, thus constituting a distributed retrieval system.

[0051] In some embodiments, a distributed retrieval method is used to retrieve similar vectors. For example, Figure 1 A schematic diagram of a retrieval system provided by existing technology. For example... Figure 1 As shown, the retrieval system includes a server node 110, a data processing node 120, and a distributed storage computing node 130.

[0052] Server node 110 is used to send retrieval requests to data processing node 120.

[0053] Data processing node 120 is used to forward retrieval requests to distributed storage computing node 130.

[0054] The distributed storage computing node 130 is used to receive retrieval requests, retrieve data similar to the retrieval request from the data stored in the local database, and send the retrieval results to the data processing node 120.

[0055] The data processing node 120 is also used to receive the search results sent by the distributed storage computing node 130, summarize the search results, obtain the final search results, and send the final search results to the server node 110.

[0056] In cases where similar vectors exist in some databases, it would be a waste of computing resources if each distributed storage computing node had to retrieve data from its local storage.

[0057] In other embodiments, the data in the database is divided into multiple partitions, and some partitions are randomly selected to retrieve similar vector data within those partitions. For example, Figure 2 A flowchart illustrating a retrieval method provided by existing technology, such as... Figure 2As shown, the method includes the following steps.

[0058] Step 210: Obtain target vector data.

[0059] The data to be retrieved is converted into vector data to obtain the target vector data.

[0060] Step 220: Determine the partition.

[0061] Divide the data in the database into multiple partitions. Randomly select several partitions as the retrieval partitions.

[0062] Step 230: Determine the search parameters.

[0063] Based on the amount of data contained in different search partitions, determine the search parameters for each search partition. This ensures that the search complexity of each search partition is positively correlated with the amount of data in that partition. For example, search parameters include, but are not limited to, search depth and the number of candidate similar data. For instance, when searching in a search partition containing a large amount of data, increase the search depth and the number of candidate similar data.

[0064] Step 240: Retrieve similar vector data.

[0065] Based on the retrieval parameters of the retrieval partition determined in step 230, similar vector data of the target vector data are retrieved from the retrieval partition determined in step 220.

[0066] In step 220, randomly selecting several partitions as search partitions may result in situations where similar vector data does not exist in the selected search partitions. For example, if the database is divided into three partitions, A, B, and C, and similar data for the target vector data exists in partition B, then randomly selecting partitions A and C will lead to searching for similar vector data in incorrect partitions, resulting in low search accuracy.

[0067] In other embodiments, distributed retrieval is performed by storing vector data using both memory and disk storage media. For example, the vector data is clustered to obtain vector data for multiple categories, as well as the centroids of each category. The vector data for multiple categories is stored on the disks of the cluster nodes, while the centroids for multiple categories are stored in the memory of the cluster nodes. Vector retrieval is performed from the cluster nodes based on similar centroids retrieved from memory.

[0068] Example, Figure 3 A schematic diagram of another retrieval system provided by existing technology. For example... Figure 3As shown, in this system, vector data in the cluster nodes is stored using both memory and disk. Memory stores the centroids of the categories to which the vector data belongs, managed using a multi-level centroid neighbor graph data structure. Disk stores all vector data belonging to the categories of the centroids.

[0069] When receiving a retrieval request, cluster nodes start from a non-empty top-level centroid neighbor graph and search downwards layer by layer for centroids similar to the retrieval request. For example, the top-level neighbor graph uses a randomly selected centroid as the entry point, and the lower-level neighbor graph uses a centroid obtained from the adjacent upper-level neighbor graph as the entry point. Based on the centroids obtained from the lower-level neighbor graphs, similar vector data to the retrieval request is calculated from the cluster nodes where the centroids reside.

[0070] In particular, when vector data similar to the retrieval request only exists in the vector data stored on some cluster nodes, retrieving similar vectors on each cluster node introduces unnecessary retrieval operations, increases the consumption of computing resources, and leads to low resource utilization.

[0071] To address the issues of low retrieval accuracy and low resource utilization, this application provides a retrieval method applied to a computing system. The computing system includes multiple computing nodes, which are used to perform retrievals on various types of databases. These computing nodes are connected to a control node. The retrieval method is executed by the control node and includes determining at least one database from the various database types based on the characteristic relationship between the retrieval request and the databases, instructing the computing nodes deploying the at least one database to perform the retrieval, and obtaining retrieval results. The retrieval results include data from at least one database that is similar to the retrieval request.

[0072] Compared to searching every database on every computing node, which wastes computing resources and reduces resource utilization, or searching every database, which results in a large data volume and low search accuracy, the method provided in this application selects databases with similar feature relationships to the search request for retrieval. This eliminates the need to search for similar data in all types of databases, reducing wasted computing resources and improving resource utilization. Searching for similar data in databases with similar feature relationships to the search request also improves search accuracy.

[0073] The following is a detailed description of a retrieval method provided in this application, with reference to the accompanying drawings.

[0074] Figure 4 This is a schematic diagram of the architecture of a retrieval system provided in this application. Figure 4 As shown, the retrieval system 400 includes a client 410, a computing cluster 420, and a storage cluster 430.

[0075] The computing cluster 420 contains multiple computing nodes 421. For example, a computing node 421 is a computing device, such as a server.

[0076] In some embodiments, each of the multiple compute nodes deploys at least one type of database. Different compute nodes deploy different types of databases.

[0077] For example, different computing nodes may deploy at least one different database. Different computing nodes may also deploy the same database and different databases. For instance, there are three types of databases A, B, and C, where database A has a larger data size. In a distributed retrieval system, there are three computing nodes: a first computing node, a second computing node, and a third computing node. Because the storage medium on the first computing node is insufficient to accommodate the large database A, database A is divided into two parts, deployed on the first and second computing nodes respectively. Database B is deployed on the second computing node, and database C is deployed on the third computing node. The first and second computing nodes may deploy the same database A and a different database B.

[0078] For example, different computing nodes may deploy different databases. For instance, there are four types of databases, A, B, C, and D, in a distributed retrieval system with four computing nodes. The four databases A, B, C, and D are deployed on the four computing nodes respectively.

[0079] In some embodiments, the computing cluster 420 is a heterogeneous computing architecture to provide high-performance computing. For example, computing nodes 421 include computing units with computing capabilities such as central processing units (CPUs), graphics processing units (GPUs), data processing units (DPUs), neural processing units (NPUs), and neural-network processing units (NPUs) to provide high-performance computing.

[0080] In this embodiment of the application, computing node 421 is used to retrieve data from the database that is similar to the data to be retrieved contained in the retrieval request.

[0081] Compute node 421 is also used to store various types of databases on local storage media.

[0082] In other embodiments, multiple computing nodes 421 are connected via network devices (such as switches, network interface cards, etc.) based on high-speed interconnect technology, enabling communication between the multiple computing nodes 421.

[0083] In some embodiments, client 410 communicates with computing cluster 420 and storage cluster 430 via network 440. For example, client 410 sends a retrieval request to computing cluster 420 via network 440, requesting computing cluster 420 to retrieve similar data. Network 440 refers to an internal enterprise network (e.g., a Local Area Network, LAN) or the Internet. Client 410 refers to a computer connected to network 440, also known as a workstation. Different clients share network resources (e.g., computing resources, storage resources).

[0084] Optionally, the computing cluster 420 also includes a control node 422. For example, the control node and the computing nodes are independent physical devices. Alternatively, the control node and multiple computing nodes may reside on the same physical device. The control node is a CPU. The multiple computing nodes include computing units such as GPUs, NPUs, and DPUs. The control node 422 is used to determine at least one database from multiple types of databases based on a retrieval request, and to send instruction messages to the computing nodes that deploy the at least one database, instructing the computing nodes to perform vector retrieval tasks and obtain retrieval results.

[0085] The control node 422 is also used to receive the search results returned by the computing node.

[0086] Control node 422 is also used to divide the data in the dataset into multiple types of databases.

[0087] Optionally, control node 422 is also used to train a probability calculation model. The probability calculation model is used to calculate the probability that data belongs to different types of databases.

[0088] In other embodiments, client 410 has client program 411 installed. Client 410 runs client program 411 and displays a user interface (UI). User 450 operates the user interface to submit a request. For example, user 450 operates the user interface to submit a search request. After receiving the search request, control node 422 sends an instruction message to at least one computing node, causing at least one computing node to perform a vector search from the database deployed on the computing node.

[0089] Optionally, the system administrator 460 may configure system information, such as the dataset required by the compute node 421 to perform the retrieval task, by calling the application platform interface (API) 412 or the command-line interface (CLI) 413 through the client 410.

[0090] Storage cluster 430 comprises multiple storage nodes 431. A storage node 431 includes one or more controllers, a network interface card (NIC), and multiple hard drives. Hard drives are used to store data. Hard drives are disks or other types of storage media, such as solid-state drives (SSDs) or shingled magnetic recording (SMR) hard drives. The NIC is used to communicate with the compute nodes 421 included in compute cluster 420. The controller is used to write data to or read data from the hard drives based on read / write data requests sent by the compute nodes 421. During the read / write process, the controller needs to translate the address carried in the read / write data request into an address that the hard drive can recognize.

[0091] Optionally, storage cluster 430 is used to store multiple types of databases. Compute node 421 obtains multiple types of databases from storage cluster 430 to facilitate retrieval.

[0092] Optionally, storage cluster 430 may also be used to store parameters. Parameters include, but are not limited to, model parameters. For example, in the case where control node 422 trains a probability calculation model, storage cluster 430 may be used to store the parameters of the probability calculation model.

[0093] Figure 4 This is merely an illustrative diagram; the embodiments of this application do not limit the device connection method or the number of devices in the retrieval system. For example, the retrieval system includes multiple clients. One client connects to multiple computing nodes. Different clients establish connections with different computing nodes.

[0094] Next, the method for deploying various types of databases provided in this application will be described in detail with reference to the accompanying drawings.

[0095] Figure 5 This is a flowchart illustrating a method for deploying multiple types of databases provided in this application. Here, we mainly focus on... Figure 4 The schematic diagram of the retrieval system is shown for illustration. For example, the method is executed by control node 422. Figure 5 As shown, the method includes the following steps.

[0096] Step 510: Divide the data in the dataset into multiple types of databases.

[0097] A dataset is a collection of structured or unstructured data, typically organized in the form of database records, tables, files, etc., and can be used for retrieval, modeling, or analysis. Data in a dataset includes, but is not limited to, text, images, audio, or video data.

[0098] In some embodiments, the data in the dataset is divided into multiple types of databases based on various data types. For example, in the field of product recommendation, multiple data types include clothing, videos, digital products, etc. The dataset is used to retrieve images, text, and videos of product types that the user is interested in, and this information is then recommended to the user.

[0099] In some embodiments, the probability that data in the dataset belongs to multiple types of databases is calculated, and the dataset is divided into multiple types of databases based on the probability. The probability that data belongs to one type of database is used to measure the similarity between the data and that type of database.

[0100] In the first implementation, the data is divided into n types of databases with higher probabilities based on the probability that the data in the dataset belongs to multiple types.

[0101] For example, calculate the probability of each data point in the dataset belonging to each of the various database types, and then assign that data point to one of the n database types with the highest probability. The dataset partitioning is complete when all data points in the dataset are assigned to the databases with the highest probabilities.

[0102] For example, the dataset contains data of five types: A, B, C, D, and E, with n = 3. The probabilities of data X belonging to type A, B, C, D, and E are 0.3, 0.3, 0.2, 0.1, and 0.1, respectively. Data X is then assigned to the database containing the three types with the highest probabilities: type A, type B, and type C.

[0103] Optionally, the value of n is specified by the user, and the value of n must satisfy the condition that n is less than the number of data types in the dataset.

[0104] In the second implementation, the dataset is divided into multiple types of databases based on the relationship between the probability and the first threshold.

[0105] For example, calculate the probability that each data point in the dataset belongs to each of the various database types, and determine the relationship between the probability of a data point belonging to a different database type and a first threshold. If the probability of a data point belonging to a particular database type is greater than the first threshold, the data is assigned to that database type. If the probability of a data point belonging to a particular database type is less than the first threshold, the data is not assigned.

[0106] For example, the dataset contains data of four types: A, B, C, and D, with a first threshold of 0.3. The probabilities of data Y belonging to types A, B, C, and D are 0.4, 0.4, 0.1, and 0.1, respectively. Since the probabilities of data Y belonging to types A and B are both greater than the first threshold of 0.3, data Y is assigned to the database containing types A and B.

[0107] In the third implementation, the data is divided into multiple types of databases based on thresholds.

[0108] For example, calculate the probability that each data point in the dataset belongs to each of the various database types, and then assign the data to each database type in descending order of probability. After each assignment, sum the probabilities; stop assigning probabilities when the sum exceeds a threshold.

[0109] For example, the dataset contains data in four categories: A, B, C, and D, with a threshold of 0.8. The probabilities of data Z belonging to categories A, B, C, and D are 0.4, 0.2, 0.3, and 0.1, respectively. Following the order of decreasing probabilities, data Z is first assigned to the database containing category A, with a sum of probabilities of 0.4, which is less than the threshold of 0.8. Next, data Z is assigned to the database containing category C, with a sum of probabilities updated to 0.7, still less than the threshold of 0.8. Then, data Z is assigned to the database containing category B, with a sum of probabilities updated to 0.9, which is greater than the threshold of 0.8. Therefore, the assignment stops, and data Z will not be assigned to the database containing category D. Thus, data Z is assigned to the databases containing categories A, B, and C.

[0110] In other embodiments, a probability calculation model is used to calculate the probability that data in the dataset belongs to multiple data types.

[0111] Optionally, an artificial intelligence model can be trained using data sampled from the dataset to obtain a probability calculation model. This model can then be used to calculate the probability distribution of data belonging to different types of databases.

[0112] For example, a portion of the data is sampled from the dataset, and labels are assigned to the sampled data. The sampled data and labels are then input into an AI model to obtain the predicted probabilities of each data point belonging to a different category. A loss function is defined for the model, which measures the difference between the predicted probabilities and the labels. The model parameters are iteratively updated using a backpropagation algorithm combined with gradient descent optimization to gradually reduce the prediction error and minimize the loss function. For example, when the error is large, the AI ​​model parameters are adjusted to reduce the error. This process continues until the AI ​​model converges, yielding the probability calculation model.

[0113] For example, sampling methods include, but are not limited to, random sampling.

[0114] For example, labels can be added to the sampled data. Labels are used to indicate the type of data. Methods for labeling include, but are not limited to, manual labeling, obtaining the original labels of the dataset, performing clustering algorithms on the sampled data, and obtaining pre-stored types from JSON (JavaScript Object Notation) files.

[0115] Optionally, the probability calculation model can be a neural network or a classifier. Classifiers include, but are not limited to, Gaussian Mixture Models (GMMs) and Support Vector Machines (SVMs).

[0116] Optionally, the data in the dataset can be encoded into a machine learning or deep learning model, converting the data into vector data. The vector data can then be used to calculate the probability that the data in the dataset belongs to various database types.

[0117] Step 520: Deploy multiple types of databases.

[0118] The control node divides the data in the dataset into multiple types of databases and deploys these databases on the compute nodes. Deployment refers to the operation of storing the databases.

[0119] In some embodiments, multiple types of databases are stored in the storage medium of the computing node. For example, in... Figure 4 Each compute node 421 of the computing cluster 420 shown stores at least one database of a different type. For example, in... Figure 4 Each computing node 421 of the computing cluster 420 shown stores different types of databases.

[0120] Optionally, multiple types of databases can be stored in cloud storage devices. For example, in... Figure 4 The database is stored in the storage cluster 430 shown.

[0121] In some embodiments, multiple types of databases are deployed based on the simulated access results of multiple types of databases.

[0122] Optionally, simulated search requests can be used to obtain simulated access results. For example, artificial intelligence models can be used to generate text or image data as simulated search requests.

[0123] The simulation access results include database simulation access results and compute node simulation access results.

[0124] The database simulation access results include the number of searches for each database type. A higher number of searches for a database indicates a greater ease of retrieval within that database. Conversely, a lower number of searches for a database indicates a less easy retrieval within that database.

[0125] The simulated access results for compute nodes include the number of times each compute node performs a retrieval task. The more times a compute node performs a retrieval task, the more computing resources it consumes and the less computing resources remain. Conversely, the fewer times a compute node performs a retrieval task, the less computing resources it consumes and the more computing resources remain.

[0126] Based on the simulated access results, databases with more retrievals are deployed on computing nodes with more computing resources, while databases with fewer retrievals are deployed on computing nodes with fewer computing resources.

[0127] For example, the probability that the data to be retrieved in the simulated retrieval request belongs to multiple types of databases is calculated. Following the method in step 610, at least one database is selected from the multiple types of databases for retrieval, and the retrieval count for the selected database is updated to obtain the simulated database access results. For example, the update method is that each time the control node selects a database, the retrieval count for that type of database is incremented by one.

[0128] For example, when a single type of database is deployed on a compute node, the number of times the compute node performs a retrieval task is equal to the number of retrievals performed by the database deployed on that compute node. When multiple types of databases are deployed on a compute node, the number of times the compute node performs a retrieval task is equal to the sum of the number of retrievals performed by the various databases deployed on that compute node.

[0129] Optionally, the control node sets up a priority queue, the length of which is the same as the number of database types. Each entry in the priority queue corresponds to a database type, recording the number of searches for that type of database. The priority queue is then sorted in descending order of search count.

[0130] Optionally, the control node can be configured with an array whose length is the same as the number of compute nodes. Each element of the array corresponds to a compute node and records the number of times the database stored in that compute node has been retrieved, i.e., the number of times the compute node has performed a retrieval task.

[0131] Optionally, the control node selects the database with the highest number of searches from the priority queue and the compute node with the fewest searches from the array. The database with the highest number of searches is then deployed on the compute node with the fewest searches. The search count for the corresponding compute node in the array is updated. For example, the control node selects database A from the priority queue, selects compute node B from the array, deploys database A on compute node B, and adds the search count for database A to the search count for compute node B in the array.

[0132] Optionally, indexes can be built for the database deployed on the compute nodes to improve retrieval efficiency when searching for similar data in the database. Index types include, but are not limited to, graph indexes, inverted indexes, tree indexes, or hash indexes.

[0133] Next, the retrieval method provided in this application will be described in detail with reference to the accompanying drawings.

[0134] Figure 6 This is a flowchart illustrating a retrieval method provided in this application. Here, we mainly focus on... Figure 4 The schematic diagram of the retrieval system is shown for illustration. For example, the method is executed by control node 422. Figure 6 As shown, the method includes the following steps.

[0135] Step 610: Based on the characteristic relationship between the search request and multiple types of databases, determine at least one database from the multiple types of databases.

[0136] In some embodiments, the control node receives a search request from a client; alternatively, the control node retrieves a search request from a storage medium. For example, the search request may be a historical search request. The search request contains data to be searched. The data to be searched includes, but is not limited to, text, images, audio, or video data.

[0137] In some embodiments, the control node determines at least one database from multiple types of databases based on the characteristic relationship between the data to be retrieved contained in the retrieval request and multiple types of databases.

[0138] For example, features are extracted from the data to be retrieved contained in the search request, the feature relationships between the data to be retrieved and multiple types of databases are calculated, and at least one database with similar feature relationships to the data to be retrieved is selected. Optionally, at least one database is selected as the target database, and data similar to the data to be retrieved is retrieved from the target database.

[0139] In some embodiments, the feature relationship between the data to be retrieved and multiple types of databases refers to the degree of similarity between the data to be retrieved contained in the retrieval request and multiple types of databases. The control node selects at least one database from the multiple types of databases based on the degree of similarity.

[0140] For example, the similarity between the data to be retrieved and multiple types of databases can be calculated, and at least one database with a high similarity to the data to be retrieved can be selected. Methods for calculating similarity include, but are not limited to, machine learning and knowledge graphs.

[0141] In some embodiments, the probability that the data to be retrieved belongs to each of multiple types of databases is calculated, and at least one database is determined from the multiple types of databases based on the probability.

[0142] In the first implementation, based on the probability that the data to be retrieved belongs to multiple types of databases, n types of databases with a higher probability of similarity to the data to be retrieved are selected.

[0143] For example, calculate the probability that the data to be retrieved belongs to each of the various database types, and identify the n database types with the highest probability of belonging to the data to be retrieved as the target databases. The value of n is greater than or equal to 1, and less than or equal to the number of database types.

[0144] Provided that the value of n is greater than or equal to 1 and less than or equal to the number of database types, the larger the value of n, the more target databases are identified, and the smaller the value of n, the fewer target databases are identified.

[0145] Optionally, the value of n can be updated based on the number of retrieval tasks performed by the computing nodes. For example, when the computing nodes perform fewer retrieval tasks, their computing resources are sufficient. In this case, increasing the value of n increases the number of target databases, thereby retrieving similar data from more target databases and improving retrieval accuracy. When the computing nodes perform many retrieval tasks, their computing resources are insufficient. In this case, decreasing the value of n reduces the number of target databases, thereby retrieving similar data from fewer target databases and alleviating the pressure on computing resources.

[0146] Optionally, the value of n can be updated based on retrieval efficiency. For example, when retrieval efficiency is low, the value of n can be reduced to decrease the number of target databases. This allows for the retrieval of similar data from fewer target databases, thereby improving the retrieval efficiency of the retrieval system.

[0147] In the second implementation, at least one database is determined based on the relationship between the probability and a second threshold.

[0148] For example, calculate the probability that the data to be retrieved belongs to each of the various database types, and determine the relationship between the probability of the data belonging to different database types and a second threshold. When the probability of the data belonging to a particular database type is greater than the second threshold, that database type is identified as the target database.

[0149] A larger second threshold indicates a stricter criterion for determining the database type of the retrieved data, resulting in fewer identified target databases. A smaller second threshold indicates a more lenient criterion for determining the database type of the retrieved data, resulting in more identified target databases.

[0150] Optionally, the second threshold can be updated based on the number of retrieval tasks performed by the computing nodes. For example, when fewer retrieval tasks are performed by the computing nodes, the second threshold can be lowered to increase the number of target databases, thereby retrieving similar data from more target databases and improving retrieval accuracy. When more retrieval tasks are performed by the computing nodes, the second threshold can be raised to reduce the number of target databases, thereby retrieving similar data from fewer target databases and alleviating the pressure on computing resources.

[0151] Optionally, the second threshold can be updated based on retrieval efficiency. For example, when retrieval efficiency is low, the second threshold can be increased to reduce the number of target databases. Retrieving similar data from fewer target databases improves the retrieval efficiency of the retrieval system.

[0152] Optionally, different configuration methods are provided for receiving user input of the second threshold. These different configuration methods include, but are not limited to, page-based or backend configuration options. Page-based options include, but are not limited to, web pages or application pages, while backend configuration options include, but are not limited to, configuration files, environment variables, or databases. For example, a second threshold selection function is provided via a page. The client receives the second threshold selected by the user through the page.

[0153] Optionally, a probability calculation model can be used to calculate the probability that the data to be retrieved belongs to each of the various database types. The data to be retrieved is input into the probability calculation model to obtain the probabilities that the data belongs to each of the various database types. The training method for the probability calculation model can be found in step 510, and will not be elaborated here.

[0154] Step 620: Instruct the computing nodes that deploy at least one database to perform a retrieval and obtain the retrieval results.

[0155] In some embodiments, similar data is retrieved from a type of database.

[0156] For example, when this type of database is deployed on a compute node, the control node sends an instruction message to the compute node where the database resides, instructing the compute node to perform a retrieval.

[0157] For example, when this type of database is deployed across multiple compute nodes, the control node sends instruction messages to the multiple compute nodes hosting the database, instructing them to perform a retrieval. For instance, it is necessary to retrieve similar data from database A, which is relatively large and deployed on the first and second compute nodes. The control node sends instruction messages to both the first and second compute nodes, instructing them to retrieve similar data from database A.

[0158] In other embodiments, similar data is retrieved from multiple types of databases.

[0159] For example, in a scenario where multiple types of databases are deployed on a single compute node, the control node sends instruction messages to the compute node hosting each type of database, instructing that compute node to perform a retrieval. For instance, it's necessary to retrieve similar data from databases A and B, which are deployed on the first compute node. The control node sends an instruction message to the first compute node, instructing it to retrieve similar data from databases A and B.

[0160] For example, when multiple types of databases are deployed across multiple compute nodes, the control node sends instruction messages to these nodes, instructing them to perform searches. For instance, to retrieve similar data from databases A and B, where database A is deployed on the first compute node and database B on the second, the control node sends instruction messages to both nodes, instructing the first node to retrieve similar data from database A and the second node to retrieve similar data from database B. The implementation of instructing compute nodes to perform searches is described below. Figure 7 As shown, step 620 includes steps 621, 622, 623 and 624.

[0161] Step 621: Send an instruction message.

[0162] In some embodiments, the instruction message includes the data to be retrieved contained in the retrieval request.

[0163] In other embodiments, the instruction message also includes a total query count and a sub-query count. The total query count represents the total number of data items similar to the data to be retrieved that need to be obtained from the target database. The sub-query count represents the number of data items similar to the data to be retrieved that need to be obtained from each database within the target database.

[0164] Optionally, the sub-query volume can be calculated based on the total query volume and the probability that the data to be retrieved belongs to the target database.

[0165] For example, the total query volume is K, and the target databases include database A and database B. Database A is deployed on the first compute node, and database B is deployed on the second compute node. The probability that the data to be retrieved belongs to database A is 0.4, and the probability that it belongs to database B is 0.4. The probabilities of the data to be retrieved belonging to database A and database B are normalized. After normalization, the probability that the data to be retrieved belongs to database A is 0.4 / (0.4+0.4) = 0.5, and the probability that it belongs to database B is 0.4 / (0.4+0.4) = 0.5. The instruction message sent by the control node to the first compute node contains a sub-query volume of 0.5K, and the instruction message sent to the second compute node contains a sub-query volume of 0.5K.

[0166] Optionally, the total query limit is set by the user. Different configuration methods are provided for users to set the total query limit. These methods include, but are not limited to, page-based or backend configuration options.

[0167] Step 622: Retrieve similar data.

[0168] In some embodiments, the computing node receives an instruction message from the control node to obtain the data to be retrieved. It then calculates the similarity between the data to be retrieved and the data contained in the target database to obtain similar data. The similarity calculation methods include, but are not limited to, Euclidean distance, cosine similarity, or inner product distance.

[0169] In some scenarios, computing nodes calculate the similarity between the data to be retrieved and each piece of data in the target database one by one to obtain similar data to the data to be retrieved.

[0170] In other scenarios, when an index is built on the target database deployed on the computing node in step 520, the computing node calculates the similarity between the data to be retrieved and the data contained in the target database based on the index of the target database, and obtains similar data to the data to be retrieved.

[0171] Optionally, in step 621, if the indication message includes the total query volume and the sub-query volume, the number of similar data obtained by the computing node in the target database is equal to the sub-query volume.

[0172] Step 623: Send the search results.

[0173] In some embodiments, the computing node sends the search results to the control node, and the search results include at least one similar data.

[0174] Optionally, the search results may also include the similarity between the data to be retrieved and each similar data. For example, the similarity obtained according to different similarity calculation methods is a specific numerical value.

[0175] Step 624: Process the search results.

[0176] In some embodiments, where the target database is deployed on a computing node, the control node receives the search results sent by the computing node. The search results obtained from the computing node are then returned as the final search results to the device that sent the search request to the control node.

[0177] In other embodiments, when the target database is deployed on two or more computing nodes, the control node receives search results from the two or more computing nodes, processes the search results, and returns the processed search results as the final search results to the device that sent the search request to the control node.

[0178] Optionally, the control node aggregates the search results. For example, if the target database is deployed on two or more computing nodes, the control node receives the search results from the two or more computing nodes, merges the search results from the two or more computing nodes, and uses the merged search results as the final search results.

[0179] Optionally, the control node deduplicates the search results. For example, if the target database is deployed on two or more computing nodes, the control node receives the search results sent by the two or more computing nodes, deletes duplicate similar data in the search results, and uses the search results after deduplication as the final search results.

[0180] Optionally, the control node reorders the search results. For example, the search results sent by the compute nodes to the control node include the similarity between the data to be searched and each similar data. When the target database is deployed on two or more compute nodes, the control node receives the search results sent by the two or more compute nodes, selects the data with higher similarity from the search results based on the similarity in the search results, and uses the data with higher similarity as the final search result.

[0181] It is understood that, in order to achieve the functions in the above embodiments, the computer device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and method steps described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0182] The above text combines Figures 4 to 7 The retrieval method provided in this application is described in detail below, and will be combined with... Figure 8This application describes the apparatus provided according to the present application. These apparatuses are used to implement the functions of the backup system in the above-described method embodiments, and therefore also achieve the beneficial effects of the above-described method embodiments. In this embodiment, the apparatus is as follows: Figure 4 The retrieval system shown is still a module (such as a chip) applied to computer equipment.

[0183] like Figure 8 As shown, the retrieval device 800 includes a communication module 810, a processing module 820, and a storage module 830.

[0184] The retrieval device 800 is used to achieve the above. Figure 4 The functions of computing cluster 420 and storage cluster 430 in the retrieval system shown are illustrated.

[0185] The communication module 810 is used to receive search requests, send instruction messages, or receive search results. For example, the communication module 810 is used to perform... Figure 7 Steps 621 and 623.

[0186] Optionally, the communication module 810 receives a search request from the client and sends it to the processing module 820. The processing module 820 determines at least one database from multiple types of databases based on the search request and sends an instruction message to the communication module 810. The communication module 810 then sends the instruction message to the computing node where the at least one database resides. The computing node retrieves similar data from the database deployed on its local storage medium, obtains the search results, and sends the search results to the communication module 810. The communication module 810 receives the search results and sends them to the processing module 820.

[0187] Processing module 820 is used to determine at least one database from multiple types of databases based on a search request, instruct computing nodes deploying at least one database to perform the search, and process the search results. For example, processing module 820 is used to execute... Figure 6 Steps 610 and 620 in the process. For example, processing module 820 is used to execute... Figure 7 Steps 610 and 624 in the process.

[0188] Storage module 830 is used to store various types of databases and retrieval results, etc.

[0189] It should be understood that the retrieval device 800 in this application embodiment is implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Software implementation is also possible. Figure 6 or Figure 7 The method shown, and its various modules, are also software modules; the retrieval device 800 and its various modules are also software modules.

[0190] The retrieval device 800 according to the embodiments of this application can correspond to the execution of the method described in the embodiments of this application, and the above and other operations and / or functions of each unit in the retrieval device 800 are respectively for implementing Figure 6 or Figure 7 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0191] Figure 9 This is a structural schematic diagram of a computer device 900 provided in this application. (See attached diagram.) Figure 9 As shown, the computer device 900 includes a processor 910, a bus 920, a memory 930, a communication interface 940, a main memory unit 950, and a processor 960. The processor 910, processor 960, memory 930, main memory unit 950, and communication interface 940 are connected via the bus 920.

[0192] It should be understood that in this embodiment, processor 910 is a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor is a microprocessor or any conventional processor.

[0193] The computer device 900 also includes a graphics processing unit (GPU), a neural network processing unit (NPU), a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of programs according to the present application. For example, the processor 960 is a GPU or an NPU.

[0194] The communication interface 940 is used to enable communication between the computer device 900 and external devices or components.

[0195] In this application, computer device 900 is used to implement Figure 4 The retrieval system shown includes a communication interface 940 for acquiring retrieval requests, sending instruction messages, and receiving retrieval results. A processor 960 is used to divide the data in the dataset into multiple types of databases based on various data types, and to determine at least one database from these databases based on the characteristic relationships between the retrieval request and the multiple types of databases. The processor 960 also stores model parameters and optimizer parameters. A processor 910 instructs computing nodes deploying at least one database to perform the retrieval and obtain the retrieval results.

[0196] Bus 920 includes a pathway for transferring information between the aforementioned components (such as processor 910, memory 950, and storage 930). In addition to the data bus, bus 920 also includes a power bus, control bus, and status signal bus. However, for clarity, all buses are labeled as bus 920 in the diagram. Bus 920 is a Peripheral Component Interconnect Express (PCIe) bus, or an Extended Industry Standard Architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cachecoherent interconnect for accelerators (CCIX), etc. Bus 920 is divided into address bus, data bus, and control bus.

[0197] As an example, computer device 900 includes multiple processors. A processor is a multi-core (multi-CPU) processor. Here, a processor refers to one or more devices, circuits, and / or computing units used to process data (e.g., computer program instructions).

[0198] It is worth noting that, Figure 9 Taking a computer device 900 comprising one processor 910 and one memory 930 as an example, the processor 910 and memory 930 are used to indicate a type of device or equipment. In a specific embodiment, the number of each type of device or equipment is determined according to business requirements. For example, the computer device 900 may include multiple GPUs or NPUs.

[0199] Memory 950 can be non-transitory memory, which can be a pool of volatile memory or a pool of non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory is random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). Memory 950 is used to store datasets, various types of databases, etc.

[0200] The memory 930 corresponds to the storage medium used in the above method embodiments for storing various types of databases, retrieval results, and other information, such as a disk, like a mechanical hard disk or a solid-state hard disk.

[0201] The aforementioned computer device 900 may be a general-purpose device or a special-purpose device. For example, computer device 900 may also be a server or other device with computing capabilities.

[0202] It should be understood that the computer device 900 according to this embodiment may correspond to the retrieval device 800 in this embodiment, and correspond to the execution according to Figure 6 or Figure 7The corresponding subject in any of the methods, and the above and other operations and / or functions of each module in the retrieval device 800 are respectively for the purpose of implementing Figure 6 or Figure 7 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0203] The method steps in this embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a computing device. Of course, the processor and storage medium can also exist as discrete components in the computing device.

[0204] Some embodiments of this application provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing computer program instructions that, when executed on a computer, cause the computer to perform one or more steps in the backup and recovery resource control method as described in any of the above embodiments.

[0205] For example, the aforementioned computer-readable storage media include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical discs (e.g., CDs (Compact Disks), DVDs (Digital Versatile Disks), etc.), smart cards, and flash memory devices (e.g., EPROMs (Erasable Programmable Read-Only Memory), cards, sticks, or key drives, etc.). The various computer-readable storage media described in the embodiments of this application may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0206] Some embodiments of this application also provide a computer program product. This computer program product includes computer program instructions carried on a non-transitory computer-readable storage medium, which, when executed on a computer, cause the computer to perform one or more steps of the data storage method as described in the above embodiments.

[0207] The beneficial effects of the computer-readable storage medium and computer program product described above are the same as those of the data storage method described in some of the above embodiments, and will not be repeated here.

[0208] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD). The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A retrieval method, characterized in that, The computing system includes multiple computing nodes, which are used to perform retrievals on various types of databases. These computing nodes are connected to a control node, and a method is executed by the control node, the method including: Based on the characteristic relationship between the retrieval request and the various types of databases, at least one database is determined from the various types of databases; The system instructs computing nodes that deploy the at least one database to perform a retrieval and obtain retrieval results, the retrieval results including data in the at least one database that is similar to the retrieval request.

2. The method according to claim 1, characterized in that, Based on the characteristic relationship between the search request and the various types of databases, at least one database is determined from the various types of databases, including: Based on the degree of similarity between the search request and the various types of databases, at least one database is determined from the various types of databases.

3. The method according to claim 2, characterized in that, Based on the similarity between the search request and the various types of databases, at least one database is determined from the various types of databases, including: Based on the probability that the data to be retrieved indicated by the retrieval request belongs to one of the multiple types of databases, at least one database is determined from the multiple types of databases.

4. The method according to any one of claims 1-3, characterized in that, Instruct computing nodes deploying at least one of the databases to perform a retrieval and obtain retrieval results, including: Send an instruction message, the instruction message being used to instruct at least one computing node to perform a retrieval, the at least one computing node deploying the at least one database; Receive the retrieval results fed back by the at least one computing node.

5. The method according to claim 4, characterized in that, The method further includes: Perform at least one operation, either deduplication or sorting, on the retrieval results returned by the at least one computing node to obtain the retrieval results.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: The data in the dataset is divided into various types of databases based on multiple data types.

7. The method according to claim 6, characterized in that, The dataset is divided into multiple database types based on various data types, including: The dataset is divided into databases of various data types based on the probability that the dataset belongs to multiple data types.

8. The method according to claim 6 or 7, characterized in that, The method further includes: The various types of databases are deployed on the multiple computing nodes based on load balancing.

9. A retrieval device, characterized in that, The device includes: The communication module is used to obtain search requests; The processing module is used to determine at least one database from the multiple types of databases based on the characteristic relationship between the retrieval request and the multiple types of databases; The processing module is further configured to instruct computing nodes that deploy the at least one database to perform a retrieval and obtain retrieval results, the retrieval results including data in the at least one database that is similar to the retrieval request.

10. The apparatus according to claim 9, characterized in that, When the processing module determines at least one database from the multiple types of databases based on the characteristic relationship between the retrieval request and the multiple types of databases, it is specifically used for: Based on the degree of similarity between the search request and the various types of databases, at least one database is determined from the various types of databases.

11. The apparatus according to claim 10, characterized in that, When the processing module determines at least one database from the multiple types of databases based on the similarity between the search request and the multiple types of databases, it is specifically used for: Based on the probability that the data to be retrieved indicated by the retrieval request belongs to one of the multiple types of databases, at least one database is determined from the multiple types of databases.

12. The apparatus according to any one of claims 9-11, characterized in that, The communication module is also used to send an instruction message, which instructs at least one computing node to perform a retrieval, wherein the at least one computing node deploys the at least one database; The communication module is also used to receive the retrieval results fed back by the at least one computing node.

13. The apparatus according to claim 12, characterized in that, The processing module is also used for: Perform at least one operation, either deduplication or sorting, on the retrieval results returned by the at least one computing node to obtain the retrieval results.

14. The apparatus according to any one of claims 9-13, characterized in that, The processing module is also used for: The data in the dataset is divided into various types of databases based on multiple data types.

15. The apparatus according to claim 14, characterized in that, When the processing module divides the data in the dataset into multiple types of databases according to multiple data types, it is also used for: The dataset is divided into databases of various data types based on the probability that the dataset belongs to multiple data types.

16. The apparatus according to claim 14 or 15, characterized in that, The processing module is also used for: The various types of databases are deployed on the multiple computing nodes based on load balancing.

17. A computer device, characterized in that, The computer device includes a memory and a plurality of processors, the memory being used to store a set of computer instructions; when the processors execute the set of computer instructions, the plurality of processors perform the operation steps of the method according to any one of claims 1-8.

18. A retrieval system, characterized in that, The retrieval system includes multiple computing nodes and at least one control node, wherein the control node is used to execute the method as described in any one of claims 1-8 to achieve database retrieval.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or instructions, which, when executed by a computer device, implement the method as described in any one of claims 1-8.

20. A computer program product, characterized in that, The computer program product includes a computer program or instructions that, when executed by a computer device, implement the method as described in any one of claims 1-8.

Images