Information query method and equipment

Abstract

The invention discloses an information query method and equipment, and the information query method comprises the steps: receiving a query vector which is determined based on query information; constructing a query plan corresponding to the query vector, wherein the query plan at least comprises a first query condition of a target text length corresponding to the target query result; executing the query plan in a target vector database to obtain a target vector corresponding to the query vector; and outputting a target query result corresponding to the target vector.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically to an information query method and device. Background Art

[0002] With the development of artificial intelligence technology, more and more large models are applied to terminal devices. For example, in application scenarios such as intelligent question answering and intelligent retrieval, it is necessary to process the query information input by the user to obtain the query results that match it. At present, when processing the query information, it is necessary to determine the final query result based on the information that is closest to it. In the process of determining the information that is closest to it, if too much candidate information is obtained, the computing overhead will increase, and if too little candidate information is obtained, the accuracy of the result will be affected, which cannot meet the actual application needs. Summary of the invention

[0003] In view of this, this application provides the following technical solutions:

[0004] An information query method, comprising:

[0005] receiving a query vector, wherein the query vector is determined based on the query information;

[0006] Constructing a query plan corresponding to the query vector, wherein the query plan at least includes a first query condition of a target text length corresponding to a target query result;

[0007] Execute the query plan in a target vector database to obtain a target vector corresponding to the query vector;

[0008] Output the target query result corresponding to the target vector.

[0009] Optionally, the target text length corresponding to the target query result is determined based on application characteristics of the query information, and the query plan further includes a second query condition for screening candidate vectors corresponding to the query vector.

[0010] Optionally, the target vector database includes multiple text blocks, and a text block length and a text vector corresponding to each text block.

[0011] Optionally, executing the query plan in a target vector database to obtain a target vector corresponding to the query vector includes:

[0012] executing a second query condition in the target vector database to determine a candidate vector related to the query vector;

[0013] Based on the length of the text block corresponding to each of the candidate vectors and the first query condition, a target vector corresponding to the query vector is determined, wherein the cumulative length of the text blocks corresponding to the target vectors is not less than the target text length.

[0014] Optionally, the second query condition represents a condition for screening candidate vectors corresponding to the query vector based on a relevance parameter between each text vector in the target vector database and the query vector.

[0015] Optionally, the second query condition represents a condition for screening candidate vectors corresponding to the query vector based on a relevance parameter between a text vector in the target vector database and the query vector determined based on a vector index.

[0016] Optionally, the vector index includes a graph index established for each text vector in the target vector database in the form of a graph, each node in the graph index represents one of the text vectors, and the edges between the nodes represent the similarity between the corresponding text vectors; executing the second query condition in the target vector database to determine the candidate vector related to the query vector includes:

[0017] Determining a correlation parameter between the corresponding text vector and the query vector based on a traversal order of the text vector corresponding to the graph index;

[0018] Based on the relevance parameter, a candidate vector is determined.

[0019] Optionally, the vector index includes an inverted index, the inverted index includes an index vector and an inverted file corresponding to the index vector, each of the inverted files includes a cluster center and a text vector corresponding to the cluster center; wherein executing the second query condition in the target vector database to determine a candidate vector related to the query vector includes:

[0020] Determining a first correlation parameter corresponding to the query vector and each cluster center based on the inverted index;

[0021] Determining candidate cluster centers based on the first correlation parameter;

[0022] A candidate vector is determined based on a second correlation parameter between the text vector corresponding to the candidate cluster center and the query vector.

[0023] An information query device, comprising:

[0024] Transmission module, processing module and target vector database;

[0025] The transmission module is used to receive a query vector and send the query vector to the processing module; the query vector is determined based on the query information;

[0026] The processing module is used to construct a query plan corresponding to the query vector; perform a query in the target vector database based on the query plan to obtain a target vector; wherein the query plan at least includes a first query condition of a target text length corresponding to a target query result;

[0027] The transmission module is further used to output the target query result corresponding to the target vector.

[0028] Optionally, the target vector database includes multiple text blocks, and a text block length and a text vector corresponding to each text block; wherein the target vector represents at least one text vector in the target vector database corresponding to the query vector, and the target query result represents result information determined based on the text blocks corresponding to each target vector. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0030] Figure 1 A flowchart of an information query method provided in an embodiment of the present application;

[0031] Figure 2 A flowchart of a method for obtaining a candidate vector based on an inverted index provided in an embodiment of the present application;

[0032] Figure 3 A schematic diagram of the structure of an information query device provided in an embodiment of the present application;

[0033] Figure 4 A schematic diagram of a target vector database provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] The terms "first" and "second" in this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0036] The embodiment of the present application provides an information query method for use in information retrieval application scenarios, and can specifically be applied to scenarios where a large model processes information, such as in RAG (Retrieval-Augmented Generation) scenarios. RGA retrieves relevant information from an external database and inputs the retrieval results as prompt information to the large model to enhance the ability of the large model to handle knowledge-intensive tasks. The information query method of the present application can reduce unnecessary computing overhead and improve the accuracy of information retrieval to improve the processing effect of the information retrieval results of the large model application.

[0037] See also Figure 1 , is a flow chart of an information query method provided in an embodiment of the present application, the method may include the following steps:

[0038] S101: Receive a query vector.

[0039] S102: Construct a query plan corresponding to the query vector.

[0040] S103: Execute the query plan in the target vector database to obtain a target vector corresponding to the query vector.

[0041] S104. Output the target query result corresponding to the target vector.

[0042] In an embodiment of the present application, the information query method can be executed by an information query device, which can be a device capable of accessing a target vector database, or a device storing the target vector database. Specifically, the information query device can be a terminal device capable of performing information retrieval of a target vector database, or a server or a server cluster. The target vector database is a database system for storing, retrieving and managing vector data, which includes vector information and text information corresponding to the vector information. The target vector database can support multiple types of data, such as text, images, videos, etc., which can be converted into vector form for storage after feature extraction.

[0043] In step S101, the query vector can be received by the transmission module of the information query device. Specifically, in order to facilitate the distinction of transmission information and efficient data transmission, the transmission module in the information query device may also include an input interface and an output interface, wherein the input interface receives the query vector, and the output interface can be used to output the query result information. The query vector is determined based on the query information. For example, in an intelligent question-answering application scenario, the query information may be a question statement input by a user. The query information is then converted into a vector format to obtain a query vector, and is input into the information query device through the input interface of the information query device.

[0044] In step S102, after obtaining the query vector, it is necessary to construct a query plan corresponding to the query vector, which represents the processing flow or a set of steps executed by the target vector database to complete the query. Specifically, the query plan may include all operations and steps for executing the query, such as the query flow, the query result determination flow, and the data output flow. In the embodiment of the present application, the target query result includes a first query plan for the target text length corresponding to the target query result. Among them, the target text length represents the total length threshold of the text block included in the target query result. The target text length can be determined based on the application scenario of the current information query, or the target text length can be determined according to the query information feature corresponding to the query vector. In one embodiment, the target text length corresponding to the target query result is determined based on the application feature of the query information. Among them, the application feature represents the application scenario feature corresponding to the query information, such as the application purpose of the query result corresponding to the query information. For example, the query result will be used as the prompt information of the large model so that the large model can output a more accurate processing result based on the prompt information. At this time, the target text length corresponding to the target query result can be determined according to the maximum number of tokens of the large model, wherein the token represents the text block read or written by the large model, and a token can be a character, a word, or a part of a word. That is, the target text length can be the total length of the maximum number of text blocks that the large model can read. Therefore, in the embodiment of the present application, the first query condition limits the total length of the text blocks included in the output target query result. Correspondingly, the query plan can also include a second query condition, which represents a condition for screening candidate vectors corresponding to the query vector, that is, the second query condition is a condition for screening data, that is, determining candidate vectors related to the query vector.

[0045] Then, in step S103, the query plan is executed in the target vector database to obtain the target vector corresponding to the query vector, that is, the target vector corresponding to the query plan is searched in the target vector database. The vectors related to the query vector are screened from the various vectors included in the target vector database, and the target vector that meets the target text length is determined according to the correlation between the screened candidate vectors and the query vector, and then the text block corresponding to the target vector is determined as the target query result. Specifically, after the candidate vectors are screened, the candidate vectors are sorted according to the similarity parameters between the candidate vectors and the query vector to obtain the most relevant candidate vectors, and according to the text block length corresponding to the most relevant candidate vector, when the cumulative length of the corresponding text block meets the target text length, the corresponding candidate vector is determined as the target vector. For example, the candidate vectors obtained by sorting according to the correlation with the query vector include: a first vector, a second vector, a third vector, a fourth vector, etc., wherein the text block length corresponding to the first vector is 3, the text block length corresponding to the second vector is 2, the text block length corresponding to the third vector is 2, the text block length corresponding to the fourth vector is 3, etc. If the target text length is set to 6, the target vector includes the first vector, the second vector and the third vector. Thereby, the text blocks corresponding to the above-mentioned target vectors are taken as the target query results, and the target query results are output through the output interface of the information query device. For example, the target query results can be output to the terminal device that generates the query information, or the target query results can be input to the next application layer, such as outputting the target query results to the large model as prompt information of the large model.

[0046] In an embodiment of the present application, in the process of determining the target vector corresponding to the query vector in the target vector database, the target vector is determined based on the set target text length, and there is no need to pre-set the corresponding number of filtered text blocks and the text similarity threshold. This can avoid problems such as high computational overhead or inaccurate query results caused by inappropriate threshold setting.

[0047] The information query method of the embodiment of the present application is described below in conjunction with the corresponding application scenario. In order to improve the efficiency of information query, the target vector database in the embodiment of the present application includes multiple text blocks, as well as the text block length and text vector corresponding to each text block. In this way, after determining the text vector corresponding to the query vector, the text block length of the text vector can also be obtained based on the information in the target vector database, which is convenient for the subsequent determination of whether the conditions corresponding to the target text length are met. Furthermore, in the case where the target vector database includes a vector index, the overhead of reading records during the vector query process is avoided. When constructing the vector index, in addition to the vector data, the vector index also includes the length of the text block corresponding to the vector. In this way, when the query obtains the related vector, the text block length of the vector can also be obtained, thereby reducing the data query overhead.

[0048] In one implementation of the embodiment of the present application, the query plan includes a first query condition of a target text length corresponding to a target query result, and also includes a second query condition for filtering candidate vectors corresponding to the query vector. The first query condition limits the cumulative length of the text block corresponding to the target vector finally filtered, and the second query condition limits the filtering condition for filtering candidate vectors related to the query vector.

[0049] Correspondingly, in one embodiment, executing a query plan in a target vector database to obtain a target vector corresponding to a query vector may include: executing a second query condition in the target vector database to determine candidate vectors related to the query vector; and determining a target vector corresponding to the query vector based on the length of a text block corresponding to each candidate vector and the first query condition.

[0050] Among them, the cumulative length of the text blocks corresponding to each target vector is not less than the target text length. In this embodiment, after receiving the query vector, the second query condition can be determined according to the data distribution information of each text vector in the target vector database. Among them, the data distribution information of the text vector is used to characterize the number of records and related contents of different types of text vectors. For example, the query vector characterization needs to query the data of a certain image type, then the second query condition can represent the query mainly on the text vector corresponding to the image type in the target vector database. Further, the second query condition can represent the condition for screening the candidate vector corresponding to the query vector based on the correlation parameter between each text vector in the target vector database and the query vector. That is, the correlation between each text vector in the target vector database and the query vector can be determined one by one. Correspondingly, in order to improve the screening efficiency, it is also possible to first determine the preliminary screening results with the query vector. At this time, the second query condition represents the condition for screening the candidate vector corresponding to the query vector based on the correlation parameter between the text vector in the preliminary screening results and the query vector.

[0051] Specifically, the preliminary filtering conditions can be determined based on the type of query information corresponding to the query vector, and then the preliminary filtering conditions are used to filter in the target vector database to obtain preliminary filtering results. The correlation parameter between each text vector in the preliminary filtering result and the query vector is calculated, and then the candidate vector corresponding to the query vector is determined based on the size of the correlation parameter. The correlation parameter can be obtained by a similarity calculation method, for example, by determining the correlation parameter between the query vector and the text vector through the cosine similarity between the two; the correlation parameter can also be determined by the Euclidean distance, Manhattan distance, etc. of the two vectors.

[0052] After determining the relevance parameters of each candidate vector and the query vector, each candidate vector can be sorted according to the relevance parameters, that is, sorted in descending order of relevance to the query vector. If the candidate vectors include the first text vector, the second text vector, the third text vector, the fourth text vector, etc., the first text vector is most relevant to the query vector, and the second text vector is second. Obtain the length of the first text block corresponding to the first text vector. If the length of the first text block is less than the preset target text length, obtain the length of the second text block corresponding to the second text vector. If the sum of the lengths of the first text block length and the second text block length is still less than the target text length, obtain the length of the third text block corresponding to the third text vector. If the sum of the lengths of the first text block length, the second text block length and the third text block length is greater than the target text length, the first text vector, the second text vector and the third text vector are used as the target vector; the first text block of the first text vector, the second text block of the second text vector and the third text block of the third text vector are used as the target query result.

[0053] It can be seen that in this embodiment, there is no need to pre-set a similarity threshold or a sorting threshold, etc. The target query result corresponding to the target vector with a higher relevance can be output according to the length of the target text, thereby avoiding the problem of excessive computational overhead or inaccurate query caused by improper threshold setting.

[0054] In another implementation of the embodiment of the present application, the second query condition represents a correlation parameter between a text vector in a target vector database and a query vector determined based on a vector index, and a condition for screening candidate vectors corresponding to the query vector. In this implementation, vector queries are performed based on vector indexes and a preset target text length. In order to improve the efficiency of vector queries, the length of the text block corresponding to each text vector can be set in the vector index of the target vector database, so that in the process of querying based on the query vector, the text block length of each text vector can be obtained to further determine whether the cumulative length of the text block of the candidate vector meets the condition corresponding to the target text length.

[0055] The vector index in the target vector database is a data structure used to quickly retrieve vectors similar to the query vector in a high-dimensional space. Correspondingly, the vector index can be determined based on different data structures, such as vector indexes including inverted indexes, tree indexes, graph indexes, etc. The information query method based on the target text length provided in the embodiment of the present application can be applied in the process of querying with different vector indexes.

[0056] In one implementation of the embodiment of the present application, the vector index includes a graph index established for each text vector in the target vector database in the form of a graph, each node in the graph index (also referred to as a bitmap index) represents a text vector, and the edges between the nodes represent the similarity between the corresponding text vectors, that is, there are edges between nodes with an associated relationship. When performing a vector query, the query vector is queried in the graph composed of the index data. Correspondingly, the second query condition is executed in the target vector database, and the processing process of determining the candidate vector related to the query vector may include: determining the correlation parameter between the corresponding text vector and the query vector based on the traversal order of the text vector corresponding to the graph index; and determining the candidate vector based on the correlation parameter.

[0057] Specifically, the graph index can be turned on, the initial value of the counter can be set to 0, and then the graph index can be traversed. The initial query resource can be determined in the target vector database according to the query requirements corresponding to the query vector. For example, the relevant vector queue for storing candidate vectors can be determined according to the bitmap index of the historical access record. For example, the record queue with the smallest distance from the query vector, the candidate record priority queue, the most recently accessed record queue, etc. can be set. Then, the vector index (i.e., graph index) can be traversed, and a piece of data can be obtained from the vector index through the query interface in the transmission module of the information query device, and the distance between the text vector in the data and the query vector can be determined. The similarity between the text vector and the query vector can be determined through the distance. The longer the distance, the lower the similarity between the two.

[0058] Then, the graph index is traversed according to the rules corresponding to the graph index. Each time a new node in the graph index is accessed, the distance between it and its corresponding neighbor nodes that have not been accessed and the query vector is calculated. Among them, when determining a neighbor node, the corresponding neighbor node can be determined according to the association relationship of the nodes in the graph index (i.e., the edge of the node). For example, the neighbor node of the current node can be determined according to the visited record bitmap. According to the distance between each node and the query vector, the entry of the corresponding node (wherein each entry corresponds to a row of data in the target database, specifically including the text vector, text block, text block length and other information of the node) is added to the corresponding node queue, such as adding it to the queue with the smallest distance. If the number of entries in the queue exceeds the first target number threshold (such as, the first target number threshold can be set to 5), the entry of the node with the largest distance from the query vector in the queue can be removed, and the maximum distance between each node in the queue and the query vector (such as represented by D1) is synchronously recorded to facilitate the screening of subsequent nodes.

[0059] If a query is performed in the recently visited nodes based on the graph index, the entries of the recently visited nodes can be added to the recently visited record queue according to the distance between the recently visited nodes and the query vector. If the number of entries corresponding to the nodes in the queue exceeds the second target number threshold (e.g., the second target number threshold can be set to 10), the entry corresponding to the node that was first added to the queue can be deleted, and after sorting, the distance between the node in the middle position of the queue and the query vector is recorded, such as represented by D2.

[0060] If D2 is greater than D1 before the traversal is completed, the counter is incremented by 1; otherwise, the counter is reset to 0. When the counter value is greater than the target threshold (such as the target threshold can be set to 30-80), the traversal can be stopped. If the entry returned by the query interface meets the query conditions, the entry of the corresponding node can be sorted by distance to obtain a candidate record queue.

[0061] Take out the entries of the corresponding nodes from the candidate record queue one by one in order, and accumulate the length of their text blocks, or accumulate the length of the specified field in their text blocks. When the sum of the lengths is less than the specified target text length, add the selected node entries to the result set. When the sum is greater than the target text length, the graph index can be closed, the relevant resources can be released, and the query results can be returned. If the sum of the lengths of the specified fields of the text blocks corresponding to the entries of all nodes in the candidate record queue is still less than the target text threshold, continue to traverse the graph index, obtain the distance between each node and the query vector, repeat the above steps, and obtain the final output result.

[0062] In the process of iterative calculation of vector distance based on graph index in the embodiment of the present application, a priority queue (such as the candidate record queue described in the above embodiment) can be maintained. The priority queue stores the distance calculation results of the vectors involved in the calculation in the sub-vector space and the query vector according to the size of the distance. After each round of distance iterative calculation, one or more vectors closest to the query vector are selected, and a predetermined number of neighboring vectors (for example, K) of the vector are used as the target of the next round of distance iterative calculation. After each round of distance iterative calculation, the priority queue will be dynamically adjusted to always put the vector closest to the query vector at the front. In this way, the most relevant vector corresponding to the query vector can be automatically selected and output according to the length of the text block corresponding to each vector in the priority queue, saving computing resources and improving the accuracy of the query. It should be noted that the target text length pre-set in the embodiment of the present application can refer to the total length of the text block corresponding to the text vector, or it can be the cumulative value of the length of the specified field in the corresponding text block. Specifically, it can be determined according to the actual application scenario.

[0063] In another implementation of the embodiment of the present application, the vector index includes an inverted index, the inverted index includes an index vector and an inverted file corresponding to the index vector, each inverted file includes a cluster center and a text vector corresponding to the cluster center; wherein the second query condition is executed in the target vector database to determine a candidate vector related to the query vector, see Figure 3 , the method for obtaining a candidate vector based on an inverted index comprises the following steps:

[0064] S201, determining a first correlation parameter corresponding to a query vector and each cluster center based on an inverted index;

[0065] S202, determining a candidate cluster center based on the first correlation parameter;

[0066] S203: Determine a candidate vector based on a second correlation parameter between the text vector corresponding to the candidate cluster center and the query vector.

[0067] For example, if the vector index is an inverted index, the processing flow can be as follows:

[0068] S301, open the vector index (i.e., inverted index), set the initial value of the counter to 0 (the counter is used to characterize the number of candidate vectors in the candidate record queue). In the case of traversal, the query resources in the target vector database can be allocated first, such as the query resources corresponding to the visited record bitmap, or the query resources corresponding to the candidate record priority queue. Calculate the distance between the query vector and the cluster center of the corresponding resource (specifically, it can be the cluster center point of each cluster center), use the distance as the first relevance parameter, and sort the cluster center points from small to large based on the distance.

[0069] S302, traverse the vector index, and obtain the entry data corresponding to each text vector and the distance to the query vector from the vector index through the query interface. The query interface first takes out the cluster center in order, and then takes out the entries corresponding to the text vector one by one from the text vector contained in the cluster center in the order of the distance between the text vector and the query vector from small to large. If the text vector has been accessed and meets the query conditions, the distance between the text vector and the query vector is calculated and added to the candidate record queue, which represents a queue sorted by vector distance.

[0070] S303: If the number of text vectors in the candidate record queue exceeds the target number threshold, stop traversal.

[0071] S304. If the condition for stopping the traversal is not met, that is, the number of text vectors in the candidate record queue does not exceed the target number threshold, then continue to execute S302. If the traversal is stopped, the text vectors are taken out one by one in the order of the text vectors recorded in the candidate record queue. If the target text length represents the length of the text block, the text block length of the taken text vector is cumulatively calculated. If the cumulative length is less than the target text length, the corresponding text vectors are continued to be taken out from the candidate record queue. If the cumulative length is greater than the target text length, the vector index is closed, the relevant resources are released, and the query results are returned. If the cumulative length of each text vector in the entire candidate record queue is less than the target text length, then continue to execute S302 to obtain the corresponding text vector.

[0072] It should be noted that the target text length can also guarantee the length of the specified field in the text block. In this case, the cumulative length of the specified fields in the text blocks of the extracted text vector can be calculated. If the cumulative length of the specified fields of each text block is greater than the target text length, these text blocks will be output as query results.

[0073] In the embodiment of the present application, when querying the query vector in the target vector database, vector streaming query logic can be added to the target vector database. When the streaming query end condition is met, for each record of the query iteration (the record includes the text vector in the target vector database), the length of its text block, or the length of the specified field in the text block, is accumulated. When the accumulated value exceeds the target text length, the text blocks of all records found, or the text of the relevant fields are fed back. When the accumulated value is less than the specified threshold, the next record is iterated. Until the target text length is met. Based on the target text length in the extended query interface, RAG can directly obtain the text block with the closest vector distance that meets the text length limit, without setting the sorting threshold or similarity threshold and the parameters of the text block in advance. There is no unnecessary computational overhead (such as when the sorting threshold topk or the similarity threshold range search is set too large, the computational overhead is large), which also ensures that the RAG question-answering effect is good, avoiding the problem of asking poor results due to insufficient data when the sorting threshold or similarity threshold is set too small.

[0074] See also Figure 3 In an embodiment of the present application, an information query device is also provided, and the information query device includes: a transmission module 401, a processing module 402 and a target vector database 403.

[0075] The transmission module 401 is used to receive the query vector and send the query vector to the processing module; the query vector is determined based on the query information;

[0076] The processing module 402 is used to construct a query plan corresponding to the query vector; based on the query plan, a query is performed in the target vector database 403 to obtain the target vector; wherein the query plan at least includes a first query condition of a target text length corresponding to the target query result;

[0077] The transmission module 401 is further used to output the target query result corresponding to the target vector.

[0078] Optionally, the target vector database 403 includes multiple text blocks, and a text block length and a text vector corresponding to each text block; wherein the target vector represents at least one text vector corresponding to the query vector in the target vector database 403, and the target query result represents the result information determined based on the text blocks corresponding to each target vector.

[0079] Further, in order to enable efficient transmission of information, the transmission module 401 may include an input interface, an output interface and a query interface, wherein the input interface is used to receive a query vector, and the output interface is used to output a target query result. The query interface is used to interact with the target vector database for data, such as the query interface can return the target vector corresponding to the query plan in the target vector database. In an embodiment of the present application, an extended query interface can be implemented, that is, a field representing the target text length is added to the query field transmitted in the query interface, so that in the process of querying, the text block of the target vector corresponding to the query vector can be determined based on the target text length, thereby outputting the query result. Some existing databases currently support interfaces for mixed queries of vectors and variables (such as taking the topk interface as an example), which can return the topk records closest to the query vector that meet the specified filtering conditions (the filtering conditions represent the conditions of the text vector that the query vector needs to obtain), as well as some field data or statistical results of these records (such as summing a certain field). Then a threshold of k is set, and k related vectors are output. However, if k is set larger, computing resources will be increased, and if k is set smaller, query accuracy will be affected.

[0080] In the embodiment of the present application, the query field in the query interface can be expanded to add a field that limits the length of the target text. For example, taking the topk query interface as an example, the parameters of the query interface may include: table name (DataTable), partition name (can be empty), scalar query expression (such as expr,lang='chinese'), query vector (xq), output domain (text domain, text partition), etc. The current interface can return the topk records closest to the query vector that meet the specified filtering conditions.

[0081] The extended query interface adds the field name for calculating the sum of text lengths (such as textlen) and the sum limit parameter (sum_limit). Correspondingly, the extended query interface can be as follows:

[0082] Search(collection(data table - vector database)_name, partition(a partition in the data table)_name, filter(filter condition)_expression, xq(query vector), textlen(calculated sum of a field), sum_limit(sum limit), output_fields(output field)……).

[0083] When performing a query, you can parse the query plan, obtain the restriction conditions for the target text length, sum the specified fields corresponding to the target text length, and verify whether the cumulative value of the specified field matches the preset length threshold. In addition, a vector calculation streaming query mode is added to provide a streaming query interface for the vector domain and a method for determining query results. Further, a cost estimator for vector streaming queries can be added according to application requirements, so as to select a query plan, vector index form, etc. suitable for the current query, thereby improving query efficiency and accuracy.

[0084] Specifically, the processing process of executing the query plan to obtain the target query result can be found in the description of the aforementioned information query method, which will not be described in detail here.

[0085] See also Figure 4 In an embodiment of the present application, a target vector database is also provided, and the target vector database may include a data file 501 and an index file 502, wherein the data file 501 stores the data content in the vector data. Specifically, in an embodiment of the present application, the data file may store a text vector and a text block corresponding to the text vector. The index file 502 is a data structure file used to accelerate vector data retrieval. The index file can be used to quickly locate the text vector related to the query vector in the data file, thereby achieving efficient data retrieval. Specifically, the data file may include fields such as text vectors, text blocks, data partitions, and data storage formats; the index file includes a vector index, and the vector index includes, in addition to the vector data, the length of the text block corresponding to the vector, so that the text block length of the corresponding vector and the cumulative text length with other vectors can be quickly calculated, thereby improving query efficiency.

[0086] Correspondingly, the process of performing a query in the target vector database based on the query plan can refer to the aforementioned information query method, which will not be described in detail here.

[0087] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0088] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0089] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0090] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An information query method, comprising: receiving a query vector, wherein the query vector is determined based on the query information; Constructing a query plan corresponding to the query vector, wherein the query plan at least includes a first query condition of a target text length corresponding to a target query result; Execute the query plan in a target vector database to obtain a target vector corresponding to the query vector; Output the target query result corresponding to the target vector.

2. According to the information query method of claim 1, the target text length corresponding to the target query result is determined based on the application characteristics of the query information, and the query plan also includes a second query condition for screening candidate vectors corresponding to the query vector.

3. According to the information query method according to claim 2, the target vector database includes multiple text blocks, and the text block length and text vector corresponding to each text block.

4. The information query method according to claim 3, wherein executing the query plan in a target vector database to obtain a target vector corresponding to the query vector comprises: executing a second query condition in the target vector database to determine a candidate vector related to the query vector; Based on the length of the text block corresponding to each of the candidate vectors and the first query condition, a target vector corresponding to the query vector is determined, wherein the cumulative length of the text blocks corresponding to the target vectors is not less than the target text length.

5. According to the information query method of claim 3, the second query condition represents a condition for screening candidate vectors corresponding to the query vector based on a correlation parameter between each text vector in the target vector database and the query vector.

6. According to the information query method of claim 3, the second query condition represents a condition for screening candidate vectors corresponding to the query vector based on a correlation parameter between the text vector in the target vector database and the query vector determined based on a vector index.

7. According to the information query method of claim 6, the vector index includes a graph index established for each text vector in the target vector database in the form of a graph, each node in the graph index represents one of the text vectors, and the edges between the nodes represent the similarity between the corresponding text vectors; The performing a second query condition in the target vector database to determine a candidate vector related to the query vector includes: Determining a correlation parameter between the corresponding text vector and the query vector based on a traversal order of the text vector corresponding to the graph index; Based on the relevance parameter, a candidate vector is determined.

8. According to the information query method of claim 6, the vector index includes an inverted index, the inverted index includes an index vector and an inverted file corresponding to the index vector, each of the inverted files includes a cluster center and a text vector corresponding to the cluster center; wherein, The performing a second query condition in the target vector database to determine a candidate vector related to the query vector includes: Determining a first correlation parameter corresponding to the query vector and each cluster center based on the inverted index; Determining candidate cluster centers based on the first correlation parameter; A candidate vector is determined based on a second correlation parameter between the text vector corresponding to the candidate cluster center and the query vector.

9. An information query device, comprising: Transmission module, processing module and target vector database; The transmission module is used to receive the query vector and send the query vector to the processing module; The query vector is determined based on the query information; The processing module is used to construct a query plan corresponding to the query vector; Based on the query plan, a query is performed in the target vector database to obtain a target vector; wherein the query plan at least includes a first query condition of a target text length corresponding to a target query result; The transmission module is further used to output the target query result corresponding to the target vector.

10. The information query device according to claim 9, wherein the target vector database comprises a plurality of text blocks, and a text block length and a text vector corresponding to each text block; wherein, The target vector represents at least one text vector in the target vector database corresponding to the query vector, and the target query result represents result information determined based on the text blocks corresponding to each target vector.

Images