Information search method and device, electronic equipment and storage medium

Abstract

The invention provides an information search method and device, electronic equipment and a storage medium, and relates to the technical field of data processing, in particular to the technical field of vector retrieval. According to the specific implementation scheme, a search vector is obtained; performing hierarchical navigation worldlet HNSW search on the search vector until a bottom map of the HNSW is entered; multi-level search is carried out on the underlying graph, search of the next level is carried out when search of each level meets the local end condition of the underlying graph, search is ended until the global end condition is met, at least one target vector corresponding to a search vector is obtained, and searched nodes corresponding to each level do not have intersection.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, specifically to the field of vector retrieval technology, and in particular to an information search method, device, electronic device and storage medium. Background Art

[0002] Hierarchical Navigable Small World graphs (HNSW) search is used to perform vector similarity retrieval, but the performance of HNSW search is poor in extreme filtering scenarios.

[0003] Existing technologies often switch between HNSW search and brute-force search by estimating the selectivity. However, the selectivity estimation-based approach may incorrectly apply HNSW and brute-force search methods in some scenarios, resulting in a decrease in the overall recall rate and speed of the algorithm. Summary of the invention

[0004] The present disclosure provides a method, device, electronic device and storage medium for information search.

[0005] According to one aspect of the present disclosure, there is provided an information search method, comprising: obtaining a search vector; performing a hierarchical navigation small world HNSW search on the search vector until entering the underlying graph of the HNSW; performing a multi-level search on the underlying graph, and performing a search on the next level when the search at each level satisfies its own local end condition, until the search is terminated by satisfying the global end condition, and obtaining at least one target vector corresponding to the search vector, wherein there is no intersection between the searched nodes corresponding to each level.

[0006] According to another aspect of the present disclosure, there is provided an information search device, including: an acquisition module, for acquiring a search vector; a first search module, for performing a hierarchical navigation small world HNSW search on the search vector until entering the underlying graph of the HNSW; a second search module, for performing a multi-level search on the underlying graph, and performing a search on the next level when the search at each level meets its own local end condition, until the search is terminated by meeting the global end condition, and obtaining at least one target vector corresponding to the search vector, wherein there is no intersection between the searched nodes corresponding to each level.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the information search method described in the above-mentioned one aspect embodiment.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, on which a computer program / instructions are stored, and the computer instructions are used to enable the computer to execute the information search method described in the above-mentioned embodiment.

[0009] According to another aspect of the present disclosure, a computer program product is provided, including a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the information search method described in the above-mentioned first embodiment is implemented.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 A flowchart of an information search method provided by an embodiment of the present disclosure;

[0013] Figure 2 A flowchart of another information search method provided by an embodiment of the present disclosure;

[0014] Figure 3 A flowchart of another information search method provided by an embodiment of the present disclosure;

[0015] Figure 4 A schematic flow chart of a process of determining a number threshold in an information search method provided in an embodiment of the present disclosure;

[0016] Figure 5 A schematic diagram of the structure of an information search device provided by an embodiment of the present disclosure;

[0017] Figure 6 The present invention is a block diagram of an electronic device for implementing the information search method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0019] The following describes the information search method, device and electronic device according to the embodiments of the present disclosure with reference to the accompanying drawings.

[0020] Figure 1 A flowchart of an information search method provided in an embodiment of the present disclosure.

[0021] like Figure 1 As shown, the information search method may include:

[0022] S101, obtaining a search vector.

[0023] It should be noted that the execution subject of the information search method in the embodiment of the present disclosure may be a hardware device with data processing capabilities and / or the necessary software required to drive the hardware device to work. Optionally, the execution subject may include a server, a user terminal and other intelligent devices. Optionally, the user terminal includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, etc. Optionally, the server includes but is not limited to a network server, an application server, and may also be a server of a distributed system, or a server combined with a blockchain, etc. The embodiment of the present disclosure is not specifically limited.

[0024] In some implementations, a search vector may be generated based on query information input by a user, such as text, images, and other information. The search vector may be obtained by vectorizing the input query information. Alternatively, the query information may be directly converted into a search vector, or key features of the query information may be extracted and converted into a search vector.

[0025] S102, performing a hierarchical navigation small world HNSW search on the search vector until entering the bottom layer graph of the HNSW.

[0026] It is understandable that the Hierarchical Navigable Small Worldgraphs (HNSW) search adopts a hierarchical structure, where the top layer has the longest edges for fast search, while the lower layers have shorter edges for accurate search. This hierarchical structure enables the algorithm to balance search speed and accuracy at different levels.

[0027] When searching for a vector, we first start from the highest layer and find the node that is most similar to the query vector. Then we gradually move down to lower layers, narrowing the search scope layer by layer. As the search process progresses, we will eventually enter the bottom layer of HNSW. In the bottom layer, the edges between nodes are shorter, indicating that the similarity between vectors is more precise.

[0028] That is, by performing HNSW search on the search vector, the search vector is searched layer by layer from the highest layer downwards in the hierarchical structure of the HNSW until entering the bottom layer graph of the HNSW.

[0029] S103, perform a multi-level search on the underlying graph, and perform a search on the next level when the search at each level meets its own local end condition, until the search ends when the global end condition is met, and obtain at least one target vector corresponding to the search vector, wherein there is no intersection between the searched nodes corresponding to each level.

[0030] In some implementations, after entering the underlying graph, the search can be divided into multiple levels based on the set local end conditions, and then the search vector can be searched at multiple levels in the underlying graph. When each level meets its own local end conditions, the search can be entered into the next level until the global end conditions are met to end the search and obtain the target vector.

[0031] Optionally, multi-level search in the underlying graph can be implemented by calculating the distance between the nodes in the underlying graph and the nodes where the search vector is located. That is, by calculating the distance between the nodes in the underlying graph and the nodes where the search vector is located, vectors corresponding to multiple nodes with smaller distances can be selected as target vectors.

[0032] Optionally, the local end condition may be determined based on a threshold value of the number of searches at each level, that is, the local end condition may be determined based on a threshold value of the number of distance calculations.

[0033] For example, three levels of search can be performed in the underlying graph, where the local end condition for the first level is that the search times reach 1000 times; the local end condition for the second level is that the search times reach 500 times; and the local end condition for the third level is that the search times reach 200 times. If the global end condition is that the search times reach 1600 times, the search ends when the third level searches 100 times, and the vectors corresponding to the one or more nodes obtained by the search are used as the target vector.

[0034] In some implementations, when performing a search at any level, a candidate node queue and a target node queue may be determined based on the distance between the search vector and the nodes in the underlying graph. The nodes in the target node queue are the top-ranked nodes in the candidate node queue. Optionally, the nodes may be sorted based on the distance between the search vector and the nodes. That is, the distance between the nodes in the target node queue and the search vector is less than the distance between the nodes in the candidate node queue and the search vector.

[0035] In some implementations, when performing a multi-level search, searched nodes may be marked so that the searched nodes will not be searched again. Alternatively, a node pool may be constructed during the search process, the searched nodes may be added to the node pool, and the node pool may be transferred to the next level so that the next level does not search the searched nodes multiple times when performing a search.

[0036] According to the information search method provided by the embodiment of the present disclosure, a HNSW search is performed on the search vector, and a multi-level search is performed on the underlying graph when entering the HNSW. When the search at each level meets its own local end condition, the search at the next level is performed until the global end condition is met and the search is terminated, and at least one target vector corresponding to the search vector is obtained. By performing a hierarchical search on the underlying graph of HNSW, the search speed of the HNSW search in the extreme filtering scenario can be improved, and the recall rate of the HNSW search can be improved, thereby improving the query efficiency and search quality of the user, and further improving the user's search experience.

[0037] Figure 2 A flowchart of an information search method provided in an embodiment of the present disclosure.

[0038] like Figure 2 As shown, the information search method may include:

[0039] S201, obtaining a search vector.

[0040] S202, performing a hierarchical navigation small world HNSW search on the search vector until entering the bottom layer graph of the HNSW.

[0041] The relevant contents of steps S201 - S202 can be found in the above embodiment and will not be described again here.

[0042] S203, performing an approximate nearest neighbor search at the i-th level, and determining the number of distance calculations in the i-th level search process.

[0043] In some implementations, an approximate nearest neighbor search can be performed on the nodes in the underlying graph of HNSW, and multi-level search can be implemented by calculating the distances between the nodes. That is, for the i-th level, an approximate nearest neighbor search can be performed on the nodes in the i-th level to determine the number of distance calculations in the search process of the i-th level.

[0044] In some implementations, the amount of computation can be reduced and the search speed can be increased by performing an approximate nearest neighbor search on nodes that have not been searched in any level. Alternatively, by determining nodes that have not been searched in the underlying graph when entering the i-th level, and determining nodes that need to be searched in the i-th level from the nodes that have not been searched in the underlying graph, an approximate nearest neighbor search can be performed on the nodes that need to be searched in the i-th level based on the search vector.

[0045] That is, the distance between the search vector and the node to be searched can be calculated, and the i-th level search result can be determined according to the distance. Optionally, when calculating the distance between the search vector and the node, the calculated distances can be sorted, and search results can be generated according to the nodes with the highest sorting results. The search results are the candidate node queue and the target node queue obtained according to the distance.

[0046] S204, in response to the number of distance calculations reaching the number threshold of the i-th level, entering the i+1-th level search, wherein the local end condition is that the number of distance calculations reaches the number threshold of the i-th level.

[0047] In some implementations, in order to save computational effort and avoid unnecessary computation leading to waste of resources, the determination of whether to perform the next level of search may be made based on the number of computations. Alternatively, the determination of whether to perform the next level of search may be made based on a threshold of the number of distance computations.

[0048] In some implementations, if the number of distance calculations at the i-th level reaches the number threshold of the i-th level, the search at the i+1-th level may be entered. In other words, the local termination condition of the search at each level is that the number of distance calculations reaches the number threshold of the i-th level.

[0049] In some implementations, when performing an approximate nearest neighbor search on a node, the searched nodes may be marked so that when performing a search on the nodes at the next level, the nodes that have not been searched and the nodes that have been searched may be clearly identified.

[0050] That is, before entering the search of the i+1th level, the nodes that have been searched at the i-th level are determined, and the nodes that have been searched at the i-th level are marked, and the nodes that have not been searched in the underlying graph are updated according to the marked searched nodes at the i-th level. For example, the searched nodes can be marked as B, and the unsearched nodes can be marked as A to distinguish whether the grounding point has been searched.

[0051] Optionally, a node pool can be generated based on the nodes in the underlying graph, and when searching at each level, the node pool is updated based on the node markings, and each level inherits the node pool of the previous level. The node pool includes marked searched nodes and unsearched nodes, so that the next level will not search for nodes searched by the previous level, saving computational effort and time.

[0052] S205, ending the search until a global end condition is met, and obtaining at least one target vector corresponding to the search vector, wherein there is no intersection between the searched nodes corresponding to each level.

[0053] The relevant contents of step S205 can be found in the above embodiment and will not be described again here.

[0054] According to the information search method provided by the embodiment of the present disclosure, by performing HNSW search on the search vector, and in the underlying graph entering the HNSW, for the i-th level, performing an approximate nearest neighbor search through the i-th level, and entering the i+1-th level search when the i-th level meets the local end condition, a multi-level search of the underlying graph is implemented until the global end condition is met and the search is ended, and at least one target vector corresponding to the search vector is obtained. Based on the nodes that have been searched and the nodes that have not been searched at each level, the distance calculation is performed on the nodes in each level, which can improve the search efficiency and save the search time.

[0055] Figure 3 A flowchart of an information search method provided in an embodiment of the present disclosure.

[0056] like Figure 3 As shown, the information search method may include:

[0057] S301, obtaining a search vector.

[0058] S302, performing a hierarchical navigation small world HNSW search on the search vector until entering the bottom layer graph of the HNSW.

[0059] S303, performing an approximate nearest neighbor search at the i-th level.

[0060] The relevant contents of steps S301 - S303 can be found in the above embodiment and will not be described again here.

[0061] S304, determining the nodes in the candidate node queue and the nodes in the target node queue when entering the i-th level.

[0062] In some implementations, the candidate node queue and target node queue when entering the i-th level can be determined based on the search results of the i-1th level, and the nodes in the candidate node queue and the nodes in the target node queue can be obtained from the candidate node queue and the target node queue.

[0063] S305 , during the search process at the i-th level, the nodes in the candidate node queue are updated based on the distance between the search vector and the node j to be searched at the i-th level.

[0064] In some implementations, during the search process at the i-th level, the node that has not been searched at the i-1th level can be determined based on the node pool at the i-1th level as the node j to be searched at the i-th level. The distance between the search vector and the node j can then be calculated, and the nodes in the candidate node queue can be updated based on the distance between the search vector and the node j to optimize the search results, thereby improving the search quality.

[0065] In some implementations, the nodes in the candidate node queue are updated by comparing the distance corresponding to node j with the distance corresponding to node k in the candidate node queue. In response to the distance corresponding to node j being less than the distance corresponding to node k in the candidate node queue, node j is added to the candidate node queue according to the distance corresponding to node j.

[0066] In some implementations, in response to there being surplus node vacancies in the candidate node queue, node j can be directly added to the candidate node queue. In response to the candidate node queue being full, the node with the largest distance is deleted from the candidate node queue, and node j is added to the candidate node queue based on the distance corresponding to node j, so as to ensure that the distance corresponding to the node in the candidate node queue is less than the distance corresponding to the node not in the node queue, so that the search results are more similar to the search vector, thereby ensuring the search quality.

[0067] S306: Update the nodes in the target node queue according to the updated nodes in the candidate node queue.

[0068] In some implementations, the distances corresponding to the updated nodes in the candidate node queue are obtained and sorted based on the distances to obtain a sorting result of the distances corresponding to the nodes from small to large, and the nodes in the target node queue can be updated according to the sorting result. In other words, the nodes in the target node queue can be re-determined according to the order of the updated nodes in the candidate node queue.

[0069] Optionally, the top k nodes may be selected from the sorting results and used as nodes in the target node queue, thereby improving the relevance of the nodes in the target node queue with the search vector and further improving the accuracy and quality of the search.

[0070] S307, determining the number of distance calculations in the i-th level search process.

[0071] S308, in response to the number of distance calculations reaching the number threshold of the i-th level, entering the search at the i+1-th level, wherein the local end condition is that the number of distance calculations reaches the number threshold of the i-th level.

[0072] S309, ending the search until a global end condition is met, and obtaining at least one target vector corresponding to the search vector, wherein there is no intersection between the searched nodes corresponding to each level.

[0073] The relevant contents of steps S307-S309 can be found in the above embodiment and will not be repeated here.

[0074] In some implementations, if the search at the i+1th level is the last level search, indicating that the first i levels failed to search for a node that matches the search vector, the search method can be switched to perform a K nearest neighbor search on the unsearched nodes to determine the final search result, thereby improving the search efficiency and reducing the search time. The final search result is the target vector corresponding to the search vector.

[0075] That is, in response to entering the final level search, the last remaining unsearched nodes on the underlying graph are used to determine the nodes that need to be searched at the final level, and a K nearest neighbor search is performed among the nodes that need to be searched at the final level based on the search vector to update the nodes in the target node queue.

[0076] In some implementations, the number of times the distance is calculated during the final-level search reaching a corresponding threshold number of times can be used as a global termination condition. In response to satisfying the global termination condition, the search is terminated, and a vector corresponding to the node in the target node queue when the search is terminated is determined as at least one target vector corresponding to the search vector.

[0077] According to the information search method provided by the embodiment of the present disclosure, a HNSW search is performed on the search vector, and a multi-level search is performed on the underlying graph in the underlying graph entering the HNSW. By obtaining the candidate node queue and the target node queue at each level of search, and updating the nodes in the node queue, and when the local end condition of the node is met, the next level of search is performed until the global end condition is met to end the search, and at least one target vector corresponding to the search vector is obtained. By updating the nodes in the candidate node queue and the target node queue according to the distance corresponding to the node, the accuracy of the search can be improved, the relevance of the search structure can be optimized, and the search quality can be improved.

[0078] Based on the above embodiments, the present disclosure can explain the process of determining the number threshold, such as: Figure 4 As shown, the process of determining the number threshold may include:

[0079] S401 , obtaining vector dimension, HNSW indexing parameters and the total number of vectors in the HNSW index as threshold influencing parameters.

[0080] It can be understood that the vector dimension refers to the number of features of the vector; the HNSW indexing parameters include the size of the nearest neighbor list used during construction, the maximum number of connections for each node, the number of index levels, etc.; the total number of vectors in the HNSW index refers to the number of vectors contained in the index.

[0081] Optionally, the vector dimension, the HNSW indexing parameter and the total number of vectors in the HNSW index may be determined from the configuration information when the HNSW search is established, and used as the threshold influencing parameters.

[0082] S402: Determine search configuration information of a first object input associated with a search vector, wherein the search configuration information at least includes the number of target vectors, a search timeout period, and a node filtering condition.

[0083] In some implementations, the search configuration information may be determined based on the search habits of the first object. Optionally, the attribute information, feature information, etc. of the first object may be used as the first description information, and the search configuration information may be determined based on the first description information. The search configuration information is determined based on the first object, and the number threshold is adjusted based on the search configuration information, so that the number threshold in the search process can be more personalized to meet the user's search experience.

[0084] In some implementations, by determining the first object associated with the search vector and determining the first description information of the first object, the description information of other objects can be obtained from the stored search configuration information, and then the search configuration information is determined by combining the first description information and the description information of the other objects. The other objects can be the second objects.

[0085] That is, by obtaining the second description information of the second object associated with the pre-stored candidate search configuration information, and according to the first description information and the second description information, the search configuration information input by the first object is obtained from the candidate search configuration information. Optionally, the candidate search configuration information can be obtained from the search configuration information library to further determine the description information.

[0086] Optionally, the search configuration information input by the first object can be determined from the candidate search configuration information based on the matching degree between the first description information and the second description information. The second description information with the highest matching degree with the first description information can be selected, and the candidate search configuration information corresponding to the second description information is the search configuration information input by the first object.

[0087] In some implementations, when obtaining search configuration information based on the first description information, in response to not obtaining the search configuration information available for the first object from the candidate search configuration information, global search configuration information is obtained as the search configuration information input for the first object, so that the HNSW search can smoothly perform multi-level search and avoid the situation where the search cannot be performed due to the absence of user configuration information.

[0088] S403: Determine the number threshold corresponding to each level according to the threshold impact parameter and the search configuration information.

[0089] In some implementations, the initial number threshold corresponding to each level can be determined based on the threshold influencing parameters, and then the initial number threshold can be adjusted according to information such as the number of target vectors, search timeout time and node filtering conditions in the search configuration information to determine the number threshold corresponding to each level.

[0090] In some implementations, in order to save search time and improve the user's search experience, the search can be monitored in real time after the search is performed, so as to adjust the number threshold according to the monitoring result. The multi-level search situation of the first object on the underlying graph is monitored, and the multi-level search situation is analyzed according to the monitoring result, and the number threshold corresponding to at least part of the levels of the first object is adjusted according to the analysis result.

[0091] For example, if the search level of the first object always falls on the third level at the end of the search, then you can consider lowering the thresholds of the first and second levels to allow the first object to quickly enter the third level. For another example, if the number of searches for the first object always slightly exceeds the number threshold corresponding to the first level, and it returns after a very short search after entering the second level, and there is still a lot of margin in the search time, then you can consider raising the number threshold of the first level to allow the first object to search multiple times in the first level to obtain more accurate search results.

[0092] In some implementations, the number threshold can also be adjusted according to the resource information of the search server. By adjusting the number threshold, it is possible to ensure that the search server allocates resources reasonably and avoids excessive consumption or idleness of resources, thereby improving the resource utilization of the server and reducing resource waste.

[0093] Optionally, by monitoring the available resources of the search server, responding that the available resources are sufficient, a supplementary search is performed based on the cached supplementary search information to update the corresponding number threshold of each level. That is, when the available resources of the search server are greater than the set threshold, it is determined that the available resources are sufficient, and the number threshold can be increased to achieve the update of the number threshold.

[0094] According to the information search method provided by the embodiment of the present disclosure, by obtaining the threshold influencing parameters in the search process and obtaining the search configuration information of the input of the first object associated with the search vector, the number threshold corresponding to each level is determined according to the threshold influencing parameters and the search configuration information, so that the number threshold in the search process is more personalized, so that the search process can be based on actual conditions, thereby optimizing the search experience.

[0095] Corresponding to the information search methods provided in the above-mentioned embodiments, an embodiment of the present disclosure also provides an information search device. Since the information search device provided in the embodiment of the present disclosure corresponds to the information search methods provided in the above-mentioned embodiments, the implementation methods of the above-mentioned information search methods are also applicable to the information search device provided in the embodiment of the present disclosure and will not be described in detail in the following embodiments.

[0096] Figure 5 A schematic diagram of the structure of an information search device provided in an embodiment of the present disclosure.

[0097] like Figure 5 As shown, the information search device 500 of the embodiment of the present disclosure includes an acquisition module 501 , a first search module 502 and a second search module 503 .

[0098] An acquisition module 501 is used to acquire a search vector;

[0099] A first search module 502 is used to perform a hierarchical navigation small world HNSW search on the search vector until entering the bottom layer graph of the HNSW;

[0100] The second search module 503 is used to perform multi-level search on the underlying graph, and to perform search on the next level when the search at each level meets its own local end condition, until the search is terminated by meeting the global end condition, and obtain at least one target vector corresponding to the search vector, wherein there is no intersection between the searched nodes corresponding to each level.

[0101] In one embodiment of the present disclosure, the second search module 503 is also used to: perform an approximate nearest neighbor search at the i-th level, and determine the number of distance calculations in the search process at the i-th level; in response to the number of distance calculations reaching the number threshold of the i-th level, enter the search at the i+1-th level, where the local end condition is that the number of distance calculations reaches the number threshold of the i-th level.

[0102] In one embodiment of the present disclosure, the second search module 503 is also used to: determine the nodes that have not been searched in the underlying graph when entering the i-th level; determine the nodes that need to be searched in the i-th level from the nodes that have not been searched in the underlying graph; and perform an approximate nearest neighbor search for the nodes that need to be searched in the i-th level based on the search vector.

[0103] In one embodiment of the present disclosure, the second search module 503 is also used to: determine the nodes that have been searched at the i-th level, and mark the nodes that have been searched at the i-th level; and update the unsearched nodes in the underlying graph based on the marked searched nodes at the i-th level.

[0104] In one embodiment of the present disclosure, each level inherits the node pool of the previous level, and the node pool includes marked nodes that have been searched and nodes that have not been searched.

[0105] In one embodiment of the present disclosure, the second search module 503 is also used to: determine the nodes in the candidate node queue and the nodes in the target node queue when entering the i-th level; during the search process of the i-th level, update the nodes in the candidate node queue based on the distance between the search vector and the node j that needs to be searched at the i-th level; and update the nodes in the target node queue based on the updated nodes in the candidate node queue.

[0106] In one embodiment of the present disclosure, the second search module 503 is further used to: in response to the distance corresponding to node j being smaller than the distance corresponding to node k in the candidate node queue, add node j to the candidate node queue according to the distance corresponding to node j.

[0107] In one embodiment of the present disclosure, the second search module 503 is further used to: in response to the candidate node queue being full, delete the node with the largest distance from the candidate node queue, and add node j to the candidate node queue according to the distance corresponding to node j.

[0108] In one embodiment of the present disclosure, the second search module 503 is further used to: re-determine the nodes in the target node queue according to the updated order of the nodes in the candidate node queue.

[0109] In one embodiment of the present disclosure, the second search module 503 is also used for: in response to currently entering the final-level search, determining the nodes that need to be searched at the final level from the last remaining unsearched nodes on the underlying graph, and performing a K-nearest neighbor search in the nodes that need to be searched at the final level based on the search vector to update the nodes in the target node queue; in response to ending the search when the global end condition is met, determining the vector corresponding to the node in the target node queue when the search ends as at least one target vector corresponding to the search vector.

[0110] In one embodiment of the present disclosure, the second search module 503 is further used to: obtain vector dimension, HNSW indexing parameter and the total number of vectors in the HNSW index as threshold influencing parameters; determine search configuration information of the first object input associated with the search vector, wherein the search configuration information includes at least the number of target vectors, the search timeout time and the node filtering condition; determine the number threshold corresponding to each level according to the threshold influencing parameters and the search configuration information;

[0111] In one embodiment of the present disclosure, the second search module 503 is also used to: determine the first description information of the first object; obtain the second description information of the second object associated with the pre-stored candidate search configuration information; and obtain the search configuration information input by the first object from the candidate search configuration information based on the first description information and the second description information.

[0112] In one embodiment of the present disclosure, the second search module 503 is further used to: in response to not obtaining the search configuration information available to the first object from the candidate search configuration information, obtain global search configuration information as the search configuration information input by the first object.

[0113] In one embodiment of the present disclosure, the device also includes: monitoring the multi-level search of the first object on the underlying map; analyzing the multi-level search, and adjusting the number threshold corresponding to at least part of the level of the first object according to the analysis result.

[0114] In one embodiment of the present disclosure, the device further includes: monitoring available resources of the search server, and in response to sufficient available resources, performing a supplementary search based on cached supplementary search information to update a corresponding number threshold of each level.

[0115] According to the information search device provided by the embodiment of the present disclosure, a HNSW search is performed on the search vector, and a multi-level search is performed on the underlying graph when entering the HNSW. When the search at each level meets its own local end condition, the search at the next level is performed until the global end condition is met and the search is ended, and at least one target vector corresponding to the search vector is obtained. By performing a hierarchical search on the underlying graph of HNSW, the search speed of the HNSW search in the extreme filtering scenario can be improved, and the recall rate of the HNSW search can be improved, thereby improving the query efficiency and search quality of the user, and further improving the user's search experience.

[0116] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0117] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0118] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0119] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program / instruction stored in a read-only memory (ROM) 602 or a computer program / instruction loaded from a storage unit 606 to a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0120] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606 such as a keyboard, a mouse, etc.; an output unit 607 such as various types of displays, speakers, etc.; a storage unit 608 such as a disk, an optical disk, etc.; and a communication unit 609 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0121] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as information search methods. For example, in some embodiments, the information search method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 606. In some embodiments, part or all of the computer program / instructions may be loaded and / or installed on the device 600 via ROM 602 and / or communication unit 609. When the computer program / instructions are loaded into RAM 603 and executed by the computing unit 601, one or more steps of the information search method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the information search method in any other appropriate manner (e.g., by means of firmware).

[0122] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs / instructions that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0124] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0126] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0127] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs / instructions running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0128] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in the disclosure can be achieved, and this document does not limit them here.

[0129] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An information search method, wherein: The method comprises: Get the search vector; Performing a hierarchical navigation small world HNSW search on the search vector until entering the bottom layer graph of the HNSW; A multi-level search is performed on the underlying graph, and when the search at each level meets its own local end condition, the search at the next level is performed until the search is terminated by meeting the global end condition, thereby obtaining at least one target vector corresponding to the search vector, wherein there is no intersection between the searched nodes corresponding to each level.

2. The method according to claim 1, wherein: The step of searching at the next level when the search at each level satisfies its own local end condition includes: Performing an approximate nearest neighbor search at the i-th level and determining the number of distance calculations in the i-th level search process; In response to the number of distance calculations reaching the number threshold of the i-th level, entering the search at the (i+1)th level, wherein the local end condition is that the number of distance calculations reaches the number threshold of the i-th level.

3. The method according to claim 2, wherein: The search for the approximate nearest neighbor at the i-th level includes: Determine nodes in the underlying graph that have not been searched when entering the i-th level; Determine the nodes to be searched in the i-th level from the nodes that have not been searched in the bottom graph; An approximate nearest neighbor search is performed for the nodes to be searched in the i-th level based on the search vector.

4. The method according to claim 3, wherein: Before entering the i+1th level search, the following steps are also included: Determine the nodes that have been searched at the i-th level, and mark the nodes that have been searched at the i-th level; According to the searched nodes marked at the i-th level, the nodes that have not been searched in the underlying graph are updated.

5. The method according to claim 4, wherein: Each level inherits the node pool of the previous level, which includes marked searched nodes and unsearched nodes.

6. The method according to any one of claims 2 to 5, wherein: The method further comprises: Determine the nodes in the candidate node queue and the nodes in the target node queue when entering the i-th level; In the search process of the i-th level, updating the nodes in the candidate node queue based on the distance between the search vector and the node j to be searched at the i-th level; The nodes in the target node queue are updated according to the updated nodes in the candidate node queue.

7. The method according to claim 6, wherein: The updating of the nodes in the candidate node queue based on the distance between the search vector and the node j to be searched at the i-th level includes: In response to the distance corresponding to the node j being smaller than the distance corresponding to the node k in the candidate node queue, the node j is added to the candidate node queue according to the distance corresponding to the node j.

8. The method according to claim 7, wherein: The adding the node j to the candidate node queue according to the distance corresponding to the node j includes: In response to the candidate node queue being full, the node with the largest distance is deleted from the candidate node queue, and the node j is added to the candidate node queue according to the distance corresponding to the node j.

9. The method according to claim 7, wherein: The updating of the nodes in the target node queue according to the updated nodes in the candidate node queue includes: The nodes in the target node queue are re-determined according to the updated order of the nodes in the candidate node queue.

10. The method according to claim 9, wherein: The method further comprises: In response to currently entering the final level search, the last remaining unsearched nodes on the bottom graph are used to determine the nodes to be searched at the final level, and a K nearest neighbor search is performed in the nodes to be searched at the final level based on the search vector to update the nodes in the target node queue; In response to the search being ended in response to the global end condition being met, a vector corresponding to a node in the target node queue when the search is ended is determined as at least one target vector corresponding to the search vector.

11. The method according to any one of claims 2 to 5, wherein: The method further comprises: Get the vector dimension, HNSW indexing parameters, and the total number of vectors in the HNSW index as threshold influencing parameters; Determine search configuration information of the first object input associated with the search vector, wherein the search configuration information at least includes the number of the target vectors, a search timeout period, and a node filtering condition; A number threshold corresponding to each level is determined according to the threshold influencing parameter and the search configuration information.

12. The method according to claim 11, wherein: The determining the search configuration information of the first object input associated with the search vector includes: Determining first description information of the first object; Acquire second description information of a second object associated with pre-stored candidate search configuration information; The search configuration information input by the first object is obtained from the candidate search configuration information according to the first description information and the second description information.

13. The method according to claim 12, wherein: The determining the search configuration information of the first object input associated with the search vector includes: In response to not obtaining the search configuration information available to the first object from the candidate search configuration information, obtaining global search configuration information as the search configuration information input by the first object.

14. The method according to claim 12, wherein: The method further comprises: Monitoring a multi-level search of the first object on the underlying graph; The multi-level search situation is analyzed, and according to the analysis result, a number threshold corresponding to at least part of the levels of the first object is adjusted.

15. The method according to claim 12, wherein: The method further comprises: The available resources of the search server are monitored, and in response to the available resources being sufficient, a supplementary search is performed based on the cached supplementary search information to update the corresponding number threshold of each level.

16. An information search device, wherein: The device comprises: An acquisition module, used for acquiring a search vector; A first search module, configured to perform a hierarchical navigation small world HNSW search on the search vector until entering the bottom layer graph of the HNSW; The second search module is used to perform multi-level search on the underlying graph, and to perform search on the next level when the search at each level meets its own local end condition, until the search is terminated by meeting the global end condition, to obtain at least one target vector corresponding to the search vector, wherein there is no intersection between the searched nodes corresponding to each level.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 15.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-15.

19. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 15 is implemented.

Images