Search methods, devices, equipment, storage media and products
By converting search requirements into multi-path search vectors, the problem of missing field content caused by single-vector search is solved, achieving higher search accuracy and coverage.
Patent Information
- Application Number
- CN202611552921.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-08
- Publication Date
- 2026-11-17
AI Technical Summary
In existing technologies, single-vector retrieval can only search one feature field of a document, resulting in the missing content of another field and reducing retrieval accuracy.
The search requirements are converted into multi-path search vectors, and the search results are retrieved through multi-path retrieval, avoiding the loss of field content caused by single-vector retrieval.
It improves retrieval accuracy and can simultaneously cover multiple feature fields or multiple model expressions of a document, overcoming the limitations of single-vector retrieval.
Smart Images

Figure CN123115174A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of search engine retrieval technology, and in particular to a retrieval method, apparatus, device, storage medium and product. Background Technology
[0002] In related technologies, vector retrieval technology often adopts single-vector retrieval, that is, the index stores only one type of single vector, and the retrieval is only performed by searching the few most similar search results from this single vector index. In other words, single-vector retrieval can only search a certain feature field of a document. However, since documents usually contain multiple feature fields, such as web pages which are generally divided into "title" and "content" fields, if only the vector corresponding to one field is searched, the content of the other field will be missing, reducing the retrieval accuracy. Summary of the Invention
[0003] The main purpose of this application is to provide a retrieval method, apparatus, device, storage medium, and product, which aims to solve the technical problem of reduced retrieval accuracy caused by a single vector index.
[0004] To achieve the above objectives, this application proposes a retrieval method, apparatus, device, storage medium, and product, wherein the retrieval method includes: In response to a search request, the search request is converted into a search vector; Multi-path retrieval is performed based on the retrieval vector to recall the corresponding retrieval results.
[0005] In one embodiment, the step of converting the search request into a search vector in response to the search request includes: In response to a search request, the search request is converted into a search vector of different types.
[0006] In one embodiment, the step of converting the search request into different types of search vectors in response to the search request includes: In response to retrieval needs, determine the corresponding query intent; Based on the query intent, determine the corresponding search features; Based on the search features, the search requirements are converted into different types of search vectors.
[0007] In one embodiment, the different types include at least one of different dimensions and different models.
[0008] In one embodiment, the query intent includes at least one of a first query intent and a second query intent, and the step of determining the corresponding query intent in response to a retrieval request includes: In response to a search request, determine whether there are any query restriction requirements in the search request; If the query restriction requirement exists, the first query intent is determined based on the query restriction requirement; In the absence of the aforementioned query restriction requirement, the retrieval requirement is identified as corresponding to the second query intent.
[0009] In one embodiment, before the step of performing multi-way retrieval based on the retrieval vector to recall the corresponding retrieval results, the method includes: Build a multi-way vector index for the text in the text library.
[0010] In one embodiment, the vector index includes at least one of a first vector index, a second vector index, and a third vector index, and the step of constructing a multi-way vector index for the text in the text library includes at least one of the following: Construct multiple first vector indices for different text contents in the text library; Construct multi-way second vector indexes of different dimensions for the text in the text library; The text in the text library is used to construct a multi-way third vector index using different models.
[0011] In one embodiment, the search results include at least one of a first search result and a second search result, and the step of performing multi-way retrieval based on the search vector to recall the corresponding search results includes at least one of the following: Based on the retrieval vector, multi-path batch retrieval is performed to recall the corresponding first retrieval result, and the retrieval results corresponding to different batches in the first retrieval result are independent of each other; Based on the retrieval vector, a multi-path synchronous retrieval is performed to recall the corresponding second retrieval result.
[0012] In one embodiment, the step of performing multi-path batch retrieval based on the retrieval vector to recall the corresponding first retrieval result includes: If there is a recall restriction requirement in the search requirements, multi-path batch search is performed based on the search vector to recall the corresponding first search result.
[0013] In one embodiment, the step of performing multi-path simultaneous retrieval based on the retrieval vector to recall the corresponding second retrieval result includes: If there is no recall restriction requirement in the search requirements, multi-path synchronous search is performed based on the search vector to recall the corresponding second search result.
[0014] In one embodiment, the step of performing multi-path simultaneous retrieval based on the retrieval vector to recall the corresponding second retrieval result includes: Based on the retrieval vector, multi-path vector retrieval is performed simultaneously; The search results retrieved during synchronous retrieval are merged to obtain the second search result.
[0015] In one embodiment, the step of fusing the search results retrieved in the synchronous retrieval to obtain the second search result includes: Based on the search features corresponding to the search requirements, adjust the weights corresponding to different paths; Based on the aforementioned weights, the search results of the synchronous search are merged and sorted. Filter a preset number of the second search results from the merged sort results.
[0016] Furthermore, to achieve the above objectives, this application also proposes a retrieval device, the retrieval device comprising: A conversion module is used to convert a retrieval request into a retrieval vector in response to the retrieval request. The recall module is used to perform multi-way retrieval based on the retrieval vector in order to recall the corresponding retrieval results.
[0017] In one embodiment, the conversion module includes: The type conversion submodule is used to convert the retrieval request into a retrieval vector of different types in response to the retrieval request.
[0018] In one embodiment, the type conversion submodule includes: The intent conversion unit is used to respond to a retrieval request, determine the corresponding query intent, which includes at least one of a first query intent and a second query intent; determine the corresponding retrieval features based on the query intent; and convert the retrieval request into different types of retrieval vectors based on the retrieval features.
[0019] In one embodiment, the intent conversion unit includes: The determination subunit is used to respond to a retrieval request by determining whether there is a query restriction requirement in the retrieval request; if the query restriction requirement exists, the first query intent is determined based on the query restriction requirement; if the query restriction requirement does not exist, the second query intent corresponding to the retrieval request is identified.
[0020] In one embodiment, the retrieval device further includes: The building module is used to construct multi-way vector indexes for text in a text library.
[0021] In addition, to achieve the above objectives, this application also proposes a retrieval device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the retrieval method as described above.
[0022] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the retrieval method described above.
[0023] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the retrieval method described above.
[0024] One or more technical solutions proposed in this application have at least the following technical effects: In contrast to related technologies, since documents typically contain multiple feature fields (e.g., web pages generally have "title" and "content" fields), retrieving only the vector corresponding to one field can lead to missing content in the other field, reducing retrieval accuracy. This application, however, responds to retrieval needs by converting the retrieval needs into retrieval vectors; then, it performs multi-path retrieval based on these retrieval vectors to retrieve the corresponding retrieval results. It is understood that after responding to retrieval needs and converting the retrieval needs into retrieval vectors, this application does not perform retrieval on a single vector, but rather performs multi-path retrieval based on that retrieval vector to retrieve the retrieval results. This avoids the loss of field content during the retrieval process caused by retrieving only a single vector, thus improving retrieval accuracy. Attached Figure Description
[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 This is a schematic diagram of the retrieval device structure of the hardware operating environment involved in the embodiments of this application; Figure 2 This is a flowchart illustrating the retrieval method of this application in Embodiment 1; Figure 3This is a flowchart illustrating Embodiment 3 of the retrieval method of this application; Figure 4 This is a flowchart illustrating the batch recall process of the retrieval method in this application; Figure 5 This is a flowchart illustrating the simultaneous recall process of the retrieval method in this application. Figure 6 This is a schematic diagram of the module structure of the retrieval device according to an embodiment of this application; The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0028] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.
[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0030] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.
[0031] Additionally, it should be noted that in this article, "return" means returning to a specific step and re-executing from that step.
[0032] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, may be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or," "and / or," "including at least one of the following," etc., as used in this application, may be interpreted as inclusive, or mean any one or any combination thereof. For example, "including at least one of the following: A, B, C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Similarly, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Exceptions to this definition only occur when the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.
[0033] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.
[0034] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”
[0035] It should be noted that step designations such as S10 and S20 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S20 first and then S10, etc., but these should all be within the protection scope of this application.
[0036] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.
[0037] Reference Figure 1 , Figure 1 This is a schematic diagram of the retrieval device structure of the hardware operating environment involved in the embodiments of this application.
[0038] like Figure 1 As shown, the retrieval device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0039] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the retrieval device and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.
[0040] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and a retrieval program.
[0041] exist Figure 1In the retrieval device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the retrieval device of this application can be set in the retrieval device, and the retrieval device calls the retrieval program stored in the memory 1005 through the processor 1001 and executes the retrieval method provided in the embodiment of this application.
[0042] Based on this, the embodiments of this application provide a retrieval method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the retrieval method of this application.
[0043] It should be noted that the execution subject in this embodiment is a retrieval device, which can be a mobile terminal or a controller or other device with data processing capabilities. This application does not impose any specific limitations.
[0044] The following explanation uses mobile terminals as the retrieval device.
[0045] It is understandable that vector retrieval technology maps unstructured data, such as known data and user demand data, into points in a high-dimensional vector space, and uses the similarity measure between vectors to achieve semantic matching. However, vector indexes built based on known data only store a single vector, so during retrieval, only the K most similar results (TopK) are retrieved from this single vector. Furthermore, known data, such as webpage content, is divided into "title" and "content" fields. In single-vector retrieval, only one field can be selected as the vector index for vector retrieval; both fields cannot be retrieved simultaneously. Therefore, a vector index built based on a single field content can only represent a single perspective. For example, if only the title content is used as the index vector, the vector retrieval process can only find the nearest neighbor in the title space, while key information in the content field cannot be effectively recalled, and vice versa. Therefore, this application, after responding to retrieval needs and converting the retrieval needs into retrieval vectors, does not only perform retrieval on a single vector, but performs multi-path retrieval based on that retrieval vector to recall retrieval results. This avoids the loss of field content during the retrieval process caused by only retrieving a single vector, thus improving retrieval accuracy.
[0046] In this embodiment, the retrieval method includes steps S10 to S20: Step S10: In response to the retrieval request, the retrieval request is converted into a retrieval vector; It should be noted that the execution entity in this embodiment is a retrieval device, which is equipped with a search engine. This search engine can provide a front-end page for users to submit search requests, and can also call the user-side interface to obtain the search requests submitted by the user. The search request can be expressed as a natural language question, keyword combination, or other forms of query input. A search vector refers to a specific numerical array that maps unstructured search requests to a high-dimensional continuous space. After receiving the user's input search request, the retrieval device converts the search request into a search vector in a high-dimensional continuous space.
[0047] Step S20: Perform multi-path retrieval based on the retrieval vector to recall the corresponding retrieval results.
[0048] Understandably, after obtaining the retrieval vector, the retrieval device will distribute the retrieval vector to multiple index queues. Each index queue will independently perform similarity calculations, and each queue will recall a preset number of texts with the highest similarity to the retrieval vector as the retrieval results for that path. The preset number can be customized based on the number of texts in the text library or the performance budget of the retrieval device.
[0049] Because this application can perform retrieval on multiple index queues based on retrieval vectors, the retrieval device can simultaneously cover multiple feature fields (such as title and content) or multiple model expressions of a document in a single retrieval request, overcoming the limitation of single-vector retrieval that can only obtain results from one feature dimension, thereby improving retrieval accuracy.
[0050] In one feasible implementation, step S20 includes: In response to a search request, the search request is converted into a search vector of different types.
[0051] It should be noted that after receiving the user's search request, the search device will convert the search request into different types of search vectors through different conversion methods.
[0052] Specifically, users can restrict the specific form of the search vector in their search requirements, such as specifying that the search vector must be output by model A in dimension a.
[0053] In one feasible implementation, the step of converting the retrieval request into different types of retrieval vectors in response to the retrieval request includes: In response to retrieval needs, determine the corresponding query intent; Understandably, query intent represents the type of index vector the user intends to retrieve or the type of index vector that best suits their retrieval needs during this search process. After receiving the user's input search request, the retrieval device determines the query intent corresponding to that request.
[0054] Based on the query intent, determine the corresponding search features; It should be noted that search features are attribute information used to characterize the focus of the search, indicating which specific vector index queues should be used for recall operations, such as focusing on specific feature fields, specific vector dimensions, or specific model representations. After determining the user's query intent, the search device mines one or more search features that match the current intent, thereby influencing the focus of the search.
[0055] Based on the search features, the search requirements are converted into different types of search vectors.
[0056] Understandably, the retrieval device, based on the specific type and number of the determined retrieval features, inputs the user's retrieval requirements into the transformation model or transformation dimension corresponding to each retrieval feature, generating different types of retrieval vectors.
[0057] In one feasible implementation, the different types include at least one of different dimensions and different models.
[0058] It should be noted that the capacity of retrieval vectors varies depending on their dimension. Low-dimensional vectors have limited capacity, retaining only high-frequency semantics while ignoring low-frequency detailed semantics. In the process of retrieval based on low-dimensional retrieval vectors, a large number of retrieval results corresponding to or related to low-dimensional retrieval vectors can be quickly retrieved. High-dimensional vectors, on the other hand, have a larger capacity, can store refined information, and retrieve retrieval results strongly related to high-dimensional retrieval vectors. This allows search engines to flexibly call matching dimensions according to the scenario, for example, converting retrieval vectors into 768-dimensional and 1024-dimensional vectors respectively.
[0059] Specifically, the retrieval device converts retrieval requirements into retrieval vectors of different dimensions by setting different output dimensions for the embedding model, such as 768-dimensional and 1024-dimensional. In this way, the retrieval vectors output by the embedding model can have different dimensions.
[0060] Understandably, A / B testing based on single-vector retrieval requires building multiple clusters to evaluate the merits of multiple models, resulting in high operational costs. However, this application generates retrieval vectors with different granular semantic representation capabilities through different models. For example, the retrieval vectors generated by model A and model B are in different semantic mapping spaces, each reflecting different semantic encoding preferences. This allows for simultaneous access to document vector indexes built based on different dimensions and models during subsequent multi-path retrieval, overcoming the limitation of single-vector retrieval being limited to fixed dimensions and models, and improving retrieval accuracy.
[0061] In one feasible implementation, the query intent includes at least one of a first query intent and a second query intent, and the step of determining the corresponding query intent in response to a retrieval request includes: In response to a search request, determine whether there are any query restriction requirements in the search request; It should be noted that query restriction requirements refer to the constraints explicitly defined by the user in their search query. For example, during A / B testing, the user might specify that search vectors be generated using model A and model B respectively for comparison of search results, thus narrowing the search scope and improving testing efficiency. After receiving the user's search query, the retrieval device parses and identifies the query query to determine whether it contains query restriction requirements.
[0062] If the query restriction requirement exists, the first query intent is determined based on the query restriction requirement; Understandably, when a retrieval device determines that there are query restrictions in the retrieval request, it will use the query restrictions as the main basis for the query intent, so that the subsequent retrieval feature selection and multi-vector retrieval process will prioritize the user's intention.
[0063] In the absence of the aforementioned query restriction requirement, the retrieval requirement is identified as corresponding to the second query intent.
[0064] It should be noted that when the retrieval device determines that there are no query restrictions in the retrieval request, it directly performs semantic parsing and intent recognition processing on the retrieval request without adding any attribute dimension constraints, and identifies the second query intent corresponding to the retrieval request.
[0065] Specifically, the identification of the second query intent can be achieved by: collecting historical search requests and corresponding retrieved search results, performing data mining on the search requests and search results to learn the relationship between search requests and query intents, and then identifying the second query intent corresponding to the search request through this relationship.
[0066] In this implementation, the same search requirement can be converted into different types of search vectors. Subsequently, these different types of search vectors can be used to match text vector indexes of corresponding types (such as different dimensions, different models, or different fields), avoiding the problem that a single type of vector is not expressive enough in a specific query scenario, and improving the accuracy of search results.
[0067] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment can be referred to the above description, and will not be repeated hereafter. Based on this, the query intent includes at least one of a first query intent and a second query intent. The step of determining the corresponding query intent in response to the retrieval request, the retrieval method further includes step S01: Step S01: Construct a multi-way vector index for the text in the text library.
[0068] As can be understood, the text library refers to the collection of all document resources managed by the search engine of the retrieval device, including each document (such as web pages, articles, product descriptions, etc.). Each vector index refers to an independent index queue, and each index queue corresponds to a specific vector type (e.g., "title-768-dimensional-model A"). The retrieval device generates multiple text vectors of different types from the text in the text library, and then stores these document vectors into the corresponding index queues according to their types, forming a multi-path vector index.
[0069] In one feasible implementation, the vector index includes at least one of a first vector index, a second vector index, and a third vector index, and step S01 includes at least one of the following: Construct multiple first vector indices for different text contents in the text library; It should be noted that different text content refers to text paragraphs or blocks that differ in content nature or theme, such as the text content corresponding to the "title" and "content" fields in a webpage. The retrieval device processes the different text content within each document in the text database separately, constructing an independent first vector index for each type of text content.
[0070] Specifically, the retrieval device can divide a webpage text into title content, body content, tag content, summary content, etc. Then, for each type of text content, it performs vectorization transformation through an embedding model to generate corresponding text vectors. These vectors are then stored in their respective first vector index queues according to their content type. Each first vector index only stores vector data corresponding to the same type of text content, and vectors of different types of text content are stored in different first vector index queues. Alternatively, the retrieval device can also identify all feature fields in the webpage text, and for the text content corresponding to each feature field, it performs vectorization transformation through an embedding model to generate corresponding text vectors.
[0071] Construct multi-way second vector indexes of different dimensions for the text in the text library; It is understandable that the retrieval device uses multiple different vector dimension specifications to vectorize the same text content or different text content in the text library, and generates multiple second vector indices from the converted text vectors. These are organized into a separate index queue, and each index queue is a second vector index. The index queues corresponding to different dimensions are stored and managed independently. For example, 768-dimensional vector representation and 1024-dimensional vector representation can be generated at the same time.
[0072] The text in the text library is used to construct a multi-way third vector index using different models.
[0073] It should be noted that the "model" refers to an embedding model, which can convert text retrieval into points in a vector space. The retrieval device uses multiple different embedding models to vectorize the same text content or different text content from the text library. For example, it can use model A and model B simultaneously to generate vector representations of the same text. For each embedding model, the retrieval device organizes the full set of document vectors generated by that model into a separate index queue; each index queue is a third-party vector index.
[0074] Furthermore, the above-mentioned different types of vector indexes can be combined to form a single vector index. For example, when constructing vector indexes for different text content, they can be constructed using different models, and the output dimensions of different models can also be set to different dimensions. For example, for the text content corresponding to field A, a vector index can be generated using model A and model B, and the output dimensions of model A and B can be set to dimension A and dimension B respectively. At this time, field A corresponds to 4 vector indexes: vector index 1 (field A, model A, dimension A), vector index 2 (field A, model A, dimension B), vector index 3 (field A, model B, dimension A) and vector index 4 (field A, model B, dimension B).
[0075] In this embodiment, a multi-path vector index is constructed for the same text in the text library, so that each document can be hit by multiple retrieval vectors in subsequent searches, thereby overcoming the limitation that only one feature vector of each document is retrieved in a single vector index.
[0076] Based on the first and second embodiments of this application, the content that is the same as or similar to the first and second embodiments described above in the third embodiment of this application can be referred to the above description and will not be repeated hereafter. Based on this, refer to... Figure 3 The search results include at least one of the first search results and the second search results. In step S20, the search method further includes steps A01-A02: Step A01: Based on the retrieval vector, perform multi-path batch retrieval to recall the corresponding first retrieval result, wherein the retrieval results corresponding to different batches in the first retrieval result are independent of each other; Understandably, the retrieval device can employ a multi-path batch retrieval mode. Specifically, instead of merging multiple retrievals into a single request, the device initiates an independent retrieval request for each vector index, or for every N vector indices, to obtain the retrieval results corresponding to that batch. The retrieval requests from different batches can be initiated sequentially or randomly, with each batch's execution process completely isolated from the others. After all batches are completed, the independently retrieved results from each batch are merged into the first retrieval result. The results of each batch maintain their vector similarity ranking. This is suitable for A / B testing scenarios. (Refer to...) Figure 4 , Figure 4 A flowchart for the phased recall process is provided.
[0077] Step A02: Based on the retrieval vector, perform multi-path synchronous retrieval to recall the corresponding second retrieval result.
[0078] It should be noted that the retrieval device can employ a multi-path synchronous retrieval mode. Specifically, in a single retrieval request, the device simultaneously distributes the retrieval vector to multiple designated vector index queues. Each queue performs similarity calculations in parallel and retrieves its own top K documents with the highest similarity to the retrieval vector. The retrieval of all queues is completed synchronously within the same request, retrieving the second retrieval result. This is suitable for multi-feature combination retrieval scenarios. (Refer to...) Figure 5 , Figure 5 A flowchart for the synchronous recall process is provided.
[0079] Specifically, the above-mentioned multi-way retrieval method is as follows: after the device converts the retrieval request into different types of retrieval vectors, it determines the index queue corresponding to different retrieval vectors according to the type of the retrieval vector, and then retrieves the retrieval vector in the corresponding index queue.
[0080] Furthermore, the selection of different search modes can be dynamically chosen based on the amount of search data and engine performance.
[0081] In one feasible implementation, step A01 includes: If there is a recall restriction requirement in the search requirements, multi-path batch search is performed based on the search vector to recall the corresponding first search result.
[0082] Step A02 includes: if there is no recall restriction requirement in the search requirements, performing multi-path synchronous search based on the search vector to recall the corresponding second search result.
[0083] It is understandable that recall restriction requirements refer to the limiting information specified by the user in the recall phase of the search request. The search device adaptively selects different multi-path search execution strategies based on whether recall restriction requirements exist in the search request. Specifically, there are two scenarios: If it is determined that there are recall restriction requirements in the search request, the search vectors are distributed to the vector index queue set in batches according to the constraints of these restrictions, and the search is performed in a multi-path batch search manner; if it is determined that there are no recall restriction requirements in the search request, the search vectors are simultaneously distributed to all available vector index queues, and the search is performed in a multi-path synchronous search manner.
[0084] In one feasible implementation, step A02 includes: Based on the retrieval vector, multi-path vector retrieval is performed simultaneously; It should be noted that the retrieval device takes the retrieval vector as input and distributes it to multiple pre-built vector index queues simultaneously within a single request. Vector similarity calculation and TopK recall operations are performed in parallel on each queue. Each queue independently completes the entire process from vector matching and similarity ranking to outputting the TopK result for that path. The retrieval of all queues is completed synchronously within the same request lifecycle, and an independent recall result set from multiple vector index queues is obtained simultaneously within a single request.
[0085] The search results retrieved during synchronous retrieval are merged to obtain the second search result.
[0086] Understandably, the retrieval device outputs its own independent TopK result set to each vector index queue, merges the text in each recall result, and performs deduplication on the repeated text according to the text identifier to obtain the second retrieval result.
[0087] In one feasible implementation, the step of fusing the search results retrieved during synchronous retrieval to obtain the second search result includes: Based on the search features corresponding to the search requirements, adjust the weights corresponding to different paths; It should be noted that after the retrieval device determines the retrieval features required for this retrieval based on the query intent, it configures corresponding weight values for each vector index queue to be retrieved based on the type and attributes of these retrieval features. If the retrieval features indicate that the user is more concerned about title matching (such as the query intent being biased towards title retrieval), then a higher weight is assigned to the path corresponding to the title vector index, while a lower weight is assigned to the path corresponding to the content vector index, and vice versa.
[0088] Based on the aforementioned weights, the search results of the synchronous search are merged and sorted. Understandably, the retrieval device first merges the text from all path recall results and performs deduplication. Then, for each candidate document, it collects its similarity score in each recalled path and multiplies it by the weight coefficient of its corresponding path to obtain a weighted score. Then, according to a preset fusion strategy (such as weighted summation, weighted average, etc.), these weighted scores are combined to calculate the final fusion score of the document. Finally, based on the final fusion scores of each candidate document, they are sorted globally in descending order to generate the fusion ranking result.
[0089] Filter a preset number of the second search results from the merged sort results.
[0090] It should be noted that the preset quantity K refers to the number of search result entries that need to be returned to the user in the end. This quantity is represented by a positive integer (e.g., 10, 20, or 50). Based on the preset quantity, the retrieval device selects the first K texts in descending order of comprehensive score from the beginning of the fused sorted result list as the final search result output, i.e., the second search result.
[0091] In this embodiment, the application supports both multi-path batch retrieval and multi-path synchronous retrieval modes, enabling the search engine to flexibly switch retrieval strategies according to different business scenarios.
[0092] This application also provides a retrieval device, please refer to... Figure 6 The retrieval device includes: The conversion module 10 is used to convert the retrieval request into a retrieval vector in response to the retrieval request; The recall module 20 is used to perform multi-way retrieval based on the retrieval vector in order to recall the corresponding retrieval results.
[0093] Optionally, the conversion module includes: The type conversion submodule is used to convert the retrieval request into a retrieval vector of different types in response to the retrieval request.
[0094] Optionally, the type conversion submodule includes: The intent conversion unit is used to determine the corresponding query intent in response to the retrieval request; determine the corresponding retrieval features based on the query intent; and convert the retrieval request into different types of retrieval vectors based on the retrieval features.
[0095] Optionally, the intent conversion unit includes: The determination subunit is used to respond to a retrieval request by determining whether there is a query restriction requirement in the retrieval request; if the query restriction requirement exists, the first query intent is determined based on the query restriction requirement; if the query restriction requirement does not exist, the second query intent corresponding to the retrieval request is identified.
[0096] Optionally, the retrieval device further includes: The building module is used to construct multi-way vector indexes for text in a text library.
[0097] Optionally, the building module includes: The text construction submodule is used to construct multiple first vector indices for different text content in the text library; construct multiple second vector indices of different dimensions for the text in the text library; and construct multiple third vector indices for the text in the text library through different models.
[0098] Optionally, the recall module includes: The pattern recall submodule is used to perform multi-path batch retrieval based on the retrieval vector to recall the corresponding first retrieval result, wherein the retrieval results corresponding to different batches in the first retrieval result are independent of each other; and to perform multi-path synchronous retrieval based on the retrieval vector to recall the corresponding second retrieval result.
[0099] Optionally, the pattern recall submodule includes: The first judgment unit is used to perform multi-path batch retrieval based on the retrieval vector to recall the corresponding first retrieval result when there is a recall restriction requirement in the retrieval requirement.
[0100] The second judgment unit is used to perform multi-path synchronous retrieval based on the retrieval vector to recall the corresponding second retrieval result when there is no recall restriction requirement in the retrieval requirement.
[0101] Optionally, the second determination unit includes: The fusion sorting subunit is used to adjust the weights corresponding to different paths based on the retrieval features corresponding to the retrieval requirements; to perform fusion sorting on the retrieval results of synchronous retrieval based on the weights; and to select a preset number of second retrieval results from the fusion sorting results.
[0102] The retrieval device provided in this application, employing the retrieval method described in the above embodiments, can solve the technical problem of retrieval. Compared with the prior art, the beneficial effects of the retrieval device provided in this application are the same as those of the retrieval method provided in the above embodiments, and other technical features in the retrieval device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.
[0103] This application provides a storage medium that stores one or more programs, which can be executed by one or more processors to implement the steps of the retrieval method described above.
[0104] The specific implementation of the storage medium in this application is basically the same as the embodiments of the above-described retrieval method, and will not be repeated here.
[0105] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0106] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0107] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A retrieval method, characterized in that, The retrieval method includes: In response to a search request, the search request is converted into a search vector; Multi-path retrieval is performed based on the retrieval vector to recall the corresponding retrieval results.
2. The retrieval method as described in claim 1, characterized in that, The step of converting the search request into a search vector in response to the search request includes: In response to a search request, the search request is converted into a search vector of different types.
3. The retrieval method as described in claim 2, characterized in that, The step of converting the search request into different types of search vectors in response to the search request includes: In response to retrieval needs, determine the corresponding query intent; Based on the query intent, determine the corresponding search features; Based on the search features, the search requirements are converted into different types of search vectors.
4. The retrieval method as described in claim 2, characterized in that, The different types include at least one of different dimensions and different models.
5. The retrieval method as described in claim 3, characterized in that, The query intent includes at least one of a first query intent and a second query intent, and the step of determining the corresponding query intent in response to the retrieval request includes: In response to a search request, determine whether there are any query restriction requirements in the search request; If the query restriction requirement exists, the first query intent is determined based on the query restriction requirement; In the absence of the aforementioned query restriction requirement, the retrieval requirement is identified as corresponding to the second query intent.
6. The retrieval method as described in claim 1, characterized in that, Before the step of performing multi-path retrieval based on the retrieval vector to recall the corresponding retrieval results, the following steps are included: Build a multi-way vector index for the text in the text library.
7. A retrieval device, characterized in that, The device includes: A conversion module is used to convert a retrieval request into a retrieval vector in response to the retrieval request. The recall module is used to perform multi-way retrieval based on the retrieval vector in order to recall the corresponding retrieval results.
8. A retrieval device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the retrieval method as described in any one of claims 1 to 6.
9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the retrieval method as described in any one of claims 1 to 6.
10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the retrieval method as described in any one of claims 1 to 6.
