Information retrieval method and device and electronic equipment

Abstract

The invention provides an information retrieval method and device and electronic equipment, and relates to the field of artificial intelligence, in particular to the technical field of large models, deep learning and natural language processing. Obtaining a target search word; determining a target information identifier and a query vector corresponding to a to-be-searched word according to the target search word; determining a target information cluster from a plurality of information clusters according to the target information identifier, and determining first candidate information from the target information cluster according to the query vector; performing retrieval in a pre-constructed information index database according to the query vector, and determining second candidate information; and determining the to-be-pushed target information according to the first candidate information and the second candidate information, so that the to-be-pushed target information is determined according to the target information identifier and the query vector, fusion of generative retrieval and metric retrieval is realized, the information retrieval capability is effectively improved, and the accuracy of an information retrieval result is improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, specifically to the fields of large models, deep learning and natural language processing, and particularly to an information retrieval method, apparatus and electronic device. Background Technology

[0002] As large language models have demonstrated their powerful capabilities in the field of complex semantic representation, information retrieval technology has gradually evolved from traditional text- and vector-based metric retrieval modes to generative retrieval modes. However, current generative retrieval modes have failed to fully capture the fine-grained semantic feature differences between different information contents, and in terms of real-time retrieval performance and resource consumption, generative retrieval still has certain shortcomings compared to metric retrieval, and cannot guarantee the accuracy of information retrieval results. Summary of the Invention

[0003] This disclosure provides an information retrieval method, an information retrieval device, an electronic device, a storage medium, and a computer program product.

[0004] According to a first aspect of this disclosure, an information retrieval method is provided, comprising: acquiring a target search term; determining a target information identifier and a query vector corresponding to the target search term based on the target search term; determining a target information cluster from multiple information clusters based on the target information identifier, and determining first candidate information from the target information cluster based on the query vector, wherein the multiple information clusters are obtained by identifying and mapping dense vectors of information; retrieving information from a pre-constructed information index based on the query vector to determine second candidate information; and determining target information to be pushed based on the first candidate information and the second candidate information.

[0005] According to a second aspect of this disclosure, an information retrieval device is provided, comprising: an acquisition module for acquiring a target search term; a first determination module for determining a target information identifier and a query vector corresponding to the target search term based on the target search term; a first retrieval module for determining a target information cluster from multiple information clusters based on the target information identifier, and determining first candidate information from the target information cluster based on the query vector, wherein the multiple information clusters are obtained by identifying and mapping dense vectors of information; a second retrieval module for retrieving information from a pre-built information index library based on the query vector to determine second candidate information; and a second determination module for determining target information to be pushed based on the first candidate information and the second candidate information.

[0006] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the information retrieval method described in the first aspect of this disclosure.

[0007] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing the computer to perform the information retrieval method described in the first aspect of this disclosure.

[0008] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program, characterized in that the computer program, when executed by a processor, implements the information retrieval method according to a first aspect of this disclosure.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0011] Figure 1 This is a schematic flowchart of an information retrieval method according to an embodiment of the present disclosure;

[0012] Figure 2 This is a schematic flowchart of an information retrieval method according to an embodiment of the present disclosure;

[0013] Figure 3 This is a flowchart illustrating a large-scale retrieval model training method according to an embodiment of the present disclosure;

[0014] Figure 4 This is a flowchart illustrating a large-scale retrieval model training method according to an embodiment of the present disclosure;

[0015] Figure 5 This is a flowchart illustrating a large-scale retrieval model training method according to an embodiment of the present disclosure;

[0016] Figure 6 This is a schematic diagram of an information retrieval system according to an embodiment of the present disclosure;

[0017] Figure 7 This is a schematic diagram of the structure of an information retrieval device according to an embodiment of the present disclosure;

[0018] Figure 8 This is a schematic block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0019] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0020] The following is a brief description of the technical field involved in the solution disclosed herein:

[0021] Artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It involves both hardware and software technologies. AI hardware technologies generally include computer vision, speech recognition, natural language processing, and related technologies such as deep learning, big data processing, and knowledge graphs.

[0022] Large models refer to machine learning models with a large number of parameters and high complexity. They require a lot of computing resources and storage space for training and storage, and often require distributed computing and special hardware acceleration technologies. Large models have stronger generalization and expressive capabilities.

[0023] Deep learning (DL) is a new research direction in the field of machine learning (ML), bringing it closer to its original goal—artificial intelligence. Deep learning learns the inherent laws and hierarchical representations of sample data; the information gained during this learning process greatly aids in interpreting data such as text, images, and sound. Its ultimate goal is to enable machines to possess analytical and learning capabilities like humans, capable of recognizing data such as text, images, and sound. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition far exceeding previous related technologies.

[0024] Natural Language Processing (NLP) is an important research direction in the field of artificial intelligence. It integrates knowledge from multiple disciplines such as linguistics, computer science, machine learning, mathematics, and cognitive psychology. It is an interdisciplinary field that combines computer science, artificial intelligence, and linguistics, encompassing two main aspects: natural language understanding and natural language generation. Its research content includes multiple levels such as characters, words, phrases, sentences, paragraphs, and texts, serving as a bridge between machine language and human language. Its aim is to enable machines to understand, interpret, and generate human language, achieving effective communication between humans and machines, and enabling computers to perform tasks such as language translation, sentiment analysis, and text summarization.

[0025] An information retrieval method according to an embodiment of the present disclosure is described below with reference to the accompanying drawings.

[0026] Figure 1 This is a schematic flowchart of an information retrieval method according to an embodiment of the present disclosure. It should be noted that the executing entity of the information retrieval method in this embodiment is an information retrieval device, which can specifically be a hardware device or software within a hardware device. The hardware device includes, for example, a terminal device or a server.

[0027] like Figure 1 As shown, the information retrieval method proposed in this embodiment includes the following steps:

[0028] S101. Obtain the target search term.

[0029] The target search term (query) can be understood as the input for the search query.

[0030] Optionally, the target search term can be the search information entered by the user on the terminal device, such as the search information entered by the user on the mobile phone, the search information entered by the user on the computer, etc.

[0031] It should be noted that this disclosure does not limit the retrieval scenario for the target search term. For example, the retrieval scenario can be an advertising retrieval scenario, a news retrieval scenario, a product retrieval scenario, or a document retrieval scenario.

[0032] For example, in an advertising search scenario, the target search term could be "headphones," while in a product search scenario, the target search term could be "high-performance mobile phone."

[0033] S102. Based on the target search term, determine the target information identifier and query vector corresponding to the target search term.

[0034] In this embodiment of the disclosure, a pre-trained retrieval model can be obtained, and the target search terms can be processed by the retrieval model to determine the target information identifier and query vector corresponding to the target search terms.

[0035] Optionally, the target search terms can be encoded using a large retrieval model to obtain the encoding information of the target search terms, and the target information identifier and query vector can be predicted based on the encoding information of the target search terms.

[0036] S103. Based on the target information identifier, determine the target information cluster from multiple information clusters, and based on the query vector, determine the first candidate information from the target information cluster. The multiple information clusters are obtained by identifying and mapping the dense vectors of the information.

[0037] In this embodiment of the disclosure, a dense vector of information can be obtained, and a hierarchical clustering algorithm can be used to map the dense vector of information to obtain multiple information clusters, wherein different information clusters correspond to different information identifiers.

[0038] Optionally, the correlation score between the query vector and the dense vector of information within the target information cluster can be obtained, and the first candidate information can be determined from the target information cluster based on the correlation score.

[0039] S104. Based on the query vector, search the pre-built information index to determine the second candidate information.

[0040] In this embodiment of the disclosure, a pre-trained retrieval model is used to generate encoding vectors for candidate information, and an information index library is constructed based on the candidate information and its encoding vectors.

[0041] Alternatively, an approximate nearest neighbor search algorithm can be used to search the information index based on the query vector to obtain the second candidate information.

[0042] S105. Based on the first candidate information and the second candidate information, determine the target information to be pushed.

[0043] In this embodiment of the disclosure, after obtaining the first candidate information and the second candidate information, the first candidate information and the second candidate information can be filtered to determine the target information to be pushed.

[0044] According to the information retrieval method of this disclosure, the method obtains a target search term, determines the target information identifier and query vector corresponding to the target search term, determines a target information cluster from multiple information clusters based on the target information identifier, and determines first candidate information from the target information cluster based on the query vector. The multiple information clusters are obtained by mapping the dense vectors of the information. A second candidate information is determined by searching a pre-built information index library based on the query vector. Finally, the target information to be pushed is determined based on the first and second candidate information. Thus, this disclosure determines the target information to be pushed based on the target information identifier and query vector, achieving a fusion of generative retrieval and metric retrieval, effectively improving information retrieval capabilities and increasing the accuracy of information retrieval results.

[0045] Figure 2 This is a schematic flowchart of an information retrieval method according to an embodiment of the present disclosure.

[0046] like Figure 2 As shown, the information retrieval method proposed in this embodiment includes the following steps:

[0047] S201. Obtain the target search term.

[0048] The above embodiment's S102 "determine the target information identifier and query vector corresponding to the target search term based on the target search term" may specifically include S202.

[0049] S202. Encode the target search terms using a pre-trained retrieval model to obtain the encoding information of the target search terms, and predict the target information identifier and query vector based on the encoding information of the target search terms.

[0050] The large retrieval model was pre-trained using deep learning algorithms.

[0051] In this embodiment of the disclosure, a first training sample can be determined, which includes a sample search term and a first sample information identifier. A second training sample can be determined, which includes a second sample information identifier and sample information. The dual-tower large model is trained based on the first and second training samples, and the retrieval large model is obtained upon completion of training.

[0052] Optionally, after obtaining the target search term, the target search term can be input into the retrieval model. The retrieval model encodes the target search term to obtain the encoding information of the target search term. Based on the encoding information, the query vector is determined, and the target information identifier is predicted according to the beam search decoding strategy of the prefix tree to ensure the effectiveness of the target information identifier.

[0053] The step S103 in the above embodiment, "determine the target information to be pushed based on the target information identifier and query vector", may specifically include S203 and S206.

[0054] S203. Based on the target information identifier, determine the target information cluster from multiple information clusters.

[0055] In this embodiment of the disclosure, a dense vector of information can be obtained, and a hierarchical clustering algorithm can be used to map the dense vector of information to obtain multiple information clusters, wherein different information clusters correspond to different information identifiers.

[0056] For example, in the context of advertising retrieval, information can be understood as advertising content, such as advertising keywords and advertising landing page fragments. By obtaining dense vectors of advertising content through the encoder of the retrieval model, and using hierarchical clustering algorithms to map multiple semantically similar dense vectors to the same identifier (Identity Document, or ID), the advertising content is identified to obtain multiple information clusters.

[0057] Among them, information identifiers have semantic hierarchical structure information.

[0058] S204. Obtain the correlation score between the query vector and the dense vector of information within the target information cluster.

[0059] Optionally, the similarity between the query vector and the dense vector of information within the target information cluster can be calculated, and based on the similarity, a relevance score can be obtained between the query vector and the dense vector of information within the target information cluster.

[0060] S205. Based on the relevance score, determine the first candidate information from the target information cluster.

[0061] In this embodiment of the disclosure, a first retrieval task can be generated based on the target information identifier and the query vector, a generative retrieval path can be initiated, and the first retrieval task can be sent to the generative retrieval path, wherein the generative retrieval path is used to execute the first retrieval task to obtain the first candidate information.

[0062] Optionally, a first retrieval task can be generated based on the target information identifier and the query vector. The first retrieval task has end-to-end retrieval capabilities and good zero-sample cold start capabilities. By activating the generative retrieval path, the first retrieval task is sent to the generative retrieval path to obtain the relevance score between the query vector and the dense vector of information within the target information cluster. The relevance scores are sorted in descending order, and the top N information from the sorted results are selected as the first candidate information. Lightweight index management reduces storage overhead and improves the efficiency of obtaining the first candidate information.

[0063] S206. Using the approximate nearest neighbor search algorithm, the information index is searched according to the query vector to obtain the second candidate information.

[0064] In this embodiment of the disclosure, a large model can be retrieved to generate encoding vectors for candidate information, and an information index library can be constructed based on the candidate information and the encoding vectors for the candidate information.

[0065] In this embodiment of the disclosure, a second retrieval task is generated based on the query vector, a metric retrieval path is initiated, and the second retrieval task is sent to the metric retrieval path, wherein the metric retrieval path is used to execute the second retrieval task to obtain second candidate information.

[0066] Optionally, a second retrieval task is sent to the metric retrieval path. The second retrieval task has good cold start efficiency and real-time recall performance. Based on the query vector, an approximate nearest neighbor search algorithm is used to search the information index, and the top K similar information is selected as the second candidate information.

[0067] S207. Determine the target information based on the first candidate information and the second candidate information.

[0068] In this embodiment of the disclosure, after obtaining the first candidate information and the second candidate information, the first candidate information and the second candidate information can be merged and sorted to obtain a sorting result. Based on the sorting result, the target information is determined, thus ensuring the sufficiency of the information retrieval results and the effectiveness of the target information.

[0069] Optionally, duplicate and similar candidate information in the first candidate information and the second candidate information can be merged to obtain the first target information. A ranking index of the first target information can be obtained, a ranking strategy can be constructed based on the ranking index, and the first target information can be ranked according to the ranking strategy to determine the target information.

[0070] Optionally, to avoid information overload, the target information can be truncated after it has been determined to determine the final target information.

[0071] For example, if there are 10 pieces of target information and the preset information quantity threshold is 5 pieces, the first 5 pieces of target information can be selected as the final target information.

[0072] According to the information retrieval method of this disclosure, the target search term is obtained, encoded by a pre-trained retrieval model to obtain the encoded information of the target search term, and the target information identifier and query vector are predicted based on the encoded information of the target search term. Based on the target information identifier, a target information cluster is determined. Based on the query vector, a first candidate information is determined from the target information cluster. Based on the query vector, a search is performed in the information index database to determine a second candidate information. Based on the first and second candidate information, the target information is determined. Thus, this disclosure determines the target information based on the first and second candidate information, realizing the fusion of generative retrieval and metric retrieval, effectively improving the information retrieval capability. While improving the efficiency of information retrieval, it also improves the accuracy of information retrieval results and enhances the user's retrieval experience. Furthermore, the above method can be widely applied to various information retrieval scenarios such as advertising retrieval, news retrieval, product retrieval, and document retrieval, improving the adaptability and flexibility of information retrieval.

[0073] The training method for the large retrieval model proposed in this disclosure will be explained below.

[0074] like Figure 3 As shown, the training method for the large retrieval model proposed in this embodiment includes the following steps:

[0075] S301. Determine the first training sample. The first training sample includes the sample search term and the first sample information identifier.

[0076] It should be noted that this disclosure does not limit the specific method for determining the first training sample, and the method can be selected according to the actual situation.

[0077] Optionally, sample search terms and first sample information identifiers can be extracted from the search logs, and a first training sample can be constructed based on the sample search terms and first sample information identifiers.

[0078] For example, in the context of ad retrieval, we can extract pairs of search terms (sample search terms) and ad IDs (first sample information identifiers) that have click relationships from the ad search logs, and use each search term and ad ID pair as a training sample in the first training sample to obtain the first training sample.

[0079] S302. Determine the second training sample, which includes the second sample information identifier and sample information.

[0080] It should be noted that this disclosure does not limit the specific method for determining the second training sample, and the method can be selected according to the actual situation.

[0081] Optionally, a second sample information identifier and sample information can be extracted from the search log, and a second training sample can be constructed based on the second sample information identifier and sample information.

[0082] For example, in the context of ad retrieval, ad IDs (second sample information identifiers) and ad content (sample information) can be extracted from ad search logs. Each ad ID and the plaintext ad content can be used as a training sample in the second training sample to obtain the second training sample.

[0083] S303. The dual-tower large model is trained based on the first training sample and the second training sample, and the retrieval large model is obtained upon completion of training.

[0084] In this embodiment of the disclosure, after obtaining the first training sample and the second training sample, the dual-tower large model can be trained based on the first training sample and the second training sample to obtain the loss information during the training process of the dual-tower large model. Based on the loss information, the model parameters of the dual-tower large model are adjusted until the training termination condition is met, and the retrieval large model is obtained.

[0085] It should be noted that this disclosure does not impose any restrictions on the setting of training termination conditions, and the model training termination conditions can be set according to the actual situation.

[0086] Optionally, the training termination condition can be set to the loss information value being less than a preset loss threshold; alternatively, the training termination condition can also be set to the number of times the model parameters of the dual-tower large model are adjusted reaching a preset number threshold.

[0087] According to the training method of the retrieval big model in the embodiments of this disclosure, a first training sample is determined, which includes a sample search term and a first sample information identifier; a second training sample is determined, which includes a second sample information identifier and sample information; and a dual-tower big model is trained based on the first and second training samples. Upon completion of training, the retrieval big model is obtained. Thus, by obtaining the first and second training samples, this disclosure realizes the joint modeling of the coarse-grained information identifier generation task and the fine-grained information matching task to obtain the trained retrieval big model, laying the foundation for accurate information retrieval in the future.

[0088] Figure 4 This is a schematic flowchart of a method for training a large retrieval model according to an embodiment of the present disclosure.

[0089] like Figure 4 As shown, the training method for the large retrieval model proposed in this embodiment includes the following steps:

[0090] S401. Determine the first training sample. The first training sample includes the sample search term and the first sample information identifier.

[0091] S402. Determine the second training sample, which includes the second sample information identifier and sample information.

[0092] S403. Based on the first training sample, train the first large model in the dual-tower large model.

[0093] In this embodiment of the disclosure, the first large model obtains the encoding information of the sample search terms based on the first training samples, obtains the prediction information identifier based on the encoding information of the sample search terms, and obtains the generation loss of the first large model based on the sample information identifier and the prediction information identifier.

[0094] Optionally, the cross-entropy loss of the first-largest model can be obtained based on the sample information identifier and the prediction information identifier, and the cross-entropy loss can be used as the generation loss of the first-largest model.

[0095] S404. Train the second largest model in the dual-tower large model based on the second training samples.

[0096] In this embodiment of the disclosure, the second large model obtains the encoding information of the sample information based on the second training sample, determines the encoding information of the first large model for the sample search term, and obtains the metric loss of the second large model based on the encoding information of the sample information and the encoding information of the sample search term.

[0097] Optionally, the similarity between the encoded information of the sample information and the encoded information of the sample search terms is obtained, and negative sampling is performed within the batch of training samples to obtain negative advertising samples that are unrelated to the sample search terms. Based on the similarity and the labels of the positive and negative samples, comparative learning is performed to obtain the loss of the second largest model.

[0098] S405. Determine the loss information of the dual-tower large model, and based on the loss information, adjust and share the parameters of the first and second large models.

[0099] In this embodiment of the disclosure, after obtaining the generation loss of the first large model and the measurement loss of the second large model, the first weight of the generation loss and the second weight of the measurement loss can be obtained. Based on the first weight and the second weight, the generation loss of the first large model and the measurement loss of the second large model are weighted and calculated to obtain the loss information of the dual-tower large model.

[0100] Alternatively, the loss information of the dual-tower large model can be obtained according to the following formula:

[0101] Loss = α × Loss 生成 +β×Loss 度量

[0102] Where Loss represents the loss information of the dual-tower large model, α represents the first weight, and Loss... 生成 The loss is generated, β is the second weight, and Loss is... 度量 To measure the loss.

[0103] In this embodiment of the disclosure, after obtaining the loss information, the parameters of the first and second major models can be continuously adjusted based on the loss information, and parameters can be shared during the parameter adjustment process.

[0104] S406. In response to the end of training, select any well-trained large model from the dual-tower large models as the retrieval large model.

[0105] Optionally, based on the loss information, the parameters of the first and second largest models are adjusted and shared. In response to the training meeting the training termination condition, i.e., the training ends, the trained first and second largest models are obtained.

[0106] It should be noted that since the first and second large models in the dual-tower large model have the same model structure and share parameters, either trained large model in the dual-tower large model can be selected as the retrieval large model.

[0107] For example, in advertising retrieval scenarios, such as Figure 5 The training process of the large retrieval model is explained as shown.

[0108] The dual-tower model consists of a first model and a second model. The first model processes search terms, while the second model processes advertising information.

[0109] Optionally, for the first major model (generative modeling), based on the generative large language model foundation, advertising retrieval is modeled as an advertising ID prediction task. First, the advertising content is ID-ized, that is, the dense vector representation of the advertising content is obtained based on the large language model encoder. Hierarchical clustering algorithm is used to map multiple semantically similar advertising vectors to the same advertising ID to generate a label (first sample identifier). A virtual token is added before the sample search term, which can be understood as classification (cls). For example, the added virtual token is [QUERY]. The sample search term is input into the first major model to obtain the encoded information of the sample search term (query vector). The probability of each advertising ID is generated based on the query vector to obtain the generation probability (GEN Probs). Based on the GEN Probs and the first sample identifier, the generation loss of the first major model is obtained.

[0110] Optionally, for the second major model (metric modeling), an integrated dual-tower architecture sharing parameters with generative modeling is adopted to learn the metric representations of search queries and advertisements respectively. First, the input of the second major model is jointly constructed using the advertisement ID and advertisement content, enabling the second model to learn both coarse-grained and fine-grained hierarchical features of advertisements simultaneously. A virtual token is added before the advertisement ID and advertisement content, for example, the added virtual token is [AD], to extract the global features of the input. After the input of the second major model is encoded, the embedding vector of the virtual token ([QUERY] and [AD]) is taken as the semantic representation, and a fully connected layer is introduced to reduce its dimensionality to improve retrieval efficiency. During training, a negative sampling strategy within a batch of training samples is used to obtain negative advertisement samples that are unrelated to the sample search terms, which are then used for comparative learning to obtain the metric loss of the second major model.

[0111] Optionally, the generation loss of the first large model and the metric loss of the second large model are weighted and calculated to obtain the loss information of the dual-tower large model. Based on the loss information, the parameters of the first large model and the second large model are adjusted and shared. In response to the end of training, any trained large model in the dual-tower large model is selected as the retrieval large model.

[0112] In summary, the training method for the retrieval big model according to the embodiments of this disclosure involves determining a first training sample, which includes sample search terms and a first sample information identifier, and determining a second training sample, which includes a second sample information identifier and sample information. Based on the first training sample, the first big model in the dual-tower big model is trained, the loss information of the dual-tower big model is determined, and based on the loss information, the parameters of the first big model and the second big model are adjusted and shared. Upon completion of training, any trained big model in the dual-tower big model is selected as the retrieval big model. Thus, this disclosure integrates the advantages of generative retrieval and metric retrieval, relies on a parameter-sharing big language model encoder base, and simultaneously applies memory and matching mechanisms to achieve joint modeling of coarse-grained information identifier generation tasks and fine-grained information content matching tasks. Through the mutual promotion of two training tasks with different coarse and fine granularities, the accuracy and reliability of the retrieval big model training process are ensured, resulting in a retrieval big model with good scalability and lower data update costs. This effectively improves information retrieval capabilities and the relevance of recalled content, laying the foundation for accurate information retrieval based on the retrieval big model in the future.

[0113] In the embodiments of this disclosure, an information retrieval system can be constructed based on the information retrieval method proposed in this disclosure. The specific process of the information retrieval system proposed in this disclosure will be explained below.

[0114] It should be noted that, for advertising retrieval scenarios, search ads connect user intent with commercial information, and their recall efficiency and accuracy directly affect user experience and advertiser benefits. As large language models have demonstrated powerful capabilities in the field of complex semantic representation, advertising recall technology has gradually evolved from the traditional text- and vector-based metric retrieval mode to a generative retrieval paradigm, realizing deep semantic interaction between search queries and ads. However, current generative retrieval methods typically map search queries to coarse-grained ad feature identifiers (such as ad IDs) and fail to fully capture the fine-grained semantic feature differences between different ad content. In addition, generative retrieval still has certain shortcomings compared to metric retrieval in terms of ad cold start timeliness, real-time recall performance, and resource consumption.

[0115] It should be noted that, for advertising retrieval scenarios, current advertising retrieval technologies are mainly divided into two categories: metric retrieval and generative retrieval. Traditional metric retrieval technologies are usually based on methods such as bag-of-words models and deep learning models to extract dense vector representations of search queries and advertising documents, and use algorithms such as approximate nearest neighbor to calculate the similarity between query vectors and document vectors, and retrieve ads related to the query based on the similarity. Typical metric retrieval methods include maximal marginal relevance (MVR). When processing advertising retrieval tasks, the above methods achieve the filtering of relevant ads through vector similarity calculation, which can meet basic retrieval needs to a certain extent. Generative retrieval technologies use autoregressive large language models to directly generate relevant ad identifiers (such as ad IDs) based on search queries. Examples include Differentiable Search Index (DSI) and generative retrieval (GenRet). These methods do not rely on step-by-step retrieval processes and have significant advantages over metric retrieval in terms of end-to-end recall efficiency.

[0116] It should be noted that, since the core objective of generative retrieval technology is to directly map user queries to coarse-grained ad identifiers rather than focusing on the actual ad content itself, it is difficult to accurately remember fine-grained ad features. At the same time, as the size of the ad corpus continues to expand, the memory confusion problem faced by generative retrieval technology becomes increasingly serious. In addition, due to its structural differences from metric retrieval methods, the retrieval results of the two are not completely overlapping. There are some ads that are relevant to the query but were not recalled by generative retrieval, thus affecting the overall ad recall effect.

[0117] For example, such as Figure 6As shown in this embodiment, for an advertising retrieval scenario, the user's search query can be obtained. The search query is preprocessed by a preprocessing module to obtain the query (target search term). A pre-trained retrieval model is called, and the target search term is processed by the retrieval model (generative metric integrated inference module) to obtain the ad ID (target information identifier) ​​and query vector. In the generative retrieval path, the ad ID and query vector are processed by the ad selection module, that is, based on the ad ID, an ad ID cluster (target information cluster) is determined. The ads within the ad ID cluster are sorted and selected based on relevance scores using the query vector to determine the first candidate ad. In the metric retrieval path, the query vector is processed by the metric retrieval module, that is, based on the query vector, a search is performed in a pre-built ad index library to determine the second candidate ad. The first and second candidate ads are merged, sorted, and truncated to obtain the target ad. The target ad, combined with the ad creative, is returned to the product front end for display to the user.

[0118] In summary, the information retrieval method according to the embodiments of this disclosure, targeting the advertising retrieval scenario, integrates the advantages of both generative retrieval and metric retrieval, and based on a shared large language model encoder foundation, achieves joint modeling of coarse-grained advertising ID generation tasks and fine-grained advertising content matching tasks. This effectively improves the relevance of advertising retrieval capabilities and recalled content, which is conducive to improving advertising monetization efficiency and enhancing the user's retrieval experience.

[0119] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0120] Corresponding to the information retrieval methods provided in the above embodiments, one embodiment of this disclosure also provides an information retrieval device. Since the information retrieval device provided in this disclosure corresponds to the information retrieval methods provided in the above embodiments, the implementation methods of the information retrieval methods are also applicable to the information retrieval device provided in this embodiment, and will not be described in detail in this embodiment.

[0121] Figure 7 This is a schematic diagram of the structure of an information retrieval device according to an embodiment of the present disclosure.

[0122] like Figure 7 As shown, the information retrieval device 700 includes: an acquisition module 710, a first determination module 720, a first retrieval module 730, a second retrieval module 740, and a second determination module 750.

[0123] Module 710 is used to obtain the target search term;

[0124] The first determining module 720 is used to determine the target information identifier and query vector corresponding to the target search term based on the target search term;

[0125] The first retrieval module 730 is used to determine a target information cluster from multiple information clusters based on the target information identifier, and to determine first candidate information from the target information cluster based on the query vector, wherein the multiple information clusters are obtained by identifying and mapping the dense vectors of the information;

[0126] The second retrieval module 740 is used to perform a retrieval in a pre-built information index based on the query vector to determine the second candidate information.

[0127] The second determining module 750 is used to determine the target information to be pushed based on the first candidate information and the second candidate information.

[0128] The device 700 is further configured to: acquire a dense vector of information; and use a hierarchical clustering algorithm to perform identifier mapping on the dense vector of information to obtain multiple information clusters, wherein different information clusters correspond to different information identifiers.

[0129] The device 700 is further configured to: generate encoding vectors for candidate information using a pre-trained retrieval model; and construct the information index library based on the candidate information and the encoding vectors for the candidate information.

[0130] The first retrieval module 730 is further configured to: obtain a correlation score between the query vector and the dense vector of information within the target information cluster; and determine the first candidate information from the target information cluster based on the correlation score.

[0131] The second retrieval module 740 is further configured to: use an approximate nearest neighbor search algorithm to search the information index database according to the query vector to obtain the second candidate information.

[0132] The device 700 is further configured to: generate a first retrieval task based on the target information identifier and the query vector; initiate a generative retrieval path and send the first retrieval task to the generative retrieval path, wherein the generative retrieval path is used to execute the first retrieval task to obtain the first candidate information.

[0133] The device 700 is further configured to: generate a second retrieval task based on the query vector; initiate a metric retrieval path and send the second retrieval task to the metric retrieval path, wherein the metric retrieval path is used to execute the second retrieval task to obtain the second candidate information.

[0134] The retrieval module 730 is further configured to: merge and sort the first candidate information and the second candidate information to obtain a sorting result; and determine the target information based on the sorting result.

[0135] The determining module 720 is further configured to: encode the target search term using a pre-trained retrieval model to obtain the encoding information of the target search term, and predict the target information identifier and query vector based on the encoding information of the target search term.

[0136] The pre-training process of the retrieval model includes: determining a first training sample, which includes a sample search term and a first sample information identifier; determining a second training sample, which includes a second sample information identifier and sample information; training the dual-tower model based on the first training sample and the second training sample, and obtaining the retrieval model upon completion of training.

[0137] The pre-training process of the retrieval large model includes: training the first large model in the dual-tower large model based on the first training sample; training the second large model in the dual-tower large model based on the second training sample; determining the loss information of the dual-tower large model, and adjusting and sharing the parameters of the first large model and the second large model based on the loss information; and selecting any trained large model in the dual-tower large model as the retrieval large model in response to the end of training.

[0138] The pre-training process of the retrieval model includes: obtaining the encoding information of the sample search terms using the first large model based on the first training samples; obtaining the prediction information identifier based on the encoding information of the sample search terms; and obtaining the generation loss of the first large model based on the sample information identifier and the prediction information identifier.

[0139] The pre-training process of the retrieval model includes: obtaining the encoding information of the sample information through the second large model based on the second training samples; determining the encoding information of the sample search terms by the first large model; and obtaining the metric loss of the second large model based on the encoding information of the sample information and the encoding information of the sample search terms.

[0140] The pre-training process of the retrieval large model includes: weighting the generation loss of the first large model and the metric loss of the second large model to obtain the loss information of the dual-tower large model.

[0141] According to the information retrieval apparatus of this disclosure, by acquiring a target search term, determining a target information identifier and a query vector corresponding to the target search term, determining a target information cluster from multiple information clusters based on the target information identifier, and determining first candidate information from the target information cluster based on the query vector, wherein the multiple information clusters are obtained by identifying and mapping dense vectors of information, and searching in a pre-built information index library based on the query vector to determine second candidate information, and determining the target information to be pushed based on the first candidate information and the second candidate information, this disclosure determines the target information to be pushed based on the target information identifier and the query vector, thereby realizing the fusion of generative retrieval and metric retrieval, effectively improving the information retrieval capability and the accuracy of information retrieval results.

[0142] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0143] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0144] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0145] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0146] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as information retrieval methods. For example, in some embodiments, the information retrieval method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the information retrieval method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the information retrieval method by any other suitable means (e.g., by means of firmware).

[0147] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0148] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0149] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0150] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0151] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0152] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0153] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the information retrieval method described above.

[0154] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0155] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An information retrieval method, wherein, The method includes: Obtain the target search term; Based on the target search term, determine the target information identifier and query vector corresponding to the target search term; Based on the target information identifier, a target information cluster is determined from multiple information clusters, and based on the query vector, a first candidate information is determined from the target information cluster, wherein the multiple information clusters are obtained by identifying and mapping the dense vectors of the information; Based on the query vector, a search is performed in a pre-built information index to determine the second candidate information; Based on the first candidate information and the second candidate information, the target information to be pushed is determined.

2. The method according to claim 1, wherein, The process of acquiring the multiple information clusters includes: A dense vector for acquiring information; Hierarchical clustering algorithm is used to map the dense vectors of the information to obtain multiple information clusters, where different information clusters correspond to different information identifiers.

3. The method according to claim 1, wherein, The construction process of the information index database includes: Generate encoded vectors of candidate information using a pre-trained large retrieval model; The information index library is constructed based on the candidate information and the encoding vector of the candidate information.

4. The method according to claim 1, wherein, The step of determining the first candidate information from the target information cluster based on the query vector includes: Obtain the correlation score between the query vector and the dense vector of information within the target information cluster; The first candidate information is determined from the target information cluster based on the relevance score.

5. The method according to claim 1, wherein, The step of retrieving the second candidate information from a pre-built information index based on the query vector includes: Using an approximate nearest neighbor search algorithm, the information index is searched according to the query vector to obtain the second candidate information.

6. The method according to any one of claims 2-5, wherein, The method further includes: A first retrieval task is generated based on the target information identifier and query vector; A generative retrieval pathway is initiated, and the first retrieval task is sent to the generative retrieval pathway, wherein the generative retrieval pathway is used to execute the first retrieval task to obtain the first candidate information.

7. The method according to any one of claims 2-5, wherein, The method further includes: A second retrieval task is generated based on the query vector; Initiate a metric retrieval pathway and send the second retrieval task to the metric retrieval pathway, wherein the metric retrieval pathway is used to execute the second retrieval task to obtain the second candidate information.

8. The method according to any one of claims 2-5, wherein, The step of determining the target information based on the first candidate information and the second candidate information includes: The first and second candidate information are merged and sorted to obtain the sorting result; The target information is determined based on the sorting results.

9. The method according to any one of claims 1-5, wherein, The step of determining the target information identifier and query vector corresponding to the target search term based on the target search term includes: The target search term is encoded using a pre-trained retrieval model to obtain the encoded information of the target search term, and the target information identifier and query vector are predicted based on the encoded information of the target search term.

10. The method according to claim 9, wherein, The pre-training process of the large retrieval model includes: Determine the first training sample, which includes the sample search term and the first sample information identifier; Determine a second training sample, which includes a second sample information identifier and sample information; The dual-tower large model is trained based on the first training sample and the second training sample, and the retrieval large model is obtained upon completion of training.

11. The method according to claim 10, wherein, The step of training the dual-tower large model based on the first and second training samples, and obtaining the retrieval large model upon completion of training, includes: Based on the first training sample, the first large model in the dual-tower large model is trained; The second largest model in the dual-tower large model is trained based on the second training sample; Determine the loss information of the dual-tower large model, and based on the loss information, adjust and share the parameters of the first large model and the second large model; Upon completion of training, any trained large model from the dual-tower large models is selected as the retrieval large model.

12. The method according to claim 11, wherein, The step of training the first large model in the dual-tower large model based on the first training sample includes: The first large model obtains the encoding information of the search terms in the sample based on the first training sample; The predicted information identifier is obtained based on the encoding information of the sample search terms; The generation loss of the first large model is obtained based on the sample information identifier and the prediction information identifier.

13. The method according to claim 12, wherein, The step of training the second largest model in the dual-tower large model based on the second training samples includes: The second large model obtains the encoded information of the sample information based on the second training sample; Determine the encoding information of the first large model for the sample search terms, and obtain the metric loss of the second large model based on the encoding information of the sample information and the encoding information of the sample search terms.

14. The method according to claim 12, wherein, The determination of the loss information for the dual-tower large model includes: The generation loss of the first large model and the metric loss of the second large model are weighted and calculated to obtain the loss information of the dual-tower large model.

15. An information retrieval device, wherein, The device includes: The acquisition module is used to acquire the target search terms; The first determining module is used to determine the target information identifier and query vector corresponding to the target search term based on the target search term; The first retrieval module is used to determine a target information cluster from multiple information clusters based on the target information identifier, and to determine first candidate information from the target information cluster based on the query vector, wherein the multiple information clusters are obtained by identifying and mapping the dense vectors of the information; The second retrieval module is used to search in a pre-built information index based on the query vector to determine the second candidate information; The second determining module is used to determine the target information to be pushed based on the first candidate information and the second candidate information.

16. An electronic device, characterized in that, Including processor and memory; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the method as described in any one of claims 1-14.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-14.

18. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-14.

Images