Method and system for item retrieval using generative model output

Abstract

Some aspects relate to techniques for performing item retrieval on a listing platform utilizing model output from a generative model. In some examples, an item retrieval input is provided to a generative model to produce a model output. The lookup is performed on the key value repository using the model output. If a matching query is found in the key value repository, query embedding corresponding to the matching query is returned. If a matching query is not found, the query embedding is obtained by generating a query embedding from the model output, or querying a known query in a known query index using the model output, the known query being used to look up the query embedding in the key value repository. One or more item inserts are identified based on the query insert. An output is provided that identifies one or more item lists corresponding to the one or more item embedding.

Description

Background Art

[0001] Listing platforms (such as e-commerce websites) are online platforms that offer products, services, digital content (e.g., music, videos), or other items to users. Such platforms typically offer a large number of items. While some items are relevant to any given user, most are not. Therefore, item retrieval for list platforms is a unique internet-centric problem that has proven difficult to fully solve. Specifically, given the large number of items available on a list platform, which items should be retrieved and in what order should they be presented to the user?

[0002] Given the vast number of items available, listing platforms include features such as search and recommendations to help users find items of interest on the platform. For example, listing platforms typically provide a search capability that receives a user query and returns search results that identify items relevant to the user query. Listing platforms also often utilize recommendation systems to recommend items that may be of interest to the user based on various information, such as the item the user is currently viewing, user attributes, and user behavior on the listing platform (e.g., previous item views, purchases, etc.). Summary of the Invention

[0003] Some aspects of the present technology relate, in particular, to performing item retrieval on a list platform using model outputs from a generative model. In some examples, an input for item retrieval is received, which is, for example, based on a user query or a seed item for recommendation. The input is provided to a generative model, which generates text to produce a model output. A lookup is performed on a key-value store using the model output. The key-value store stores queries as keys and stores query embeddings as corresponding values. If a query matching the model output is identified in the key-value store, the query embedding stored in association with the matching query is retrieved. In some aspects, if a matching query for the model output is not found in the key-value store, the model output is used to search a known query index to identify known queries related to the model output, and the query embedding is retrieved from the key-value store by performing a lookup using the known query. In some aspects, if a matching query for the model output is not found in the key-value store, a query embedding is generated by providing the model output to an embedding model.

[0004] After obtaining one or more query embeddings, one or more item embeddings are identified based on similarity to the query embeddings. Each item embedding corresponds to an item list on the listing platform. An output is provided that identifies the item list corresponding to the item embedding. For example, a search result identifying the item list may be provided as a response to the user query, or the item list may be provided as a recommendation.

[0005] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The present technology is described in detail below with reference to the accompanying drawings, in which:

[0007] Figure 1 is a block diagram of an exemplary system according to some embodiments of the present disclosure;

[0008] Figure 2 is a block diagram illustrating an example process for performing item retrieval for a listing platform using model output from a generative model according to some embodiments of the present disclosure;

[0009] Figure 3 is a block diagram illustrating another example process for performing item retrieval for a listing platform using model output from a generative model according to some embodiments of the present disclosure;

[0010] Figure 4 is a flow chart illustrating a method for performing item retrieval for a listing platform using model output from a generative model according to some embodiments of the present disclosure;

[0011] Figure 5 is a flow chart illustrating another method for performing item retrieval for a listing platform using model output from a generative model according to some embodiments of the present disclosure; and

[0012] Figure 6 is a block diagram of an exemplary computing environment suitable for use with embodiments of the present disclosure. DETAILED DESCRIPTION Overview

[0013] While item retrieval systems are useful tools for locating items on listing platforms, shortcomings in existing search and recommendation technologies used by conventional item retrieval systems often lead to unnecessary consumption of computing resources (e.g., I / O costs, network packet generation costs, throughput, memory consumption, etc.). For example, current search systems often fail to effectively locate items of interest to users. This may be a result of search engines' limited understanding of queries. Search systems typically operate on keyword-based queries and may not fully understand some queries (such as complex natural language queries submitted by some users). While some search systems offer filtering capabilities to refine search results, the available filters may be insufficient or unintuitive. In the context of recommendations, recommended items provided by recommendation systems are often too similar or too different from the seed items, and there is no method to adjust the distance between recommended items and the seed items. While some recommendation systems personalize recommended items for users, this personalization is unavailable in many scenarios, such as when users navigate to an item page from a third-party search engine. Furthermore, item lists on listing platforms often include unstructured data, making it difficult for item retrieval systems to return relevant item lists in search and recommendation scenarios.

[0014] Due to the shortcomings of existing search and recommendation technologies on list platforms, users often need to submit multiple queries to find the desired item list. For example, a user may initiate a first query to the list platform's search engine, and the search engine returns a set of search results. The user can browse the search results and select certain search results to access the corresponding item lists. The selection of the search results will cause the retrieval of the corresponding item lists. In addition, in some cases, the application is launched to render the data associated with the item list. In the context of recommendations, when the recommended item lists are insufficient, the user may choose to view certain item lists and find that these lists are not what the user is looking for. This often leads users to turn to query-based search, which, as discussed above, may involve issuing a large number of queries in an attempt to identify relevant item lists.

[0015] This repeated input leads to increased computing resource consumption, among other things. For example, repeated user queries incur packet generation costs, adversely impacting computer network communications. Each time a user issues a query, the query content, or payload, is typically supplemented with header information or other metadata within the packet in TCP / IP and other protocol networks. Therefore, while this functionality multiplies with all the inputs required to retrieve the desired data, repeatedly generating this metadata and sending it across the computer network incurs throughput and latency costs. In some cases, this repeated input (e.g., repeated clicks, selections, or queries) increases storage device I / O (e.g., excessive physical read / write head movement on non-volatile disks). This is because each time a user enters unnecessary information (e.g., multiple queries), the computing system often must contact the storage device to perform a read or write operation, which is time-consuming, error-prone, and can ultimately wear out components such as the read / write head. Furthermore, if users repeatedly issue queries, the cost is high because processing the queries consumes significant computing resources. For example, some item retrieval systems may require computing a query execution plan each time a query is issued, potentially requiring the system to find the cheapest query execution plan to fully execute the query. This reduces throughput and increases network latency, potentially wasting valuable time.

[0016] In view of these deficiencies in existing search and recommendation technologies, aspects of the technology described herein improve the functionality of computers themselves by providing a solution in which an item retrieval system utilizes model outputs from a generative model to facilitate returning item lists for search and recommendation applications on a listing platform.

[0017] According to some configurations, the item retrieval system receives input for item retrieval. The input can be based on a user query, for example, in the case of search, or a seed item, for example, in the case of recommendation. The input is provided to a generative model, which generates a text string as a model output. In some cases, the model output identifies one or more global items, each of which names an item (e.g., iPhone 15). In some cases, the model output includes one or more item descriptions (e.g., 4K 55-inch OLED TV).

[0018] The item retrieval system uses the model output to identify a list of relevant items on the listing platform to return as search results or recommendations. Specifically, the item retrieval system obtains a query embedding for the model output and identifies the item embedding based on similarity to the query embedding. In order to reduce latency and operate at a large scale on the listing platform, query embeddings and item embeddings are generated offline by the embedding model. For example, query embeddings can be generated for historical queries in the query log of the listing platform and / or for model outputs previously generated by the generative model based on various inputs previously received by the system. The key-value store stores queries (e.g., historical queries or previously generated model outputs) as keys and query embeddings as corresponding values. Item embeddings are generated for item lists available on the listing platform and can be stored in the item embedding data repository together with an item list identifier that identifies the corresponding item list.

[0019] At runtime, to obtain a query embedding for a given model output, a lookup is performed on the key-value store using the model output. If a query matching the model output (i.e., a matching query) is identified in the key-value store, the query embedding stored as a value with the matching query is retrieved. If no matching query is found in the key-value store, a query embedding for the model output can be obtained in a variety of different ways. In some aspects, a known query index is searched to identify one or more known queries related to the model output. A known query is a query for which a query embedding has been generated and stored in the key-value store. For example, a fuzzy search can be used to search the known query index to identify known queries that are related to the model output but do not exactly (or nearly exactly) match the model output. The known queries identified from the search are used to perform a lookup on the key-value store to retrieve the query embedding stored with the known query. In other aspects, when no matching query is found for the model output in the key-value store, a query embedding is generated at runtime by providing the model output to an embedding model.

[0020] The query embeddings obtained for the model output are used to identify similar item embeddings. This may involve vector search techniques such as cosine similarity and k-nearest neighbors. Output is generated based on the identified item embeddings. For example, item list identifiers associated with the identified item embeddings may be used to generate search results or recommendations, thereby identifying the item lists identified by those item list identifiers. In some cases, a similarity metric determined for each item embedding (i.e., indicating how similar the item embedding is to the query embedding) may be used to rank the search results or recommendations.

[0021] Various aspects of the technology described herein provide many improvements over existing item retrieval techniques. For example, computational resource consumption is improved compared to the prior art. Specifically, a generative model is used to generate text that can better match item lists on a listing platform relative to an initial input (e.g., a user query or seed item). For example, the model output can include text identifying global products or product descriptions for identifying more relevant item lists as search results or recommendations. The use of query embeddings for the model output and item embeddings for the item lists further improves the relevance of search results and recommendations. In addition, generating query embeddings and item embeddings offline reduces latency and allows for large-scale item retrieval on large listing platforms.

[0022] The relevance of the item lists returned using the various aspects of the technology described herein eliminates (or at least reduces) repeated user queries, search result selections, and rendering of item lists relative to conventional techniques. This is because a relevant item list is provided, eliminating the need for the user to continually enter various search queries to access search results and / or continually make item selections to obtain more information about the presented item list. Therefore, the various aspects of the technology described herein reduce computing resource consumption, such as packet generation costs. For example, a user query (e.g., an HTTP request) will only need to traverse a computer network once (or a relatively small number of times relative to the prior art). Specifically, the content or payload of the user query is supplemented with header information or other metadata within a packet in a TCP / IP and other protocol network, and is only performed once for the initial user query. This packet of the user query is only sent over the network once or a relatively small number of times. Therefore, there is no repeated metadata generation and continuous packet transmission over the computer network.

[0023] Likewise, aspects of the technology described herein improve storage device or disk I / O and query execution functionality because they only require a single access to the disk (or a smaller number of accesses relative to existing search technologies). As described above, deficiencies in existing search and recommendation technologies result in repeated user queries, search result selection, and item list rendering. This results in multiple traversals of the disk. In contrast, the aspects described herein reduce storage device I / O because the user only provides minimal input, so the computing system does not have to frequently contact the storage device to perform read or write operations. For example, the system can utilize model outputs from a generative model to identify a list of items that meet the user's intent from a single user query (or from fewer queries relative to existing technologies) and / or a single round of providing recommendations. Therefore, because disk I / O is significantly reduced, wear and tear on components such as read / write heads is less severe. Example of an item retrieval system using generative model output

[0024] Referring now to the accompanying drawings, Figure 1 is a block diagram illustrating an exemplary system 100 for performing item retrieval on a list platform by utilizing generative model outputs according to an embodiment of the present disclosure. It should be understood that this arrangement and other arrangements described herein are set forth as examples only. Other arrangements and elements (e.g., machines, interfaces, functions, commands, and groupings of functions) can be used to supplement or replace the arrangements and elements shown, and some elements can be omitted entirely. In addition, many of the elements described herein are functional entities that can be implemented as discrete or distributed components or in combination with other components and in any suitable combination and location. The various functions performed by one or more entities described herein can be performed by hardware, firmware, and / or software. For example, the various functions can be performed by a processor that executes instructions stored in a memory.

[0025] System 100 is an example of a suitable architecture for implementing some aspects of the present disclosure. System 100 includes a user device 102, a listing platform 104, and an item retrieval system 106, among other components not shown. Figure 1 Each of the user device 102, listing platform 104, and item retrieval system 106 shown in FIG may include one or more computer devices, as discussed below. Figure 6 The computing device 600. Figure 1 As shown, the user device 102, the list platform 104, and the item retrieval system 106 can communicate via a network 110, which can include, but is not limited to, one or more local area networks (LANs) and / or wide area networks (WANs). Such network environments are common in offices, enterprise-wide computer networks, intranets, and the Internet. It should be understood that within the scope of the present technology, any number of user devices and servers can be employed within the system 100. Each can include a single device or multiple devices collaborating in a distributed environment. For example, the list platform 104 and the item retrieval system 106 can each be provided by multiple server devices that collectively provide the functionality of the list platform 104 and the item retrieval system 106 as described herein. In addition, other components not shown can also be included within the network environment.

[0026] The user device 102 can be a client device located on the client side of the operating environment 100, while the list platform 104 and the item retrieval system 106 can be located on the server side of the operating environment 100. The list platform 104 and / or the item retrieval system 106 can each include server-side software that is designed to work in conjunction with the client software on the user device 102 to implement any combination of the features and functions discussed in this disclosure. For example, the user device 102 can include an application 108 for interacting with the list platform 104 and / or the item retrieval system 106. The application 108 can be, for example, a web browser or a dedicated application for providing functionality (such as the functionality described herein). This division of the operating environment 100 is provided to illustrate an example of a suitable environment, and for every implementation, it is not required that any combination of the list platform 104 and the item retrieval system 106 be maintained as separate entities. For example, in some aspects, the item retrieval system 106 is part of the list platform 104. While operating environment 100 illustrates a configuration in a networked environment with separate user devices, listing platforms, and item retrieval systems, it should be understood that other configurations combining aspects of the various components may be employed.

[0027] User device 102 may include any type of computing device that a user can use. For example, in one aspect, the user device may be a computer system described herein. Figure 6 The type of computing device 600 depicted. By way of example and not limitation, user device 102 may be embodied as a personal computer (PC), a laptop computer, a mobile or mobile device, a smartphone, a tablet computer, a smartwatch, a wearable computer, a personal digital assistant (PDA), an MP3 player, a global positioning system (GPS) or device, a video player, a handheld communication device, a gaming device or system, an entertainment system, an in-vehicle computer system, an embedded system controller, a remote control, a home appliance, a consumer electronic device, a workstation, or any combination of the devices depicted, or any other suitable device. A user may be associated with user device 102 and may interact with listing platform 104 and / or item retrieval system 106 via user device 102.

[0028] The listing platform 104 may be implemented using one or more server devices, one or more platforms with corresponding application programming interfaces, a cloud infrastructure, etc. The listing platform 104 generally provides item listings to user devices (e.g., user device 102), describing items (physical or digital) available for purchase, rental, streaming, downloading, etc. For example, the listing platform 104 may include an e-commerce platform where products or services are listed and available for purchase by a user of the user device 102 upon navigating to the listing platform 104. As other examples, the listing platform 104 may include a rental platform that lists various items for rental (e.g., equipment, tools, real estate, vehicles, contract labor), or a media platform that lists digital content items (e.g., digital content for streaming / downloading).

[0029] The functionality of the listing platform 104 includes providing an interface capable of presenting item listings to users of the listing platform 104. The item listing data repository 124 stores item listings of items available for sale / rental / consumption via the listing platform 104. Each item listing may include a description associated with the item, including one or more of the following: monetary price, reviews, item image, shipping options, ratings, item condition, item size, item color, etc. In various aspects, each item is associated with one or more categories, including meta-categories and leaf categories. For example, meta-categories can be divided into sub-categories (or sub-categories), while leaf categories cannot be divided.

[0030] The item retrieval system 106 utilizes generative models when performing item retrieval (e.g., searching and recommending) for the listing platform. Figure 1 As shown, the project retrieval system 106 includes a generative model component 112, a query lookup component 114, a query search component 116, an embedding component 118, a project search component 120, and a user interface component 122. The components of the project retrieval system 106 may be in addition to other components that provide other additional functionality beyond the features described herein. The project retrieval system 106 may be implemented using one or more server devices, one or more platforms with corresponding application programming interfaces, cloud infrastructure, etc. Although in Figure 1 In the configuration shown, the item retrieval system 106 is shown as being separate from the listing platform 104 and the user device 102, but it should be understood that in other configurations, some functionality of the item retrieval system 106 may be provided on the listing platform 104 and / or the user device 102. In addition, while components 112, 114, 116, 118, 120, 122 are shown as being part of the item retrieval system 106, in other configurations, one or more components may be provided by the listing platform 104 or the user device 102. Figure 1 The components may be provided by a single entity or by multiple entities.

[0031] In some aspects, the functions performed by the components of the item retrieval system 106 are associated with one or more applications, services, or routines. Specifically, such applications, services, or routines may operate on one or more user devices, servers, may be distributed across one or more user devices and servers, or may be implemented in the cloud. Furthermore, in some aspects, these components of the item retrieval system 106 may be distributed across a network (including one or more servers and client devices), in the cloud, and / or may reside on a user device. Furthermore, these components, the functions performed by these components, or the services performed by these components may be implemented at an appropriate abstraction layer (e.g., an operating system layer, an application layer, a hardware layer, etc. of a computing system). Alternatively or in addition, the functions of these components and / or aspects of the techniques described herein may be performed, at least in part, by one or more hardware logic components. For example, but not limited to, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like. Additionally, while functionality is described herein with respect to specific components shown in the example system 100, it is contemplated that in some aspects the functionality of these components may be shared or distributed across other components.

[0032] Based on the interaction between a user device (e.g., user device 102) and the list platform 104, input for item retrieval is received by the item retrieval system 106. In the case of a search, a user may submit a user query to perform a search for items on the list platform 104. For example, a user may submit a text-based or voice-based user query via an application 108 on the user device 102 and provide input to the item retrieval system 106 based on the user query. In the case of a recommendation, the item retrieval system 106 may receive input based on a seed item (or multiple seed items). For example, a user may choose to view a list page for an item and access recommendations included on the list page, and input identifying the item (e.g., an item title or an item description) may be received. In some aspects, the input may be supplemented with additional information, such as user information of the current user interacting with the list platform 104 (e.g., attribute or behavioral information about the user), chat history from the user's current or previous chat sessions, and information from other users.

[0033] Given an input for an item retrieval, the generative model component 112 of the item retrieval system 106 uses the generative model to generate an output for querying a list of available items on a listing platform. The model output includes a text string generated by the generative model. In some instances, the text string identifies a "global product" or "global item," for example, which is a real-world entity such as an iPhone 15 or any other product with a UPC number or an ISBN number. The text string can identify a global product by name without identifying any specific item listing from the listing platform. The model output for a given input can identify a single global item or multiple global items. In another aspect, instead of identifying a global item by name or other identifier, the text string output by the generative model can include an item description that provides item attributes. For example, the item description for a "4k 55-inch OLED TV" includes attributes that specify the resolution, screen size, and screen type of the TV.

[0034] In some aspects, the generative model component 112 generates prompts using the received input for item retrieval and provides the prompts to the generative model to generate model outputs. The prompts may include text strings that instruct the generative model on how to generate the model outputs. The prompts used for a given input may be based on the type of item retrieval being performed (e.g., search or recommendation). For example, in a recommendation scenario, the prompt may include a text string such as "Find items related to [seed item from input]." As another example, in a search scenario, the prompt may include a text string such as "Assuming you are an expert recommender system, find the best items for this user query."

[0035] In some cases, prompts can be generated based on supplementary information associated with the input (e.g., user information). For example, suppose a user submits the query "Show me shirts I can wear to an event on Friday night." In this case, the system can retrieve supplementary information, such as user attributes (e.g., "gender: male," "age: 35," and "size: medium"), as well as information about the type of event on Friday night based on the user's calendar. This supplementary information can then be used to generate prompts that enable the generative model to generate appropriate outputs.

[0036] The generative models used by the generative model component 112 to provide output for performing item retrieval may include language models, which comprise a set of statistical or probabilistic functions used to perform natural language processing (NLP) to understand, learn, and / or generate human natural language content. For example, a language model can be a tool for determining the probability of a given word sequence appearing in a sentence or natural language sequence. Simply put, it can be a model trained to predict the next word in a sentence. When a language model is trained on a large amount of data and / or has a large number of parameters, it is referred to as a large language model (LLM). Some examples of LLMs include Google's BERT and OpenAI's GPT-3 and GPT-4. These models have capabilities ranging from writing simple papers to generating complex computer code (all with limited or no supervision). Therefore, LLMs can include deep neural networks, which are extremely large (billions to hundreds of billions of parameters) and are trained on large amounts of text to understand, process, and generate human natural language. These models can predict future words in a sentence, allowing them to generate sentences in a manner similar to human speech and writing.

[0037] According to some aspects, the generative model used by generative model component 112 comprises a neural network. As used herein, a neural network comprises multiple operational layers, including an input layer and an output layer, and any number of hidden layers between the input and output layers. Each layer comprises neurons. Different types of layers and networks connect neurons in different ways. Neurons have weights, activation functions, and outputs, where the activation function defines the output of the neuron given a given input (including weights). Weights are adjustable parameters that cause the network to produce the correct output.

[0038] In some configurations, the generative model used by the generative model component 112 is a pre-trained model (e.g., GPT-4) that has not yet been fine-tuned. In other configurations, the generative model is a model built and trained from scratch, or a pre-trained model that has already been fine-tuned. In such configurations, the generative model can be trained or fine-tuned using training data. For example, the training data can include data pairs where an input is paired with a true output to train the generative model to generate output text corresponding to the true output. During training, the weights associated with each neuron can be updated. Initially, the generative model can include random weight values ​​or pre-trained weight values, which are adjusted during training. In one aspect, the generative model is trained using backpropagation. The backpropagation process includes a forward pass, a loss function, a backward pass, and weight updates. This process is repeated using the training data. For example, each iteration can include providing an input text string to the model, generating an output text string from the generative model, comparing the output text string from the generative model to the true output text string paired with the input text string (e.g., calculating a loss), and updating the model based on the comparison. The goal is to update the weights of each neuron (or other model component) so that the generative model produces a specific form of output, such as identifying a global item or providing a textual description of the item. Once training is complete, the weights associated with a given neuron can remain fixed. Other data passed between neurons can change in response to a given input. Retraining the network with additional training data can update one or more weights in one or more neurons.

[0039] Given a model output from the generative model component 112, the query lookup component 114 attempts to find a query embedding for the model output by performing a lookup in the key-value store 126. The key-value store 126 is a data repository that maps known queries, stored as keys, to corresponding query embeddings, stored as values. A known query is a text string query for which a query embedding has been generated. Each query embedding in the key-value store 126 is a vector representation of the corresponding known query generated by the embedding component 118, as described in further detail below. For a given model output from the generative model component 112, the query lookup component 114 uses the model output to perform a lookup to determine whether a matching query is stored in the key-value store 126. In some aspects, a matching query is a known query with a text string that exactly matches the text string of the model output from the generative model component 112. If a match exists, the query lookup component 114 retrieves the query embedding stored as a value associated with the matching query in the key-value store 126.

[0040] The query search component 116 uses the model output from the generative model component 112 to search the known query index 128 to identify known queries (or queries) that are related to the model output but do not exactly match the model output. As previously described, a known query is a query whose query embedding has been generated by the embedding component 118 and stored in the key-value store 126. The query search component 116 is used in certain configurations where no matching query for the model output is found in the key-value store 126. In such configurations, when no matching query is found in the key-value store 126, the query search component 116 queries the known query index 128 based on the model output to identify one or more known queries related to the model output. The query search component 116 and the known query index 128 can employ any of a variety of search techniques and architectures. By way of example only, and not limitation, the query search component 116 and the known query index 128 may comprise an Elasticsearch platform that uses fuzzy matching to identify one or more known queries in the known query index 126 that are related to the model output.

[0041] The embedding component 118 employs one or more embedding models to generate query embeddings for queries and item embeddings for item listings available on the listing platform 104. In some aspects, the same embedding model is used to generate both query embeddings and item embeddings, although different embedding models can be used in other configurations.

[0042] The embedding model comprises a machine learning model (e.g., a neural network) that converts input data into vector representations (referred to herein as embeddings) in an embedding space (sometimes referred to as a latent vector space). The embedding space of the embedding model provides a multi-dimensional space in which similarities between embeddings can be determined, for example, based on geometric distances between embeddings in the embedding space. Thus, a query embedding generated by the embedding model comprises a vector representation of the query in the embedding space of the embedding model, and an item embedding generated by the embedding model comprises a vector representation of a list of items in the embedding space of the embedding model. The embeddings generated by the embedding component 118 allow for determination of similarities between query embeddings and item embeddings for item retrieval purposes, as will be described in further detail below.

[0043] In some aspects, the embedding component 118 generates query embeddings for queries offline (i.e., not at runtime when processing input for item retrieval). For example, the embedding component 118 can access a query log of the listing platform 104 that stores historical queries submitted by users for searching for item listings on the listing platform 104, and the embedding component 118 can generate query embeddings for these historical queries. Each of these query embeddings is then stored in the key-value store 126 in association with the corresponding query that generated the query embedding (each query is stored as a key in the key-value store 126). Each query for which a query embedding is generated can also be indexed as a known query in the known query index 128. In some cases, the embedding component 118 accesses a model output generated by the generative model component 112, where there is no matching query for the model output in the key-value store 126 (e.g., from the log of the model output), and the embedding component 118 generates a query embedding for the model output. Each of these query embeddings is then stored in a key-value store 126 in association with the corresponding model output that generated the query embedding (each model output is stored as a key in the key-value store 126). Each model output that generates the query embedding can also be indexed as a known query in the known query index 128.

[0044] While in some aspects the embedding component 118 can generate query embeddings offline, in other configurations the embedding component 118 can generate query embeddings for the model output at runtime. For example, when the query lookup component 114 does not find a matching query for the model output in the key-value store 126, the model output can be provided to the embedding component 118 to generate a query embedding for the model output, and the query embedding can be used for item retrieval instead of searching for similar known queries in the known query index 128. In some cases, the model output and the query embedding generated by the embedding component 118 at runtime can be added to the key-value store 126, and the model output can be indexed as a known query in the known query index 128.

[0045] For item embeddings, the embedding component 118 can take as input any of a variety of available data for each item list available on the list platform 104. For example, text data from an item list (e.g., an item title and an item description) can be provided as input to an embedding model to generate item embeddings for the item list. In some aspects, a multimodal embedding model can be used, which can, for example, take as input an image of an item in addition to the text of the item. The item embeddings generated by the embedding component 118 can be stored in the item embedding data repository 130 for item retrieval purposes, as described below. Each item embedding can be stored along with an item list identifier that identifies the corresponding item list.

[0046] According to various aspects, the embedding component 118 can use an embedding model that is pre-trained, built from scratch, or fine-tuned from a pre-trained model. In some aspects, when the same embedding model is used for both query embedding and item embedding, the embedding model can be a neural network trained or fine-tuned using a contrastive learning method, where the training data includes pairs of queries and item lists. A given pair in the training data can be a positive example (i.e., the item list is relevant to the query) or a negative example (i.e., the item list is irrelevant to the query). During training, a loss function (e.g., a contrastive loss function or a triplet loss function) can be used to cause the embedding model to position the embeddings of positive examples close together in the embedding space, while separating the embeddings of negative examples from each other in the embedding space.

[0047] Given the query embedding identified for the model output, the item search component 120 uses the item embeddings of the item lists on the list platform 104 stored in the item embedding data store 130 to identify an item list. In the case where the initial input to the item retrieval is a user query, the item search component 120 identifies an item list to return as a search result in response to the user query. In the case where the initial input to the item retrieval is a recommended seed item, the item search component 120 identifies an item list to provide as a recommendation.

[0048] In some aspects, the item search component 120 determines a similarity measure between a given query embedding and the item embeddings in the item embedding data repository 130. Various techniques for determining similarity can be used, such as cosine similarity and k-nearest neighbors. The item retrieval component 120 selects one or more item embeddings based on the similarity. Item list identifiers stored in association with each selected item embedding in the item embedding data repository 130 can be retrieved and used to identify the corresponding item list to generate search results or recommendations in response to the initial input of the item retrieval. In some cases, the search results or recommendations returned are ranked at least in part based on the similarity of each item embedding to the query embedding (e.g., based on the similarity measure).

[0049] The user interface component 122 provides one or more user interfaces for interacting with the listing platform 104 and / or the item retrieval service 106. Figure 1104 , but in some configurations, the user interface component 122 can be part of the listing platform 104. The user interface component 122 provides one or more user interfaces to a user device (e.g., user device 102). In some cases, the user interface can be presented on the user device 102 via an application 108, which can be a web browser or a dedicated application for interacting with the listing platform 104 and / or the recommendation service. For example, the user interface component 122 can provide a user interface for providing search results and / or recommendations identifying a listing of items selected by the item retrieval system 106, etc.

[0050] Figure 2 A block diagram illustrating a process 200 for performing item retrieval using a generative model according to one aspect of the present technology is provided. The process 200 includes both an offline process 202 and a runtime process 204. Figure 2 As shown, the offline process 202 includes an embedding service 206 that generates query embeddings and item embeddings, which are stored in an offline embedding data store 208. The queries that generate the query embeddings may include, for example, historical queries from a query log and / or previous model outputs. The item embeddings are generated for item lists on a list platform. The queries and corresponding query embeddings are stored as key-value pairs in a key-value store 216, and the queries are also indexed as known queries in a known query index 220. In addition, the item embeddings are stored in an item embedding data store 224 in association with the item list identifier.

[0051] At runtime 204, an initial input is received at the item retrieval service 210, such as a user query in the case of a search or a seed item in the case of a recommendation. The item retrieval service 210 may be provided by, for example, Figure 1 The project retrieval system 106 provides. Figure 2 For example, the project retrieval service 210 coordinates the runtime processes of multiple services to perform project retrieval.

[0052] The item retrieval service 210 provides input to the generative model service 212, which may correspond to Figure 1 The generative model component 112 of the generative model service 212 employs the generative model to generate text based on the input to produce a model output. In some cases, this includes generating prompts using the input and providing the prompts to the generative model. The model output is returned to the item retrieval service 210, which provides the model output to the query lookup service 214, which may correspond to Figure 1The query lookup component 114 of the query lookup service 214 determines whether there is a matching query in the key-value store 216. If a matching query is found in the key-value store 216, the query embedding stored in association with the matching query is returned to the item retrieval service 210.

[0053] If no matching query is found in the key-value store 216, a no-match indication is returned to the item retrieval service 210, which then provides the model output to the query search service 218, which may correspond to Figure 1 The query search component 116 of the embodiment of the present invention is provided. The query search service 218 uses the model output to search the known query index 220 to identify the known query and returns the known query to the item retrieval service 210. The known query is provided to the query lookup service 214, which performs a lookup on the key-value store 216 using the known query to identify the corresponding query embedding and returns the query embedding to the item retrieval service 210.

[0054] The item retrieval service 210 provides the query embedding to the item search service 222, which may correspond to Figure 1 Item search component 120. Item search service 222 searches item embedding repository 224 to identify item embeddings similar to the query embedding, for example, using vector search techniques. An indication of an item list corresponding to the identified item embedding (e.g., using an item list identifier) ​​is returned to item retrieval service 210. Item retrieval service 210 provides an output identifying the item list. In the case of a search, the output can be used to provide search results. In the case of a recommendation, the output can be used to provide item list recommendations.

[0055] Next go to Figure 3 , a block diagram illustrating another process 300 for performing item retrieval using a generative model according to one aspect of the present technology is provided. Similar to process 200, process 300 includes both an offline process 302 and a runtime process 304. Compared to process 200, if no match is found for the model output in the key-value store 316, the runtime process 304 generates a query embedding from the model output using a runtime embedding service 318, rather than performing a known query search.

[0056] like Figure 3As shown, the offline process 302 includes an embedding service 306 that generates query embeddings and item embeddings, which are stored in an offline embedding data store 308. The queries used to generate the query embeddings may include, for example, historical queries from a query log and / or previous model outputs. The item embeddings are generated for the item list on the list platform. The query and the corresponding query embedding are stored as a key-value pair in a key-value store 316. In addition, the item embedding is stored in an item embedding data store 322 in association with the item list identifier.

[0057] At runtime 304, an initial input is received at the item retrieval service 310, such as a user query in the case of a search or a seed item in the case of a recommendation. The item retrieval service 310 may be provided by, for example, Figure 1 The project retrieval system 106 provides. Figure 3 In the example of , the project retrieval service 310 coordinates the runtime processes of multiple services to perform project retrieval.

[0058] The item retrieval service 310 provides input to the generative model service 312, which may correspond to Figure 1 The generative model component 112 of the generative model component 112. The generative model service 312 uses the generative model to generate text based on the input to produce a model output. In some cases, this includes using the input to generate prompts and providing the prompts to the generative model. The model output is returned to the item retrieval service 310, which provides the model output to the query lookup service 314, which can correspond to Figure 1 The query lookup component 114 of the query lookup service 314 determines whether there is a matching query in the key-value store 316. If a matching query is found in the key-value store 316, the query embedding stored in association with the matching query is returned to the item retrieval service 310.

[0059] If no matching query is found in the key-value store 316, a no-match indication is returned to the item retrieval service 310, which then provides the model output to the runtime embedding service 318, which may correspond to Figure 1 The runtime embedding service 318 includes an embedding model that uses the model output to generate query embeddings and returns the query embeddings to the item retrieval service 310. In some cases, the model output and the query embeddings are also added to the key-value store 316 as key-value pairs.

[0060] The project retrieval service 310 provides the query embedding returned by the query lookup service 314 (if there is a match in the key-value store 316) or the query embedding generated by the runtime embedding service 318 (if there is no match in the key-value store 316) to the project search service 320, which may correspond to Figure 1 Item search component 120. Item search service 320 searches item embedding repository 322 to identify item embeddings similar to the query embedding, for example, using vector search techniques. An indication of an item list corresponding to the identified item embedding (e.g., using an item list identifier) ​​is returned to item retrieval service 310. Item retrieval service 310 provides an output identifying the item list. In the case of a search, the output can be used to provide search results. In the case of a recommendation, the output can be used to provide item list recommendations.

[0061] Although Figure 2 and Figure 3 We discussed using a single query embedding to identify item embeddings, but it should be understood that any number of query embeddings can be used. For example, the model output may identify multiple global items, and a query embedding may be obtained for each global item. As another example, Figure 2 The query search service 218 can identify multiple known queries for the model output and can obtain a query embedding for each of these known queries. Example of an item retrieval method using the output of a generative model

[0062] Now refer to Figure 4 , provides a flowchart illustrating a method 400 for performing item retrieval using a generative model. The method 400 may be performed by, for example Figure 1 The method 400 and any other method described herein are performed by the item retrieval system 106. Each block of the method 400 and any other method described herein comprises a computing process performed using any combination of hardware, firmware, and / or software. For example, the various functions may be performed by a processor executing instructions stored in a memory. The methods may also be embodied as computer-usable instructions stored on a computer storage medium. The methods may be provided by a standalone application, a service, or a hosted service (standalone or in combination with another hosted service), or a plug-in to another product, to name a few.

[0063] As shown in box 402, input for item retrieval is received. The input can include, for example, a user query in the case of a search or a seed item in the case of a recommendation. The user query can include, for example, a text-based query or a voice-based query. The input is provided to a generative model, which generates text to provide a model output, as shown in box 404. In some aspects, the process can include generating prompts using the initial input received at box 402 and providing the prompts to the generative model. The prompts can be generated based on the type of item retrieval task. For example, one type of prompt can be used for search-based item retrieval, while another type of prompt can be used for recommendation-based item retrieval.

[0064] A lookup is performed on the key-value store using the model output, as shown in block 406. The key-value store stores known queries as keys and corresponding query embeddings as values ​​for each known query. The lookup determines whether there is a known query that matches the model output (i.e., a matching query). Therefore, a determination is made at block 408 as to whether there is a matching query in the key-value store for the model output. If there is a matching query, the query embedding stored in the key-value store in association with the matching query is retrieved, as shown in block 410. In some cases, multiple matching queries can be identified based on the model output, and a query embedding for each of these matching queries can be returned. For example, the model output can identify multiple global projects, and a matching query can be identified for each global project identified in the model output.

[0065] Alternatively, if no matching query exists, a search is performed on a known query index using the model output, as shown in block 412. The known query index is a data repository that indexes known queries in a manner that allows identification of known queries that are related to, but not exactly matching, the model output. For example, a fuzzy search can be performed using the model output as input to identify one or more known queries that are similar to the model output. A search is performed on a key-value store using the known queries identified from searching the known query index, as shown in block 414. The search identifies queries in the key-value store that match the known query. As shown in block 416, the query embedding stored in the key-value store associated with the matching query is retrieved. In some cases, the search performed at block 412 returns multiple known queries, and a search is performed to retrieve the query embedding for each of these known queries.

[0066] Although Figure 4

[0045] Although not shown, in some embodiments, query embeddings can be returned from both matching queries identified in the key-value store for the model output and known queries identified from the known query index using the model output. For example, the model output may identify multiple global items. Matching queries can be identified in the key-value store for one or more of these global items from the model output, and query embeddings can be identified for each of those matching queries. Additionally, matching queries cannot be identified for one or more other global items in the model output, but known queries can be identified from the known query index for these other global items and used to retrieve additional query embeddings from the key-value store.

[0067] As shown in block 418, a vector search is performed on the item embedding data store using the query embedding obtained at blocks 410 and / or 416. In some aspects, the vector search identifies one or more item embeddings based on the similarity of the item embeddings to the query embedding. Any of a variety of vector search techniques (e.g., cosine similarity and k-nearest neighbors) can be used to identify the one or more item embeddings. In some cases, a similarity metric is determined for each of the plurality of item embeddings, and one or more item embeddings are selected based on their corresponding similarity metrics.

[0068] An output is provided in response to the initial input received at block 402, as shown in block 420. The output is based on the item lists corresponding to the one or more item embeddings identified at block 418. For example, where the initial input is a user query, the output may include one or more search results that identify the one or more item lists corresponding to the one or more item embeddings. Where the initial input is a recommended seed item, the output may include one or more recommendations that identify the one or more item lists corresponding to the one or more item embeddings. For example, the item lists identified in the output may be determined based on an item list identifier stored in an item embedding repository in association with each item embedding returned from the search performed at block 418.

[0069] Next go to Figure 5 , provides a flowchart illustrating another method 500 for performing item retrieval using a generative model. The method 500 may be performed by, for example Figure 1The item retrieval system 106 is executed. As shown in box 502, input for item retrieval is received. The input may include, for example, a user query in the case of a search or a seed item in the case of a recommendation. The user query may include, for example, a text-based query or a voice-based query. The input is provided to a generative model, which generates text to provide a model output, as shown in box 504. In some aspects, the process may include: generating prompts using the initial input received at box 502, and providing the prompts to the generative model. The prompts may be generated based on the type of item retrieval task. For example, one type of prompt may be used for search-based item retrieval, while another type of prompt may be used for recommendation-based item retrieval.

[0070] A lookup is performed on the key-value store using the model output, as shown in block 506. The key-value store stores known queries as keys and corresponding query embeddings as values ​​for each known query. The lookup determines whether there is a known query that matches the model output (i.e., a matching query). Therefore, a determination is made at block 508 as to whether there is a matching query in the key-value store for the model output. If there is a matching query, the query embedding stored in the key-value store in association with the matching query is retrieved, as shown in block 510. In some cases, multiple matching queries can be identified based on the model output, and a query embedding for each of these matching queries can be returned. For example, the model output can identify multiple global projects, and a matching query can be identified for each global project identified in the model output.

[0071] Alternatively, if there is no matching query, a query embedding is generated from the model output, as shown in block 512. For example, the model output can be provided to an embedding model, which generates a query embedding based on the model output. In some cases, multiple query embeddings can be generated for the model output. For example, the model output can identify multiple global items, and a query embedding can be generated for each global item identified in the model output.

[0072] although Figure 5

[0045] Although not shown, in some embodiments, query embeddings can be returned from both matching queries identified in the key-value store for the model output and query embeddings generated by the embedding model based on the model output. For example, the model output may identify multiple global projects. For one or more of these global projects from the model output, matching queries can be identified in the key-value store, and query embeddings can be identified for each of those matching queries. Additionally, matching queries may not be identified for one or more other global projects in the model output, but query embeddings can be generated by the embedding model for each of those other global projects.

[0073] As shown in block 514, a vector search is performed on the item embedding data store using the query embedding obtained at blocks 510 and / or 512. In some aspects, the vector search identifies one or more item embeddings based on the similarity of the item embeddings to the query embedding. Any of a variety of vector search techniques (e.g., cosine similarity and k-nearest neighbors) can be used to identify the one or more item embeddings. In some cases, a similarity metric is determined for each of the plurality of item embeddings, and one or more item embeddings are selected based on their corresponding similarity metrics.

[0074] An output is provided in response to the initial input received at block 502, as shown in block 516. The output is based on the item lists corresponding to the one or more item embeddings identified at block 514. For example, where the initial input is a user query, the output may include one or more search results that identify the one or more item lists corresponding to the one or more item embeddings. Where the initial input is a recommended seed item, the output may include one or more recommendations that identify the one or more item lists corresponding to the one or more item embeddings. For example, the item lists identified in the output may be determined based on an item list identifier stored in an item embedding repository in association with each item embedding returned from the search performed at block 514. Exemplary Operating Environment

[0075] Having described the implementation of the present disclosure, the following describes an exemplary operating environment in which embodiments of the present technology can be implemented in order to provide a general context for various aspects of the present disclosure. Figure 6 , an exemplary operating environment for implementing embodiments of the present technology is shown and generally designated as computing device 600. Computing device 600 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the technology. Neither should computing device 600 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated.

[0076] The technology may be described in the general context of computer code or machine-usable instructions (including computer-executable instructions, such as program modules) executed by a computer or other machine (e.g., a personal data assistant or other handheld device). Generally, program modules, including routines, programs, objects, components, data structures, etc., refer to code that performs specific tasks or implements specific abstract data types. The technology can be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialized computing devices, and the like. The technology may also be practiced in distributed computing environments, where tasks are performed by remote processing devices linked through a communications network.

[0077] refer to Figure 6, computing device 600 includes a bus 610 that directly or indirectly couples the following devices: memory 612, one or more processors 614, one or more presentation components 616, input / output (I / O) ports 618, I / O components 620, and an illustrative power supply 622. Bus 610 may represent one or more buses (e.g., an address bus, a data bus, or a combination thereof). Although for clarity, Figure 6 The various boxes of FIG are represented by lines, but in reality, it is not so clear to depict the various components, and metaphorically, the lines would more accurately be gray and fuzzy. For example, a presentation component such as a display device can be considered an I / O component. In addition, a processor has memory. The inventors recognize that this is the nature of the art and reiterate that Figure 6 The figures illustrate only exemplary computing devices that can be used in conjunction with one or more embodiments of the present technology. No distinction is made between categories such as "workstation," "server," "laptop," "handheld device," etc., as all of these categories are in the Figure 6 are considered within the scope of and refer to "computing devices".

[0078] The computing device 600 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by the computing device 600 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0079] Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage devices, magnetic cassettes, magnetic tape, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to store the desired information and which can be accessed by the computing device 600. The terms "computer storage medium" and "computer storage media" do not themselves include signals.

[0080] Communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal (such as a carrier wave or other transport mechanism), and includes any information transmission media. The term "modulated data signal" refers to a signal that has one or more of its characteristics set or changed in such a manner as to encode the information in the signal. By way of example, and not limitation, communication media include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.

[0081] Memory 612 includes computer storage media in the form of volatile and / or non-volatile memory. Memory can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, and the like. Computing device 600 includes one or more processors that read data from various entities such as memory 612 or I / O components 620. Presentation component 616 presents data indications to a user or other device. Exemplary presentation components include a display device, a speaker, a printing component, a vibration component, and the like.

[0082] I / O ports 618 allow computing device 600 to logically couple with other devices, including I / O components 620, some of which may be built-in. Illustrative components include a microphone, joystick, game controller, satellite dish, scanner, printer, wireless device, etc. I / O components 620 can provide a natural user interface (NUI) that processes in-air gestures, voice, or other physiological input generated by the user. In some cases, the input can be transmitted to an appropriate network element for further processing. The NUI can implement any combination of the following: voice recognition, touch and stylus recognition, facial recognition, biometric recognition, gesture recognition both on and near the screen, in-air gestures, head and eye tracking, and touch recognition associated with the display on computing device 600. Computing device 600 can be equipped with a depth camera, such as a stereo camera system, an infrared camera system, an RGB camera system, or combinations thereof, for gesture detection and recognition. In addition, computing device 600 can be equipped with an accelerometer or gyroscope capable of detecting motion.

[0083] The present technology has been described with respect to specific embodiments which are intended in all respects to be illustrative rather than restrictive. Alternative embodiments will become apparent to those skilled in the art to which the technology pertains without departing from the scope of the present technology.

[0084] Having identified various components used herein, it should be understood that any number of components and arrangements may be employed to implement the desired functionality within the scope of this disclosure. For example, for clarity of concept, the components in the embodiments shown in the figures are represented by lines. Other arrangements of these and other components may also be implemented. For example, although some components are depicted as single components, many elements described herein may be implemented as discrete or distributed components or in combination with other components, and may be implemented in any suitable combination and position. Some elements may be omitted entirely. In addition, the various functions performed by one or more entities described herein may be performed by hardware, firmware, and / or software, as described below. For example, the various functions may be performed by a processor executing instructions stored in a memory. Therefore, other arrangements and elements (e.g., machines, interfaces, functions, sequences, and functional groupings) may be used in addition to or in place of the arrangements and elements shown.

[0085] The embodiments described herein may be combined with one or more specifically described alternatives. Specifically, the claimed embodiments may include references to more than one other embodiment in the alternatives. The claimed embodiments may specify further limitations of the claimed subject matter.

[0086] The subject matter of embodiments of the present technology is described herein with specificity to satisfy statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter may also be embodied in other ways, to include different steps or combinations of steps similar to those described in this document, in conjunction with other prior art or future technologies. Furthermore, although the terms "step" and / or "box" may be used herein to refer to different elements of the method employed, such terms should not be interpreted as implying any particular order among or between the various steps disclosed herein, unless and except where an order of the various steps is explicitly described.

[0087] For purposes of this disclosure, the word "including" has the same broad meaning as the word "comprising," and the word "access" includes "receiving," "referencing," or "retrieving." Furthermore, the word "communicating" has the same broad meaning as the word "receiving" or "sending," which is facilitated by a software or hardware-based bus, receiver, or transmitter using the communication media described herein. Furthermore, unless otherwise indicated to the contrary, words such as "a," "an," and "an" include the plural as well as the singular. Thus, for example, the constraint of "a feature" is satisfied where there is one or more features. Furthermore, the term "or" includes conjunctive, disjunctive, and both (thus, a or b includes a or b, and a and b).

[0088] For the purposes of the detailed discussion above, embodiments of the present technology are described with reference to a distributed computing environment; however, the distributed computing environment described herein is merely an example. Components may be configured to perform novel embodiments of the present technology, where the term "configured to" may refer to being "programmed" to perform specific tasks or implement specific abstract data types using code. Furthermore, while embodiments of the present technology may generally refer to the technical solution environments and schematics described herein, it will be appreciated that the described technology can be extended to other implementation contexts.

[0089] From the foregoing, it will be seen that the present technology is well adapted to attain all of the ends and objectives set forth above, as well as other advantages that are obvious and inherent to the systems and methods. It will be understood that certain features and subcombinations are useful and may be employed without reference to other features and subcombinations. This is contemplated by and is within the scope of the claims.

Claims

1. One or more computer storage media storing computer-usable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising: receiving input for item retrieval on the list platform; causing a generative model to generate text using the input to provide a model output; The query embeddings for the model output are retrieved from the key-value store via: Perform a lookup on the key-value store using the model output, In response to finding a matching query for the model output in the key-value store, retrieving a first query embedding stored in the key-value store in association with the matching query, wherein the first query embedding is provided as the query embedding for the model output, and In response to an absence of a matching query for the model output in the key-value store, performing a search on a known query index using the model output to identify a known query, and retrieving a second query embedding stored in the key-value store in association with the known query, wherein the second query is provided as a query embedding for the model output; identifying one or more item embeddings based on similarity to a query embedding output for the model; and An output is provided that identifies one or more item lists corresponding to the one or more item embeddings.

2. One or more computer storage media according to claim 1, wherein: The input to the item retrieval comprises a user query from a user device, and wherein the output comprises one or more search results identifying the one or more item lists.

3. The one or more computer storage media of claim 1 , wherein: The input to the item retrieval includes a seed item, and wherein the output includes one or more recommendations identifying the one or more item lists.

4. The one or more computer storage media of claim 1 , wherein: Causing the generative model to generate the text to provide the model output includes: generating a prompt using the input retrieved from the item; and The prompt is provided as input to the generative model.

5. One or more computer storage media according to claim 4, wherein: The prompt instructs the generative model to generate a text string identifying one or more global items.

6. The one or more computer storage media of claim 1, wherein: The key-value store and the known query index store a plurality of known queries, and wherein at least a portion of the plurality of known queries include historical queries from a query log of the listing platform.

7. The one or more computer storage media of claim 1, wherein: The key-value store and the known query index store a plurality of known queries, and wherein at least a portion of the plurality of known queries comprise model outputs previously generated by the generative model using inputs from previous item retrievals.

8. The one or more computer storage media of claim 1, wherein: When no matching query for the model output is found in the key-value store, the operations further include: causing the embedding model to generate a new query embedding for the model output; and The new query embedding is stored in the key-value store in association with the model output.

9. The one or more computer storage media of claim 8, wherein: The operations further include: The model outputs are stored as known queries in the known query index.

10. A computer-implemented method comprising: receiving input for item retrieval on the list platform; causing a generative model to generate text using the input to provide a model output; Obtain query embeddings for the model output as follows: Perform a lookup on the key-value store using said model output, In response to finding a matching query for the model output in the key-value store, retrieving a query embedding stored in the key-value store in association with the matching query, and using the query embedding retrieved from the key-value store as the query embedding for the model output, and In response to there not being a match for the query in the key-value store, causing the embedding model to generate a new query embedding, and using the new query embedding as the query embedding for the model output; identifying one or more item embeddings based on similarity to a query embedding output for the model; as well as An output is provided that identifies one or more item lists corresponding to the one or more item embeddings.

11. The computer-implemented method of claim 10, wherein: The input to the item retrieval comprises a user query from a user device, and wherein the output comprises one or more search results identifying the one or more item lists.

12. The computer-implemented method of claim 10, wherein: The input to the item retrieval includes a seed item, and wherein the output includes one or more recommendations identifying the one or more item lists.

13. The computer-implemented method of claim 10, wherein: Causing the generative model to generate the text to provide the model output includes: generating a prompt using the input retrieved from the item; and The prompt is provided as input to the generative model.

14. The computer-implemented method of claim 10, wherein: The key-value store stores a plurality of known queries, and wherein at least a portion of the plurality of known queries include historical queries from a query log of the listing platform.

15. The computer-implemented method of claim 10, wherein: The key-value store stores a plurality of known queries, and wherein at least a portion of the plurality of known queries comprise model outputs previously generated by the generative model using previous inputs of item retrieval.

16. The computer-implemented method of claim 10, wherein: When no matching query for the model output is found in the key-value store, the operations further include: The new query embedding is stored in the key-value store in association with the model output.

17. A computer system comprising: one or more processors; as well as One or more computer storage media storing computer-usable instructions that, when used by the one or more processors, cause the computer system to perform operations comprising: receiving input for item retrieval on the list platform; causing a generative model to generate text using the input to provide a model output; identifying a first query embedding from a key-value store, the first query embedding corresponding to a first known query in the key-value store that matches a first portion of the model output; Obtain a second query embedding based on a second portion of the model output; identifying one or more item embeddings based on the first query embedding and / or the second query embedding; and An output is provided that identifies one or more item lists corresponding to the one or more item embeddings.

18. The computer system of claim 17, wherein: A first portion of the model output identifies a first item, and a second portion of the model output identifies a second item.

19. The computer system according to claim 17, wherein: Obtaining the second query embedding includes: determining a second known query from the known query index based on a relevance to the second portion of the model output; and A second query embedding corresponding to the second known query is identified from the key-value store.

20. The computer system of claim 17, wherein: Obtaining the second query embedding includes: Cause the embedding model to generate the second query embedding using a second portion of the model output.

Images