Auto-generated product recommendations based on questions and answers

Abstract

An automatic technique is disclosed for improving the quality of presented responses by highlighting relevant shopping recommendations. Shopping recommendations may be highlighted within the response itself or as an auxiliary list of suggestions. In the context of a question-and-answer pair, a model is described for selecting phrases from the response text (sequential strings of words called noun phrases) that point to potential products that are likely to represent relevant shopping recommendations. The noun phrases are then ranked in order of importance. The top-ranked noun phrases are used to search for products displayed associated with that noun phrase. Clicking or tapping on a highlighted noun phrase launches a shopping-related flow, such as presenting a widget with product recommendations or performing a search in a search engine.

Description

[Background technology]

[0001] Question answering (QA) is a popular feature in e-commerce services that many customers use as part of their shopping journey. QA involves building systems that automatically respond to questions posed by users in natural language. Part of QA is natural language processing, which involves processing text, understanding the meaning of specific words, understanding the conversational context of those words, and drawing inferences from the text and content. Once natural language processing is complete, a response can be provided that may be useful to the user when generating a new search for a specific product. For example, a user can generate a question such as "How do I remove dog hair from furniture?" and an automated response such as "Try a lint brush or rubber squeegee" can be provided. The user can then use that response to generate an appropriate search in the search box. While the QA experience is useful, it could be improved with further automation. [Brief explanation of the drawings]

[0002] [Figure 1] FIG. 1 is a user interface (UI) diagram in which noun phrases are highlighted and associated with products using connecting arrows. [Figure 2] FIG. 1 is a diagram of a UI that includes noun phrases in a response associated with a product and n-grams related to reviews of the product. [Figure 3] FIG. 1 is a diagram of a system showing a UI interacting with a backend system that includes a semantic similarity model. [Figure 4] Further details of the semantic similarity model are shown in Figure 3 . [Figure 5] We present a specific example of the semantic similarity model in Figure 4. [Figure 6] 3 and 4 illustrate obtaining click data for training a semantic similarity model. [Figure 7]FIG. 1 is a diagram of an example system showing multiple virtual machine instances running in a multi-tenant environment with a semantic similarity model running on a server computer. [Figure 8] 1 is a flowchart for generating a display having products associated with n-grams, according to one embodiment. [Figure 9] 10 is a flowchart according to another embodiment for generating a display having products associated with n-grams. [Figure 10] 1 depicts a generalized example of a suitable computing environment in which the described innovations may be implemented. DETAILED DESCRIPTION OF THE INVENTION

[0003] In QA, responses can stimulate customer interest and expand opportunities for shopping behavior. However, current solutions do not provide an easy and natural bridge for customers from asking a question to shopping activities. An automatic technique for improving the quality of presented responses by highlighting relevant shopping recommendations is disclosed. Shopping recommendations can be highlighted within the response itself or as an auxiliary list of suggestions. In the context of a question-and-answer pair, a model is described for selecting phrases from the response text (sequences of words called n-grams) that point to potential products that are likely to represent relevant shopping recommendations. From the n-grams, noun phrases can be extracted. A noun phrase is a type of n-gram that contains a noun. The noun phrases are then ranked in order of score. The highest-scoring noun phrase is used to search for products displayed associated with that noun phrase. Clicking (in a web interface) or tapping (in a mobile device) on a highlighted noun phrase launches a shopping-related flow, such as presenting a widget with product recommendations or performing a search in a search engine. Although the embodiments described herein focus on noun phrases, such embodiments are equally applicable to any type of n-gram.

[0004] FIG. 1 illustrates a UI 100 used for e-commerce. A user can enter a question 110 into an input text bar 112 and select a search indicator 116. While shown as a question, the input text bar can also receive a search query or a voice recognition command. A response 120 associated with the question is displayed. The response 120 can have a particular noun phrase, such as a noun phrase 130, highlighted by underlining, changing color, changing formatting, etc. The user has the option to select the noun phrase by clicking, hovering, etc. The highlighted noun phrase 130 is accompanied by an arrow 132 pointing toward the product search results 140, creating an association between the noun phrase 130 and the product search results 140. While the association is indicated using an arrow, other associations can be used, such as using color, text, or other formatting. The product search results 140 can be displayed automatically or after the user selects a noun phrase. While only a single noun phrase is displayed, a response can have multiple noun phrases, and a product search can be performed after the user selects a particular noun phrase of interest. One advantage of UI 100 is that users do not need to reformulate their search query, but instead can simply select the appropriate noun phrase to continue searching for the desired product. Furthermore, users can ask questions without necessarily thinking about products, but the response provides useful product recommendations to start the user's shopping journey. In some cases, non-product content, such as instructional videos, photos, reviews, etc., can be associated with the noun phrase.

[0005] FIG. 2 shows another example of a UI 200 in which a question 210 is entered into an input text bar 212. A response 220 is automatically generated, highlighting two n-grams, including noun phrase 1 and n-gram 1. Noun phrase 1 contains multiple words, including “Company A’s latest mouse,” while n-gram 1 contains only a single word (an adjective), including the word “comfortable.” The UI 200 can associate some n-grams with products and other n-grams with content, such as reviews. For example, noun phrase 1 is identified as containing product-related terms, and a search for the product generates search result 230. However, n-gram 1 is identified as an adjective, resulting in the display of user review 240. Thus, multiple n-grams can be included in a single response, and n-grams can be associated with various types of supplemental information, such as products, reviews, instructional videos, etc. Additionally, as in FIG. 1, associations can be created between n-grams and supplemental information, such as arrows indicating that the two are paired or otherwise associated together. Once the product is found, an add to cart button 250 may be displayed, as shown at 230, to allow the user to easily purchase the product.

[0006] FIG. 3 illustrates a system 300 for generating the UIs of FIGS. 1 and 2. The UI 302 typically resides on a client computer (not shown) connected to a network 310, such as the Internet. One or more backend server computers 318 are used to receive questions 314 from the UI 302 and, as shown in FIG. 1, to provide resulting answers 316 with highlighted noun phrases and related products. The questions 314 may be user-generated or system-generated and combined with the user's question selection. In either case, the questions 314 are received from the UI 302 on a server computer running an answer generation model 320. The answer generation model 320 can process text, understand the meaning of specific words, understand the conversational context of those words, and draw inferences from the text and content. The answer generation model 320 can then generate an answer 326 to the question 314. The answer generation model 320 can also select stored questions received from third parties, although the answers 326 are further enriched with supplemental data before being returned to the UI 302. For example, the responses 326 are sent to a server computer running the semantic similarity model 330. In offline mode, the semantic similarity model 330 can be trained using click training data 340. As described further below, the click training data can relate to searches (by third parties) for products and the different search terms used that led to the discovery of the same product. Because the same product was found, the search terms in the different searches are assumed to be related. Thus, the click training data 340 can be automatically generated using click data from multiple users of an e-commerce website. Further alternative training data 342 can include manually selected training data to supplement the click training data 340. Once trained, the semantic similarity model 330 can parse the responses 326 and search for noun phrases 348 within the responses. The noun phrases can then be ranked using techniques described below to provide ranked noun phrases 350.The ranked noun phrases 350 may be provided to a product search engine 360, which can use the noun phrases to find related products in a product database 362 containing product image data. The product search engine 360 ​​provides product images 370 to a UI generator 372. The noun phrases 348 are also provided to the UI generator 372 by either the semantic similarity model 330 or the product search engine 360. In either case, the UI generator 372 receives associations (or linkings) describing which noun phrases are linked to which product images 370. The UI generator 372 may then provide the noun phrases, highlighted and linked to the corresponding products, in a response 316. The response 316 is returned through the network 310 and displayed in the user interface 302. Although described as a question / answer, the UI 302 can receive search queries via text or voice commands. Additionally, while FIG. 3 is described with reference to noun phrases, it can be extended to other types of n-grams. Typically, noun phrases (a type of n-gram) are identified from the n-grams and used to determine the product.

[0007] 4 shows the semantic similarity model 330 in more detail. The response 326 is input to a noun phrase identifier 408, which parses the response to extract noun phrases. The noun phrase identifier 408 can be a natural language processing (NLP) modeling tool. The NLP modeling tool can include a multi-layer perceptron (MLP) classifier, which includes an underlying neural network for classifying parts of the response, and a semantic segmentation algorithm for identifying noun phrases and word dependencies within the noun phrases. The output of the noun phrase identifier 408 is a number of noun phrases 410 contained in the response 326. The semantic similarity model 330 also includes multiple semantic similarity modules 412, 414, and 416, which generate scores based on combinations of inputs including one of the noun phrases 410 and a question (input to module 412), one of the noun phrases 410 and a response (input to module 414), and one of the noun phrases 410 and a question and response (input to module 416). Instead of multiple parallel semantic similarity modules 412, 414, and 416, a single semantic similarity model can be used to perform the serial operation. Each of the semantic similarity modules 412, 414, and 416 generates one or more scores 420, 422, and 424 based on the combination. As described further below, the semantic similarity module 416 calculates two scores based on different combinations of questions, responses, and noun phrases. Additional or fewer scores can be calculated based on the design. The controller 430 receives the scores and ranks the noun phrases based on the scores to generate the output 350. Click training data 340 may be received by the controller and used as feedback from past results to train 440 the semantic similarity modules 412, 414, and 416. The training data 440 may result in adjusting the weightings within the semantic similarity modules 412, 414, and 416.The training data 440 can be obtained offline using several pre-trained Bidirectional Encoding Representation of Sentences by Transformers (BERT) models to represent candidate keywords, questions, and responses using vectors.

[0008] FIG. 5 is an example of how the semantic similarity model 330 can convert a question-answer pair 510 into a ranked output 350. First, the semantic similarity model 330 receives both the question 314 and the answer 326 (shown as question-answer pair 510) from the answer generation module 320 (FIG. 3). At 520, multiple noun phrases are identified using the noun phrase identifier 408. The noun phrase is highlighted by placing a box around it, as can be seen, for example, in the words "regularly moistened." Other noun phrases are also identified, including "floor cleaner," "porcelain flooring," and "for years." At 530, a score is generated using four combinations of inputs to the semantic similarity module. A first combination of inputs at 540 includes the noun phrase and the question. A second combination of inputs at 542 includes the noun phrase and the answer. A third combination of inputs at 544 includes the noun phrase that combines the question and the answer. The fourth combination at 546 also includes noun phrases with question and answer combinations that were calculated separately and then averaged. These combinations can be repeated for each noun phrase identified at 520. The different combinations above can be generally summarized using the following formula: SimScore = Sim(ngram, Q) * z + Sim(ngram, A) * (1 - z), where z is a hyperparameter selected between 0 and 1. Thus, z is a weighting parameter that defines the relative weight of the question or answer as input to the semantic similarity module. The controller 430 accumulates all scores as the output of the semantic similarity module and generates a ranking 350 based on the scores. As described below, the ranking 350 can be used to identify products, which can then be displayed in the UI 100 (FIG. 1), as indicated at 140.

[0009] FIG. 6 is an example of obtaining training data from click data. In particular, different searches from different customers can be determined to be related through similar results. Thus, noun phrases in the input terms are also related. The related input words can then be used as training data 340. In a first search 610, a first user enters a question 613, "Can I replace my car seat base?", in an input text bar 612. A response 614 is shown, and a product 616 is displayed. Of particular importance in this example is that the search identified product ID1. The user can click (or otherwise select) product ID1, having confirmed that the product is correctly linked to the searched phrase 613. A second search 630, performed by the same or a different customer, uses a search query that includes standard search terms 632 rather than a question. Notably, no response is provided, but the results of the search are shown at 640. The search finds the same product ID1. When the customer clicks on the product, the search terms 632 are then linked to the product. Additionally, search term 632 (and any noun phrases therein), question 613 (and any noun phrases therein), and product ID 1 are all associated together. This association can then be used as click training data 340.

[0010] FIG. 7 is a diagram of a computing system of a network-based computing service provider 700 illustrating one environment in which embodiments described herein can be used. By way of background, a computing service provider 700 (i.e., a cloud provider) can supply computing and storage capacity as a service to a community of end recipients. In an exemplary embodiment, the computing service provider environment may be established by or on behalf of an organization for an organization. That is, the computing service provider environment 700 may provide a "private cloud environment." In another embodiment, the computing service provider 700 supports a multi-tenant environment in which multiple customers operate independently (i.e., a public cloud environment). Generally speaking, the computing service provider 700 may provide the following models: Infrastructure as a Service ("IaaS"), Platform as a Service ("PaaS"), and / or Software as a Service ("SaaS"). Other models may be offered. In the IaaS model, the computing service provider 700 may provide computers and other resources as physical or virtual machines. Virtual machines may be run as guests by a hypervisor, as described further below. The PaaS model supplies a computing platform that may include an operating system, a programming language execution environment, a database, and a web server. Application developers may develop and run their software solutions on the computing service provider platform without the costs of purchasing and managing the underlying hardware and software. The SaaS model enables the installation and operation of application software on the computing service provider.In some embodiments, end customers access computing service provider environment 700 using networked client devices, such as desktop computers, laptops, tablets, smartphones, etc., running a web browser or other lightweight client application. Those skilled in the art will recognize that computing service provider 700 may be described as a "cloud" environment.

[0011] In some implementations of the techniques of this disclosure, computer service provider 500 may be a cloud provider network. A cloud provider network (sometimes simply referred to as a "cloud") refers to a pool of network-accessible computing resources (such as compute, storage, and networking resources, applications, and services), which may be virtualized or bare metal. A cloud may provide convenient, on-demand network access to a shared pool of configurable computing resources that can be programmatically provisioned and released in response to customer commands. These resources can be dynamically provisioned and reconfigured to adjust to fluctuating loads. Thus, cloud computing can be viewed as both applications delivered as a service over a publicly accessible network (such as the Internet or a cellular communication network) and the hardware and software in the cloud provider's datacenter that provides those services.

[0012] Instead of purchasing, owning, and maintaining their own data centers and servers, cloud computing allows organizations to acquire technology such as computing power, storage, databases, and other services as needed. Cloud provider networks offer users an on-demand, scalable computing platform over the network, allowing customers, for example, to have scalable "virtual computing devices" at their disposal through the use of computing servers and block store servers. These virtual computing devices have the attributes of personal computing devices, including hardware (various types of processors, local memory, random access memory ("RAM"), hard disk and / or solid-state drive ("SSD") storage), a selected operating system, network capabilities, and preloaded application software. Each virtual computing device may also virtualize its console input / output ("I / O") (e.g., keyboard, display, and mouse). This virtualization allows customers to configure and use their virtual computing devices as if they were personal computing devices by connecting to them using computer applications such as browsers, application programming interfaces, and software development kits. Unlike a personal computing device, where a customer has a fixed amount of hardware resources available, the hardware associated with a virtual computing device can be scaled up or down depending on the resources the customer needs. Customers may choose to deploy their own virtual computing systems to provide network-based services for their own use and / or for use by their customers or clients.

[0013] A cloud provider network may be formed as several regions, where a region is a distinct geographic area where the cloud provider clusters its data centers. Each region may include two or more availability zones connected to each other via a private high-speed network such as a fiber optic connection. An availability zone (also called an availability domain or simply a "zone") refers to an isolated failure domain that includes one or more data center facilities with a separate power source, separate network, and separate cooling from other availability zones. A data center refers to a physical building or enclosure that houses and provides power and cooling to the servers of the cloud provider network. Availability zones within a region are preferably located far enough apart from each other so that the same natural disaster does not take more than one availability zone offline at the same time. Customers may connect to availability zones in the cloud provider network through a publicly accessible network (e.g., the Internet, a cellular network) via a transit center (TC). TCs are primary backbone locations linking customers to cloud provider networks and may be located on other network provider facilities (e.g., internet service providers, telecommunications providers) and securely connected to availability zones (e.g., via VPN or direct connections). Each region may operate two or more TCs for redundancy. Regions are connected to a global network that includes a private network infrastructure (e.g., fiber connections controlled by the cloud provider) that connects each region to at least one other region. The cloud provider network can deliver content from points of presence outside of, but networked with, these regions via edge locations and regional edge cache servers.This partitioning and geographic distribution of computing hardware enables cloud provider networks to offer customers global, low-latency resource access with a high degree of fault tolerance and stability.

[0014] A cloud provider network may implement various computing resources or services that implement the disclosed techniques for TLS session management, including elastic computing cloud services (referred to in various embodiments as elastic computing services, virtual machine services, computing cloud services, computing engines, or cloud computing services), data processing service(s) (e.g., map-reduce, dataflow, and / or other large-scale data processing techniques), data storage services (e.g., object storage services, block-based storage services, or data warehouse storage services), and / or any other type of network-based service (not shown, which may include various other types of storage, processing, analytics, communication, event handling, visualization, and security services). Resources (e.g., compute resources and storage resources) required to support the operation of such services may be requested by customers of the cloud provider network and provisioned in accounts associated with the cloud provider, as opposed to resources that may be provisioned in the customer's account.

[0015] In particular, the illustrated computing service provider 700 includes multiple server computers 702A-D. While only four server computers are shown, any number may be used, and large centers may include thousands of server computers. The server computers 702A-D may provide computing resources for running software instances 706A-D. In one embodiment, the instances 706A-D are virtual machines. As known in the art, a virtual machine is a software-implemented instance of a machine (i.e., a computer) that runs applications like a physical machine. In the virtual machine example, each of the servers 702A-D may be configured to run a hypervisor 708 or another type of program configured to enable the execution of multiple instances 706 on a single server. Furthermore, each of the instances 706 may be configured to run one or more applications.

[0016] Although the embodiments disclosed herein are described primarily in the context of virtual machines, it should be understood that other types of instances may be utilized with the concepts and techniques disclosed herein. For example, the techniques disclosed herein may be utilized with storage resources, data communication resources, and other types of computing resources. The embodiments disclosed herein may also execute all or part of an application directly on a computer system without utilizing a virtual machine instance.

[0017] One or more server computers 704 may be dedicated to executing software components for managing the operation of the server computers 702 and the instances 706. For example, the server computers 704 may execute a management component 710. A customer may access the management component 710 to configure various aspects of the operation of the instances 706 they have purchased. For example, a customer may purchase, rent, or lease instances and modify the instance's configuration. A customer may also specify settings regarding how the purchased instances scale in response to demand. The management component may further include a policy document to implement customer policies. An autoscaling component 712 may scale the instances 706 based on customer-defined rules. In one embodiment, the autoscaling component 712 allows a customer to specify scale-up rules to use when determining when new instances should be instantiated and scale-down rules to use when determining when existing instances should be terminated. The autoscaling component 712 may consist of several subcomponents that run on different server computers 702 or other computing devices. The autoscaling component 712 monitors available computing resources via an internal management network and can change the available resources based on need.

[0018] The deployment component 714 may be used to assist a customer in deploying a new instance 706 of a computing resource. The deployment component may access account information associated with the instance, such as the account holder's name, credit card information, and country. The deployment component 714 may receive a configuration from a customer that includes data describing how the new instance 706 should be configured. For example, the configuration may specify one or more applications to be installed on the new instance 706, comprise scripts and / or other types of code to be executed to configure the new instance 706, provide caching logic that specifies how application caches should be created, and other types of information. The deployment component 714 may utilize customer-provided configuration and caching logic to configure, provision, and launch the new instance 706. The configuration, caching logic, and other information may be specified by the customer using the management component 710 or by providing this information directly to the deployment component 714. An instance manager may be considered part of the deployment component.

[0019] Customer account information 715 may include any desired information associated with a customer of the multi-tenant environment. For example, customer account information may include a unique identifier for the customer, a customer address, billing information, licensing information, customization parameters for launching instances, scheduling information, auto-scaling parameters, previous IP addresses used to access the account, etc.

[0020] A network 730 may be utilized to interconnect server computers 702A-702D and server computer 704. Network 730 may be a local area network (LAN) and may be connected to a wide area network (WAN) 740 by which end customers may access service provider 700. It should be understood that the network topology depicted in Figure 7 is simplified and that many more networks and networking devices may be utilized to interconnect the various computing systems disclosed herein.

[0021] The semantic similarity model 330 can be executed on a server computer within the computing service provider 700. Additionally, a separate server computer 752 can perform the data acquisition necessary to train the semantic similarity model 330 and obtain click training data 340. As noted above, the training data can be obtained by considering search terms, noun phrases used in questions, and any other queries that result in the discovery of the same or similar products.

[0022] FIG. 8 is a flowchart for recommending products in a UI, according to one embodiment. At process block 810, a question is received from a client computer. For example, in FIG. 1, the client computer can be associated with UI 100, and a user can input a question into the UI, such as via input text bar 112 (FIG. 1). Alternatively, the user can select an automatically generated question in UI 100. At process block 820, a response to the question can be received. For example, in FIG. 3, response 326 is generated by response generation model 320 and received by semantic similarity model 330. At process block 830, a pool of candidate n-grams is extracted from the response. For example, in FIG. 4, noun phrase identifier 408 may identify noun phrases, as displayed at 520 in FIG. 5. At process block 840, the n-grams are ranked using the semantic similarity model. 5, the semantic similarity model can rank noun phrases using multiple algorithms 540, 542, 544, and 546 that create different scores based on the combination of questions, answers, and noun phrases input to the semantic similarity model. Such scores can be used to generate ranked output 350. Finally, in process block 850, display elements in a UI can be generated that show products associated with the n-grams, as shown in FIG.

[0023] FIG. 9 is a flowchart according to another embodiment for recommending products in a UI. At process block 910, a question is received from a user interface. For example, in FIG. 3, question 314 is entered by a user into UI 302. Question 314 may be received by semantic similarity model 330. At process block 920, a response associated with the question is received. For example, in FIG. 3, semantic similarity module 330 receives response 326 from response generation model 320. At process block 930, n-grams are extracted from the generated response. For example, in FIG. 5, noun phrases are identified as shown at 520. At process block 940, scores for the n-grams are generated. For example, in FIG. 4, semantic similarity modules 412, 414, and 416 may generate scores 420, 422, and 424, respectively. At process block 950, the n-grams are ranked based on the scores. For example, the ranked scores can be seen at 350. Finally, products are searched for and selected based on the rankings at 960. For example, as shown in Figure 3, a product search engine 360 ​​can use the ranked noun phrases 350 to find product images 370 to display.

[0024] 10 depicts a generalized example of a suitable computing environment 1000 in which the described innovations may be implemented. The computing environment 1000 is not intended to suggest any limitation as to scope of use or functionality, as the innovations may be implemented in a variety of general-purpose or special-purpose computing systems. For example, the computing environment 1000 may be any of a variety of computing devices (e.g., a desktop computer, a laptop computer, a server computer, a tablet computer, etc.).

[0025] Referring to FIG. 10, a computing environment 1000 includes one or more processing units 1010, 1015 and memory 1020, 1025. In FIG. 10, this basic configuration 1030 is included within the dotted line. The processing units 1010, 1015 execute computer-executable instructions. A processing unit may be a general-purpose central processing unit (CPU), a processor in an application-specific integrated circuit (ASIC), or any other type of processor. In a multi-processing system, multiple processing units execute computer-executable instructions to increase processing power. For example, FIG. 10 shows a central processing unit 1010 and a graphics processing unit or co-processing unit 1015. The tangible memory 1020, 1025 may be volatile memory (e.g., registers, cache, RAM), non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.), or some combination of the two accessible by the processing unit(s). The memories 1020, 1025 store software 1080 in the form of computer-executable instructions suitable for execution by a processing unit(s) that implements one or more innovations described herein. For example, a semantic similarity model can be implemented in the software 1080.

[0026] A computing system may have additional characteristics. For example, computing environment 1000 includes storage 1040, one or more input devices 1050, one or more output devices 1060, and one or more communication connections 1070. An interconnection mechanism (not shown), such as a bus, controller, or network, interconnects the components of computing environment 1000. Typically, operating system software (not shown) provides an operating environment for other software executing in computing environment 1000 and coordinates the operation of the components of computing environment 1000.

[0027] Tangible storage 1040 may be removable or non-removable and may be a magnetic disk, magnetic tape or cassette, CD-ROM, DVD, or other medium that can be used to non-transitory store information and that can be accessed within computing environment 1000. Storage 1040 stores instructions for software 1080 that implements one or more of the innovations described herein.

[0028] Input device(s) 1050 may be a keyboard, a mouse, a touch input device such as a pen or trackball, a voice input device, a scanning device, or another device that provides input to computing environment 1000. Output device(s) 1060 may be a display, a printer, speakers, a CD writer, or another device that provides output from computing environment 1000.

[0029] The communication connection(s) 1070 enable communication over a communication medium to another computing entity. The communication medium conveys information such as computer-executable instructions, audio or video input or output, or other data in a modulated data signal. A modulated data signal is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, the communication medium may use an electrical, optical, RF, or other carrier.

[0030] Although some operations of the disclosed methods are described in a particular order for preferred presentation, it should be understood that the description of this manner encompasses rearrangement unless specific language described below requires a particular order. For example, operations described sequentially may, in some cases, be rearranged or performed simultaneously. Moreover, for simplicity, the accompanying figures may not show the various ways in which the disclosed methods can be used in conjunction with other methods.

[0031] Any of the disclosed methods may be implemented as computer-executable instructions stored on one or more computer-readable storage media (e.g., one or more optical media disks, volatile memory components (such as DRAM or SRAM), or non-volatile memory components (such as flash memory or a hard drive)) and executed on a computer (e.g., any commercially available computer, including a smartphone or other portable device containing computing hardware). The term computer-readable storage medium does not include communication connections such as signals and carrier waves. Any of the computer-executable instructions for implementing the disclosed techniques, as well as any data created and used during the practice of the disclosed embodiments, may be stored on one or more computer-readable storage media. The computer-executable instructions may be a dedicated software application or part of a software application accessed or downloaded, for example, via a web browser or other software application (e.g., a remote computing application). Such software may be implemented, for example, using one or more networked computers, on a single local computer (e.g., any suitable commercially available computer) or in a networked environment (e.g., via the Internet, a wide area network, a local area network, a client-server network (e.g., a cloud computing network), or other such network).

[0032] For clarity, only certain selected aspects of software-based implementations are described. Other details that are well known in the art are omitted. For example, it should be understood that the disclosed technology is not limited to any particular computer language or program. For example, aspects of the disclosed technology may be implemented by software written in C++, Java, Perl, or any other suitable programming language. Similarly, the disclosed technology is not limited to any particular computer or hardware type. Specific details of suitable computers and hardware are well known and need not be described in detail in this disclosure.

[0033] It should be understood that instead of software, one or more hardware logic components can perform, at least in part, any of the functions described herein. For example, without limitation, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard packages (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0034] Additionally, any of the software-based embodiments (e.g., comprising computer-executable instructions that cause a computer to perform any of the disclosed methods) can be uploaded, downloaded, or remotely accessed through appropriate communications means, including, for example, the Internet, the World Wide Web, an intranet, a software application, cable (including fiber optic cable), magnetic communications, electromagnetic communications (including RF, microwave, and infrared communications), electronic communications, or other such communications means.

[0035] The disclosed methods, apparatus, and systems should not be construed as limiting in any way. Instead, the present disclosure is directed to all novel and non-obvious properties and aspects of the various disclosed embodiments, both alone and in various combinations and subcombinations with respect to one another. The disclosed methods, apparatus, and systems are not limited to any particular aspect or property or combination thereof, nor are the disclosed embodiments required to exhibit any one or more particular advantages or to solve any further problems.

[0036] For example, the following numbered paragraphs describe some embodiments:

[0037] Paragraph 1. A method of recommending a product, comprising: receiving a query from a client computer entered through an input text box of a user interface (UI); receiving a response to said query; extracting from the response a pool of candidate noun phrases containing the word; inputting the question, the response, and the candidate noun phrases into a semantic similarity model; ranking the noun phrases using the semantic similarity model; Identifying products to display in the UI for the highest ranked noun phrases; and generating the product associated with the noun phrase for display.

[0038] Paragraph 2. The method of Paragraph 1, further comprising training the semantic similarity model using a first search including the question and a second search including a search term, wherein both the first search and the second search result in the discovery of the same product.

[0039] Paragraph 3. The method of any preceding paragraph, wherein the ranking includes using a plurality of different input sequences to the semantic similarity model to receive a plurality of numerical scores.

[0040] Paragraph 4. The method of any preceding paragraph, wherein the plurality of different input sequences includes a first of the candidate noun phrases and the question, and a first of the candidate noun phrases and the response.

[0041] Paragraph 5. The method of any preceding paragraph, further comprising training the semantic similarity model using click data received at an e-commerce website.

[0042] Paragraph 6. A method comprising: receiving a query from a user interface; receiving an associated response to said question; extracting n-grams from the generated responses; generating a score for the n-gram using at least one semantic similarity module that receives the n-gram and one or both of the question and the response; ranking the n-grams using the scores; and retrieving and selecting products based on the rankings.

[0043] Paragraph 7. The semantic similarity module receives the following inputs: the n-grams and the question, the n-grams and the response, and 7. The method of paragraph 6, including the n-gram, the question, and the response.

[0044] Paragraph 8. The method of any of Paragraphs 6 to 7, further comprising training the plurality of semantic similarity modules using click data from product queries on an e-commerce website.

[0045] Paragraph 9. The method of any of Paragraphs 6 to 8, further comprising training the plurality of semantic similarity modules using a first search including a question and a second search including a search term, wherein both the first search and the second search result in the discovery of the same product.

[0046] Paragraph 10. The method of any of Paragraphs 6 to 9, further comprising displaying the selected products in a user interface (UI).

[0047] Paragraph 11. The method of any of Paragraphs 6 to 10, wherein the n-grams are identified using a natural language processing (NLP) modeling tool, and the n-grams include noun phrases.

[0048] Paragraph 12. The method of any of Paragraphs 6 to 11, further comprising associating the selected product on a user interface (UI) with at least one of the n-grams.

[0049] Paragraph 13. The method of any of Paragraphs 6 to 12, wherein the ranking is based on which n-grams are most likely to be associated with the product.

[0050] Paragraph 14. The method of any of Paragraphs 6 to 13, further comprising adjusting weights of the semantic similarity module using a pre-trained Bidirectional Encoding Representation of Sentences with Transformers (BERT) model.

[0051] Paragraph 15. One or more computer-readable media containing computer-executable instructions that, when executed, cause a computing system to: generating a response to a user's question; extracting noun phrases from the response using a natural language processing (NLP) model; inputting the extracted noun phrase and the user's question into a semantic similarity model to determine products associated with the noun phrase; and One or more computer-readable media that cause a method to be performed, the method including transmitting an image of the determined product for display in association with a corresponding one of the selected noun phrases.

[0052] Paragraph 16. One or more computer-readable media according to Paragraph 15, wherein the method further comprises inputting the extracted noun phrases and the responses into a semantic similarity model to generate a score, and using the score to determine the product.

[0053] Paragraph 17. One or more computer-readable media according to any of Paragraphs 15 to 16, wherein the semantic similarity model is used to generate a plurality of scores using a combination of the extracted noun phrases and a combination of the user's question and the response.

[0054] Paragraph 18. One or more computer-readable media according to Paragraph 17, wherein the generated scores are used to rank the noun phrases.

[0055] Paragraph 19. One or more computer-readable media according to Paragraph 18, wherein the noun phrase that is highest ranked in the ranking of the noun phrases is used to search for the determined product.

[0056] Paragraph 20. One or more computer-readable media according to any of Paragraphs 15 to 19, wherein the method further comprises training the plurality of semantic similarity models using click data from product queries on an e-commerce website.

[0057] In view of the many possible embodiments to which the principles of the disclosed invention may be applied, it should be recognized that the illustrated embodiments are only examples of the invention and should not be construed as limiting the scope of the invention. Accordingly, the inventors claim as their invention all that falls within the scope of these claims.

Claims

1. 1. A method comprising: receiving a query from a user interface; receiving an associated response to said question; extracting n-grams from the generated responses; generating a score for the n-gram using at least one semantic similarity module that receives the n-gram and one or both of the question and the response; ranking the n-grams using the scores; and retrieving and selecting products based on the rankings.

2. The semantic similarity module receives the following inputs: the n-gram and the query, the n-gram and the response, and The method of claim 1 , comprising the n-grams, the questions, and the responses.

3. The method of claim 1 , further comprising training a plurality of the semantic similarity modules using click data from product queries of an e-commerce website.

4. 10. The method of claim 1, further comprising training a plurality of the semantic similarity modules using a first search including a question and a second search including a search term, wherein both the first search and the second search result in the discovery of the same product.

5. The method of claim 1 , further comprising displaying the selected product in a user interface (UI).

6. The method of claim 1 , wherein the n-grams are identified using a natural language processing (NLP) modeling tool, and the n-grams include noun phrases.

7. The method of claim 1 , further comprising associating the selected product on a user interface (UI) with at least one of the n-grams.

8. The method of claim 1 , wherein the ranking is based on which n-grams are most likely to be associated with the product.

9. 10. The method of claim 1, further comprising adjusting weights of the semantic similarity module using a pre-trained Bidirectional Encoding Representation of Sentences by Transformers (BERT) model.

10. One or more computer-readable media containing computer-executable instructions that, when executed, cause a computing system to: generating a response to a user's question; extracting noun phrases in the response using a natural language processing (NLP) model; inputting the extracted noun phrase and the user's question into a semantic similarity model to determine products associated with the noun phrase; and The one or more computer-readable media causing a method to be performed, the method including transmitting an image of the determined product for display in association with a corresponding one of the selected noun phrases.

11. 11. The one or more computer-readable media of claim 10, wherein the method further comprises inputting the extracted noun phrases and the responses into the semantic similarity model to generate a score and using the score to determine the product.

12. 11. The one or more computer-readable media of claim 10, wherein the semantic similarity model is used to generate a plurality of scores using a combination of the extracted noun phrases and a combination of the user's question and the response.

13. The one or more computer-readable media of claim 12 , wherein the generated scores are used to rank the noun phrases.

14. 14. The one or more computer-readable media of claim 13, wherein the noun phrase that is highest ranked in the ranking of the noun phrases is used to search for the determined product.

15. 11. The one or more computer-readable media of claim 10, wherein the method further comprises training the plurality of semantic similarity models using click data from product queries of an e-commerce website.

Images