Automatic delivery of customer assistance at physical locations

Abstract

The present application relates to automated delivery of customer assistance at a physical location. A system is described that identifies items purchased by a user of a device at a physical location associated with a merchant based on contextual information associated with the device. The system performs an autonomous search query for product information predicted to assist the user in completing a purchase of the item from the merchant at the physical location. The system transmits the product information to the device for subsequent output. The system determines whether a likelihood that the user will complete the purchase in response to receiving the product information satisfies a likelihood threshold, and if the likelihood threshold is not satisfied, executes a remote assistance module accessed by the device to provide a virtual environment in which a person provides additional information required by the user to complete the purchase.

Description

[0001] Description of the case

[0002] This application is a divisional application of Chinese invention patent application No. 201680078277.3, filed on November 14, 2016. Technical Field

[0003] The present disclosure relates to automated delivery of customer assistance at a physical location. Background Art

[0004] Despite many technological advances in the online shopping experience, the way customers typically purchase items from physical (or so-called "brick and mortar") stores has remained relatively unchanged for many years. If a customer needs assistance, he or she may attempt to find an in-store assistant to provide the information he or she needs. In some cases, it may not always be easy to find an in-store assistant, and if the user has questions about an item, he or she may turn to a mobile device to manually perform a search (e.g., at an online retail website) for any additional information the customer may need before completing the purchase in-store. In addition to being inconvenient and time-consuming, manual searches may not always yield the exact answers or additional information the user needs to complete the purchase. Summary of the Invention

[0005] In one example, the present disclosure relates to a method comprising determining, by a computing device, that the computing device is at a physical location associated with a merchant; determining, by the computing device based on contextual information associated with the computing device, product information that is predicted to assist a user of the computing device in completing a purchase of a product from the merchant at the physical location; determining, by the computing device, a degree of likelihood that the user will complete the purchase in response to receiving the product information; outputting, by the computing device, an indication of the product information; and in response to determining that the degree of likelihood does not satisfy a likelihood threshold, executing, by the computing device, a remote assistance module to provide a virtual environment in which a person provides additional information required for the user to complete the purchase.

[0006] In another example, the present disclosure relates to a method comprising identifying, by a computing system, an item that a user of the computing device intends to purchase from the physical location based on contextual information associated with a computing device located at a physical location associated with a merchant and based on a plurality of items for sale at the physical location; executing, by the computing system, an autonomous search query for item information that is predicted to assist the user in completing a purchase of the item from the merchant at the physical location; determining, by the computing system, in response to receiving the item information, whether a degree of likelihood that the user will complete the purchase satisfies a probability threshold; sending, by the computing system, the item information to the computing device and subsequently outputting, by the computing device, the item information; and, in response to determining that the degree of likelihood does not satisfy the probability threshold, executing, by the computing system, a remote assistance module accessed by the computing device to provide a virtual environment in which a person provides additional information required by the user to complete the purchase.

[0007] In another example, the present disclosure relates to a computing device comprising at least one processor; input and output devices configured to present a user interface associated with the computing device; a context module operable by the at least one processor to: determine that the computing device is at a physical location associated with a merchant; and obtain context information associated with the computing device; a user interface module operable by the at least one processor to: receive, based on the context information associated with the computing device, product information that is predicted to assist a user of the computing device in completing a purchase of a product from the merchant at the physical location, the product information having been assigned a degree of likelihood that the user will complete the purchase in response to receiving the product information; output an indication of the product information via the input and output devices; and in response to determining that the degree of likelihood does not satisfy a likelihood threshold, execute a remote assistance module to provide a virtual environment using the input and output devices in which a person provides additional information required by the user to complete the purchase.

[0008] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the disclosure will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a conceptual diagram illustrating an example system for providing customer assistance to users of computing devices at physical locations in accordance with one or more aspects of the present disclosure.

[0010] Figure 2 is a block diagram illustrating an example computing system configured to provide customer assistance to users of computing devices at physical locations, in accordance with one or more aspects of the present disclosure.

[0011] Figures 3A to 3Dis a diagram illustrating an example graphical user interface presented by an example computing device configured to provide customer assistance to a user of the computing device at a physical location, according to one or more aspects of the present disclosure.

[0012] Figure 4 is a flow diagram illustrating example operations performed by an example computing system configured to provide customer assistance to users of computing devices at physical locations, in accordance with one or more aspects of the present disclosure.

[0013] Figure 5 is a flow diagram illustrating example operations performed by an example computing device configured to provide customer assistance to a user of the computing device at a physical location, in accordance with one or more aspects of the present disclosure. DETAILED DESCRIPTION

[0014] In general, the techniques of this disclosure can enable a computing device to access multiple layers of customer assistance services for obtaining product information related to items sold by a merchant when the computing device is located at a physical location associated with the merchant. A computing system can analyze contextual information associated with the computing device and determine the precise location of the computing device relative to the merchant's physical location (e.g., a "brick and mortar store"). Based on the location of the computing device (and other contextual information), the system can infer the items that the user of the computing device intends to purchase from the store.

[0015] For example, as part of the first layer of customer assistance services, the system may obtain product information that is predicted to assist the user in completing a product purchase (e.g., by performing an autonomous search, such as a query that does not require explicit user input). If the system determines that the product information is likely to help the user or is likely to cause the user to complete a product purchase, the system may cause the computing device to automatically present the product information to the user via an artificial intelligence (AI) or predictive information user interface (e.g., as a pop-up or notification). The AI or predictive information user interface may enable the system to receive additional information from the computing device (e.g., via voice and text-based input) and interact with the user of the computing device to attempt to answer all questions the user may have about the product. As the user interacts with the AI interface, if the system determines that the additional product information is likely to help the user or is likely to cause the user to complete a product purchase, the AI may continuously update and present additional product information that answers the user's questions.

[0016] However, if the system cannot obtain product information with a sufficient degree of certainty for the user to complete the purchase, the system may infer that the user requires a higher level of customer assistance. As part of the second layer of customer assistance services, the system may invoke or execute a remote assistance module accessed by the computing device to provide a virtual environment (e.g., text-based or voice-based) in which a person can provide additional information needed by the user to complete the purchase. For example, the system may cause the computing device to present a chat window from which the user can interact with a human expert located at a remote call center to obtain answers to questions he or she may have about the product.

[0017] Finally, the system may determine (automatically or in response to input from the user or the remote assistance module) that the user may require a higher level of customer assistance. As part of the third tier of customer assistance services, the system may schedule (e.g., by sending a notification to an in-store scheduling system) in-person assistance to a physical location. As a result of the scheduling, a customer service assistant who is an expert on the merchandise may arrive at the physical location of the store to provide the answers or information the user needs to complete the purchase.

[0018] Throughout this disclosure, examples are described in which a computing device and / or computing system analyzes information associated with a computing device and its user (e.g., context, location, speed, search queries, etc.) only when the computing device receives permission from the user of the computing device to analyze the information. For example, in the scenarios discussed below, before a computing device or computing system can collect or utilize information associated with a user, the user can be provided with an opportunity to provide input to control whether programs or features of the computing device and / or computing system can collect and utilize user information (e.g., information about the user's current location, current speed, etc.), or to instruct the device and / or system whether and / or how to receive content that may be relevant to the user. Additionally, certain data can be processed in one or more ways before being stored or used by the computing device and / or computing system to remove personally identifiable information. For example, the user's identity can be processed so that personally identifiable information about the user cannot be determined, or, if location information is obtained, the user's geographic location can be generalized (such as to a city, zip code, or state level) so that the user's specific location cannot be determined. Thus, the user can control how information about the user is collected and used by the computing device and computing system.

[0019] Figure 1is a conceptual diagram illustrating system 100 as an example system for providing customer assistance to users of computing devices at physical locations, in accordance with one or more aspects of the present disclosure. System 100 includes an information server system ("ISS") 160 in communication with a merchant server system ("MSS") 180 and computing device 110 via a network 130. Although system 100 is shown as distributed among ISS 160, MSS 180, and computing device 110, in other examples, features and techniques attributed to system 100 may be performed internally by local components of computing device 110. Similarly, ISS 160 may include certain components and perform various techniques that will be attributed to MSS 180 and computing device 110 in the following description.

[0020] Network 130 represents any public or private communication network for transmitting data between computing systems, servers, and computing devices, such as cellular, Wi-Fi, and / or other types of networks. ISS 160 may communicate with computing device 110 via network 130 contextual information associated with computing device 110, which is information ISS 160 needs to provide customer assistance services to computing device 110 when computing device 110 is at or near a physical location associated with a merchant. ISS 160 may further communicate with MSS 180 via network 130 to obtain merchant information regarding specific products and promotions offered by the merchant at the physical location, which is information ISS 160 also needs to provide customer assistance services accessed by computing device 110. Computing device 110 may receive information associated with customer assistance services provided by ISS 160 via network 130, such as product information that ISS 160 infers will likely assist the user of computing device 110 in purchasing products at the physical location.

[0021] Network 130 may include one or more network hubs, network switches, network routers, or any other network devices operatively coupled to one another to provide for information exchange between ISS 160, MSS 180, and computing device 110. Computing device 110, ISS 160, and MSS 180 may transmit and receive data across network 130 using any suitable communication technology. ISS 160, MSS 180, and computing device 110 may each be operatively coupled to network 130 using respective network links. The links coupling computing device 110, MSS 180, and ISS 160 to network 130 may be Ethernet or other types of network connections, and such connections may be wireless and / or wired.

[0022] Computing device 110 represents an individual mobile or non-mobile computing device. Examples of computing device 110 include a mobile phone, a tablet computer, a laptop computer, a desktop computer, a server, a mainframe computer, a set-top box, a television, a wearable device (e.g., a computerized watch, computerized glasses, computerized gloves, etc.), a home automation device or system (e.g., a smart thermostat or home assistant), a personal digital assistant (PDA), a portable gaming system, a media player, an e-book reader, a mobile television platform, a car navigation and entertainment system, or any other type of mobile, non-mobile, wearable, and non-wearable computing device configured to receive information via a network such as network 130.

[0023] The computing device 110 includes a user interface device (UID) 112, a user interface (UI) module 120, and a context module 122. The modules 120-122 may perform the described operations using software, hardware, firmware, or a combination of hardware, software, and firmware resident in and / or executing at the respective computing device 110. The computing device 110 may utilize multiple processors or multiple devices to execute the modules 120-122. The computing device 110 may execute the modules 120-122 as virtual machines executing on underlying hardware. The modules 120-122 may be executed as one or more services of an operating system or computing platform. The modules 120-122 may be executed as one or more executable programs on the application layer of the computing platform.

[0024] The UID 112 of the computing device 110 can be used as an input and / or output device for the computing device 110. The UID 112 can be implemented using various technologies. For example, the UID 112 can be used as an input device using a presence-sensitive input screen, such as a resistive touch screen, a surface acoustic wave touch screen, a capacitive touch screen, a projected capacitive touch screen, a pressure-sensitive screen, an acoustic pulse recognition touch screen, or another presence-sensitive display technology. In addition, the UID 112 can include microphone technology, infrared sensor technology, or other input device technology for receiving user input.

[0025] The UID 112 may function as an output (e.g., display) device using one or more display devices such as a liquid crystal display (LCD), a dot matrix display, a light emitting diode (LED) display, an organic light emitting diode (OLED) display, electronic ink, or any similar monochrome or color display capable of outputting visual information to a user of the computing device 110. Additionally, the UID 112 may include speaker technology, haptic feedback technology, or other output device technology for outputting information to the user.

[0026] UID 112 may include a presence-sensitive display that can receive tactile input from a user of computing device 110. UID 112 can detect one or more gestures from the user (e.g., the user touches or points to one or more locations of UID 112 with a finger or stylus). UID 112 can present output to the user, for example, at the presence-sensitive display. UID 112 can present the output as a graphical user interface (e.g., user interface 114), which can be associated with the functionality provided by computing device 110 and / or the service that computing device 110 is accessing.

[0027] For example, UID 112 may present a user interface (e.g., user interface 114) related to a customer assistance service provided by ISS 160, which is accessed by UI module 120 on behalf of computing device 110. In some examples, UID 112 may present a user interface related to autonomous search functionality provided by UI module 120, or other features of a computing platform, operating system, application, and / or service executed at or accessible from computing device 110 (e.g., an electronic messaging application, an Internet browser application, a mobile or desktop operating system, etc.).

[0028] The UI module 120 can manage user interactions with the UID 112 and other components of the computing device 110, including interactions with the ISS 160, in order to provide autonomous search results at the UID 112. When a user of the computing device 110 views output and / or provides input at the UID 112, the UI module 120 can cause the UID 112 to output a user interface, such as the user interface 114 (or other example user interfaces), for display. The UI module 120 and the UID 112 can receive one or more indications of input from the user at different times and when the user and the computing device 110 are in different locations as the user interacts with the user interface. The UI module 120 and the UID 112 can interpret the input detected at the UID 112 and can relay information about the input detected at the UID 112 to, for example, one or more associated platforms, operating systems, applications, and / or services executing at the computing device 110 to cause the computing device 110 to perform functions.

[0029] UI module 120 may receive information and instructions from one or more associated platforms, operating systems, applications, and / or services executing at computing device 110 and / or one or more remote computing systems, such as ISS 160 and MSS 180. In addition, UI module 120 may act as an intermediary between one or more associated platforms, operating systems, applications, and / or services executing at computing device 110 and various output devices (e.g., speakers, LED indicators, audio or electrostatic tactile output devices, etc.) of computing device 110 to generate output (e.g., graphics, flashing lights, sounds, tactile responses, etc.) utilizing computing device 110.

[0030] exist Figure 1 In the example of FIG, user interface 114 is a graphical user interface associated with a customer assistance service provided by ISS 160 and accessed by computing device 110. Figure 1 As shown, user interface 114 presents "product information" that ISS 160 predicts will help the user complete the purchase of the product when at a physical location (e.g., a store) associated with the merchant. User interface 114 can present the product information in various forms such as text, graphics, content cards, images, etc. UI module 120 can cause UID 112 to output user interface 114 based on data received by UI module 120 from ISS 160 via network 130. UI module 120 can receive graphical information (e.g., text data, image data, etc.) for presenting user interface 114 as input from ISS 160 and instructions from ISS 160 for presenting graphical information within user interface 114 at UID 112.

[0031] The context module 122 can collect context information associated with the computing device 110 to define a context for the computing device 110. In particular, with respect to the system 100, the context module 122 is primarily used to define a context for the computing device 110, which indicates the location of the computing device 110 physically relative to a physical location associated with a merchant (e.g., a "brick and mortar store" of the merchant). However, the context module 122 can be configured to define any type of context that specifies characteristics of the physical and / or virtual environment of the computing device 110 at a particular time.

[0032] As used throughout this disclosure, the term "contextual information" is used to describe any information that can be used by context module 122 to define the characteristics of the virtual and / or physical environment that a computing device and its user may experience at a particular time. Examples of contextual information are numerous and may include: sensor information obtained by sensors of computing device 110 (e.g., position sensors, accelerometers, gyroscopes, barometers, ambient light sensors, proximity sensors, microphones, and any other sensors); communication information sent and received by a communication module of computing device 110 (e.g., text-based communications, audible communications, video communications, etc.); and application usage information associated with applications executed at computing device 110 (e.g., application data associated with the applications, internet search history, text communications, voice and video communications, calendar information, social media posts, and related information, etc.). Other examples of contextual information include signals and information obtained from transmitting devices external to computing device 110. For example, context module 122 may receive beacon information transmitted from an external beacon located at or near a physical location of a business via a radio or communication unit of computing device 110. As described in more detail below, context module 122 may use beacon information to define a context for computing device 110 that indicates the precise location of computing device 110 within the interior space of a merchant's physical store.

[0033] Based on the contextual information collected by context module 122, context module 122 may define a context for computing device 110 that places computing device 110 at a precise location within the physical location of a merchant or the interior space of a physical store. Context module 122 may rely on merchant information obtained from MSS 180 via network 130 to supplement the contextual information obtained by context module 122 when determining the context of computing device 110. For example, by correlating contextual information with merchant information (e.g., an electronic map or store layout, a list of beacon identifiers and their relative locations within the store, etc.), context module 122 may determine the precise location defined by the context of computing device 110 as a specific row or aisle within a physical store, the nearest point of interest within a physical store, a department within a physical store, the store's coordinate location, an altitude, a floor, a specific checkout station within a store, or any other type of location identifier associated with the store.

[0034] For example, a merchant may place multiple beacons at various locations throughout a physical store and record the locations and corresponding beacon identifiers associated with each beacon as merchant information in the MSS 180. Although primarily described herein as using beacons, a merchant may use other types of transmission devices, such as wireless communication units (e.g., and radio frequency identifier [RFID] transmitters, other types of radio transmitters and receivers) cellular towers, amplifiers, etc. In any case, when the computing device 110 is located at or near a physical store, the context module 122 can receive beacon information from one or more of the plurality of beacons. The beacon information can include a corresponding beacon identifier.

[0035] Context module 122 may use triangulation techniques to determine a proximity score associated with each of the plurality of beacons and determine that the beacon with the highest proximity score is closest to the precise location of computing device 110 within the merchant's store. For example, for each beacon signal received by context module 122, context module 122 may determine the signal strength associated with that particular beacon. Because signal strength alone may not be the most reliable way to determine the nearest beacon, context module 122 may ignore certain beacon signals and / or rely on other context information associated with computing device 110 to determine the nearest beacon. As an example, context module 122 may discard beacon signals with very high signal strengths (e.g., greater than 90 percentile) and very low signal strengths (e.g., less than 10 percentile). Context module 122 may use signal strengths along with accelerometer data, barometer data, wireless communication signals, detected service set identifiers (SSIDs), and other context information received by computing device 110 to determine the proximity score associated with each beacon. Using the proximity score for each beacon, the context module 122 can triangulate the precise location of the computing device 110 from within the merchant's store.

[0036] As the signal strength of beacon signals changes and other contextual information changes, context module 122 can infer movement associated with computing device 110 and update the location of computing device 110 accordingly. For example, as a user moves through rows or aisles of a store with computing device 110, context module 122 can determine subsequent beacon signal strengths, subsequent accelerometer data, and subsequent other contextual information, and re-triangulate the location of computing device 110 onto or near different beacons.

[0037] Context module 122 may transmit the current context of computing device 110 to ISS 160 via network 130, based on which ISS 160 may use the context to provide customer assistance to the user of computing device 110 and / or perform autonomous searches for information (e.g., product information, promotional information, etc.) related to the context of computing device 110. For example, context module 122 may send an indication of the precise location of computing device 110 within a merchant's store to ISS 160. The indication of the precise location may correspond to the coordinate location of computing device 110, a department location within a physical store, a row location, a trash bin location, an aisle location, the nearest entrance or exit, the location of a cashier, or any other type of location information that ISS 160 may use to infer the relative location of computing device 110 within a merchant's physical store.

[0038] ISS 160 and MSS 180 represent any suitable remote computing systems capable of both sending and receiving information to and from a network, such as network 130, such as one or more desktop computers, laptop computers, mainframe computers, servers, cloud computing systems, etc. ISS 160 hosts (or at least provides access to) customer assistance services associated with a physical merchant. MSS 180 hosts (or at least provides access to) merchant information associated with the physical merchant, such as inventory, merchandise locations, promotions, customer lists, etc.

[0039] Computing device 110 may communicate with ISS 160 via network 130 to access customer assistance services provided by ISS 160. ISS 160 may communicate with MSS 180 via network 130 to obtain merchant information required to provide customer assistance services to computing device 110. In some examples, ISS 160 represents a cloud computing system that provides access to customer assistance services via the cloud.

[0040] exist Figure 1 In the example of FIG. 1 , MSS 180 includes a merchant information data store 182. The information stored by data store 182 can be searchable and / or categorized. MSS 180 can provide access to the information stored at data store 182 as a cloud-based data access service to devices connected to network 130, such as ISS 160 and computing device 110. For example, ISS 160 can provide a request to MSS 180 for information from data store 182 and, in response to providing input, receive the information stored at data store 182 via network 130. Computing device 110 can request a mapping or list of beacon identifiers and their locations within a physical store and receive the mapping of the beacon list as merchant information from MSS 180 via network 130.

[0041] Examples of merchant information stored at data store 182 include the locations of merchant stores, merchandise inventory associated with merchandise sold by the merchant at each location of the merchant store (e.g., quantity and location within the store), electronic store maps, store layouts, beacon locations, beacon identifiers, and other information associated with the merchant associated with MSS 180. Other examples of merchant information include promotions, coupons, or other discount offers associated with the merchandise inventory. Other examples of merchant information include customer information, such as loyalty program information, customer lists, transaction histories, and other information related to individual customers and their interactions and purchase history with the merchant.

[0042] When data store 182 contains information associated with an individual customer or when the information is generalized across multiple customers, all personally identifiable information that links the information back to an individual, such as name, address, telephone number, and / or email address, may be removed before being stored in MSS 180. MSS 180 may further encrypt the information stored at data store 182 to prevent access to any information stored therein. Additionally, if a customer affirmatively consents to the collection of such information, MSS 180 may only store the collection of information associated with the customer. MSS 180 may further provide the customer with the opportunity to revoke consent, and in such event, MSS 180 may cease collecting or otherwise retaining information associated with that particular customer.

[0043] exist Figure 1 In the example of ISS 160, ISS 160 includes a search module 164 and an assistance module 166. Modules 164 and 166 together provide a customer assistance service accessible to computing device 110 for automatically providing product information associated with products sold at the physical location of a merchant associated with MSS 180. For example, through the customer assistance service, modules 164 and 166 may not require scanning a barcode or Quick Response (QR) code printed on a product label to obtain information about the product. Instead, when computing device 110 is in proximity to one or more beacons installed near the product (e.g., when the user may be standing in front of the product displayed on a table, wall, cabinet, or shelf), modules 164 and 166 may cause computing device 110 to automatically display product information about the product that the user may be considering purchasing. Modules 164 and 166 may invoke a user interface at computing device 110 that allows the user (customer) to quickly find the item he is looking for (e.g., by displaying a carousel of the five closest items with item information (e.g., a brief description; ratings; reviews; a drop-down bar for size, color; additional stock availability; and other item information).

[0044] Although shown as part of ISS 160, in some examples, the operations performed by modules 164 and 166 may be performed by UI module 120 of computing device 110. In other words, in some examples, UI module 120 may include functionality and perform operations similar to those performed by modules 164 and 166.

[0045] Modules 164 and 166 may perform the described operations using software, hardware, firmware, or a combination of hardware, software, and firmware resident in and / or executing at ISS 160. ISS 160 may execute modules 164 and 166 using multiple processors, multiple devices, as virtual machines executing on underlying hardware, or as one or more services of an operating system or computing platform. In some examples, modules 164 and 166 may be executed as one or more executable programs at the application layer of the computing platform of ISS 160.

[0046] Search module 164 may perform an autonomous search query based at least in part on the context of computing device 110 to identify product information determined to be relevant to the user of computing device 110. In other words, search module 164 may obtain product information that may be relevant to the user of computing device 110 given the current context of computing device 110 without receiving an explicit request from the user. This type of search is sometimes referred to as a "parameter-less" or "autonomous" search.

[0047] For example, when a user of computing device 110 stands in front of a merchandise shelf in a merchant's physical store, search module 164 may obtain information about one or more merchandise that is closest to the location of computing device 110 .

[0048] Search module 164 can determine the context of computing device 110 based on information obtained from context module 122. For example, search module 164 can receive an indication of the current location of computing device 110 and / or the nearest beacon via network 130. Search module 164 can generate a search query based on the context and perform a search for information relevant to the search query. For example, search module 164 or assistance module 166 on behalf of search module 164 can query MSS 180 for a list of products that are closest to the nearest beacon or the current location of computing device 110. Search module 164 can use the list of product names or other merchant information received from MSS 180 to formulate the search query.

[0049] Search module 164 may use natural language processing, machine learning, and / or other artificial intelligence techniques to learn and model user behavior, including what types of product information users of computing device 110 and other computing devices typically search for in a particular context. For example, search module 164 may generate search queries that are likely to obtain similar product information obtained by prior search queries that are representative of other computing devices at or near the physical location of computing device 110. By learning and modeling users' searches in different contexts, search module 164 may generate one or more rules for automatically generating search queries for a particular context that are likely to find product information that the user of computing device 110 is seeking.

[0050] The search module 164 may perform an Internet search based on the automatically generated search query to identify product information relevant to the search query. After performing the search, the search module 164 may output the product information returned from the search (e.g., autonomous search results) to the assistance module 166 before sending the product information to the computing device 110.

[0051] In some examples, search module 164 may generate a search query for product information based on additional information about the user of computing device 110, in addition to the context of computing device 110. For example, search module 164 may generate an autonomous search query based on user information including: search history, communication information (e.g., email, text, calendar, social media, instant chat, etc.), online and physical store purchase history, an electronic shopping list associated with the user, an electronic notepad associated with the user, a reminder list associated with the user, an electronic shopping list, purchase history, or communication information associated with family members of the user, as well as any other information about the user available to search module 164. For example, search module 164 may obtain information from MSS 180 indicating that the user of computing device 110 previously purchased a desk lamp during a previous visit to a merchant's store. In response to receiving contextual information from the context module 122 indicating that the user is located at or near a department of a merchant's store that sells light bulbs, the search module 164 can formulate a search query that is more likely to produce product information about light bulbs that are compatible with desk lamps that the user previously purchased from the merchant's store, rather than general product information about light bulbs in general.

[0052] The natural language processing machine learning system of the search module 164 can score or rank the product information based on how relevant the information is to any individual product. For example, if the product information only relates to light bulbs in general, the machine learning system of the search module 164 can assign a medium ranking or score to the product information, while if the product information specifically relates to the product sold on the shelf in front of the user (e.g., light bulbs), the machine learning system of the search module 164 can assign a higher ranking or score to the product information. In this way, the search module 164 can indicate whether the product information is more or less likely to assist the customer in purchasing a particular product.

[0053] The machine learning system of search module 164 can rely on training data (e.g., based on feedback obtained from presenting similar product information to users of other computing devices in similar contexts) to train and learn which types of product information are more or less likely to assist users in completing product purchases. For example, if, after presenting product information to a user for a particular product, the machine learning system of search module 164 receives merchant information from MSS 180 indicating that the user purchased the product, the machine learning system can rely on this positive feedback to provide similar product information in similar contexts. If, on the contrary, that is, if the machine learning system of search module 164 receives merchant information from MSS 180 indicating that the user did not purchase the product, the machine learning system can rely on this negative feedback to modify the product information before providing it to subsequent users in similar contexts, or to avoid providing similar product information in similar contexts. In other words, the degree or score that search module 164 assigns to product information indicating whether the user will complete a purchase in response to receiving the product information is determined at least in part based on whether other users completed a purchase of the product in response to receiving the product information.

[0054] Assistance module 166 can manage customer assistance services provided by ISS 160 to computing device 110, which is used to deliver product information that the user of computing device 110 may find helpful in completing a purchase of a product when located near a merchant's physical location. That is, assistance module 166 can receive context from context module 122 and, based on the context, determine that computing device 110 is located at a physical location associated with a merchant. Assistance module 166 can further determine (e.g., based on the context and other merchant information obtained from MSS 180) the products that the user of the computing device intends to purchase from a store. For example, assistance module 166 can query MSS 180 for information about one or more products in the inventory of a merchant closest to the location of computing device 110.

[0055] In response to determining that the computing device 110 is at a physical location associated with a merchant and as part of a first tier of customer assistance services, the assistance module 166 can invoke the search module 164 to automatically perform an autonomous search query for product information predicted to assist the user in completing a purchase of one or more products from the merchant at the physical location. The assistance module 166 can send an indication of one or more products that are near the computing device 110, and in response, the assistance module 166 can receive a score, probability, or other degree of likelihood associated with the product information obtained by the search module 164 for each of the one or more products.

[0056] If the assistance module 166 determines that the product information is likely to help the user or may result in the user completing the purchase of the product, the assistance module 166 may send the product information to the UI module 120 via the network 130 for presentation to the user (e.g., as the user interface 114). For example, the assistance module 166 may determine whether a degree of likelihood that the user will complete the purchase in response to receiving the product information satisfies a threshold. The degree of likelihood may correspond to a probability, ranking, score, etc. assigned by the search module 164 when the machine learning system of the search module 164 determines the relevance of the product information to a product that the user may want to purchase from the merchant's physical location.

[0057] In response to determining that the likelihood level satisfies a likelihood threshold (e.g., a fifty percent threshold, 0.7 / 1.0, etc.), assistance module 166 may cause computing device 110 to output an indication of the product information. For example, assistance module 166 may send the product information and instructions for packaging the product information to UI module 120 of computing device 110 via network 130 in the form of an information card, and UI module 120 may cause the information card to automatically surface for display at UID 112 and be displayed to the user. The information card may include a carousel of one or more items closest to the product on the shelf in front of the user, with a brief description, ranking, reviews, and a drop-down bar for size, color, etc. to check availability.

[0058] In the event that assistance module 166 determines that the product information identified by search module 164 is unlikely to assist the user in completing the product purchase, assistance module 166 may infer that the user requires a higher level of customer assistance and invoke a second-tier customer assistance service. In other words, in response to determining that the likelihood associated with the product information returned from search module 164 does not meet a likelihood threshold, assistance module 166 may execute a remote assistance module to provide a virtual environment (e.g., text-based, voice-based, etc.) in which a human customer service representative associated with the assistance service interacts with the user and provides additional information required by the user to complete the purchase. For example, assistance module 166 may cause UI module 120 of computing device 110 to present a chat window graphical user interface at UID 112, from which the user can interact with a human expert located in a remote call center to obtain answers to questions that the user of computing device 110 may have regarding the product.

[0059] Finally, if the assistance module 166 determines that a human customer service representative cannot provide the user with the exact product information they need to complete their purchase in-store, the assistance module 166 can escalate to support provided by a human located in the physical store as a third level of customer assistance. For example, the assistance module 166 can dispatch in-person assistance to the physical location (e.g., the exact aisle and / or bin) of the computing device 110 associated with the merchant in response to determining that the remote human assistance module is unable to provide the additional information the user needs to complete their purchase via the virtual environment. The assistance module 166 can send a notification to an in-store dispatch system operated by the MSS 180, which notifies a local, in-store human expert (e.g., via a paging system, etc.). As a result of the dispatch, the customer service assistant, who is a product expert, can arrive at the physical location in the store to provide the answers or product information the user needs to complete their purchase.

[0060] Despite the many technological advances in the online shopping experience, the way customers typically purchase goods from physical (or so-called "brick and mortar") stores has remained relatively unchanged for many years. A customer may spend time walking through the store's aisles searching for a specific item they want to purchase. If a customer needs assistance or has a question about an item, they may embark on a seemingly endless journey to find a sales associate qualified to answer the question or provide assistance. In other cases, while in a physical store, a customer may simply turn to their mobile device to manually perform a search (e.g., on an online retailer's website) for any additional information they may need before completing their purchase in-store. With the option to purchase virtually anything online, and perhaps due to the time wasted in-store searching for an item on the shelf, looking for a sales associate, or searching for the additional information needed to make a purchase, some retailers may find that their customers are becoming less interested, if not increasingly dissatisfied, in purchasing goods from their physical stores.

[0061] To address the aforementioned issues associated with the current physical store shopping experience, the technology of the present disclosure can provide a customer assistance service from which a computing device can automatically obtain the product information a user needs to complete a purchase of a product from a merchant's physical location. The customer assistance service automatically escalates the service level through multiple layers of service (e.g., from an artificial intelligence system, to a remote human operator, to an on-site human expert) until the user obtains the information he or she needs. In this way, the system does not require scanning barcodes or QR codes, and the user does not have to expend effort searching for a salesperson who may or may not even be able to answer the customer's questions. Furthermore, because the system automatically provides product information and customer assistance, users of the system do not need to interact with their devices to perform manual searches (e.g., on an online retail website) for any additional information that the customer may need before completing a purchase in a store. Users of the system can provide fewer inputs to their devices to browse product information and perform manual searches for information. With less input from the user, the example system can enable computing devices to conserve energy and use less battery power compared to other systems that do not have access to the customer assistance service described herein.

[0062] Figure 2 is a block diagram illustrating an ISS 260 as an example computing system configured to provide user assistance to users of computing devices at physical locations, according to one or more aspects of the present disclosure. Figure 1 A more detailed example of ISS 160 is given below. Figure 1 100 is described in the context of the system 100. Figure 2Only one particular example of an ISS 260 is shown, and many other examples of ISS 260 may be used in other situations and may include a subset of the components included in the example ISS 260 or may include Figure 2 Additional components not shown.

[0063] ISS 260 provides computing device 110 with a conduit where computing devices, such as computing device 110, can access user assistance services to automatically receive product information relevant to the current context of the computing device. Figure 2 As shown in the example of FIG, ISS 260 includes one or more processors 270, one or more communication units 272, and one or more storage devices 274. The storage device 274 of ISS 260 includes the context module 222, the search module 264, and the assistance module 266. Within the assistance module 266, the storage device 274 includes a machine assistance module 268A, a remote human assistance module 268B, and an in-person assistance scheduling module 268C (collectively referred to as "modules 268"). Modules 222, 264, and 266 include at least Figure 1 The storage device 274 of the ISS 260 also includes a merchant information data store 282 and a user information data store 284.

[0064] Communication channel 276 can interconnect each of components 270, 272, and 274 for inter-component communication (physically, communicatively, and / or operationally). In some examples, communication channel 276 can include a system bus, a network connection, an inter-process communication data structure, or any other method for transmitting data.

[0065] The one or more communication units 272 of the ISS 260 may communicate with the user via a communication interface such as Figure 1 The network 130 transmits and / or receives network signals on one or more networks such as Figure 1 10. For example, ISS 260 may use communication unit 272 to send and / or receive radio signals over network 130 to exchange information with computing device 110. Examples of communication unit 272 include a network interface card (e.g., such as an Ethernet card), an optical transceiver, a radio frequency transceiver, a GPS receiver, or any other type of device that can send and / or receive information. Other examples of communication unit 272 may include a shortwave radio, a cellular data radio, a wireless Ethernet network radio, and a universal serial bus (USB) controller.

[0066] Storage device 274 can store information for processing during operation of ISS 260 (e.g., ISS 260 can store data accessed by modules 222, 264, 266, and 268 during execution of ISS 260). In some examples, storage device 274 is temporary storage, meaning that the primary purpose of storage device 274 is not long-term storage. Storage device 274 on ISS 260 can be configured as volatile memory for short-term storage of information and, therefore, does not retain stored contents if power is removed. Examples of volatile memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and other forms of volatile memory.

[0067] In some examples, storage device 274 also includes one or more computer-readable storage media. Storage device 274 can be configured to store a larger amount of information than volatile memory. Storage device 274 can also be configured as a non-volatile memory space for long-term storage of information and retain information after power on / off cycles. Examples of non-volatile memory include magnetic hard disks, optical disks, floppy disks, flash memory, or electrically programmable memory (EPROM) or electrically erasable and programmable (EEPROM) memory. Storage device 274 can store program instructions and / or data associated with modules 222, 264, 266, and 268.

[0068] One or more processors 270 may implement functionality and / or execute instructions within ISS 260. For example, processor 270 on ISS 260 may receive and execute instructions stored by storage device 274 that implement the functionality of modules 222, 264, 266, and 268. These instructions, when executed by processor 270, may cause ISS 260 to store information within storage device 274 during program execution. Processor 270 may execute the instructions of modules 222, 264, 266, and 268 to perform autonomous searches and rank autonomous search results based, at least in part, on dynamic characteristics of the computing devices for which the autonomous search results are targeted. That is, modules 222, 264, 266, and 268 may be operated by processor 270 to perform various actions or functions of ISS 260 described herein.

[0069] The information stored in data stores 282 and 284 can be searchable and / or categorized. For example, one or more modules 222, 264, 266, and 268 can provide input requesting information from one or more of data stores 282 and 284 and, in response to the input, receive information stored in data stores 282 and 284. ISS 260 can provide access to the information stored in data stores 282 and 284 as a cloud-based data access service to devices connected to network 130, such as computing device 110. When data stores 282 and 284 contain information associated with an individual user or when the information is generalized across multiple users, any personally identifiable information linking the information back to an individual—such as name, address, telephone number, and / or email address—can be removed before storage in ISS 260. ISS 260 can further encrypt the information stored in data stores 282 and 284 to prevent access to any information stored therein. Additionally, ISS 260 can only store information associated with users of the computing devices if such users have affirmatively consented to the collection of such information. ISS 260 may further provide the user with an opportunity to revoke consent, and in such event, ISS 260 may cease collecting or otherwise retaining information associated with that particular user.

[0070] Merchant information data store 282 is similar to merchant information data store 182 of MSS 180 and stores merchant information that ISS 260 can use to provide customer assistance services to computing device 110. Examples of merchant information stored at data store 282 include the locations of merchant stores, merchandise inventory associated with merchandise sold by the merchant at each location of the merchant store (e.g., quantity and location within the store), electronic store maps, store layouts, beacon locations, beacon identifiers, and other information associated with the merchant. Other examples of merchant information include promotions, coupons, or other discount offers associated with the merchandise inventory. Other examples of merchant information may include customer information, such as loyalty program information, customer lists, transaction histories, and other information related to individual customers and their interactions and purchase history at the merchant.

[0071] User information data store 284 may store information associated with a user of computing device 110. In some examples, search module 264 may rely on additional information about the user of computing device 110 outside the context of computing device 110 to generate a search query for product information. For example, search module 264 may rely on user information contained in data store 284 to generate an autonomous search query. Examples of user information stored in data store 284 include: search history, communication information (e.g., email, text, calendar, social media, instant chat, etc.), online and physical store purchase history, an electronic shopping list associated with the user, an electronic notepad associated with the user, a reminder list associated with the user, an electronic shopping list, a purchase history, or communication information associated with family members of the user, and any and all other information search module 264 may obtain about the user from the user's interactions with computing device 110 and / or the user's interactions with merchants.

[0072] For example, search module 264 may obtain information from MSS 180 indicating that a family member associated with the user of computing device 110 previously purchased a particular type of electric toothbrush during a previous visit to a merchant's store. In response to receiving contextual information from context module 222 indicating that the user is located at or near a department of the merchant's store that sells toothbrush replacement heads, search module 264 may formulate a search query that is more likely to yield product information about replacement heads that are compatible with the electric toothbrushes purchased by the user's family members rather than product information about other replacement heads that are incompatible with the family member's toothbrush purchases.

[0073] In some examples, in response to receiving a prior search query associated with a product, search module 164 may determine product information that is predicted to assist a user in completing a purchase of the product for sale at a physical location associated with a merchant. For example, a search history associated with a user of computing device 110 stored in data store 284 may indicate that the user was searching for stores with parts for a particular brand of lawn mower before arriving at the merchant's location. When formulating an autonomous search query, search module 164 may reverse the query to obtain information about products compatible with the particular brand of lawn mower mentioned in the search history.

[0074] In some examples, search module 164 may determine product information that is predicted to assist a user in completing a purchase of a product for sale at a physical location associated with a merchant based at least in part on search results obtained from search queries associated with the product that have been executed by other computing devices at the physical location. For example, data store 284 may include a history of searches performed by users of other computing devices while the other computing devices were at the merchant's physical location. The search history may be tagged with location information indicating the nearest beacon or precise location within the merchant's store where these searches were performed. Search module 264 may infer that if a particular search query has been executed at the location of computing device 110 with a particular frequency, then the user of computing device 110 may also find the same type of product information that the other users are searching for.

[0075] Context module 222 may receive context information associated with computing device 110 via network 130, and similar to context module 122 of computing device 110, context module 222 may generate a context associated with computing device 110. With respect to customer assistance services provided by ISS 260, a primary purpose of context module 222 may be to determine the precise location of a computing device, such as computing device 110, when the computing device is located within an interior space of a physical location associated with a merchant. In other words, context module 222 may be configured to precisely locate the interior location of a computing device within a store (e.g., to a specific aisle and / or nearest bin) even if a GPS signal is unavailable.

[0076] For example, based on contextual information received from computing device 110, context module 222 can determine that computing device 110 is at a physical location of a merchant. In some examples, when computing device 110 is at or near a physical location associated with a merchant, the contextual information received from computing device 110 can include at least one of accelerometer data, wireless communication data, or beacon information obtained by sensors and radios of computing device 110 from multiple beacons and wireless communication devices. Context module 222 can track the signal strengths associated with multiple beacons and discard beacon signals that are inconsistent beacon signals. Context module 222 can assign higher proximity scores to beacons with higher signal strengths and lower proximity scores to beacons with lower signal strengths.

[0077] Although described as accelerometer data, other sensor data may be used to determine the location of the computing device 110. Examples of other sensor data include barometer data, ambient light sensor data, proximity sensor data, gyroscope sensor data, etc. Examples of wireless communication data include Cellular radio data (e.g. 3G, 4G, etc.), radio frequency chip, infrared receiver data, near field communication data (NFC), etc.

[0078] The context module 222 can use the accelerometer data and / or wireless communication data to adjust and improve the proximity score associated with each of the plurality of beacons. For example, the context module 222 can use the beacon strength in addition to the accelerometer data and / or wireless communication data to determine the proximity score for each beacon.

[0079] The context module 222 may infer that the location of the computing device 110 is closest to a single beacon when the beacon has a higher signal strength than other beacons. However, if there are multiple beacons with high signal strengths of approximately equal value, the context module 222 may weight the wireless communication and / or accelerometer data more highly when determining the proximity score for each of the multiple beacons. In other words, in response to determining that the beacon information is associated with multiple beacons, the context module 222 may prioritize the wireless communication data over the beacon information for determining that the computing device 110 is at a physical location.

[0080] In some examples, in response to receiving user input to dismiss product information, context module 222 may deprioritize beacon information for determining future physical locations of the computing device. For example, context module 222 may receive feedback information from assistance module 266 that context module 222 subsequently uses to improve the proximity score determined for the beacon. Assistance module 266 may receive information from computing device 110 indicating that a user of computing device 110 dismissed or ignored product information presented to the user as part of a customer assistance service provided by ISS 260. Assistance module 266 may share the received information with context module 222. Context module 222 may adjust its proximity score calculations for future contexts based on whether the user dismissed or did not dismiss the product information presented to the user. If assistance module 266 determines that the user did not perceive the product information as useful, context module 222 may infer that the beacon information used to determine the location of computing device 110 is inaccurate (or at least less accurate than wireless communication and accelerometer data).

[0081] In some examples, context module 222 can use merchant information (e.g., from MSS 180) to determine the location of computing device 110 within a physical space associated with a merchant. For example, context module 222 can obtain data indicating the location of each of a plurality of beacons and wireless communication devices relative to an interior area of the merchant's physical location. In response to identifying the nearest beacon or wireless communication device, context module 222 can determine that the location of computing device 110 corresponds to the location of the nearest beacon or wireless communication device.

[0082] In some examples, the context module 222 can use beacon information obtained from low-coverage, high-frequency beacons (e.g., 900 MHz) to calibrate accelerometer data received from the computing device 110. For example, a merchant can place low-coverage, high-frequency beacons that are easily detected when the computing device is very close (e.g., nearby), but not detectable at a distance. For example, such beacons can be radio beacons that operate in the 900 MHz spectrum, other general radio spectrum, or any other suitable spectrum obtained through government licenses, and also have very low output power, so that the receiving device needs to be close (e.g., near the aisle where the beacon is located) to receive the signal. The merchant can place low-coverage, high-frequency beacons at choke points in the store (e.g., entrances, exits, major traffic areas, etc.). When the context module 222 detects beacon information from one or more beacons located in the choke point, the context module 222 can calibrate the accelerometer data received from the computing device 110 to the specific location and calculate the relative X, Y, and Z displacement from the choke point. In this way, the context module 222 can eliminate movement in non-aisle areas or other locations where merchandise is not typically present to determine the location of the computing device 110. In addition to low-coverage, high-frequency beacons, other types of transmission devices, such as RFID transmission devices, can be used at choke points, and the context module 222 can similarly calibrate the accelerometer data when receiving RFID information from the computing device 110.

[0083] Assistance module 266 may manage customer assistance services that ISS 260 provides to computing device 110 for delivering product information that a user of computing device 110 may find helpful in completing a purchase of a product while in the vicinity of a merchant's physical location. Figure 2 In the example of FIG, the assistance module 266 includes a machine assistance module 268A.

[0084] The machine assistance module 268A provides a first level of support or customer assistance to the user of the computing device 110. For example, if the assistance module 266 infers that the user of the computing device 110 may be looking for product information that will assist in completing a purchase, the machine assistance module 268A may automatically surface the product information for presentation to the user of the computing device 110.

[0085] The machine assistance module 268 includes an artificial intelligence-based system and / or a machine learning-based natural language processing system that parses the product information data store 282 for data about items offered for sale by merchants associated with the physical location of the computing device 110, data about current inventory levels in a specific store at the physical location and in other stores of the merchant, and data about transactions associated with specific items.

[0086] In some examples, the machine assistance module 268A may receive additional information from the computing device 110 (e.g., via voice and / or text-based input from the user), and the artificial intelligence system of the assistance module 266 may engage with the user of the computing device 110 based on the additional information to try and obtain specific product information that answers all questions the user may have about the product. For example, after surfacing the product information at the UID 112, the UI module 120 may receive an indication of voice input from the user of the computing device 110 that includes audio data representing the question the user is asking. The machine assistance module 268A may break down the audio data into specific parts of the sentence structure and pass the various parts of the audio data through a natural language processing algorithm to determine the subject, object, verb, and other important parts of the sentence. The machine assistance module 268 may then input the sentence parts into an inference algorithm to find appropriate matches. In addition, the machine assistance module 268A may extract inventory levels for the item the user is querying from the merchant information data store 282.

[0087] In some examples, the machine-assistance module 268A can utilize contextual information received by the context module 222 to further disambiguate which item(s) the user is querying. For example, the context module 222 can share the most recent accelerometer data received from the computing device 110, as well as the beacon information and proximity scores, with the machine-assistance module 268A, and the machine-assistance module 268A can determine items that are near the location of the computing device 110. For example, the machine-assistance module 268A can use the contextual information received by the context module 222 to discern ambiguous item names or descriptions in questions from the user.

[0088] The machine assistance module 268A may determine a score associated with the product information obtained in response to a question from the user. If the machine assistance module 268A fails to obtain a satisfactory answer (e.g., the score is below a threshold for matching any existing objects), the assistance module 266 may escalate the customer assistance service to the next level of service. In other words, in response to determining that the likelihood of the user completing a product purchase in response to receiving specific product information does not meet the likelihood threshold, the assistance module 266 may execute the remote human assistance module 268B.

[0089] In some examples, the machine-assistance module 268A may determine the beacon closest to the merchant's physical location based on the context of the computing device 110 and the merchant information stored at the data storage 282. For example, the machine-assistance module 268A may query the data storage 282 for locations and receive an indication of the nearest beacon at the merchant's physical location. The machine-assistance module 268A may further query the data storage 282 for the nearest beacon and obtain information about items for sale within a threshold distance (e.g., one foot, one meter, etc.) of the beacon. The machine-assistance module 268A may identify the item being purchased from the merchant at the physical location from among the items located within the threshold distance from the nearest beacon.

[0090] In some examples, the machine-assistance module 268A may identify items being purchased from merchants at a physical location from among items located within a threshold distance from a nearest beacon, and further based at least in part on the user's search history. In other words, if several items are sold within a threshold distance of a nearest beacon, the machine-assistance module 268A may determine whether any of the nearest items correspond to items indicated in the user's search history stored at the data store 284. The machine-assistance module 268A infers whether the user of the computing device 110 is located near several items that appear in the recently performed search history and that he or she is most likely looking to purchase.

[0091] In some examples, the assistance module 266 may call or execute the remote human assistance module 268B in response to receiving user input to discard the product information. In other words, the computing device 110 may detect user input at the UID 112 indicating that the user is not watching or listening to the product information presented at the UID 112, or may receive user input at the UID 112 indicating that the product information is not helpful to the user (e.g., if the user provides voice or touch input to the computing device 110 indicating as much). In any case, the assistance module 266 may receive an indication of the user input detected by the computing device 110 and discern that the machine assistance module 268A does not provide the type of service that the user may expect or need. In response to determining that the user of the computing device 110 may be dissatisfied with the product information provided by the machine assistance module 268A, the assistance module 266 may execute the remote human assistance module 268B.

[0092] Remote human assistance module 268B represents a remote assistance module for providing a virtual environment in which a human can manually provide the additional information a user needs to complete a purchase. The virtual environment provided by remote human assistance module 268B can include at least one of a text-based communication environment, a video-based communication environment, or a voice-based communication environment. In other words, module 268B can cause UI module 120 of computing device 110 to present a text-based, video-based, or voice-based chat user interface for communicating with a person in a remote call center. A human can access similar information to that accessed by machine assistance module 268—only a human can also use their intuition to make a better judgment or educated guess as to whether the product information they can obtain for a user will be helpful.

[0093] Despite Figure 2 In some examples, remote human assistance module 268B may be part of UI module 120 of computing device 110. In other words, computing device 110 may include a local remote assistance module and the remote assistance module may be operated by at least one processor of computing device 110 to provide a virtual environment in which a human provides additional information needed by the user to complete the purchase.

[0094] In some examples, if the assistance module 266 determines that the user may not have received sufficient information to complete the purchase, the assistance module 266 may invoke the in-person assistance scheduling module 268C. In response to determining that the remote human assistance service cannot provide the additional information the user needs to complete the purchase, the assistance module 266 may send a notification to the merchant system (e.g., MS 180) to schedule in-person assistance at a physical location associated with the merchant. For example, a person in a call center supported by the virtual environment provided by module 268B may provide input to the device of the user of computing device 110 to send a message to the assistance module 260 indicating that the user requires a higher level of service. In some examples, invoking the in-person assistance scheduling module 268C may cause module 268C to send an indication or other notification to MSS 180 (or some other merchant system) that the user of the computing device requires in-person assistance and the physical location at which the user requires in-person assistance. The message sent by module 268C to MSS 180 may indicate the exact aisle and / or bin range where the user of computing device 110 is located and may cause an in-store employee to be directed to that location.

[0095] Figures 3A-3Dis a conceptual diagram illustrating example graphical user interfaces 314A-314D presented by an example computing device 310 configured to provide user assistance to a user of the computing device 310 at a physical location in accordance with one or more aspects of the present disclosure. The computing device 310 is Figure 1 An example of a computing device 110 of the system 100 is shown. Figure 1 The system 100 is described in the context of Figures 3A-3D as follows.

[0096] exist Figures 3A-3D In the example of FIG, computing device 310 is a mobile phone or tablet device. Computing device 310 includes a UID 312 configured to display user interfaces 314A-314D.

[0097] User interface 310A includes first product information that is automatically surfaced and presented to the user of computing device 310 in response to computing device 110 determining that computing device 310 is at a physical location associated with a merchant and further in response to determining that the first product information enables the user to complete a purchase of the product from the merchant at the physical location in response to receiving the product information. For example, the first product information may include a brief product description, reviews, rankings, price information, inventory information, etc. User interface 310A also includes an input box for receiving text input (or voice input) from the user if the user wishes to enter a question the user would like answered about a nearby product. For example, when the user of computing device 310 clicks the "Ask me anything" button in user interface 314A, they can initiate a chat with the intelligent auto-response system of computing device 310, which can understand the user's written or spoken questions and respond to the user with a written or audible response (e.g., using natural language processing, deep learning, machine learning, and / or other artificial intelligence technologies).

[0098] For example, Figure 3B Computing device 310 is shown presenting user interface 310B, which includes predictive information presented at UID 312 in response to computing device 110 receiving audio input representing a question asked by a user of computing device 310. Computing device 310 may present the predictive information in response to determining that the likelihood associated with the predicted information meets a likelihood threshold for assisting the user in completing a purchase of an item. In other words, computing device 310 may only present the predicted information if computing device 310 is more certain that the user will find the predicted information useful.

[0099] Figure 3CThe figure shows computing device 310 presenting user interface 314C at UID 312 in response to determining that the predictive information obtained by computing device 310 in response to the user's question does not have a high enough score or a degree of likelihood that it is available. In this case, computing device 310 has executed a remote human assistance module for providing a virtual environment in which a human provides additional information needed by the user to complete the purchase. Figure 3C As shown, the computing device 310 presents a chat window, through which the user of the computing device 310 can communicate with Bob (a person) located in a remote call center to obtain product information that the user may need to complete the purchase of the product.

[0100] Figure 3D Computing device 310 presents user interface 314D at UID 312, illustrating that the remote human assistance module is unable to provide the additional information required by the user to complete the purchase. Figure 3D In the example of FIG, computing device 310 automatically schedules in-person assistance to a physical location associated with a merchant to provide the product information needed by the user of computing device 310 to purchase the product.

[0101] Figure 3D Also shown is that when computing device 310 is located at a physical location associated with a merchant, computing device 310 provides customer engagement as part of a customer assistance service provided to a user of computing device 310. Embedded in user interface 314D is a coupon that the computing device provides to the user as a courtesy (e.g., if customer service is delayed, cannot answer the user's question, etc.). By providing coupons in this manner, merchants acting through computing device 310 are able to provide high-quality engagement with customers.

[0102] Figure 4 is a flow diagram illustrating example operations 400-440 performed by an example computing system, such as ISS 160, configured to provide customer assistance to users of computing devices at physical locations, in accordance with one or more aspects of the present disclosure. Figure 1 The system 100 is described in the context of Figure 4 For example, according to one or more aspects of the present disclosure, ISS 160 may perform operations 400-440.

[0103] exist Figure 4In the example of FIG4 , ISS 160 may identify an item that a user of a computing device intends to purchase from a physical location associated with a merchant based on contextual information associated with the computing device located at the physical location and from a plurality of items for sale at the physical location (400). For example, ISS 160 may receive a context from context module 122 of computing device 110 indicating that computing device 110 is located in a store of a merchant. By determining the items closest to the location of computing device 110 and / or based on other information about the user of computing device 110, ISS 160 may determine the items that the user intended to purchase at the store of the merchant.

[0104] ISS 160 may perform an autonomous search query (410) for product information that is predicted to assist the user in completing a purchase of a product from a merchant at a physical location. For example, ISS 160 may use the context of computing device 110, information about the user of computing device 110, and merchant information to form a query for information about products that are closest to computing device 110 and perform an autonomous search for information about the products. ISS 160 may assign a likelihood score or degree to the product information returned from the autonomous search, the score or degree indicating whether the user will complete the purchase of the product in response to receiving the product information.

[0105] ISS 160 may determine whether the likelihood that the user will complete the purchase in response to receiving the product information satisfies a likelihood threshold 420. For example, the likelihood threshold may be 70 percent, 50 percent, 10 percent, etc.

[0106] ISS 160 may send the product information to the computing device, and the computing device may then output the product information (430). For example, in some examples, ISS 160 may only send the product information if the product information has a likelihood that satisfies a threshold. In other examples, ISS 160 may send the product information regardless of whether the product information satisfies the likelihood threshold. In either case, when ISS 160 sends the product information, ISS 160 may send the product information along with instructions for formatting the product information as an information card that computing device 110 may automatically surface at UID 112.

[0107] In response to determining that the likelihood level does not meet the likelihood threshold, ISS 160 may execute a remote assistance module accessed by the computing device to provide a virtual environment in which a person provides additional information needed by the user to complete the purchase (440). For example, if the product information does not have a likelihood level that meets the threshold, ISS 160 may escalate the user's request for assistance to a person in a back room at a remote call center or merchant's location with whom the user can chat to obtain the product information he or she needs to complete the purchase.

[0108] Figure 5 is a flow diagram illustrating example operations 500-550 performed by an example computing device, such as computing device 110, configured to provide customer assistance to a user of the computing device at a physical location, in accordance with one or more aspects of the present disclosure. Figure 1 The system 100 is described in the context of Figure 5 For example, according to one or more aspects of the present disclosure, computing device 110 may perform operations 500 - 550 .

[0109] exist Figure 5 In the example of FIG. 5 , computing device 110 may determine that computing device 110 is at a physical location associated with a merchant ( 500 ). For example, computing device 110 may receive contextual information including beacon information, accelerometer data, wireless communication data, etc., and determine, based on the contextual information, an aisle, row, bin, table, etc. within the merchant's store that is closest to the location of computing device 110 .

[0110] Computing device 110 may determine product information predicted to assist a user of computing device 110 in completing a purchase of a product from a merchant at a physical location (510) based on contextual information associated with computing device 110. For example, computing device 110 may formulate an autonomous search query for product information associated with products that are closest to the location of computing device 110. Results from the autonomous search may include product information.

[0111] In some examples, computing device 110 may receive an indication of a user-provided query associated with an item. For example, a user may speak or type a question into a user interface presented at UID 112. Computing device 110 may search for and determine item information based at least in part on the user-provided query.

[0112] Computing device 110 may determine, in response to receiving the product information, a degree of likelihood that the user will complete the purchase (520). For example, computing device 110 may assign a score or ranking to the product information that indicates how relevant it is to the product or how likely the information is to assist the user in purchasing the product from the merchant location.

[0113] In some examples, if a query is received from a user, computing device 110 may determine a degree of likelihood that the user will complete a purchase in response to receiving the product information based at least in part on whether the product information will satisfy the query. For example, if the product information has a 90% certainty that it is likely to satisfy the query (e.g., answer the question), computing device 110 may assign a higher score to the product information, and if the product information has only a 70% certainty that it is likely to satisfy the query (e.g., answer the question), assign a lower score to the product information.

[0114] Computing device 110 may output an indication of the product information (530). For example, in some examples, if the degree of likelihood is high enough to warrant presentation to the user, computing device 110 may cause UID 112 to present the product information (e.g., as graphical user interface 114). For example, if the product information has a 90% certainty that it satisfies the user query, computing device 110 may present the product information. And if the product information has only a 70% certainty that it satisfies the user query, computing device 110 may refrain from presenting the product information. In other examples, computing device 110 may cause UID 112 to automatically present the product information without determining whether the degree of likelihood must satisfy a likelihood threshold indicating that the likelihood is high enough to warrant presentation to the user.

[0115] When computing device 110 automatically presents product information to the user, the user may provide input to computing device 110 (e.g., by typing or speaking a question to computing device 110). Computing device 110 may continue to attempt to present product information that answers the user's question and has a degree of likelihood that satisfies the likelihood threshold.

[0116] In response to determining that the likelihood level does not meet the likelihood threshold, computing device 110 may execute a remote assistance module to provide a virtual environment in which a human provides additional information required by the user to complete the purchase (540). For example, if computing device 110 determines that the likelihood level associated with the product information is insufficient to warrant presentation to the user, computing device 110 may enhance assistance to the user by invoking a remote human call center to answer questions and provide the product information to the user.

[0117] In response to determining that the remote human assistance module cannot provide the additional information required by the user to complete the purchase, the computing device 110 may send a notification to the merchant system to schedule in-person assistance to a physical location associated with the merchant (550). For example, the computing device 110 may receive information via input from the user or from a person associated with the remote human assistance module indicating that the user needs to schedule an expert to the user's location to provide the user with personal, in-person customer assistance to help the user complete the purchase of the item from the physical location associated with the merchant. The computing device 110 may send a notification to the scheduling system within the merchant location to cause an expert who can answer questions about the item to arrive at the physical location of the computing device 110.

[0118] In some examples, the techniques of this disclosure may further enable other elements of customer assistance services not described herein. For example, as a further part of customer engagement, ISS 160 or computing device 110 may determine which aisles the user of computing device 110 has visited based on the user's movements and other contextual information obtained from computing device 110. If ISS 160 and / or computing device 110 determine that the user's movements or movements indicate that he or she has visited sufficiently high-end aisles (e.g., aisles where the average value of items in the aisles significantly exceeds the user's normal shopping average), ISS 160 and / or computing device 110 may provide a coupon to the user (e.g., displayed at UID 112).

[0119] In yet another example, a merchant may equip a shopping cart provided to a customer with a device that uses the rotation of the cart to power the device (e.g., like a generator) and activate an RFID reader in the cart. Assuming that all high-cost items in the store are RFID-tagged (e.g., for theft detection), the ISS 160 may use wireless communication (e.g., to read the RFID tags) when an item is placed in the cart. ) receives information from the cart RFID reader, along with accelerometer data indicating the direction of motion of the cart wheels to further determine a match with the existing user acceleration and accurately locate the items in the user's shopping cart. In response to determining the items in the cart, ISS 160 can provide highly targeted coupons and related items to computing device 100. For example, if ISS 160 determines that a television has been placed in the cart, ISS 160 can provide coupons for UDMI or audio-video cables.

[0120] Clause 1. A method comprising: determining, by a computing device, that the computing device is at a physical location associated with a merchant; determining, by the computing device based on contextual information associated with the computing device, product information that is predicted to assist a user of the computing device in completing a purchase of a product from the merchant at the physical location; determining, by the computing device, a degree of likelihood that the user will complete the purchase in response to receiving the product information; outputting, by the computing device, an indication of the product information; and in response to determining that the degree of likelihood does not meet a likelihood threshold, executing, by the computing device, a remote assistance module to provide a virtual environment in which a person provides additional information required by the user to complete the purchase.

[0121] Clause 2. The method according to clause 1 further includes: in response to determining that the remote human assistance module cannot provide the additional information required by the user to complete the purchase, sending a notification by the computing device to the merchant system to request in-person assistance at a physical location associated with the merchant.

[0122] Clause 3. The method of any of clauses 1-2, wherein: the computing device receives an indication of a user-provided query associated with a product, wherein: the product information is further determined based at least in part on the user-provided query; and

[0123] The degree of likelihood that a user will complete a purchase in response to receiving the item information is determined based at least in part on whether the item information will satisfy the query.

[0124] Clause 4. A method according to any of clauses 1-3, wherein determining that the computing device is at the physical location of the merchant is based on at least one of accelerometer data, wireless communication data, or beacon information; and wherein the method further comprises at least one of: in response to determining that the beacon information is associated with multiple beacons, the computing device prioritizing the wireless communication data over the beacon information for determining that the computing device is at the physical location; or in response to receiving user input to discard product information, the computing device deprioritizing the beacon information for determining that the computing device is at the physical location in the future.

[0125] Clause 5. The method of any of clauses 1-4, wherein outputting instructions for product information including merchant information comprises further instructions for outputting merchant information including a promotional coupon or advertisement for purchasing the product at the physical location.

[0126] Clause 6. The method according to any of clauses 1-5 further includes: determining, by the computing device, a beacon closest to the physical location of the merchant; and identifying, by the computing device, the item as a specific item from a plurality of items for sale within a threshold distance of the beacon based at least in part on the user's search history.

[0127] Clause 7. A method according to clause 6, wherein the item is further determined based on at least one of: communication information associated with the user; the user's purchase history; an electronic shopping list associated with the user; or an electronic shopping list, purchase history, or communication information associated with a family member of the user.

[0128] Clause 8. The method of any of clauses 1-7, wherein the item information predicted to assist the user in completing a purchase of the item for sale at the physical location is further determined in response to receiving a prior search query associated with the item.

[0129] Clause 9. The method of any of clauses 1-8, wherein the product information is determined based at least in part on search results obtained from search queries associated with the product that have been executed at the physical location by other computing devices.

[0130] Clause 10. The method of any of clauses 1-9, wherein the virtual environment provided by the remote assistance module comprises at least one of a text-based communication environment, a video-based communication environment, or a voice-based communication environment.

[0131] Clause 11. The method of any of clauses 1-10, wherein the degree of likelihood that the user will complete a purchase in response to receiving the product information is determined at least in part based on whether other users completed purchases of the product in response to receiving the product information.

[0132] Clause 12. The method of any one of clauses 1-11, wherein the remote assistance module is further executed in response to receiving user input to discard the product information.

[0133] Clause 13. A method comprising: identifying, by a computing system, an item that a user of the computing device intends to purchase from the physical location based on contextual information associated with a computing device located at a physical location associated with a merchant and contextual information from a plurality of items for sale at the physical location; executing, by the computing system, an autonomous search query for item information that is predicted to assist the user in completing a purchase of the item from the merchant at the physical location; determining, by the computing system, whether a degree of likelihood that the user will complete the purchase in response to receiving the item information satisfies a likelihood threshold; sending, by the computing system, the item information to the computing device and the item information subsequently output by the computing device; and in response to determining that the degree of likelihood does not satisfy the likelihood threshold, executing, by the computing system, a remote assistance module accessed by the computing device to provide a virtual environment in which a person provides additional information required by the user to complete the purchase.

[0134] Clause 14. The method according to clause 13 further includes: in response to determining that the remote human assistance service cannot provide the additional information required by the user to complete the purchase, sending a notification by the computing system to the merchant system to schedule in-person assistance to a physical location associated with the merchant.

[0135] Clause 15. The method of clause 14, wherein scheduling in-person assistance to a physical location associated with the merchant comprises sending, by the computing system, information that a user of the computing device requires in-person assistance and the physical location to the merchant system.

[0136] Clause 16. The method according to any of clauses 13-15 further includes: determining, by a computing system, a beacon closest to the physical location of a merchant; and identifying, by the computing system, the item as a specific item from among a plurality of items for sale within a threshold distance of the beacon based at least in part on a user's search history.

[0137] Clause 17. The method of any of clauses 13-16, further comprising: adjusting, by the computing system, the degree of likelihood that the user will complete the purchase in response to determining whether users of other computing devices dismissed previously presented product information at the physical location.

[0138] Item 18. A computing device comprising: at least one processor; input and output devices configured to present a user interface associated with the computing device; a context module, the context module being operable by the at least one processor to: determine that the computing device is at a physical location associated with a merchant; and obtain context information associated with the computing device; a user interface module, the user interface module being operable by the at least one processor to: receive, based on the context information associated with the computing device, product information that is predicted to assist a user of the computing device in completing a purchase of a product from the merchant at the physical location, the product information having been assigned a degree of likelihood that the user will complete the purchase in response to receiving the product information; output an indication of the product information via the input and output devices; and in response to determining that the degree of likelihood does not meet a likelihood threshold, execute a remote assistance module to provide a virtual environment using the input and output devices in which a person provides additional information required by the user to complete the purchase.

[0139] Clause 19. The computing device of clause 18, wherein the user interface module is further operable by the at least one processor to, in response to determining that the remote human assistance module is unable to provide additional information required by the user to complete the purchase, dispatch in-person assistance to a physical location associated with the merchant.

[0140] Clause 20. A computing device according to any of clauses 18-19, wherein: the computing device includes a remote assistance module, and the remote assistance module is operable by at least one processor of the computing device to provide a virtual environment; or the remote computing system includes a remote assistance module, and the user interface module is operable by at least one processor of the computing device to access the virtual environment as a service provided by the remote assistance module.

[0141] Clause 21. A computing device comprising means for performing any of the methods of clauses 1-12.

[0142] Clause 22. A computer-readable storage medium comprising instructions that, when executed, cause at least one processor of a computing device to perform any of the methods of clauses 1-12.

[0143] Clause 23. A computing system comprising means for performing any of the methods of clauses 13-17.

[0144] Clause 24. A computer-readable storage medium comprising instructions that, when executed, cause at least one processor of a computing system to perform any of the methods of clauses 13-17.

[0145] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit.

[0146] Computer-readable media may include computer-readable storage media corresponding to tangible media such as data storage media, or communication media including any medium that facilitates the transfer of a computer program from one place to another, for example, according to a communication protocol. In this manner, computer-readable media may generally correspond to (1) a non-transitory, tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. A computer program commodity may include computer-readable media.

[0147] As an example and not by way of limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, flash memory or any other storage medium that may be used to store desired program code in the form of an instruction or data structure and that may be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technologies such as infrared, radio and microwaves, then coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwaves are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connection, carrier wave, signal or other transient media, but are directed to non-transient tangible storage media. Disks and optical disks (disks and discs) as used herein include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks and blue-ray discs, wherein disks (disks) typically copy data magnetically, while optical discs (disks) copy data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0148] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules. Furthermore, these techniques may be implemented entirely in one or more circuits or logic elements.

[0149] The technology of the present disclosure can be implemented in a variety of devices or apparatuses including wireless handsets, integrated circuits (ICs), or a set of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize the functional aspects of devices configured to perform the disclosed technology, but do not necessarily need to be implemented by different hardware units. Instead, as described above, the various units can be combined in hardware units or provided by a collection of interoperable hardware units - including one or more processors as described above - in combination with appropriate software and / or firmware.

[0150] Various embodiments have been described. These and other embodiments are within the scope of the following claims.

Claims

1. A method for automatic product identification and presentation, the method being implemented by one or more processors and comprising: determining, based on sensor data from one or more sensors of a computing device, that the computing device is located at a physical store; identifying items of interest to the user based on stored information based on one or more computing interactions of the user prior to arriving at the physical store; determining that the item is available at the physical store based on accessing stored store information associated with the physical store; and In response to identifying the item based on the one or more computing interactions of the user, determining that the item is available at the physical store, and determining that the computing device is located at the physical store: automatically executing, while the computing device is located at the physical store, an autonomous search query based on the product to identify product information responsive to the autonomous search query; determining whether a degree of likelihood that the user will complete the purchase of the product in response to receiving the product information satisfies a likelihood threshold; as well as In response to determining that the degree of likelihood satisfies the likelihood threshold: The computing device is caused to automatically present the product information based on the product information in response to the autonomous search query, wherein the product information is predicted to assist a user of the computing device in completing a purchase of the product at the physical store.

2. The method according to claim 1, further comprising: In response to determining that the degree of likelihood does not satisfy the likelihood threshold, causing the computing device to execute a remote assistance module that causes the computing device to engage with a remote computing system via which a person interacting with the remote computing system provides additional information related to the item.

3. The method according to claim 1, further comprising: In response to determining that the degree of likelihood does not satisfy the likelihood threshold, an electronic notification is caused to be transmitted over a network, wherein the electronic notification requests in-person assistance at the physical location.

4. The method according to claim 1, wherein The autonomous search query is determined based at least in part on one or more searches based on one or more search queries that have been performed at the physical store by other computing devices.

5. The method according to claim 4, wherein Determining the product information is based at least in part on search results obtained in response to one or more of the search queries executed by the other computing device at the physical store.

6. The method according to claim 1, in, Based on the sensor data from one or more sensors of the computing device, determining that the computing device is located at a physical store further comprises: determining that the computing device is located at a precise location within the physical store; and wherein determining that the item is available at the physical store based on accessing stored store information associated with the physical store comprises: It is determined that the item is within a threshold distance of the precise location.

7. The method according to claim 6, wherein: Determining the precise location where the computing device is located within the physical store includes: determining, based on at least some of the sensor data, detected signal strengths of beacons within the physical store; and The precise location is determined based on the detected signal strength of the beacon.

8. A computing device comprising: input and output devices configured to present a user interface associated with the computing device; Multiple sensors; a memory for storing instructions; at least one processor executing stored instructions for: determining, based on sensor data from one or more of the sensors, that the computing device is located at a physical store; identifying an item of interest based on one or more interactions via the computing device prior to arriving at the physical store; Determining that the product is available at the physical store; and In response to identifying the item based on the one or more interactions, determining that the item is available at the physical store, and determining that the computing device is located at the physical store: automatically executing, while the computing device is located at the physical store, an autonomous search query based on the product to identify product information responsive to the autonomous search query; determining whether a degree of likelihood that the user will complete the purchase of the product in response to receiving the product information satisfies a likelihood threshold; as well as In response to determining that the degree of likelihood satisfies the likelihood threshold: Automatically presenting product information at the user interface, wherein the product information is presented based on the product information in response to the autonomous search query, and wherein the product information is directed to completing a purchase of the product at the physical store.

9. A method for automatic product identification and presentation, the method comprising: identifying, by a computing system, an item that a user of the computing device wants to purchase from a physical location associated with a merchant based on sensor data from one or more sensors of the computing device, based on contextual information associated with the computing device being located at the physical location and based on a plurality of items for sale at the physical location; automatically executing, by the computing system, an autonomous search query for product information predicted to assist the user in completing a purchase of the product from the merchant at the physical location, the autonomous search query being based at least in part on one or more search queries executed by other computing devices at the physical location; determining, by the computing system, that a likelihood that the user will complete the purchase in response to receiving the product information satisfies a likelihood threshold; sending, by the computing device, an indication of the product information to the computing device via one or more networks for subsequent output by the computing device; as well as In response to determining that the likelihood level does not meet the likelihood threshold, a remote assistance module accessed by the computing device is executed by the computing system to provide a user interface of a virtual environment in which a person provides additional information required by the user to complete the purchase.

10. The method according to claim 9, further comprising: In response to determining that the remote assistance module cannot provide the additional information required by the user to complete the purchase, a notification is sent by the computing system to a merchant system to schedule in-person assistance at the physical location associated with the merchant.

11. The method according to claim 10, wherein: Scheduling the in-person assistance to the physical location associated with the merchant includes sending, by the computing system, an indication that the user of the computing device requires the in-person assistance and the physical location to a merchant system.

12. The method according to claim 9, further comprising: determining, by the computing system, a beacon closest to the physical location of the merchant; and The item is identified, by the computing system, as a particular item from a plurality of items for sale within a threshold distance of the beacon based at least in part on a search history of the user.

13. The method according to claim 9, further comprising: The degree of likelihood that the user will complete the purchase is adjusted by the computing system in response to determining whether users of other computing devices dismissed a previous presentation of the item information at the physical location.

14. A computing device comprising means for performing the method according to any one of claims 1-7 and 9-13.

Images