Improving interaction with an electronic chat interface

Abstract

A method includes, for each of a plurality of network resources, receiving, at a communication server, data indicative of a characteristic of the respective network resource; detecting, based on the received data, that the respective network resource provides functionality for live assistance by a third-party content provider through a chat user interface on the respective network resource; and storing, in a database, an entry indicative that the respective network resource has the functionality. The method includes receiving, from a user, a request to access a particular network resource hosted by a particular third-party content provider; determining, based on a stored entry in the database representing the particular network resource, that the network resource provides functionality for live assistance by the particular third-party content provider through a particular chat user interface on the network resource; and initiating a chat session between the user and the third-party content provider.

Description

[0001] This case is a divisional application of the invention patent application filed on June 13, 2017, with application number 201780084780.4 and invention title "Improved Interaction with Electronic Chat Interface". Background Technology

[0002] This document describes a chat interface through which users can communicate with content providers offering access to resources or content. Chat interfaces may vary across content providers and may be implemented and displayed differently. Furthermore, existing chat interfaces may suffer some or all functional loss when displayed on mobile devices. Additionally, virtually no resources are available when a chat session is disconnected. Summary of the Invention

[0003] According to at least one embodiment of this disclosure, a method is provided, comprising: receiving voice input from a user via a communication server, the voice input being submitted via a microphone of a personal digital assistant; obtaining search results using the voice input via the communication server; determining, via the communication server and based on analysis of network resources provided by a third-party content provider, that a third-party content provider corresponding to one of the search results provides real-time interactive assistance via a chat function through a chat user interface by detecting that the network resources include user interface elements initiating a connection with a real-time chat system provided by the third-party content provider; and, when the search results are delivered to the user, providing the user with access to the third-party content provider's chat function via the personal digital assistant. The provided chat functionality includes the option to initiate a chat session without requiring the user to first request network resources; receiving confirmation from the user that they have requested to initiate a chat session via a personal digital assistant; and initiating a chat session between the user and a proxy of the content provider via a communication server without redirecting the user to network resources provided by a third-party content provider. The option to initiate a chat session includes providing a voice inquiry via the personal digital assistant, which asks the user whether they wish to connect to or chat with a proxy that can provide the user with more information from the content provider. Receiving confirmation from the user that they have requested to initiate a chat session includes receiving an audio response to the voice inquiry via the personal digital assistant.

[0004] According to at least one embodiment of this disclosure, a system is provided, comprising: a gateway for storing interaction data corresponding to client-side input in a chat session; and a communication server for communicating with a client device, wherein the communication server facilitates a chat session between the client device and a content provider, and performs operations including: receiving voice input from a user of the client device, the voice input being submitted via a microphone of a personal digital assistant; obtaining search results using the voice input; and, based on analysis of network resources provided by a third-party content provider, determining, by detecting that the network resources include user interface elements initiating a connection to a real-time chat system provided by the third-party content provider, a third-party content provider corresponding to one of the search results provides real-time interactive assistance via a chat function through a chat user interface; and when the... When the search results are delivered to the user, the user is provided with the option to initiate a chat session using a chat function provided by a third-party content provider via a personal digital assistant (PDA), without the user first requesting network resources; the PDA receives confirmation from the user that they have requested to initiate a chat session; and a chat session is initiated between the user and a proxy of the content provider via a communication server without redirecting the user to network resources provided by the third-party content provider. Providing the option to initiate a chat session includes providing a voice inquiry via the PDA, which asks the user whether they wish to connect to a proxy that can provide the user with more information from the content provider or chat with the proxy. Receiving confirmation from the user that they have requested to initiate a chat session includes receiving an audio response to the voice inquiry via the PDA.

[0005] According to at least one embodiment of this disclosure, a non-transitory computer-readable medium is provided that stores instructions, which, when executed by one or more computing devices, cause the one or more computing devices to perform operations, the operations including: receiving voice input from a user, the voice input being submitted via a microphone of a personal digital assistant; obtaining search results using the voice input; and, based on analysis of network resources provided by a third-party content provider, determining, by detecting that the network resources include user interface elements initiating a connection to a real-time chat system provided by the third-party content provider, that a third-party content provider corresponding to one of the search results provides real-time interactive assistance via a chat function through a chat user interface; and, when the search results are delivered to a user, via a personal digital assistant... The digital assistant provides users with the option to initiate a chat session using chat functionality provided by a third-party content provider without requiring the user to first request network resources; receives confirmation from the user that they have requested to initiate a chat session via the personal digital assistant; and initiates a chat session between the user and an agent of the content provider without redirecting the user to network resources provided by the third-party content provider. Providing the option to initiate a chat session includes providing a voice inquiry via the personal digital assistant, the voice inquiry asking the user whether they wish to connect to an agent that can provide the user with more information from the content provider or chat with the agent; and receiving confirmation from the user that they have requested to initiate a chat session includes receiving an audio response to the voice inquiry via the personal digital assistant.

[0006] In general, one innovative aspect of the subject matter described in this specification can be implemented as a method comprising: receiving a search request for content from a user by a communication server; in response to the search request, obtaining search results on a displayed webpage in the form of multiple selectable content items, each searchable content item containing a link to a corresponding webpage resource associated with the search request. The method includes, based on information obtained from a database relating to the characteristics of multiple network resources, determining, by the communication server, that a specific network resource has the capability to be provided with real-time assistance by a specific third-party content provider through a specific chat user interface on the specific network resource; for said specific network resource, automatically providing a chat interface on the displayed webpage, the chat interface being rendered and / or controlled by the communication server to be displayed adjacent to the corresponding selectable content item on the displayed webpage. In response to user interaction with the chat interface from the displayed page, initiating a chat session between the user and the specific third-party content provider associated with the specific network resource by the communication server.

[0007] These and other implementations may each optionally include one or more of the following features. In some implementations, the method includes populating a database with database entries related to web page resources. Populating the database includes, for each of the plurality of web pages, receiving data at a communication server indicating the characteristics of the corresponding network resource; the communication server detecting, based on the received data, that the corresponding network resource provides real-time assistance functionality via a chat user interface on the corresponding network resource provided by a third-party content provider; and the communication server storing entries in the database indicating that the corresponding network resource has functionality. The step of populating the database is performed periodically to capture data of web pages not yet included in the database.

[0008] In some implementations, a chat session includes a first session between a communication server and a user, and a second session between the communication server and a third-party content provider. In some implementations, the method may include: the communication system detecting inactivity in the chat session for a predetermined period; the communication system detecting that at least one of the first or second sessions has ended; the communication system determining, based on contextual data of the chat session, that the user has not yet completed the chat session; storing data indicating the entire first session in a database; in response to detecting that the second session has ended, the communication system initiating a third session between the communication server and the third-party content provider as part of the chat session; and the communication system providing the third-party content provider with data indicating at least a portion of the first session, enabling the user to restart the chat session with the third-party content provider.

[0009] In some implementations, the method includes receiving input from a user via a chat user interface in a first session at a communication server; and providing data indicating user input in a second session to a third-party content provider by the communication server. In some implementations, the request to access a specific network resource includes voice input. In some implementations, the method includes receiving second voice input from a user in the first session by a communication system; providing data indicating a text representation of the second voice input to a third-party content provider in the second session; receiving text input from a third-party content provider in the second session between the communication system and the third-party content provider by the communication system; and providing data indicating a voice representation of the text input from the third-party content provider to the user in the first session by the communication system.

[0010] In some implementations, the chat user interface is a chat widget. In some implementations, the functionality of identifying a specific network resource through a specific chat user interface on that network resource, provided by a specific third-party content provider, includes: accessing the network resource by a communication system; identifying the widget on the network resource by the communication system; and identifying a label in the widget's code by the communication system, which indicates that the network resource is being provided with real-time assistance by a third-party content provider through the chat user interface on the network resource.

[0011] In some implementations, real-time assistance is provided by human representatives of specific third-party content providers.

[0012] In some implementations, the method includes, for each of multiple chat sessions initiated for a specific network resource, a content item metric of the chat session determined by the communication system based on a unique identifier of the content item that directs the user to the specific network resource. The method includes the communication system using the content item metric to generate an expression of the content item metric; and the communication system providing the expression to a third-party content provider for display.

[0013] In some implementations, the function of determining that a specific network resource is provided with real-time assistance by a specific third-party content provider through a specific chat user interface on the specific network resource includes: accessing the network resource by a communication system; determining by the communication system that a new window detached from the window accessing the network resource has been opened; and determining in the new window an instructive text phrase that instructs the network resource to be provided with real-time assistance by a third-party content provider through a chat user interface on the network resource.

[0014] In some implementations, users access chat sessions via a network-connected device. The network-connected device can be a mobile device. The network-connected device can be a desktop device. In some implementations, the network-connected device does not have a screen.

[0015] In some implementations, initiating a chat session between a user and a specific third-party content provider involves a communication server receiving a first unique session identifier for the first session of the chat session from the third-party content provider, and the communication server providing the first unique session identifier to the third-party content provider to facilitate a second session.

[0016] Another innovative aspect of the subject matter described in this specification can be implemented as a system comprising: a device communicatively connected to a network; a third-party content provider server communicatively connected to said network; and a communication server performing operations implementing the methods described above.

[0017] Another innovative aspect of the subject matter described in this specification can be implemented in at least one computer-readable storage medium encoded with executable instructions, which, when executed by at least one processor, cause at least one processor to perform operations implementing the methods described above.

[0018] Another innovative aspect of the subject matter described in this specification can be implemented as a method comprising: for each of a plurality of web pages, receiving data at a communication server indicating the characteristics of the corresponding network resource; the communication server detecting, based on the received data, the functionality provided by the corresponding network resource to a third-party content provider through a chat user interface on the corresponding network resource; and the communication server storing entries in a database indicating the functionality of the corresponding network resource. The method includes the communication server receiving from a user a request to access a specific network resource managed by a specific third-party content provider; the communication server determining, based on entries stored in the database representing the specific network resource, the functionality provided by the specific network resource to a specific third-party content provider through a specific chat user interface on the specific network resource; and the communication server initiating a chat session between the user and the third-party content provider.

[0019] In some implementations, a chat session includes a first session between a communication server and a user, and a second session between the communication server and a third-party content provider. The method may include receiving input from the user through a chat user interface at the communication server for the first session; providing data indicating user input for the second session to the third-party content provider by the communication server; and maintaining state information of the chat session between the user and the third-party content provider. The method may also include: the communication system detecting inactivity for a predetermined period; the communication system detecting that at least one of the first or second session has ended; the communication system determining that the user has not yet completed the chat session based on context data of the chat session; storing data indicating the entire first session in a database; initiating a third session between the communication server and the third-party content provider as part of the chat session in response to detecting that the second session has ended; and the communication system providing data indicating at least a portion of the first session to the third-party content provider, enabling the user to restart the chat session with the third-party content provider.

[0020] In some implementations, the request to access a specific network resource includes voice input. The method may include: receiving second voice input from a user in a first session by a communication system; providing data indicating a text representation of the second voice input to a third-party content provider in a second session; receiving text input from the third-party content provider in a second session between the communication system and the third-party content provider; and providing data indicating a voice representation of the text input from the third-party content provider to the user in the first session by the communication system.

[0021] In some implementations, the chat session initiated by the communication server between the user and a specific third-party content provider includes the communication server receiving a first unique session identifier for the first session of the chat session from the third-party content provider, and the communication server providing the first unique session identifier to the third-party content provider to facilitate a second session.

[0022] In some implementations, the request is provided in response to the selection of content items that are displayed and linked to specific web resources in response to a search request. The chat interface can be displayed on a webpage that shows the results and content items in response to the search request.

[0023] In some implementations, the chat user interface is a chat widget. In other implementations, the functionality of detecting relevant network resources and providing real-time assistance from a third-party content provider through the chat user interface on the relevant network resource includes: accessing the network resource by the communication system; identifying the widget on the network resource by the communication system; and identifying a label in the code of the widget on the network resource by the communication system, the label indicating that the network resource provides real-time assistance from a third-party content provider through the chat user interface on the network resource.

[0024] In some implementations, real-time assistance is provided by a human representative of a specific third-party content provider. In some implementations, the method includes: for each of multiple chat sessions initiated for a specific network resource, the communication system determining a content item metric of the chat session based on a unique identifier of the content item that directs the user to the specific network resource; the communication system generating an expression of the content item metric using the content item metric; and the communication system providing the expression to the third-party content provider for display.

[0025] In some implementations, the function of detecting a corresponding network resource by providing real-time assistance from a third-party content provider through a chat user interface on the corresponding network resource includes: accessing the network resource by a communication system; determining by the communication system that a new window detached from the window currently accessing the network resource has been opened; and determining an instruction text phrase in the new window that instructs the network resource to provide real-time assistance from a third-party content provider through a chat user interface on the network resource.

[0026] In some implementations, users access chat sessions via a network-connected device. The network-connected device can be a mobile device. The network-connected device can be a desktop device. In some implementations, the network-connected device does not have a screen.

[0027] Another innovative aspect of the subject matter described in this specification can be implemented as a system comprising: a device communicatively connected to a network; a third-party content provider server communicatively connected to the network; and a communication server for operation. Operation includes, for each of a plurality of web pages, receiving data indicating the characteristics of the corresponding network resource; detecting, based on the received data, the functionality of the corresponding network resource being provided with real-time assistance by the third-party content provider through a chat user interface on the corresponding network resource; and storing in a database entries indicating that the corresponding network resource has such functionality. Operation includes receiving from a user a request to access a specific network resource managed by a specific third-party content provider; determining, based on the entries stored in the database representing the specific network resource, the functionality of the specific network resource being provided with real-time assistance by a specific third-party content provider through a specific chat user interface on the specific network resource; and the communication server initiating a chat session between the user and the third-party content provider.

[0028] Another innovative aspect of the subject matter described in this specification can be implemented in at least one computer-readable storage medium encoded with executable instructions, which, when executed by at least one processor, cause at least one processor to perform operations. The operations include, for each of a plurality of web pages, receiving data indicating the characteristics of the corresponding network resource; detecting, based on the received data, the functionality of the corresponding network resource provided with real-time assistance by a third-party content provider through a chat user interface on the corresponding network resource; and storing in a database entries indicating that the corresponding network resource has such functionality. The operations also include receiving from a user a request to access a specific network resource managed by a specific third-party content provider; determining, based on the entries stored in the database representing the specific network resource, the functionality of the specific network resource provided with real-time assistance by a specific third-party content provider through a specific chat user interface on the specific network resource; and initiating a chat session between the user and the third-party content provider by a communication server.

[0029] The subject matter described in this specification can be implemented in specific embodiments to achieve one or more of the following advantages. The proposed system for initiating and maintaining message sending and receiving sessions provides efficient and optimized techniques for identifying and interacting with resources that provide chat interface functionality. The proposed techniques, on the one hand, implement a chat interface comprising a rendered chat session managed by a controller or communication server / gateway, enabling a seamless, reliable, and scalable chat interface on the user interface terminal or chat interface functionality provided by the resource or content provider, regardless of browser capabilities, service provider, or operating system used by the user. The system reduces the amount of computational resources required to initiate and maintain chat sessions between users and content providers. The system makes the chat interface more robust by maintaining separate sessions between the user and the gateway, and between the content provider and the gateway, and storing session data to re-establish disconnected chat sessions. Furthermore, the gateway can keep the session open in the event of a temporary loss of connection, thus improving the reliability of the chat connection. If the chat session cannot be kept open and must be re-established, the user can continue the re-established chat session without interruption and without having to input redundant data already provided through the previously established chat session. Furthermore, by providing a chat interface without redirecting users to the content provider's page, the system reduces network traffic associated with users searching for chat functionality. This system improves upon computer-related technologies and chat interfaces because users can search for messaging functions within each content provider's page or connect and communicate with the content provider when disconnected, without experiencing frustration.

[0030] Furthermore, the gateway system improves upon computer-related technologies by providing the ability to extend chat functionality to multiple content providers without requiring strict integration. The system allows content providers to maintain some degree of individuality and control the corresponding chat interface. The ability to conduct chat conversations without using a screen enables hands-free communication, reducing the burden on users. Additionally, the system can post-process chat messages to perform other actions using integrated applications. For example, if the response in a chat conversation is "We have scheduled an appointment for 5 PM," the system can automatically detect the existence of a calendar event and add a calendar entry, inquire about existing conflicts, and perform various other related tasks. Actions derived from chat conversations can be logged in and / or provided for analysis, the results of which can be used to influence future chat interface expressions. The gateway system reduces user wait times because users with slow network connections no longer need to access the content provider's webpage and request a rendered page. Instead, only communication is transmitted, thus reducing network traffic and latency. Attached Figure Description

[0031] Figure 1 It is a block diagram of a system through which users can contact content providers via various devices.

[0032] Figure 2 This is a sample user interface that provides a chat interface to the user.

[0033] Figure 3 This is a flowchart illustrating an example process for initiating and maintaining a chat session between a user and a third-party content provider.

[0034] Figure 4 This is a block diagram of an example computing system.

[0035] The same reference numerals and labels in the various figures indicate the same elements. Detailed Implementation

[0036] This document describes methods, systems, and apparatus for improving the connection strength and quality of communication sessions by providing a gateway between a user and the content provider with whom the user is communicating. The gateway relays content (e.g., messages) received at one of the user's communication interfaces to another communication interface of the content provider, and maintains session information by retaining relevant data about the conversation between the user and the content provider, rather than directly providing a session between the user and the content provider without maintaining session data, so that a disconnection on either the user's or the content provider's side terminates the session and prevents the chat session from being re-established.

[0037] In some situations, conversations between client devices may be interrupted or disconnected, causing frustration for users seeking help and for content providers attempting to provide it. For example, a consumer calling an internet provider for help with an internet connection problem might lose connection with the representative they were speaking to. The consumer might try to call back, but might be connected to a different representative, requiring the consumer to explain the situation again, resulting in an overall inefficient and cumbersome experience. Here, a gateway can detect and store specific data about the conversation between the user and the content provider to reduce the amount of time spent by the user and the content provider trying to recover information and progress made before the disconnection. This also improves electronic chat applications / functions as computer-related technologies by preventing the loss of data previously communicated using the chat interface in the event of a failure in the chat interface (e.g., connection failure or other interruption of the communication link).

[0038] An electronic document is data that presents a set of content on a client device. Examples of electronic documents include web pages, word processing documents, portable file format (PDF) files, images, videos, search results pages, and subscription feeds. Native applications (such as "apps") installed on mobile phones, tablets, or desktop computing devices are also examples of electronic documents. Electronic documents contain digital components displayed for user consumption.

[0039] As used throughout this document, the term "digital component" refers to a separate unit of digital content or digital information (such as a video clip, audio clip, multimedia clip, image, file, or other unit of content). Digital components can be electrically stored in a physical storage device as a single file or a collection of files, and can take the form of video files, audio files, multimedia files, image files, or text files and contain advertising information, making advertising a type of digital component. Typically, a digital component is defined or provided by a single provider or resource (such as an advertiser, publisher, or other content provider), but can be a combination of content from multiple resources. Digital components from multiple different sources can be combined into a single electronic file (e.g., a collection of various different digital components), and portions of various digital components from different sources can be combined with information extracted from search results (or other parts of content) into a single digital component.

[0040] Search engines can direct users to results that link to various web pages of a content provider. Users can contact the content provider, for example, by calling the phone number provided on the specific content provider's webpage. In another example, users can contact the content provider by sending an email or mail to the specific address listed on the content provider's webpage.

[0041] In some examples, users can contact content providers through a user interface provided on a specific content provider's webpage. This user interface could be, for example, a chat interface through which users can communicate with a representative of the specific content provider. Many webpages have different formats and contain different placements of interface elements that direct users to the chat interface. For example, a webpage might include a button that directs users to the chat interface.

[0042] The chat interface generally includes a text box through which the user provides input to the other party they are communicating with. Additionally, other user interface elements for submitting and editing user input can also be included within the chat interface; these elements can be small components that must be accessed through navigating user interface menus. For example, commonly used interface elements include small buttons for sending messages or switching keyboards, or menus for selecting content to send, such as images or video data.

[0043] Some web pages may require users to navigate a URL tree to reach a chat interface. For example, user interface elements that direct users to a chat interface where they can contact content providers may be displayed on a webpage other than the content provider's main website. These user interface elements may be located in hidden positions or resized to make it difficult for users to select them to access the chat interface.

[0044] Chat interfaces can be difficult to use on mobile devices because the screen area on mobile devices is typically smaller than that of desktop client devices. For example, the screen area on a smartphone is generally smaller than the screen area on a desktop computer with a display. User interface elements that are smaller on desktop client devices can be many times smaller on mobile devices and are more difficult to select.

[0045] In some examples, a webpage may provide a link to a chat interface that opens in a new window. When viewed on a small screen on a mobile device, interacting with the chat interface can be difficult. Users may find themselves completely unable to manipulate the chat interface to interact with it, leading to user frustration and diminishing the value of providing a chat interface for many content providers.

[0046] When a chat session is initiated between a user and a content provider representative, a disconnection typically means the user must provide any information they previously provided to the content provider to return to the same point in the conversation when the disconnection occurred. For example, through the chat interface of a chat session, a user seeking support for a particular type of amateur radio (ham radio) can provide a representative of the manufacturer of the amateur radio they are using with their registration number, their desired broadcast frequency, and a description of the problem they are experiencing. If the user's chat session with the manufacturer's representative is disconnected, the user must typically navigate to the manufacturer's webpage, find the user interface elements that direct them to the manufacturer's chat interface, and once a new chat session is established, must again provide another representative of the manufacturer with all the information they previously presented (i.e., their registration number, their desired broadcast frequency, and a description of the problem they are experiencing).

[0047] This document describes methods, systems, and apparatus for reducing the amount of computer resources required to initiate and resume chat sessions between users and content providers. In some cases, a content provider's website includes user interface elements that direct users to a chat interface and initiate a chat session between the user and a representative of the content provider. For example, a tractor company's website may include a button that connects a user to an online representative of the tractor company. The button may be displayed in various ways, including as a navigation menu item, as a prominent feature on the tractor company's website's "Support" or "Contact Us" page, or as an action suggestion displayed based on the user's behavior while visiting the tractor company's website.

[0048] The system identifies content providers, such as tractor companies, that offer chat options with their representatives, and stores data instructing content providers' websites to offer chat interface options. Once a website has been identified as offering chat session functionality, such as online chat sessions or automated chat sessions with an answer database created using a messaging machine learning model, the system can instruct the webpage to provide chat functionality. For example, the system can indicate in search results that a website presents its chat session functionality.

[0049] A message sending and receiving machine learning model can be a model that receives response data from users and content providers as input. Message sending and receiving machine learning models can use any of a variety of models, such as decision trees, linear regression models, logistic regression models, neural networks, classifiers, support vector machines, inductive logic programming, ensembles of models (e.g., using techniques such as bagging, augmentation, random forests, etc.), genetic algorithms, Bayesian networks, etc., and can be trained using various methods, such as deep learning, perceptrons, association rules, inductive logic, clustering, maximum entropy classification, learned classification, etc. In some examples, message sending and receiving machine learning models can use supervised learning. In other examples, they use unsupervised learning.

[0050] Web pages can provide user interface elements through which users can initiate chat sessions with content providers. Once a user selects a user interface element to initiate a chat session with a content provider, the system identifies the characteristics of the chat session to be used when initiating and facilitating the chat session. For example, the system can store identifiers such as: the device identifier of the client device provided by the user to the chat interface, the session identifier of the specific session between the client device and a specific representative of the content provider, and the device identifier of the device through which the specific representative of the content provider inputs data into the chat interface.

[0051] The system can act as a gateway for chat sessions between users and content providers. The system initiates a first chat session with the user and a second, separate chat session with the content provider, relaying messages between the user and the content provider using data indicating input provided in both the first and second chat sessions.

[0052] This document will discuss text-based chat interfaces throughout, but the techniques discussed here can be applied to voice chat sessions initiated using computing devices such as personal digital assistants (PDAs). For example, a PDA can receive audio input (e.g., spoken queries or commands) from a user via its microphone and provide search results, and / or other information to the user using audio output from the PDA's speaker. If the PDA is implemented in a computing device that includes a display, or connected (e.g., wirelessly or wired) to or displays information on another device, the PDA can also display visual information, such as one or more search results, and / or other information related to the audio input. Note that a personal digital assistant (PDA) can be implemented on multiple different physical devices (e.g., mobile phones, tablets, audio / video streaming devices, desktop PDA units, and / or other devices), and input can be received by one of the devices (e.g., a desktop PDA unit), while audio and / or video output can be displayed by a different device (e.g., a mobile phone). Similarly, some of the information provided in response to a query or command can be output by one of the devices (e.g., audio information output by a desktop PDA unit), while different parts of the information (e.g., visual information) can be displayed by a different device (e.g., a mobile phone or a display containing an audio / video streaming device).

[0053] As part of the audio output and / or visual information, the PDA can present options to the user to initiate a chat session. For example, the PDA can audioly ask the user if they want to connect or chat with an agent who can provide them with more information, and the user can verbally respond affirming their desire to connect to the agent (e.g., by saying the words "yes" or "connect me to the agent"). In response to the user's affirmation of their desire to connect to the agent, the PDA can initiate a text chat session (e.g., using a mobile device) and / or initiate a telephone call to the agent (e.g., using a mobile device and / or a desktop PDA unit configured to make a voice call, for example, via Internet Protocol telephony). When the user is connected to the agent, the technologies discussed throughout this document can be used to maintain the chat session (e.g., text or voice), thereby enabling the chat session to continue seamlessly even if it is interrupted. In the case of a voice chat session, speech recognition technology can be used to translate voice interaction into text, making the technologies discussed herein also applicable to these situations.

[0054] Figure 1 This is a block diagram of a system 100 in which a user can communicate with a content provider. System 100 includes a mobile device 102, a client device 104, a screenless assistive device 106 (e.g., a desktop PDA unit), a gateway 110, a sample webpage 120, and a content provider system 130. The mobile device 102, client device 104, and screenless assistive device 106 are connected to the gateway 110 and the content provider system 130 over networks such as a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof.

[0055] Mobile device 102, client device 104, and screenless auxiliary device 106 are each electronic devices capable of requesting and receiving resources over a network. Examples of mobile device 102 include mobile phones, smartphones, tablet computers, and other portable mobile devices through which users can connect to content providers over a network. Examples of client device 104 include personal computers, mobile workstations, and laptop computers. Mobile device 102, client device 104, and screenless auxiliary device 106 typically include user applications such as web browsers to facilitate the sending and receiving of data over a network; however, local applications executed by client device 104 can also facilitate the sending and receiving of data over a network.

[0056] Mobile device 102, client device 104, and screenless assistive device 106 can submit search queries to a search system, such as a search engine, over a network. In response, the search system accesses a search index to identify electronic documents related to the search query. The search system identifies the electronic documents in the form of search results and returns the search results to mobile device 102, client device 104, or screenless assistive device 106 on the search results page.

[0057] Search results are data generated by a search system that identifies electronic documents responding to a specific search query and include active links (e.g., hypertext links) that cause client devices to request data from specified web locations (e.g., URLs) in response to user interaction with the search results. Example search results may include a webpage title, text snippets, or portions of an image extracted from a webpage, along with the webpage's URL. Another example search result may include the title of a downloadable application, text snippets describing the downloadable application, an image depicting the user interface of the downloadable application, and / or a URL indicating the location from which the application can be downloaded to mobile device 102, client device 104, or screenless assistive device 106.

[0058] In some cases, the search system may be part of or interact with an app store (or online portal) from which applications can be downloaded for installation on the client device, in order to display information about downloadable applications related to the submitted search query. Like other electronic documents, the search results page may contain one or more digital component slots, where digital components (such as video clips, audio clips, images, or other digital components that can be formatted as advertisements) can be displayed.

[0059] Gateway 110 facilitates chat sessions and comprises a backend and a frontend. The backend of Gateway 110 provides the infrastructure for the chat session and communicates separately from content providers and users. For example, the backend can use data entered by the user through a client device to chat with the content provider, and similarly, data entered by a representative of the content provider to chat with the user. The backend interface can be a common format or package agreed upon by the user or content provider, such as a script-specific application programming interface (API), a specific Android application package (.apk), or various other suitable packages.

[0060] The front end of gateway 110 can be an interface through which one of the users or content providers can provide input to be sent and displayed to the other. For example, the front end could be a chat interface displayed separately to the user and the content provider. The front end can be customized for both the user and the content provider and can have default configuration settings or can be adjusted by either the user or the content provider. For example, the user's front end could include larger text boxes and bold fonts, while the content provider's front end could include additional functionality, such as remote access and control of the user's device.

[0061] Gateway 110 identifies each webpage that provides messaging functionality and stores data indicating that the webpage provides this functionality. The identification of webpages that provide messaging functionality is generally done before the webpage is displayed as a search result, rather than during the search request runtime.

[0062] Web pages that provide messaging functionality typically include user interface elements that allow users to initiate messaging sessions, such as buttons to start a chat session. For example, web page 120 is shown containing button 122, which a user can select to begin a chat session with a content provider. Web page 120 may also contain various other indications that provide or support messaging functionality, such as key text phrases. For example, web page 120 may contain the text "Chat with someone now."

[0063] Gateway 110 can also identify whether webpage 120 provides messaging or chat functionality by detecting certain script characteristics of webpage 120—such as code snippets that cause a new window to open when a user selects a specific user interface element. Another characteristic of webpage 120 that can be used to determine whether it provides chat functionality is whether it includes widgets displayed on it that link to different websites or providers. For example, if webpage 120 provides widgets for communicating with known chat service providers, then webpage 120 likely provides chat functionality. In addition to other communication information, widgets may also contain identifiers for specific sessions, such as identifiers for specific client devices, identifiers for specific situations, etc.

[0064] Features such as links to certain domains known to provide messaging services also indicate that webpage 120 may provide messaging functionality. For example, if a webpage links to a known automated chat service, gateway 110 can mark webpage 120 as supporting messaging functionality.

[0065] When gateway 110 receives the initial search request, it accesses stored data to determine whether the displayed search results provide messaging functionality. If a particular webpage provides messaging functionality, gateway 110 generates a user interface element displaying the search results, which indicates the type of messaging functionality provided. When this user interface element is selected, a messaging session is initiated without directing the user to the search results webpage.

[0066] Content provider system 130 is associated with a specific content provider and can be a network destination that provides resources in response to a search query. For example, content provider system 130 may manage resources such as files that a user can download when searching for a specific query. Content provider system 130 includes a user interface through which a representative of the content provider can interact with the user. In some embodiments, the user interface is a chat interface through which the representative can communicate with the user in real time.

[0067] Gateway 110 reduces the amount of computing resources required to initiate and maintain chat sessions between users and content providers. Furthermore, by reducing the need for users to navigate to the content provider's webpage, gateway 110 reduces network traffic associated with user search chat functionality and with the data that must be transmitted about the content provider's webpage.

[0068] Figure 2 This is an example user interface 200 for a chat session between a user and a content provider. The chat session can be conducted through a system such as system 100, which includes at least two devices connected by network communication. The chat session can be initiated, facilitated, and maintained through a communication gateway such as gateway 110.

[0069] Search query 202 is data entered by a user into a search system such as a search engine, indicating a query that the user is interested in seeing relevant results. The search engine may provide results 210, 220, 230, and 240 on the search results page. Each of results 210, 220, 230, and 240 is related to search query 202, and in some implementations, the results may be sorted based on their relevance to search query 202.

[0070] Each of results 210, 220, 230, and 240 includes corresponding user interface elements 212, 222, 232, and 242 that direct the user to a chat interface with the associated content provider. User interface elements 212, 222, 232, and 242 can be customized to display the associated content provider's webpage for each element. Customization can be based on various attributes of a specific associated webpage, such as font, text size, or color. For example, each of user interface elements 212, 222, 232, and 242 has a theme associated with the corresponding search results webpages 210, 220, 230, and 240.

[0071] When a user selects one of user interface elements 212, 222, 232, or 242, gateway 110 initiates a chat session between the user and the content provider offering the corresponding search results webpages 210, 220, 230, and 240. Figure 2 In the example shown, the user selects a user interface element 212 associated with the search results webpage 210. The chat session is then rendered and managed by the gateway. This enables the gateway, or the server associated with it, to implement control and any necessary protocols and data structures to seamlessly ensure scalability, security, reliability, and storage associated with the chat session, regardless of the browser, operating system, or any other user device or system currently used by the user and / or content provider.

[0072] Once the user selects user interface element 212, gateway 110 initiates a chat session with the content provider of the search results webpage 210. Gateway 110 identifies webpage 120 and the user who has selected user interface element 212 and stores data indicating identifiers for webpage 120 and the user. Gateway 110 may also store various other attributes of the chat session, such as the length of the session, a first session identifier for the chat session with the user, and a second separate session identifier for the chat session with the content provider.

[0073] To facilitate maintaining sessions between users and content providers when gateway 110 manages numerous different conversations between many different users and content providers, gateway 110 directly directs from one session with a user to another session with a content provider. Gateway 110 uses stored data indicating session identifiers to appropriately forward messages between users and content providers.

[0074] Gateway 110 can receive input in various formats, format the input, and provide the input to a chat session between the user and the content provider. For example, a user can send a text message to a specific phone number to initiate a chat with the content provider. The specific phone number can be associated with the content provider, and receiving a text message at that specific phone number can trigger the initiation of a chat session with the content provider. For example, receiving a text message at a specific phone number can trigger the system to contact gateway 110 and provide user input as input for the chat session. When the user receives a response from the content provider, gateway 110 can provide the response to the user as a text message. The user can also provide responses to the content provider via text message, email, or various other suitable communication methods.

[0075] Gateway 110 can process input from users or content providers in various formats. For example, if a user interacts with gateway 110 using mobile device 102 and selects user interface element 212 to initiate an email conversation with a content provider, a chat session with the content provider can be initiated, and user input and responses can be displayed within the email client interface.

[0076] In some implementations, if an existing conversation exists, gateway 110 can provide a specific identifier, such as a phone number that the user can provide. For example, if the user is already connected to a content provider, gateway 110 can detect user input in response to the content provider, even if the user input is provided to different applications such as text messaging applications or email clients. The user's input can then be translated into input to be provided to the conversation.

[0077] When a session is established, the chat interface can be displayed without directing the user to webpage 210, thereby reducing the number of network calls and / or the amount of data transmitted to establish the chat session. For example, the chat interface can be displayed on search results page 204. As described above, the chat interface can include user interface elements through which the user can input data. For example, the chat interface can include text boxes and buttons where the user can submit input, such as responses or questions to be provided to the content provider. Additional interface elements, such as file attachments and screen sharing, can be provided within the chat interface.

[0078] The chat interface can be rendered within the search results page 204 using various rendering methods such as headless rendering. The chat interface receives input from users or content providers and provides input to gateway 110 for transmission.

[0079] When a session is established, specific metrics for the chat interface method that connects to the content provider can be calculated, such as click-through rate (CTR), conversion rate, average traffic, and time to visit. For example, CTR associated with user interface elements of the chat functionality can be calculated. In some implementations, the content provider can access reports generated using the calculated metrics. For instance, the content provider can review reports on the performance of the chat interface method and adjust the chat interface's campaign budget based on the reported metrics.

[0080] When used with a screenless assistive device, the screenless assistive device 106 can facilitate conversations between users and content providers using the same chat session infrastructure as mobile device 102 and client device 104 (which may be fixed). Because the screenless assistive device 106 does not contain a screen, the methods for receiving and outputting data involve additional steps. The screenless assistive device 106 can receive various types of appropriate input, such as voice input and touch input. For example, the screenless assistive device 106 can receive spoken commands via its microphone. Spoken commands can be processed, and text data representing voice input can be extracted. In some implementations, the screenless assistive device 106 processes voice input, while in others, the gateway 110 or a remote system processes voice input.

[0081] Gateway 110 can then use text data as input for a session with the content provider. When gateway 110 receives a response from the content provider, the response can be processed into output in the same format as the input provided by the screenless assistive device 106. For example, if the screenless assistive device 106 has received voice input, then the screenless assistive device 106 can receive data containing voice data as output.

[0082] In an exemplary application of system 100, a user can provide voice input to mobile device 102 by saying, "I want to speak with ABC's customer service." Mobile device 102 can then display search results, which may include web pages identified as providing messaging functionality along with user interface elements that allow users to initiate messaging sessions.

[0083] Users can continue their conversations with content providers on different devices. For example, if a user initiates a messaging session with a content provider on mobile device 102, the user can continue the conversation on a screenless assistive device 106. Gateway 110 can detect the device from which the user provides input or accesses the messaging session and select an appropriate output method such as text, voice, or images.

[0084] By maintaining chat sessions separately for the user and the content provider, gateway 110 reduces the amount of computational resources required to re-enter and provide information previously provided by the user or content provider. For example, a user can continue the chat session without interruption and without needing to enter redundant data previously entered through an established chat session. Additionally, gateway 110 improves the robustness of the chat session by maintaining separate sessions between the user and the content provider, ensuring that a temporary disconnection of either session does not terminate the chat session.

[0085] In some examples, gateway 110 preserves the conversation between the user and the content provider by storing data from the conversation while it is in progress. If the user and the content provider become disconnected, gateway 110 uses at least some of the data stored for the conversation up to the point when the user and the content provider became disconnected to re-establish the conversation between them.

[0086] When a chat session is re-established, gateway 110 can engage in dialogue with the content provider (or provide the content provider with the previous chat session) without further input from the user, allowing the content provider to catch up on their previous conversation. For example, gateway 110 can automatically provide stored data indicating the user's device information and problem description, so that the next representative of the content provider to which the user is connected is informed of the user's situation, allowing the user to continue their conversation with the representative without repeating previously provided information. Gateway 110 can use a machine learning model trained on previously provided answers from the user to automatically determine the appropriate response to the content provider. The machine learning model can receive the content provider's response as input and output various suitable responses.

[0087] In some examples, gateway 110 integrates with existing messaging service providers and performs server-side integration by storing specific session data that allows users to pick up the conversation at specific points, such as when a user leaves the conversation. For example, once gateway 110 has established a new messaging session, it can provide the stored data to different content provider representatives.

[0088] In some implementations, gateway 110 may not be able to update the new representative to the point in the previous conversation. For example, the new representative's response to a particular segment of information may differ from the previous representative's response, and gateway 110 may not be able to redirect the conversation in a direction where gateway 110 can provide all the information that the user was previously able to provide to the representative. When gateway 110 is unable to provide all the information that the user previously provided, gateway 110 may provide the user with the context of the conversation that took place without the user present.

[0089] Gateway 110 can also perform client-side integration by storing session data or user input. When the conversation has been caught up or updated to what Gateway 110 can do, Gateway 110 can provide the stored data to the user, giving the user context with the understanding of the situation by a representative of the content provider.

[0090] Gateway 110 can detect when a user is inactive or unavailable. For example, if a user leaves to attend a meeting or leaves their device without informing a content provider representative, gateway 110 can mark the situation as inactive. Gateway 110 can determine whether a user is inactive based on various characteristics of the current conversation. Such characteristics include a threshold time period elapsed between the content provider's last response and the user's last response, certain phrases entered by the user, certain phrases entered by the content provider, or various event data associated with users already connected to gateway 110. For example, if a user has scheduled a meeting during the conversation, gateway 110 can determine that the user is in a meeting when they are unresponsive for a threshold time period.

[0091] When a conversation is marked as inactive, the gateway can notify the content provider that the user has left. For example, if a user is detected as inactive, the gateway 110 can respond on behalf of the user to the content provider representative, asking the content provider representative to standby and maintain the connection for a few minutes.

[0092] In some examples, gateway 110 may mark a situation as resolved rather than inactive. For instance, if a user does not respond for a threshold amount of time, but an indication has been generated in the response provided by the user that the content provider has resolved the user's problem, such as the user replying "Great, that's all, thank you," then gateway 110 may mark the conversation as resolved. Gateway 110, or the mobile device 102, client device 104, or screenless assistive device 106 through which the user communicates with the content provider, may also request input from the user or content provider to confirm whether the problem has been resolved. For example, screenless assistive device 106 may provide voice output to the user separately from the conversation between the user and the content provider to ask whether the problem has been resolved. If the user provides an affirmative response, screenless assistive device 106 may provide a response to gateway 110, and gateway 110 may mark the conversation as resolved and will not prompt otherwise, another response from the user.

[0093] In some examples, if the user is only temporarily unavailable, gateway 110 may attempt to maintain the conversation with the content provider's representative. For instance, if the user receives a phone call on mobile device 102, gateway 110 may respond to the content provider on behalf of the user, such as, "I'll be right back, could you please wait a moment?" Gateway 110 may then attempt to maintain the chat session until the user's next response.

[0094] Figure 3 This is a flowchart of an example process 300 for initiating and maintaining a chat session between a user and a third-party content provider. In some implementations, process 300 can be implemented using a messaging system such as system 100. For example, process 300 can be... Figure 1 The gateway 110 is used to implement this. In some implementations, process 300 can be implemented as instructions stored on a non-transitory computer-readable medium, and when the instructions are executed by one or more servers, the instructions can cause one or more servers to perform the operation of process 300.

[0095] Process 300 begins when the communication server receives data (302) indicating the characteristics of the corresponding network resource for each of a plurality of network resources. For example, gateway 110 may receive data indicating the characteristics of a particular webpage, including whether the webpage contains a specific phrase, widget, code snippet, etc.

[0096] When, for each of multiple network resources, the communication server detects based on received data that the corresponding network resource provides real-time assistance functionality via a chat user interface on the corresponding network resource through a third-party content provider, process 304 continues. In some examples, the chat user interface is a chat widget. Real-time assistance can be provided by a human representative of a specific third-party content provider. The chat user interface can be displayed on a webpage showing results and content items in response to a search request. In some examples, the user accesses the chat session via a network-connected device. The network-connected device can be a mobile device, a desktop device, or any other various network-connected device. In some examples, the network-connected device does not have a screen.

[0097] In some examples, the detection includes accessing a network resource, determining that a new window detached from the window accessing the network resource has opened, and identifying a text phrase in the new window that indicates the network resource has the capability to provide real-time assistance via a chat user interface on the network resource from a third-party content provider.

[0098] In some examples, the detection includes accessing a web resource, identifying a widget on the web resource, and identifying a label in the code of the widget on the web resource that indicates that the web resource provides real-time assistance through a chat user interface on the web resource provided by a third-party content provider.

[0099] When the communication server stores an entry in the database indicating that the corresponding network resource has functionality for each of the multiple network resources (306), process 300 continues. For example, gateway 110 may store an entry in the database indicating that a specific webpage has information sending and receiving capabilities.

[0100] When the communication server receives a request from the user to access a specific web resource managed by a specific third-party content provider (308), process 300 continues. For example, the user may choose to be directed to a search result link for a web resource associated with a specific content provider. In some examples, the request is provided in response to the selection of content items, as well as content items linked to the specific web resource in response to the search request. In some examples, the request to access the specific web resource includes voice input.

[0101] When the communication server determines, based on entries stored in the database representing a specific network resource, that the specific network resource is provided with real-time assistance by a specific third-party content provider through a specific chat user interface on the specific network resource (310), process 300 continues.

[0102] Process 300 continues when the communication server initiates a chat session (312) between the user and the third-party content provider. For example, gateway 110 can initiate a chat session between the user and the content provider. In some examples, the chat session includes a first session between the communication server and the user, and a second session between the communication server and the third-party content provider. In some examples, initiation involves receiving a first unique session identifier for the chat session from the third-party content provider, and the communication server providing the first unique session identifier to the third-party content provider to facilitate the second session.

[0103] In some examples, process 300 includes receiving input for a first session via a chat user interface and providing data instructing a third-party content provider on user input for a second session. Process 300 may also include, for each of multiple chat sessions initiated for a specific web resource, determining a content item metric for the chat session based on a unique identifier of the content item that directs the user to the specific web resource; generating an expression of the content item metric using the content item metric; and providing the expression to the third-party content provider for display. For example, the content item metric may include click-through rates associated with the chat functionality.

[0104] In some examples, process 300 includes the detection of no activity within a predetermined time period, and the termination of at least one of the first or second sessions. Process 300 includes determining, based on contextual data of the chat session, that the user has not yet completed the chat session, and storing data in the database indicating the overall first session. Process 300 may continue by initiating a third session between the communication server and the third-party content provider as part of the chat session in response to the detection that the second session has ended, and by providing data from the communication system to the third-party content provider indicating at least a portion of the first session, enabling the user to restart their chat session with the third-party content provider.

[0105] In some examples, process 300 further includes receiving a second voice input from the user in the first session. Process 300 continues by providing data indicating a text representation of the second voice input to a third-party content provider in the second session. Process 300 continues by receiving text input from the third-party content provider in the second session between the communication system and the third-party content provider. Process 300 may continue by providing data indicating a voice representation of the text input from the third-party content provider to the user in the first session.

[0106] Figure 4This is a block diagram of an example computer system 400 that can be used to perform the operations described above. System 400 includes a processor 410, memory 420, storage device 430, and input / output device 440. Each of the processor 410, memory 420, storage device 430, and input / output device 440 may be interconnected, for example, using a system bus 450. Processor 410 is capable of processing instructions executed within system 400. In one implementation, processor 410 is a single-threaded processor. In another implementation, processor 410 is a multi-threaded processor. Processor 410 is capable of processing instructions stored in memory 420 or on storage device 430.

[0107] Memory 420 stores information within system 400. In one implementation, memory 420 is a computer-readable medium. In another implementation, memory 420 is a volatile memory cell. In yet another implementation, memory 420 is a non-volatile memory cell.

[0108] Storage device 430 is capable of providing massive storage for system 400. In one implementation, storage device 430 is a computer-readable medium. In various other implementations, storage device 430 may include, for example, a hard disk drive, an optical disk drive, a storage device shared over a network by multiple computing devices (e.g., a cloud storage device), or some other high-capacity storage device.

[0109] Input / output device 440 provides input / output operations for system 400. In one implementation, input / output device 440 may include one or more network interface devices, such as an Ethernet card, a serial communication device, such as an RS-232 port, and / or a wireless interface device, such as an 802.11 card. In another implementation, the input / output device may include a driver device configured to receive input data and send output data to other input / output devices, such as a keyboard, printer, and display device 460. However, other implementations, such as mobile computing devices, mobile communication devices, set-top box television client devices, etc., may also be used.

[0110] although Figure 4 The example processing system has been described, but the implementation and functional operation of the subject matter described herein can be implemented in other types of digital electronic circuits or in computer software, firmware or hardware, including the structures disclosed herein and their equivalents, or combinations thereof.

[0111] Electronic files (hereinafter referred to as files for the sake of brevity) do not necessarily correspond to documents. Files can be stored as part of a document that contains other files, as a single document dedicated to a discussion, or as multiple collaborative documents.

[0112] Embodiments and operations of the subject matter described herein can be implemented in digital electronic circuits or in computer software, firmware, or hardware, encompassing the structures disclosed herein and their equivalents, or combinations thereof. Embodiments of the subject matter described herein can be implemented as one or more computer programs—i.e., one or more modules of computer program instructions—encoded on one or more computer storage media for execution by or control of the operation of a data processing device. Alternatively or additionally, program instructions can be encoded on artificially generated propagated signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device for execution by the data processing device. The computer storage medium can be or is contained in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or combinations thereof. Furthermore, although the computer storage medium is not a propagated signal, it can be a source or destination of computer program instructions encoded in artificially generated propagated signals. Computer storage media may also be one or more separate physical components or media (such as multiple CDs, disks or other storage devices), or contained in one or more separate physical components or media (such as multiple CDs, disks or other storage devices).

[0113] The operations described in this specification can be performed by a data processing device on data stored on one or more computer-readable storage devices or on data received from other sources.

[0114] The term "data processing device" encompasses all types of apparatus, devices, and machines that process data, including, for example, one or more programmable processors, one or more computers, one or more systems-on-a-chip, or combinations thereof. The device may also contain special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). In addition to hardware, the device may also contain code that creates the execution environment for the computer program discussed, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, cross-platform runtime environments, virtual machines, or combinations thereof. The device and execution environment can implement a variety of different computing model infrastructures, such as network services, distributed computing infrastructures, and grid computing infrastructures.

[0115] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, objects, or other units suitable for use in a computing environment. A computer program may, but does not necessarily, correspond to a document in a documenting system. A program may be stored in a portion of a document containing other programs or data (e.g., one or more scripts stored in a markup language file), in a single document dedicated to the program in discussion, or in multiple collaborative documents (e.g., documents storing one or more modules, subroutines, or code sections). A computer program can be deployed to execute on one computer or on multiple computers located at a single site or distributed across multiple sites and interconnected by a communications network.

[0116] The processes and logic flows described herein can be performed by one or more programmable processors executing one or more computer programs to act by manipulating input data and producing outputs. The processes and logic flows can also be performed by special-purpose logic circuitry (e.g., FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits)), and the device can be implemented as special-purpose logic circuitry (e.g., FPGAs or ASICs).

[0117] Processors suitable for executing computer programs include, for example, both general-purpose and special-purpose microprocessors. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor that acts according to instructions and one or more memory devices that store instructions and data. Generally, a computer will also include one or more mass storage devices—such as disks, magneto-optical disks, or optical disks—for storing data, or operatively coupled to the computer to receive data from or transfer data to mass storage devices—such as disks, magneto-optical disks, or optical disks, or both. However, a computer does not necessarily have to have such devices. Furthermore, a computer can be embedded in another device, for example, to name a few, a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (such as a Universal Serial Bus (USB) flash drive). Applicable devices for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices—such as EPROM, EEPROM, and flash memory devices; disks—such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by dedicated logic circuitry or integrated into dedicated logic circuitry.

[0118] To provide interaction with the user, embodiments of the subject matter described herein can be implemented on a computer having a display device, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user, and a keyboard and pointing device, such as a mouse or trackball, through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form including sound, speech, or tactile input. Furthermore, the computer can interact with the user by sending files to or receiving files from a device used by the user, for example by sending web pages to a web browser on the user's client device in response to a request received from a web browser.

[0119] Embodiments of the subject matter described herein can be implemented in computing systems that include backend components, middleware components, or frontend components, or any combination of such backend, middleware, or frontend components. The backend component may be, for example, a data server; the middleware component may be, for example, an application server; and the frontend component may be, for example, a client computer with a graphical user interface or a web browser through which a user can interact with the implementation of the subject matter described herein. The components of the system can be interconnected via any form of digital data communication medium, such as a communication network. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), inter-network networks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0120] A computing system may include clients and servers. Clients and servers are generally geographically distant and typically interact via a communication network. The relationship between clients and servers is manifested by computer programs running on respective computers and having a client-server relationship with each other. In some embodiments, the server transmits data (e.g., HTML pages) to a client device (e.g., for the purpose of displaying data to a user interacting with the client device and receiving user input from the user interacting with the client device). Data generated at the client device (e.g., the result of user interaction) can be received from the client device at the server.

[0121] Although this specification contains many details of specific implementations, these should not be construed as limiting the scope of the invention or the scope of the claims, but are intended solely to describe features of particular embodiments of the invention. Certain features described herein in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in various suitable sub-combinations. Furthermore, although features may be described above as operating in certain combinations and even initially claimed in the same manner, in some cases one or more features from the claimed combination may be removed from the combination, and the claimed combination may be adapted for sub-combinations or variations thereof.

[0122] Similarly, although operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring such operations to be performed in the specific order or sequence shown, or to perform all the operations shown, in order to achieve the desired result. In some environments, multitasking and parallel processing can be advantageous. Furthermore, the separation of various system components in the embodiments described above should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated into a single software product or packaged into multiple software products.

[0123] Therefore, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions described in the claims may be performed in a different order and still achieve the desired result. Additionally, the processes depicted in the appended drawings do not necessarily require a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

Claims

1. A method for interacting with a chat user interface, comprising: Voice input is received from the user via a communication server, and the voice input is submitted through the microphone of the personal digital assistant; Search results obtained via voice input are obtained through a communication server; In the case where the network resources provided by the third-party content provider include user interface elements, and the user interface elements are used to initiate a connection with a real-time chat system provided by the third-party content provider, the communication server determines that the third-party content provider provides real-time interactive assistance through the chat function via the chat user interface, wherein the third-party content provider corresponds to one of the search results; When the search results are delivered to the user, the user is given the option to initiate a chat session using chat functionality provided by a third-party content provider via a personal digital assistant, without the user having to request network resources first; Through a personal digital assistant, it receives confirmation from the user that they have requested to initiate a chat session; and A chat session is initiated between the user and the content provider's agent via a communication server, without redirecting the user to network resources provided by a third-party content provider. The option to initiate a chat session includes providing a voice inquiry via a personal digital assistant, which asks the user whether they want to connect to or chat with a proxy that can provide the user with more information from the content provider; and The confirmation that a user has requested to initiate a chat session includes receiving a voice response to the voice inquiry via a personal digital assistant.

2. The method according to claim 1, wherein: A personal digital assistant is implemented across two or more different devices, including a first device and a second device, wherein the first device does not include a display and the second device includes a display; The option to initiate a chat session includes providing a voice inquiry via a first device, which asks the user whether they want to connect to a proxy that can provide the user with more information from the content provider or chat with the proxy; as well as Receiving confirmation that a user has requested to initiate a chat session includes receiving an audio response to the voice inquiry via a second device.

3. The method according to claim 1, wherein: A personal digital assistant is implemented across two or more different devices, including a first device and a second device, wherein the first device does not include a display and the second device includes a display; The option to initiate a chat session includes providing a voice inquiry via a first device, which asks the user whether they want to connect to a proxy that can provide the user with more information from the content provider or chat with the proxy; as well as Receiving confirmation that a user has requested to initiate a chat session includes receiving interaction with a chat widget displayed on the second device's screen via a second device.

4. The method of claim 1, wherein initiating a chat session includes initiating an Internet Protocol (IP) voice call with the agent.

5. The method according to claim 1, comprising: Maintaining the state of the chat session includes recording communications between the user and the agent during the chat session; The chat session is determined to end before the interaction between the agent and the user is completed; Initiate a follow-up chat session with the content provider's subsequent agent; and Provide the status of the chat session to subsequent agents, the status of the chat session including at least a portion of the recorded communications between the user and the agent.

6. The method of claim 5, wherein determining that a chat session ends before the interaction between the agent and the user is completed includes determining that there has been no activity via the chat function for at least a predetermined period of time.

7. A system for interacting with a chat user interface, comprising: The gateway stores interactive data corresponding to client-side input in a chat session; A communication server communicates with a client device, wherein the communication server facilitates a chat session between the client device and a content provider, and performs operations including: Voice input is received from the user of the client device, and the voice input is submitted through the microphone of the personal digital assistant; Get search results using voice input; In the case where the network resources provided by the third-party content provider include user interface elements, and the user interface elements are used to initiate a connection with a real-time chat system provided by the third-party content provider, it is determined that the third-party content provider provides real-time interactive assistance through the chat function via the chat user interface, wherein the third-party content provider corresponds to one of the search results; When the search results are delivered to the user, the user is given the option to initiate a chat session using chat functionality provided by a third-party content provider via a personal digital assistant, without the user having to request network resources first; Through a personal digital assistant, it receives confirmation from the user that they have requested to initiate a chat session; and A chat session is initiated between the user and the content provider's agent via a communication server, without redirecting the user to network resources provided by a third-party content provider. The option to initiate a chat session includes providing a voice inquiry via a personal digital assistant, which asks the user whether they want to connect to or chat with a proxy that can provide the user with more information from the content provider; and The confirmation that a user has requested to initiate a chat session includes receiving a voice response to the voice inquiry via a personal digital assistant.

8. The system according to claim 7, wherein: A personal digital assistant is implemented across two or more different devices, including a first device and a second device, wherein the first device does not include a display and the second device includes a display; The option to initiate a chat session includes providing a voice inquiry via a first device, which asks the user whether they want to connect to a proxy that can provide the user with more information from the content provider or chat with the proxy; as well as Receiving confirmation that a user has requested to initiate a chat session includes receiving an audio response to the voice inquiry via a second device.

9. The system according to claim 7, wherein: A personal digital assistant is implemented across two or more different devices, including a first device and a second device, wherein the first device does not include a display and the second device includes a display; The option to initiate a chat session includes providing a voice inquiry via a first device, which asks the user whether they want to connect to a proxy that can provide the user with more information from the content provider or chat with the proxy; as well as Receiving confirmation that a user has requested to initiate a chat session includes receiving interaction with a chat widget displayed on the second device's screen via a second device.

10. The system of claim 7, wherein initiating a chat session includes initiating an Internet Protocol (IP) voice call with the agent.

11. The system according to claim 7, wherein the operation includes: Maintaining the state of the chat session includes recording communications between the user and the agent during the chat session; The chat session is determined to end before the interaction between the agent and the user is completed; Initiate a follow-up chat session with the content provider's subsequent agent; and Provide the status of the chat session to subsequent agents, the status of the chat session including at least a portion of the recorded communications between the user and the agent.

12. The system of claim 11, wherein determining that a chat session ends before the interaction between the agent and the user is completed includes determining that there has been no activity via the chat function for at least a predetermined period of time.

13. A non-transitory computer-readable medium storing instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations, the operations including: Voice input is received from the user and is submitted via the microphone of the personal digital assistant; Get search results using voice input; In the case where the network resources provided by the third-party content provider include user interface elements, and the user interface elements are used to initiate a connection with a real-time chat system provided by the third-party content provider, it is determined that the third-party content provider provides real-time interactive assistance through the chat function via the chat user interface, wherein the third-party content provider corresponds to one of the search results; When the search results are delivered to the user, the user is given the option to initiate a chat session using chat functionality provided by a third-party content provider via a personal digital assistant, without the user having to request network resources first; Through a personal digital assistant, it receives confirmation from the user that they have requested to initiate a chat session; and Initiate a chat session between the user and the content provider's agent without redirecting the user to network resources provided by a third-party content provider. The option to initiate a chat session includes providing a voice inquiry via a personal digital assistant, which asks the user whether they want to connect to or chat with a proxy that can provide the user with more information from the content provider; and The confirmation that a user has requested to initiate a chat session includes receiving a voice response to the voice inquiry via a personal digital assistant.

14. The non-transitory computer-readable medium according to claim 13, wherein: A personal digital assistant is implemented across two or more different devices, including a first device and a second device, wherein the first device does not include a display and the second device includes a display; The option to initiate a chat session includes providing a voice inquiry via a first device, which asks the user whether they want to connect to a proxy that can provide the user with more information from the content provider or chat with the proxy; as well as Receiving confirmation that a user has requested to initiate a chat session includes receiving an audio response to the voice inquiry via a second device.

15. The non-transitory computer-readable medium according to claim 13, wherein: A personal digital assistant is implemented across two or more different devices, including a first device and a second device, wherein the first device does not include a display and the second device includes a display; The option to initiate a chat session includes providing a voice inquiry via a first device, which asks the user whether they want to connect to a proxy that can provide the user with more information from the content provider or chat with the proxy; as well as Receiving confirmation that a user has requested to initiate a chat session includes receiving interaction with a chat widget displayed on the second device's screen via a second device.

16. The non-transitory computer-readable medium of claim 13, wherein initiating a chat session includes initiating an Internet Protocol voice telephone call with an agent.

17. The non-transitory computer-readable medium of claim 13, wherein the operation comprises: Maintaining the state of the chat session includes recording communications between the user and the agent during the chat session; The chat session is determined to end before the interaction between the agent and the user is completed; Initiate a follow-up chat session with the content provider's subsequent agent; and Provide the status of the chat session to subsequent agents, the status of the chat session including at least a portion of the recorded communications between the user and the agent.

Images