A system, method and computer readable medium for processing a conversation

Abstract

Embodiments of the present disclosure relate to a system, method, and computer-readable medium for processing conversations. The technology discussed herein allows an agent providing conversation-based services to converse with a user via an arbitrary mode. A central service, programmed to be able to communicate via an arbitrary mode, collects from the agent the information required to act on behalf of the agent. To ensure that the agent has maximum presence between modes, the central service provides auxiliary services such as language translation, fidelity upgrades and / or downgrades, modal constraint construction, and the like. In addition, the central service provides session tracking via a user profile stored at the intercommunication, so that if a user changes the mode through which the user is conversing with the agent, the central service can maintain the context of the conversation in the new mode. In addition, the agent can communicate with other agents via the central service, allowing other agents to provide services to the agent or add their services to an ongoing conversation with the user.

Description

[0001] Related Applications Citations

[0002] This application is a divisional application of the invention patent application with international application number PCT / US2017 / 023496, international application date March 22, 2017, date of entry into the Chinese national phase September 18, 2018, Chinese national application number 201780018284.9, and the invention name being “A system, method and computer-readable medium for processing conversations”. Technical Field

[0003] Embodiments of the present disclosure relate to a system, method, and computer-readable medium for processing a conversation. Background Art

[0004] A chatbot is a computer program that operates to communicate conversationally in a manner designed to mimic natural human conversation. Initially a novelty, chatbots are now commonly used as customer service for website visitors. They can provide information in response to user queries and can in some cases complete tasks on behalf of the user, such as completing or canceling an order. In order to provide chatbot functionality in a communication channel, developers must code the chatbot functionality into each communication channel for which they desire the chatbot functionality. The functionality of a particular chatbot is limited to the communication channel for which it is designed. Summary of the invention

[0005] This Summary is provided to introduce simplified concepts related to multimodal agents communicating with each other. This Summary is not intended to identify essential features of the claimed subject matter, nor is it intended for use in determining the scope of the claimed subject matter.

[0006] This article describes a technology for extending the modes through which agents can communicate or act. A central service referred to herein as intercommunication allows agents to provide conversation-based services associated with agents via any mode by collecting information needed on behalf of the agent from the agent and by being configured to be able to communicate via any mode. In order to ensure that the agent has maximum presence between modes, intercommunication provides auxiliary services such as language translation, fidelity upgrades and / or downgrades, modal constraint construction, etc. Intercommunication is dynamic, so intercommunication can be programmed to communicate via new modes as they are developed. In addition, intercommunication provides session tracking via user profiles stored at the intercommunication, so that if the user changes the mode in which the user is communicating with the agent, the intercommunication maintains the context of the conversation across the new mode. In addition, agents can communicate with other agents via intercommunication, allowing other agents to provide services to the agent or add their services to an ongoing conversation with the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The detailed description is described with reference to the accompanying drawings. In the drawings, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears. The same reference numbers in different drawings indicate similar or identical items.

[0008] Figure 1 is a block diagram depicting an example environment that may be used to enable multimodal agents to communicate with each other.

[0009] Figure 2 is a block diagram depicting an example device that can enable multi-channel agents to communicate with each other according to various examples.

[0010] Figure 3A is an example user interface for displaying an agent library that can be used to link agents with channels associated with a user profile.

[0011] Figure 3B is a sample user interface for managing channel associations with an agent.

[0012] Figure 3C is an example user interface for linking an agent to a user profile via a specific channel.

[0013] Figure 3D is an example user interface for granting a proxy permission to access a user's groups and messages.

[0014] Figure 4 is an example user interface illustrating interaction with a directory agent via a telephone service channel.

[0015] Figure 5 is an example user interface illustrating interaction with a directory agent via a group project application.

[0016] Figure 6 is a sample user interface for registering agents to allow intercommunication on behalf of agents.

[0017] Figure 7 is a sample user interface for managing channels linked to a proxy.

[0018] Figure 8 is a sample user interface for linking agents to channels.

[0019] Fig. 9 is a flow chart illustrating an example process for creating a conversation session between at least one agent and a user.

[0020] Fig.10 is a flow diagram illustrating an example process for facilitating a conversation between at least one agent and a user associated with a user profile.

[0021] Fig.11is a flow chart illustrating an example process for facilitating a conversation between at least one agent and a channel. DETAILED DESCRIPTION

[0022] Overview

[0023] The present disclosure relates to techniques for providing multimodal agents to communicate with each other. Examples described herein provide techniques for allowing at least one agent to provide conversation-based services to users over multiple channels without coding at least one agent for each communication channel. These techniques allow developers to write instructions for an agent that are unaware of the channels over which the agent can conduct a conversation. This changes the structure of the instructions that make up the agent itself, as the portion of the instruction that includes channel-specific communication instructions can be deleted, simplified, or ignored.

[0024] In some examples, the technology can provide this functionality by being programmed to relay communications through any channel and by being provided with information associated with the agent that is sufficient to represent the agent. In some examples, the technology can utilize the agent's API endpoint or some other access point that allows the technology to call the agent's services, such as including responses to user queries. In addition, the technology can be provided with information sufficient to communicate on behalf of the agent through the channel. Such information may include, for example, a client ID and a client token generated for the agent when the agent is registered with the channel. In some examples, the technology converts communications received from the agent and sent to the user device through the channel into a channel suitable for processing conversations with the user. The technology removes the necessity to recode the agent for each channel, for each channel the developer wishes to provide the ability to talk to the agent.

[0025] These technologies not only reduce the burden on developers, but also allow specific agents to have a portable presence across channels. For example, the technology allows users to have a conversation with an agent across channels without losing the context of the conversation. For example, in the middle of a conversation, a user can switch channels and continue the conversation, where the user left in another channel, without restarting the agent. In some examples, the technology may include one or more of a conversation tracking, a communication record, and / or a user profile containing an associated channel to ensure that the user can talk to the same agent without losing the context of the conversation. The examples described herein may also provide auxiliary services to maximize the number of channels in which the agent has presence. The example technology may also detect the presence of a service trigger in the communication of a user corresponding to the agent's service. The technology allows the introduction or provision of an agent and / or an agent's services without having to interrupt the conversation to query the agent.

[0026] The term "technology" may refer to systems, methods, computer-readable media encoded with instructions, modules and / or algorithms, as well as hardware logic (e.g., field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs)), etc., as permitted by the context described above and throughout the document.

[0027] Schematic environment

[0028] Figure 1 is a block diagram depicting an example environment 100 that can be used to implement multimodal agent intercommunication. Environment 100 includes distributed computing resources 102(1)-102(N) that can communicate with each other and with external devices via one or more networks 106. In some examples, distributed computing resources 102(1)-102(N) can implement the techniques described herein. For example, distributed computing resources 102(1)-102(N) can include a central service (herein "intercommunication 104") for accomplishing intercommunication of multimodal agents, which facilitates conversations between at least one agent 112(1) and at least one user device 108 across multiple channels 110 without encoding a version of at least one agent 112(1) for each communication channel. In some examples, example environment 100 can alternatively include a single computing resource, such as, for example, distributed computing resource 102(1) that implements the techniques herein. In some examples, some or all of the techniques described for distributed computing resources 102(1)-102(N) can additionally or alternatively be implemented by one or more client devices 108. In some examples, client device 110 may be programmed as a central service.

[0029] In some examples, distributed computing resources 102(1)-102(N) include computing devices such as distributed computing resources 102(1)-102(N). Examples support scenarios in which distributed computing resources 102(1)-102(N) may include one or more computing devices operating in a cluster and / or other grouping configuration to share resources, balance loads, improve performance, provide failover support and / or redundancy, and / or for other purposes. Although illustrated as servers, distributed computing resources 102(1)-102(N) may include a wide variety of device types and are not limited to any particular type of device. For example, distributed computing resources 102(1)-102(N) may include any type of computing device having one or more processing units operably connected to a computer-readable medium, an I / O interface, and a network interface. Distributed computing resources 102(1)-102(N) may have an agent intercommunication framework stored thereon, and distributed computing resources 102(1)-102(N) may have an agent intercommunication framework stored thereon that collectively constitutes intercommunication 104.

[0030] In some examples, the network 106 may include a public network such as the Internet, a private network such as an institutional and / or personal intranet, or some combination of private and public networks. The network 106 may also include any type of wired and / or wireless network, including but not limited to a local area network (LAN), a wide area network (WAN), a satellite network, a wired network, a Wi-Fi network, a WiMax network, a mobile communication network (e.g., 3G, 4G, etc.) or any combination thereof. The network 106 may utilize communication protocols, including packet-based and / or datagram-based protocols, such as Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP) and / or other types of protocols. In addition, the network 106 may also include many devices that facilitate network communications and / or form the basis of network hardware, such as switches, routers, gateways, access points, firewalls, base stations, repeaters, backbone devices, and the like. In some examples, the network 106 may include devices that can be connected to a wireless network, such as a wireless access point (WAP). Examples support connection through a WAP that sends and receives data on various electromagnetic frequencies (eg, radio frequencies), including WAPs that support the Institute of Electrical and Electronics Engineers (IEEE) 1302.11 standard (eg, 1302.11g, 1302.11n, etc.) and other standards.

[0031] The example environment 100 also includes a client device 108. In some examples, the client device 108 may include, but is not limited to, a desktop computer, a server computer, a network server computer, a personal computer, a mobile computer, a laptop computer, a tablet computer, a wearable computer, an implanted computing device, a telecommunications device, an automotive computer, a network-enabled television, a thin client, a terminal, a personal data assistant (PDA), a game console, a gaming device, a workstation, a media player, a personal video recorder (PVR), a set-top box, a camera, an integrated component included in a computing device, an appliance, or any other type of computing device, such as one or more separate processor devices, such as a CPU-type processor (e.g., a microprocessor), a GPU, or an accelerator device. The client device 108 may be any device through which a user may participate in a conversation with another entity.

[0032] In some examples, client device 108 communicates via channel 110, through which a user can participate in a conversation with another entity. Figure 1 Channel 110 is depicted as communicating with client device 108 via network 106 , but the term “channel” as used herein refers to any mode or method of electronic communication.

[0033] Examples of channels 110 may include social network channels 110(1) (e.g., etc.), project collaboration channel 110(2) (e.g., etc.), email service 110(3) (e.g., etc.), SMS and / or voice services 110(4), or client device 108 may participate in a conversation with another entity (e.g., AOL Instant Messenger, Gtalk, Any other channel 110(O) for conversations with the same or similar parties.

[0034] Example environment 100 also includes an agent 112 communicatively coupled to distributed computing resources 102(1)-102(N) via network 106. Agent 112 may include any conversation service, where, in at least one example, a "conversation" may include an exchange of natural language. In some examples, a "conversation" may include artificial language and / or computer code. In some examples, a "conversation" may include an exchange of files. Some examples of agent 112 may include chatbots; intelligent personal assistants (e.g., etc.); Active Directory Agents (e.g., agents that respond to user requests for information stored in an Active Directory); Payment Agents (e.g., agents that facilitate payments on behalf of users); Search Engine Agents (e.g., agents that respond to user queries with search engine results); Joke Agents (e.g., agents that respond with jokes); Goods Ordering Agents (e.g., agents that complete goods orders on behalf of user actions); Image Annotation Agents; Calendar Agents; Agent Manufacturing Agents; Agent Import Agents; Office Software Agents; Reminder Agents; Internet of Things Control Agents; Agents that process and manage map and direction information; Agents for use in note-taking, i.e., agents that record and organize audio information in electronic form; Personal Information Management Agents; Agents for organizing and accessing telephone numbers, addresses and other personal contact information; agents for accessing, browsing and searching online databases; agents for providing consumer resources for searching, locating, rating, evaluating and providing guidance for the purchase, consumption and use of a wide range of consumer products, services and information through global communications networks and using any connection access mode in any format (i.e., mobile communications device, desktop Internet, email, voice or social media channels); agents for global positioning and for providing travel instructions; agents for enabling hands-free use of mobile phones through voice recognition; agents for travel arrangements; agents to assist in making reservations at hotels and restaurants, agents providing weather information, etc.

[0035] In some examples, agent 112 is proprietary computer software in the fields of natural language, speech, speaker, language, voice recognition, and voiceprint recognition; computer services, namely assisting others in implementing, using, and operating computer hardware, computer software, and consumer electronics; technical support services, namely troubleshooting computer software problems; computer services, namely providing assistance in developing computer systems, databases, and applications; providing information in the field of computer hardware and software development; application service provider (ASP) services, characterized by hosting other people's computer software applications; or ASP services characterized by computer software and providing online non-downloadable software, namely computer software used to create, author, distribute, download, send, receive, play, edit, extract, encode, decode, display, store, and organize text, graphics, images, audio, video, and multimedia content, and electronic publications.

[0036] In some examples, agent 112 may include software, hardware, human resources, or any combination thereof. In some examples, agent 112 may include all or part of Amazon Mechanical ("MTurk") or other crowdsourced human resources. For example, agents registered with intercommunication 104 at distributed computing resources 102(1)-102(N) can utilize the MTurk API to submit MTurk requests and return results via intercommunication. In some examples, agents can use client devices 108 to act on behalf of users. In some examples, the scope of permissions of the agent to act on behalf of the user depends on the permissions granted to the agent by the user. For example, an opt-in prompt can be provided to use the agent's services.

[0037] In some examples, the agent may include software that is encoded for a single standard or channel. That is, the software that makes up the agent may not be aware of the type of channel through which the agent communicates, and therefore the agent may be encoded to function using one type of channel, or may be encoded for the channel without any specificity. Since in some examples, intercommunication 104 transforms messages sent to and from the agent 112 into a format corresponding to the corresponding channel 110, the agent does not need to be encoded for a specific channel, and does not need to be encoded for each channel. In some examples, the techniques discussed herein facilitate communication between a user device and at least one agent 112 (1) via any one of a plurality of channels, the agent being a single application. In other examples where intercommunication 104 is not used, the agent includes one application for each channel, wherein the various applications that make up the agent are each encoded to conform to a channel. In other examples where intercommunication 104 is not utilized, the agent includes multiple modules that are programmed to conform messages to multiple channels, respectively.

[0038] although Figure 1 A distributed system is depicted in which agent 112 is depicted as communicating with distributed computing resources 102 via network 106, but in some examples, agent 112 may be stored in distributed computing resources 102(1)-102(N). In some examples, agent 112 may be stored remotely, as shown. For example, agent 112 may be part of a web service, accessible by an application programming interface (API).

[0039] Figure 2An example device 200 is depicted, which may represent one or more computing resources of a distributed computing resource 102, a user device 108, or other computing device that implements some or all of the multimodal agent intercommunication techniques described herein. The example device 200 may include any type of computing device having one or more processing units 202 operably connected to a computer-readable medium 204. The connection may be via a bus 206, which in some cases may include a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and one or more of a variety of local, peripheral, and / or independent buses, or via another operably connected. The processing unit 202 may represent, for example, a microprocessor, a controller, or any other suitable type of processor for processing computer-executable instructions to control the operation of the device in order to facilitate multi-channel agent intercommunication. In some examples using a system-on-chip architecture, the processing unit 202 may include one or more fixed function blocks (also referred to as accelerators) that implement a portion of the method using intercommunication 104 in hardware (rather than software or firmware).

[0040] Computer readable media 204 includes two types of computer readable media, namely computer storage media and communication media. Computer storage media may include volatile and non-volatile, non-transient machine-readable, removable and non-removable media implemented in any method or technology for storage of information (in compressed or uncompressed form), such as computer (or other electronic device) readable and / or executable instructions, data structures, program modules and / or other data for performing the processes or methods described herein. Computer storage media include, but are not limited to, hard drives, floppy disks, optical disks, CD-ROMs, DVDs, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, flash memories, magnetic and / or optical cards, solid-state memory devices and / or other types of physical machine-readable media suitable for storing electronic instructions.

[0041] In contrast, communication media embodies computer readable instructions, data structures, program modules, and / or other data in a modulated data signal such as a carrier wave and / or other transport mechanism. As defined herein, computer storage media does not include communication media.

[0042] Example device 200 may include, but is not limited to, a desktop computer, a server computer, a web server computer, a personal computer, a mobile computer, a laptop computer, a tablet computer, a wearable computer, an implanted computing device, a telecommunications device, an automotive computer, a network-enabled television, a thin client, a terminal, a personal data assistant (PDA), a game console, a gaming device, a workstation, a media player, a personal video recorder (PVR), a set-top box, a camera, an integrated component included in a computing device, an appliance, and / or any other kind of computing device, such as one or more discrete processor devices, such as a CPU-type processor (e.g., a microprocessor), a GPU, and / or an accelerator device.

[0043] In some examples, as shown with respect to device 200, computer-readable medium 204 may store instructions executable by processing unit 202, which may represent a CPU incorporated into device 200. Computer-readable medium 204 may also store instructions executable by an external CPU-type processor, executable by a GPU, and / or executable by an accelerator, such as a field programmable gate array (FPGA)-type accelerator, a digital signal processing (DSP)-type accelerator, and / or any internal or external accelerator.

[0044] The executable instructions stored on the computer-readable medium 202 may include, for example, an operating system 208, an intercommunication framework 210, and other modules, programs and / or applications that are loadable and executable by the processing unit 202. The intercommunication framework 210 may include an agent registration and discovery module 212, a conversation conversion / relay module 214, a communication log module 216, a session tracking module 218 and / or an auxiliary service module 220. These technologies can be deployed in more or less modules. As described above, the functions described herein may be performed at least in part by one or more hardware logic components such as accelerators. For example, but not limited to, the schematic type of hardware logic components that can be used include FPGAs, application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), etc. For example, an accelerator may represent a hybrid device, such as a hybrid device from XILINX or ALTERA including a CPU core embedded in an FPGA architecture.

[0045] In the illustrated example, the computer-readable medium 204 also includes a data repository 222. In some examples, the data repository 222 includes a data store, such as a database, a data warehouse, and / or other types of structured or unstructured data stores. In some examples, the data repository 222 includes a relational database with one or more tables, indexes, stored procedures, etc. to enable data access. The data repository 222 can store data for operations of processes, applications, components, and / or modules stored in the computer-readable medium 204 and / or executed by the processor 202 and / or accelerators such as modules 212-220. For example, the data repository 216 can store proxy databases, proxy data, user profiles, value-added service data, version data, iteration data, clock data, and other state data stored and accessed by the intercommunication framework 210. Alternatively, some or all of the above reference data may be stored on separate memory, such as memory of distributed computing resources 102(1)-102(N), client device 108, agent 112, an on-board memory CPU type processor (e.g., a microprocessor), an on-board memory GPU, an on-board memory FPGA type accelerator, an on-board memory DSP type accelerator, and / or an on-board memory another accelerator.

[0046] The device 200 may also include one or more input / output (I / O) interfaces 224 to allow the device 200 to communicate with input / output devices such as user input devices including peripheral input devices (e.g., keyboards, mice, pens, game controllers, microphones or audio input devices, touch input devices (e.g., touch screens, touchpads, etc.), gesture input devices, cameras, etc.), and / or output devices including peripheral output devices (e.g., displays, printers, audio speakers, tactile output, bone conduction for audio sensations, etc.). In some examples, the I / O interfaces 224 may be used to communicate with client devices 108 or agents 112. The device 200 may also include one or more network interfaces 226 to enable communication between the computing device 200 and other networked devices such as client devices 108 and / or agents 112. Such network interfaces 226 may include one or more network interface controllers (NICs) and / or other types of transceiver devices for sending and / or receiving communications over a network such as the network 106.

[0047] The example device 200 includes an example intercommunication framework 210 that can be distributed or stored individually on the device 200, which, as described above, can include one or more devices such as distributed computing resources 102(1)-102(N). Some or all of the modules can be used on a remote device and / or distributed computing resource 102, can be accessed from a remote device and / or distributed computing resource 102, or can be stored on a remote device and / or distributed computing resource 102, which can be configured as a cloud service system. In some examples, the intercommunication framework 210 includes modules 212-220 as described herein, which provide multi-channel agent intercommunication. In some examples, any number of modules can be used, and the techniques described herein used by one module can be used by a greater or lesser number of modules.

[0048] The agent registration and discovery module 212 may include computer executable instructions that, when executed, program the processing unit 202 to register an agent and facilitate discovery of an agent by a user. In some examples, to register an agent, the agent registration and discovery module 212 may be configured to receive information about the agent that it may store in an agent directory. In some examples, the agent registration and discovery module 212 may store the agent directory in the data repository 222 or in a separate memory. In some examples, the information that makes up the agent directory may include one or more of the following: agent name, agent description, agent API endpoint reference, agent key for linking channels (e.g., client token, client ID, etc.), agent account information (e.g., if the agent key is not available, the necessary miscellaneous data of the intercommunication 104 is used to act on behalf of the agent), agent publisher identity, agent publisher email, agent associated website, agent icon, agent owner, identification of the agent language, topic tags, and / or indication of desired value-added services.

[0049] In some examples, intercommunication 104 can call the proxy API via an API endpoint reference to send and receive messages, files, and / or other communications on behalf of the proxy. The call to the proxy API by intercommunication 104 can also cause the proxy to perform services associated with the proxy. It is expected that other suitable configurations for docking with the proxy can be used. For example, if the proxy includes and / or relies on human resources, other interfaces, such as user interfaces, such as graphical user interfaces, natural language interfaces, menus, forms, command lines, can be used at least in part. In some examples, the proxy registration and discovery module 212 can also be used to register new channels. For example, if a new channel is developed, the proxy registration and discovery module 212 can receive information about the new channel, which is sufficient to convert messages sent from the new channel to the proxy and received from the proxy to be relayed on the new channel. Such information may include an API endpoint, a proxy key for accessing the new channel, a channel communication protocol, a channel constraint, a channel fidelity, etc. In some examples, the added new channel can be used for all users (i.e., published to all developers) or the added new channel can be selectively available only to certain users (e.g., authorized users) or in certain scenarios (e.g., when certain conditions are met).

[0050] In some examples, the agent registration and discovery module 212 can facilitate linking an agent to a channel. As used herein, "linking" can include authentication, association, providing sufficient information so that the intercommunication framework 210 can facilitate communication between at least one agent 112(1) and at least one user device, such as by sending and receiving messages on behalf of the at least one agent 112(1) via a channel, etc. In some examples, the agent registration and discovery module 212 is configured to receive agent keys (e.g., client ID and client secret or token) for the channel to which the agent has been linked. In some examples, the module can provide guidance for establishing an account for the agent on the channel and / or authenticating the agent on the channel (e.g., an authorization scheme for interacting with the channel's services via its exposed API). In some examples, once registered via the agent registration and discovery module 212, the intercommunication 104 can use the data stored in the agent directory to issue requests on behalf of the agent, such as requests to send and / or receive messages and / or files over the channel.

[0051] In addition, in some examples, the agent registration and discovery module 212 can also collect information to include in the user profile, and the agent registration and discovery module 212 can maintain the information in the data repository 222 or other storage. In some examples, the user profile can include one or more of the following items: user identification, user device association, user device status, status of the user device's interaction with the agent, status of the user device's interaction with the channel, agents linked to the channel associated with the user (e.g., agents that have been added to the conversation by the user, agents with which the user communicates directly, etc.), the user's language, the user's biographical data, communication logs associated with the user, analysis related to the communication logs developed by the auxiliary service module 220, etc. In some examples, the intercommunication framework 210 can use the user profile to customize communications to the user based at least in part on one or more data that make up the user profile described above.

[0052] In some examples, the conversation transition / relay module 214 may include computer executable instructions that can configure the processing unit 202 to facilitate a conversation between at least one agent 112(1) and at least one user device across multiple channels without requiring a bot to be coded for each communication channel. In some examples, the conversation transition / relay module 214 may include computer executable instructions that can retrieve data from an agent directory (e.g., the agent side of the intercommunication layer) and / or a user profile (e.g., the client side of the intercommunication layer). In some examples, the conversation transition / relay module 214 acts as a proxy layer between at least one agent 112(1) and at least one user device via a channel. In some examples, the conversation transition / relay module 214 can act on behalf of the agent to send and / or receive communications and / or take other actions over a channel (e.g., communications between the intercommunication 104 and the user device via a channel) and send and / or receive communications from at least an API endpoint associated with the agent (e.g., communications between the agent and the intercommunication) to facilitate a conversation between the agent and the user device. In some examples, the conversation transition / relay module 214 can send and / or receive communications and / or take other actions on behalf of the agent. In some examples, the conversation transition / relay module 214 can take such actions on behalf of the agent with other agents or through other services (such as, for example, search engine requests, order processing, manual input requests, etc.). For example, the conversation transition / relay module 214 can send requests to other agents on behalf of the agent and relay the responses of the other agents to the agent.

[0053] In some examples, the conversation transition / relay module 214 is configured to ensure that the conversation between the agent and the user device via the channel complies with the constraints of the channel and / or supports different channel fidelity even when the agent is written to support limited channel fidelity. For example, the conversation transition / relay module 214 may include computer executable instructions that can configure the processing unit 202 to make the communication provided from the agent to the channel comply with the constraints of the channel and vice versa. For example, if the agent is for If a related service is written that allows posting of images, and the channel for dialog with the agent is an SMS channel, the dialog conversion / relay module 214 can comply with the constraints of the SMS channel and convert the message from the agent into a 140-character message and include the image as a link in the 140-character message, which is a hyperlink to the image content. Therefore, there is no need to rewrite the SMS channel with Agents of related services. Similarly, in some examples, the conversation conversion / relay module 214 can modify the fidelity of received and sent communications based on the agent and / or channel fidelity, respectively. Low-fidelity channels include channels with sparser content, such as, for example, SMS, messaging, email, web pages, surveys, other questionnaires, other forms, etc. High-fidelity channels have richer content, such as, for example, voice, video conferencing, holograms, etc. In some examples, the conversation conversion / relay module 214 can be docked with the auxiliary service module 220 to facilitate conversations between agents and channels of different fidelity. For example, if the agent is written in the form of a web-based questionnaire, modules 214 and 220 can use suitable text-to-speech conversion technology to convert the questions of the questionnaire into natural conversation language audio to provide to the user via the channel and convert the received voice or other responses from the user into text input to provide to the agent. Similarly, audio can be converted to video via voice-to-video technology.

[0054] This example also illustrates the capabilities of the intercommunication framework 210 for providing multimodal conversations. In some examples, the conversation conversion / relay module 214, the auxiliary service module 220, the session tracking module 218, and / or other modules can determine the communication format that is most suitable for the channel and context. For example, a high-fidelity channel is capable of sending and receiving multiple forms of communication, but not all forms or certain forms are useful in certain situations. For example, a high-fidelity channel can simultaneously communicate video, voice, and text. The intercommunication framework 210 can know the fidelity of the channel (e.g., through information provided about the channel and stored at the intercommunication 104) and can communicate via all of these formats at the same time. Alternatively, the intercommunication framework 210 can select one or more formats or switch between communication formats based on the context. For example, if part of the agent's response during a conversation with a user includes information that needs to be saved later, it may be more appropriate to provide the information via text so that the user can more easily capture it, even if the user was previously video chatting or voice calling with the agent. In some examples, the intercommunication framework 210 can convert between communications in various formats or add communications in various formats. In some examples, the user may instruct the intercommunication 104 to convert between or add communication formats. For example, the user may have a conversation with the agent via an email channel, but the user may want the robot to continue the conversation with the user or another person via a phone call. The conversation conversion / relay module 214 may convert from an email message to a voice conversation sent to a channel. The conversation conversion / relay module 214 may also convert communications received from a channel into a form understandable to the agent and / or the agent's API. In addition, in order to facilitate the conversion of communications that are appropriately formatted for a particular channel, the conversation conversion / relay module 214 may store a database including channel format information (e.g., architecture information, protocol information, fidelity information, encryption architecture information, etc.). In some examples, the database may further include an association between the required information for formulating communications on the channel (e.g., text data constituting a message for a text channel, video data constituting a streaming video message for a video channel, etc.), agent output, and an auxiliary service for invoking the agent output to convert the channel acceptable format.

[0055] In addition, the intercommunication framework 210 can ensure that the conversation context of the conversation can be maintained through changes in the channel that the user is using. In some examples, the intercommunication framework 210 can capture and store communications via the communication log module 216.

[0056] In some examples, the intercommunication framework 210 can determine the best format for communication based at least in part on the conversation context of the conversation (e.g., from natural language processing provided by the auxiliary service module 220, etc.), the user's context, the usage context, the user device state, the channel state, the proxy service type, the service trigger type, the information contained in the user profile, the information in the proxy directory, etc.

[0057] A user can initiate a conversation with an agent in various ways, depending on the channel in which the user wants to talk to the agent. In some examples, the user may not know the channel type (e.g., the user encounters a kiosk linked to an agent). In some examples, for some channel types, the user can initiate communication with an account associated with the agent. For example, the user can send an SMS to a phone number associated with the agent, send an email to an email account associated with the agent, send a chat to a profile associated with the agent, send a direct message to a social network profile associated with the agent, etc. In some examples, a user with a user profile maintained by the intercommunication framework 210 can associate an agent with the user profile to gain access to the agent. In some examples, the user can add the agent as a contact within a channel. In some examples, the agent can have multiple identities as contacts in multiple channels. In some examples, by adding an agent contact with multiple identities across multiple channels to one channel, the user can add the agent to all multiple channels. In other examples, the user can additionally or alternatively initiate communication with the agent by adding the agent to a contact list or by adding the agent to a discussion group. For example, the user can add the agent to a group discussion or other collaborative communication channel.

[0058] In some examples, the conversation conversion / relay module 214 can transfer the conversation between channels. In some examples, the decision to move the conversation to another channel, to add a communication modality to a channel, or to add a channel can be based at least in part on data collected by the session tracking module 218. In some examples, the session tracking module 218 can track session data, such as, for example, channels used by users, activity / inactivity status of client devices and / or channels, analysis of usage of channels and / or client devices (e.g., time of day, location, purpose of channels and / or client devices being used), channels that are more or less commonly used on different clients, dialog data, identities of the parties communicating in the dialog, etc. In some examples, the session tracking module 218 can detect a change in a channel in use by a user based at least in part on one or more of the following: amount of user activity, cessation of communications via one channel and commencement or continuation of communications in a second channel, time of day, activity status, a channel indicated by the user as the channel in use, a preferred channel, a channel from which a recent communication has been received, or a channel that is preferred as the channel in use based at least in part on one or more of time, the user's location, an IP address associated with communications received from the channel, a calendar event, or a state of the device.

[0059] In some examples, the communication log module 216 may capture at least a portion of the communications that make up the conversation between at least one agent 112(1) and a user device. In some examples, the communication log module 216 may store the communications in the communication log. In some examples, the communication log module 216 may retain the communications in the communication log for a period of time from the time they were first stored, and then the communication log module 216 may delete the data (e.g., the duration of the conversation, a user-defined duration, until the user is de-associated with the agent, until the user profile associated with the communication log is deleted or deactivated, for one week, for one year, for five years). In some examples, the communication log module 216 may encrypt the communication log. In some examples, the communication log may be secured by the security policy of the device 200. In some examples, the communication log module 216 may provide the user with an option to start, stop, or never log communications. The communication module 216 may also allow the user to select the format, channel, and user device from which the communication log may be captured and / or stored. In some examples, the user may also select value-added services (e.g., services of the auxiliary service module 220) and / or agents with permission to access the communication log.

[0060] In some examples, the communication log can be accessed by the auxiliary service module 220, the dialogue conversion / relay module 214, or the session tracking module 218 to facilitate a smooth dialogue between the agent and the user. For example, the auxiliary service module 220 may include a message intelligence service that uses natural language processing, disambiguation, and / or other processing methods to detect service triggers, the user's cultural background, and / or other biographical data about the user from the communication log and / or ongoing dialogue. As mentioned herein, any stimulus for interacting with an agent may be referred to as a "service trigger", whether through a direct query to the agent, a dialogue with the agent, detecting the suitability of the service provided by the agent as described above, etc. In some examples, a service trigger is part of a user communication associated with one or more services of one or more agents. In order to identify a service trigger, the auxiliary service module 220 may include natural language processing, speech to text technology, video to text technology, and / or other semantic disambiguation technology and technology that converts the dialogue into a format that can be used by a natural language processing service to identify when the agent's service may be relevant. In some examples, the auxiliary service module 220 may be programmed to continuously monitor the dialogue for service triggers. In other examples, the auxiliary service module 220 may be programmed to monitor service-triggered conversations at the direction of the user, during times of day, on specific channels, between the user and a specific agent, for agents with a specific level of permission, and the like.

[0061] For example, if one user sends a communication including a time, place, person, and location to another user, the auxiliary service module 220 can detect one or a combination of these to include a service trigger associated with a service that is associated with an agent in the agent directory. For example, one or a combination of time, place, person, and location can be related to a service trigger of one or more of a calendar service, an open time lookup service, a food ordering service, a guide service, etc. In this example, the conversation conversion / relay module 214 can ask one or both of the users whether they want to add the meeting data to their calendar, or whether they want to be introduced to an agent with whom they are not associated and provide services associated with the detected service trigger. In some examples, the agent can "switch" or introduce other agents. For example, the intercommunication framework 210 and / or an agent in a conversation with a user can determine that the detected service trigger is either outside the scope of the service associated with the agent, or another agent can provide a more robust service of the type of the detected service trigger. In these instances, the agent and / or intercommunication framework 210 may provide for introducing one or more other agents associated with the service associated with the service trigger (e.g., adding one or more other agents to a dialog or query agent-to-agent while keeping the dialog box between the user and the agent open without adding the agent of the query) or may provide for changing the entity with which the user is dialoging (i.e., "switching"). In some examples, the message intelligence service of the auxiliary service module 220 may detect topics, tasks, entities, intents, etc. that may be used as service triggers.

[0062] In some examples, the auxiliary services module 220 may also detect cultural context and / or other biographical data associated with the user from a communication log or an ongoing conversation with at least one agent 112(1). In some examples, the cultural context may include the language, dialect, geographic location, demographic information, education level, occupation, estimated gender, estimated social gender, etc. spoken and / or understood by the user. The auxiliary services module 220 may use the cultural context to modify communications to and from the agent to accommodate the cultural context. For example, the auxiliary services module 220 may include a translation service to translate the user's language into the language of the robot and / or other service and vice versa and / or modify the vocabulary, tone, pitch, social gender and / or intonation, etc. of the communications that constitute the conversation, regardless of whether the communication is provided to the agent and / or the communication is provided to the channel.

[0063] In some examples, the auxiliary service module 220 may also include facial detection and recognition services, emotion detection services, beauty detection services, computer vision services, video understanding and conversion services, image understanding and conversion services, speech recognition services, speaker identification services, spelling check services, language understanding intelligent services (LUIS), web language model services (e.g., extended natural language processing), etc. In some examples, the service may be embodied as an API or accessible by an API. In some examples, the services that make up the auxiliary service module 220 described herein may receive communications sent from an agent to a user or communications received from a user that are expected to be directed to the agent as input. Whether it is an agent or a user, the service can relay its output to the desired party. In some examples, the auxiliary service module 220 may include a machine learning algorithm to enhance one or more services of the auxiliary service module 220 and / or other modules of the intercommunication framework 210 using one or more communication logs and / or ongoing conversations.

[0064] In an example instance for illustrating some of these technologies, an agent may include a burrito ordering agent. A user may discover the presence of an agent, such as through an online agent gallery or through one of various methods of physical advertising, and contact the agent via a channel or add the agent to a channel used by the user. The agent registration and discovery module 212 may relay a message received from a user device to the agent via a channel related to a burrito order. In this example, a burrito order includes multiple service triggers. A burrito order includes multiple parts, such as the identity of an entity from which a burrito can be purchased, information about the burrito to be ordered, payment, and delivery / pickup arrangements. Each of these details of an order may serve as a service trigger for a service that may involve one or more agents. In some examples, the agent registration and discovery module 212 may facilitate a conversation between at least one agent whose service may be applied to a user, and facilitate inquiries between agents in order to formulate a satisfactory conversation (e.g., a burrito restaurant agent may contact a payment agent and a search engine map agent to arrange payment and provide driving directions to a restaurant, respectively). In other examples, the agent registration and discovery module 212 can facilitate the inclusion of a second or more agents in the conversation with the user (e.g., the user is introduced to other agents, or the user can be switched to another agent for a conversation with the agent about the services provided by the agent). The agent can request the interworking framework 210 to query other bots (e.g., a scheduling agent, a payment agent) or other services (e.g., a search engine) in order to formulate a response to the user device in the conversation about the burrito order.

[0065] Sample User Interface

[0066] Figures 3A-3DAn example user interface 300 is depicted through which technology can be deployed. In particular, Figures 3A-3D Depicted is an example user interface for a website designed to allow a user to link an agent to a user profile via a particular channel (with the channel that has been so linked also illustrated). Figure 3A A portion of an example agent library is depicted, the agent library including a description of a particular agent (302), an agent name (304), an agent description (306), an agent hashtag (308), and channels to which the agent has been linked (310). In some examples, the agent library may be publicly visible and discoverable. In some examples, the agent library or certain agents of the plurality of agents that make up the agent library may not be discoverable by the public and may require a user profile to view. In some examples, the agent library may facilitate discovery of agents by users and provide information about channels linked to a user profile that also links the user profile to the agent.

[0067] For example, Figure 3A A directory agent is depicted, which is designed to provide information to the user, which is about information from the company's active directory. The library includes an "enterprise" theme tag to allow users to filter agents by the theme tag. In the depicted example, the user has linked the directory agent to a user account (icon 312) on a mixed media service channel (e.g., Skype, etc.), an email service (icon 314), and a telephone service (e.g., SMS service, call service, etc.) (icon 316). Note that in some examples, the user's channel account can be associated with a profile of the user maintained by intercommunication. In some examples, intercommunication 104 can maintain a user profile including user account associations without storing user credentials for the user account. In some examples, intercommunication 104 can maintain a user profile including user account associations by storing a username or similar publicly discoverable data rather than private data such as, for example, a password.

[0068] Figure 3B Depicted is an example user interface for managing channels that agents can link to. The example user interface displays user information for three channels to which a directory agent has been linked. The example user interface displays an option to connect to a project collaboration channel called "Group Project". By selecting this link, the user interface transitions to Figure 3C, which may provide an authentication interface for providing credentials to a project collaboration channel. In some examples, credentials are provisioned directly to the channel and intercommunication 104 facilitates authentication. In some examples, intercommunication 104 may be unaware of the credentials because intercommunication 104 may store the agent's API endpoint and the agent's channel credentials. In this example, the user only needs to grant the agent access to messages and / or groups and / or be authenticated on a user profile stored by the intercommunication. In some examples, a user may initiate communication with an agent by sending a message to an account associated with the agent and / or adding the agent to a conversation. Figure 3D Depicted is an example user interface for authorizing an agent to access messages and / or groups, thereby linking the agent to a channel associated with a user profile.

[0069] The agent registration and discovery module 212 may include computer executable instructions for configuring the processing unit 202 to render the user interface 300 via the display (via the I / O interface 224 ).

[0070] Figure 4 An example user interface 400 is depicted for conversing with an example agent (directory agent) via a telephone service channel (e.g., an SMS channel). In some examples, as in the illustrated example, the conversation with the agent can be conducted via messaging linked to the agent's phone number, phone number, API endpoint, and the agent's key stored by intercommunication 104 in some examples. Figure 4 In the depicted example, the conversation includes a user query 402 for directory information. The intercommunication, which has access to the agent's API endpoints and keys and is therefore able to receive and send messages on behalf of the agent, can detect whether the user device from which the message has been sent has permissions to access resources accessible to the agent, such as an active directory in this example. Although the user can log into the intercommunication 104 user profile, the agent may additionally require authentication in some examples. In the depicted example, the agent responds to the user query 402 with a request 404 for further authentication via the intercommunication 104. Upon authentication, the agent continues the conversation and responds to the user query.

[0071] Figure 5An example user interface 500 for a conversation with an example agent (director agent) through a group collaboration application is depicted. In this example, the example agent has previously been added to the discussion as a member "colleague" of the group discussion. In some examples, in similar examples, intercommunication 104 can provide value-added services such as natural language processing and / or disambiguation to detect when a service associated with an agent is relevant to the conversation. Based at least in part on this detection, intercommunication 104 can form a query to the agent to obtain a response from the agent that intercommunication 104 can provide to the group. In some examples, intercommunication 104 can format the query to conform to queries that are typically made to the agent. For example, members of the group can make comments about a topic without querying the agent, and comments made by members of the group may not elicit a useful response from the agent without modification. Intercommunication using natural language processing can detect that the topic is related to the agent's service and can form a query about the topic to the agent and provide the agent response to the group as an agent suggestion, where the agent suggestion is a communication from the agent when the user does not request information.

[0072] Figure 5 An example of an "introduction" between agents is also depicted. At 502, Jenny asks a question that may be interpreted as being related to the individual's schedule and, due to the nature of the schedule, the individual's location. In some examples, intercommunication 104 may facilitate a conversation with a second agent having services related to the service trigger. Intercommunication 104 may facilitate a conversation with the second agent when the first agent does not provide services that constitute the service trigger and the second agent provides services that constitute the service trigger, when the user has indicated a preference to the second agent for one or more service types and the service trigger is associated with one of those service types, when the second service provides a more adequate or satisfactory service, etc.

[0073] Returning to the example, the directory agent may provide Jenny information from an active directory related to the meeting (e.g., individuals may be met), and intercommunication 104 may insert a second agent (e.g., a calendar agent) into the dialog box. In some examples, intercommunication 104 may request permission to introduce the second agent before facilitating a conversation with the second agent. In some examples, intercommunication 104 may be configured to associate agents on behalf of the user so that "handoffs" and "introductions" may proceed more smoothly. In other examples, intercommunication 104 may request the user to associate the second agent with a channel. In another example not depicted, the directory agent may query the calendar agent via the intercommunication framework 210 to receive data related to the service trigger at 502 and relay the calendar agent's response to the intercommunication framework 210, in some examples, the intercommunication framework 210 may relay the message to the channel on behalf of the directory agent.

[0074] Figure 6An example user interface 600 for registering an agent is illustrated. In some examples, the received information can be stored in an agent directory. User interface 600 illustrates some examples of user interfaces for collecting data discussed above with respect to agent registration and discovery module 212. Figure 6 Also depicted is the ability of the agent registrar to select value-added services to activate the agent (602). In some examples, these options are shown for subgroups and / or up to all services. In other examples, these options are not displayed for value-added services. Module 212 may include computer executable instructions for configuring processing unit 202 to draw user interface 600 via a display (via I / O interface 224).

[0075] Figure 7 An example user interface 700 for managing channels linked to an agent is illustrated. In some examples, channels, addresses, enabled states, and / or settings may be stored in an agent directory or may otherwise be available to intercommunication 104. Module 212 may include computer executable instructions for configuring processing unit 202 to draw user interface 700 via a display (via I / O interface 224).

[0076] Figure 8 An example user interface 800 for linking an agent to a channel is illustrated. In this example, the channel is a "hybrid media service". The example also includes instructions for authenticating the agent on the channel so that the intercommunication 104 can issue requests on behalf of the agent, such as requests to send and / or receive messages and / or files on the channel. In this example, the instructions may include instructions for creating a channel account, registering an agent application, obtaining a client ID and client token assigned by the channel (e.g., revealed in a client id parameter in a redirect URL provided in response to registration), and providing the client ID and client token to the intercommunication 104 (i.e., to authorize the intercommunication 104 to operate on behalf of the agent). Module 212 may include computer executable instructions for configuring the processing unit 202 to draw the user interface 800 via a display (via I / O interface 224).

[0077] Example Processing

[0078] Figure 9-11 Exemplary processes 900, 1000, and 1120 are illustrated, which may be performed in whole or in part. These processes may, but need not, be performed in Figure 1 100% of the environment and / or use Figure 2 Device 200 executes.

[0079] Fig. 9An exemplary process 900 for implementing multi-channel agent intercommunication is depicted. At 902, intercommunication 104 can store a user profile in any manner described herein. At 904, intercommunication 104 can link one or more channels to the user profile in any manner described herein. In some examples, the exemplary process 900 can alternatively or additionally link one or more channels to an agent at 904.

[0080] At 906, intercommunication 104 can maintain a directory of multiple agents in any manner described herein. In some examples, the multiple agents are associated with providing user services in response to a conversation with a user, and the multiple agents are linked to one or more channels. In some examples, the services can include providing information in response to a service trigger contained in the conversation with the user, acting on behalf of the user, initiating actions outside the system, etc. At 908, intercommunication 104 can create a conversation session between at least one agent 112(1) of the multiple agents and the user. In some examples, the conversation session is configured to receive and send messages at an API endpoint of at least one endpoint, and messages received from the API endpoint (i.e., from the agent) and sent to the API endpoint (i.e., from the agent) can be relayed to the user device via the channel (e.g., via the API of the channel). In some examples, the messages can be relayed based at least in part on an association of at least one agent 112(1) with a user profile associated with the channel, the user, and / or the user device.

[0081] Fig.10 An illustrative process 1000 for implementing multi-channel agent intercommunication is depicted. At 1002, intercommunication 104 can store a user profile having one or more communication channels associated therewith in any manner described herein. At 1004, intercommunication 104 can receive a service trigger from a communication channel associated with the user profile. In some examples, the service trigger is proprietary (e.g., no disambiguation or further processing is required to identify the service trigger - the service trigger is self-identifying). In some examples, the service trigger is latent (e.g., the service trigger is contained in a communication received from the channel, but the communication must first be processed in any manner described herein to identify the service trigger), and the illustrative process 1000 can also include processing the received service trigger to identify at least one agent 112(1) associated with a service corresponding to the service trigger.

[0082] At 1006, intercommunication 104 may identify at least one agent 112(1) associated with a service corresponding to the service trigger in any manner described herein. In some examples, agents of the plurality of agents may have service identifiers associated with the plurality of agents to expose their functionality to intercommunication 104 and facilitate accurate identification of agents providing services corresponding to the service trigger. In some examples, the service trigger may include a single or multiple items. In some examples, intercommunication 104 may perform optimization of the service trigger items and the service identifier to identify at least one agent 112(1) associated with the service corresponding to the service trigger.

[0083] At 1008, intercommunication 104 may facilitate a conversation between at least one agent 112(1) and a user associated with the user profile in any manner described herein.

[0084] Fig.11 An illustrative process 1000 for implementing multi-channel agent intercommunication is depicted. At 1102, intercommunication 104 can intercept communications in a channel in any manner described herein. In some examples, "in a channel" can include communications input to a channel and output to intercommunication 104 through a channel API. In some examples, "in a channel" can include intercepting communications within a channel service. In some examples, intercommunication 104 can store intercepted communications in a communication log in any manner described herein.

[0085] At 1104, intercommunication 104 may identify a service trigger from the intercepted communication in any manner described herein. At 1106, intercommunication 104 may identify at least one agent 112(1) associated with a service corresponding to the service trigger in any manner described herein. At 1108, intercommunication 104 may facilitate a conversation between the at least one agent 112(1) and the channel in any manner described herein.

[0086] Sample Clauses

[0087] A. A system comprising: one or more processors; a network interface; and a computer-readable medium having computer-executable instructions stored thereon, which, when executed, program the one or more processors to perform operations, the operations comprising: storing a user profile of a user, the user profile being associated with a channel through which the user is able to communicate; maintaining a directory of multiple agents, the directory of multiple agents including the agents and including references to services provided by the multiple agents, the multiple agents including at least one non-human agent; linking multiple channels through which the non-human agents are able to communicate to at least one non-human agent; receiving a communication including a service trigger from a channel associated with the user profile via the network interface, the service trigger including a reference to the non-human agent or at least one of the services provided by the non-human agent; and creating a conversation session between the at least one non-human agent and the user via the network interface using a channel associated with the user profile and linked to the at least one non-human agent based at least in part on the service trigger.

[0088] B. A system as described in paragraph A, wherein the directory also includes references to application programming interface (API) endpoints associated with multiple agents, and wherein creating a conversation session includes: receiving a message from the user through the channel; relaying the message to the at least one non-human agent via the API endpoint associated with the at least one non-human agent; receiving a response from the at least one non-human agent via the API endpoint; and relaying the response to the user through the channel on behalf of the at least one non-human agent.

[0089] C. A system as described in paragraphs A or B, wherein the channel is a first channel, and the operation further includes: capturing a conversation between the at least one non-human agent and the user via the conversation session; detecting a change in activity on the first channel; detecting a change in activity on a second channel of the multiple channels, the second channel also being associated with the user profile; and maintaining the conversation session between the at least one non-human agent and the user using the second channel, wherein maintaining the conversation session includes continuing the conversation between the non-human agent and the user, and routing messages through the second channel via an API endpoint associated with the at least one non-human agent, the second channel being linked to the at least one non-human agent.

[0090] D. A system as described in any of paragraphs AC, wherein detecting a change in activity includes: monitoring or receiving one or more of the activity states of a channel; a user indication; a preferred channel; a channel from which most communications are being received; a channel from which recent communications have been received; or a channel that is preferred as the channel being used based at least in part on one or more of the following: time, location of the user, IP address associated with communications received from the channel, calendar event, or state of the device.

[0091] E. The system of any of paragraphs AD, wherein the operations further include formatting communications between the at least one non-human agent and the user to conform to the standards of the channel.

[0092] F. A system as described in any of paragraphs AE, wherein the at least one non-human agent is stored at a location remote from the system and communicates with the system via the network interface.

[0093] G. A system as described in any of paragraphs AF, wherein the operations further include: capturing communications in a channel associated with the user profile; identifying the service trigger from the captured communications; and selecting the at least one non-human agent based at least in part on a reference to a service provided by the at least one non-human agent corresponding to a keyword associated with the service trigger.

[0094] I. A system as described in any of paragraphs AH, wherein the service trigger includes a first service trigger, and the operation also includes: identifying a second service trigger associated with a service provided by at least a second agent of the plurality of agents from the conversation session; and adding the second agent to the conversation session, the adding including: relaying messages sent to and from the at least one non-human agent and the user, respectively, to the second agent via an API endpoint associated with the second agent; and relaying messages from the second agent to the at least one non-human agent and the user through the channel via the API endpoint associated with the at least second agent.

[0095] J. A system as described in any of paragraphs AI, wherein the at least one non-human agent is a first agent, and the operation further includes: receiving an indication from the first agent that a request received from the channel via the conversation session is outside the scope of services provided by the first agent; identifying at least a second agent that provides a service associated with the request; and adding the second agent to the conversation session, the addition comprising: relaying messages sent to and from the first agent and the user, respectively, to the second agent via an API endpoint associated with the second agent; and relaying messages from the second agent to the first agent and the user through the channel via the API endpoint associated with the at least second agent.

[0096] K. A system as described in any of paragraphs AJ, wherein creating the conversation session includes at least one of the following: receiving a message addressed to the at least one non-human agent from the user through the channel, the message including: contact information of the at least one non-human agent corresponding to the channel; receiving a request from the user to add the at least one non-human agent to the conversation on the channel; or receiving a request to link the at least one non-human agent to the channel.

[0097] L. A system as described in any of paragraphs AK, wherein the operations also include: maintaining a directory of channels; adding information corresponding to the channels to the directory of the channels, the information being used to link an agent or user profile to the channels; receiving and storing information related to agents in the agent directory; receiving one or more client IDs or client tokens of a web service associated with an agent, at least one of the client IDs or client tokens allowing the system to: relay messages on behalf of the agent; and call services provided by the agent; and receiving one or more client IDs or client tokens associated with authentication of an agent on a channel, at least one of the client IDs or client tokens allowing the system to relay messages through the channel on behalf of the agent.

[0098] M. A computer-readable medium having instructions stored thereon that, when executed, program one or more processors to perform operations, the operations comprising: registering an agent, the agent comprising different instructions that, when executed, program one or more processors to perform operations including providing services in response to a natural language conversation, and the registration comprising receiving an application programming interface (API) endpoint associated with the agent and authentication credentials corresponding to multiple channels on which the agent has been authenticated to interact with channel services; receiving requests from multiple channels for a conversation with the agent; and using the API endpoint and the authentication credentials of the agent to relay conversations between the agent and the multiple channels.

[0099] N. A computer-readable medium as described in paragraph M, wherein relaying a conversation between the agent and a channel of the plurality of channels comprises: receiving a message from the channel on behalf of the agent using a subset of the authentication credentials; relaying the message to the agent via the API endpoint; receiving a response from the agent via the API endpoint; and relaying the response to the channel on behalf of the agent using the authentication credentials.

[0100] O. Computer-readable media as described in paragraphs M or N, wherein using the authentication credentials includes authenticating a central service to act on behalf of the agent, the central service: registering the agent, receiving a request to communicate with the agent, receiving a message, relaying the message, receiving a response, and relaying the response.

[0101] P. A computer-readable medium as described in any of paragraphs MO, the operation also including: storing a user profile, the user profile having one or more channels associated with it; and tracking a conversation session between the agent and the channel associated with the user profile, the tracking comprising associating a conversation identifier with the agent, the channel, and the user profile.

[0102] Q. A computer-readable medium as described in any of paragraphs MP, the operation also including: receiving a message from a different channel associated with the user profile; and continuing the conversation between the agent and the user by using the different communication channel, wherein the continuation includes additionally associating the different channel with the conversation identifier, or alternatively associating the different channel with the conversation identifier instead of the channel.

[0103] R. A method comprising: intercepting communications in a channel; identifying a service trigger from the intercepted communications; identifying at least one agent associated with a service corresponding to the service trigger; and relaying a conversation between the at least one agent and a channel endpoint associated with the channel.

[0104] S. A method as described in paragraph R, wherein the service trigger includes one or more of the following items: a keyword associated with the service, a topic corresponding to the service, a receiving user location corresponding to the service, a time corresponding to the service, a data type corresponding to the service, a request to add an agent to the conversation, a reference to an agent, or a request for a service.

[0105] T. The method of paragraphs R or S also includes: identifying a change from a first channel to a second channel; and relaying a conversation between at least one agent and an endpoint associated with the second channel based at least in part on the change in channel.

[0106] U. The method described in any of paragraphs RT further includes: detecting the fidelity of the channel or one or more of the constraints of the channel; and making the response of the at least one agent conform to the fidelity of the channel or one or more of the constraints of the channel.

[0107] V. A computer-readable medium having computer-executable instructions thereon which, when executed, configure a computer to perform the method described in any of paragraphs RU.

[0108] W. A system comprising: a component for processing, a component for storing, and a component for performing any steps of the method described in any of paragraphs RU.

[0109] in conclusion

[0110] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claims.

[0111] The module described herein represents instructions that can be stored in any type of computer-readable medium and can be implemented in software and / or hardware. All of the above methods and processes can be implemented and fully automated via software code modules and / or computer-executable instructions executed by one or more computers or processors, hardware or some combination thereof. Alternatively, some or all of the methods can be implemented in special-purpose computer hardware.

[0112] Unless specifically stated otherwise, conditional language such as "may," "can," "could," or "might" is understood in context to present certain examples as including, while other examples do not, certain features, elements, and / or steps. Thus, such conditional language is generally not intended to imply that certain features, elements, and / or steps are in any way required for one or more examples, or that one or more examples must include logic for deciding, with or without user input or prompting, whether certain features, elements, and / or steps are included or to be performed in any particular example.

[0113] Unless specifically stated otherwise, joint language such as the phrase "at least one of X, Y, or Z" should be understood to present an item, term, etc. that can be X, Y, or Z, or any combination thereof. Unless explicitly described as being in the singular, "a" and "an" refer to both the singular and the plural.

[0114] Any routine description, element or box in the flowcharts described herein and / or depicted in the accompanying drawings should be understood to potentially represent a module, segment or code portion that includes one or more computer executable instructions for implementing a specific logical function or element in the routine. Alternative implementations are included within the scope of the examples described herein, in which elements or functions may be deleted, or performed in a different order from that illustrated or discussed, including substantially synchronously or in reverse order, depending on the functions involved as will be understood by those skilled in the art.

[0115] It should be emphasized that many changes and modifications may be made to the above examples, and the elements therein should be understood as other acceptable examples. All such modifications and variations are intended to be included within the scope of this disclosure and protected by the accompanying claims.

Claims

1. A system for processing a conversation, comprising: one or more processors; Network interface; as well as A computer-readable medium having computer-executable instructions stored thereon, the computer-executable instructions, when executed, causing the one or more processors to perform operations comprising: maintaining a plurality of communication channels for electronic communications via the network interface; maintaining a directory of a plurality of agents, each agent being a computer program that, when executed, implements a dialog-based service to provide information; identifying a conversation session between a first agent and a user device via a first communication channel of the plurality of communication channels, wherein the first communication channel is associated with a plurality of content formats; determining a first preferred content format to utilize during the conversation session based on the context of the conversation session and the plurality of content formats of the first communication channel; converting one or more messages from the first agent to the first preferred content format for communication to the user device via the first communication channel; maintaining the dialog session between the first agent and the user device; identifying an updated context of the dialog session; determining a second preferred content format to utilize during the conversation session based on the updated context of the conversation session and the plurality of content formats of the first communication channel; and One or more messages from the first agent are converted to the second preferred content format for communication to the user device via the first communication channel. 2 . The system of claim 1 , wherein the second preferred content format is different from the first preferred content format.

3. The system of claim 1, wherein the first agent communicates using a first agent content format. 4 . The system of claim 3 , wherein the first proxy content format is different from both the first preferred content format and the second preferred content format.

5. The system of claim 1, wherein the plurality of content formats includes natural conversational language audio, web-based forms, text-based communications, video, and images.

6. The system of claim 1, wherein the first preferred content format is one of a voice call or a video chat.

7. The system of claim 6, wherein the second preferred content format is a text communication.

8. The system of claim 7, wherein the second preferred content format is utilized while simultaneously continuing to utilize the first preferred content format.

9. One or more computer storage media which, when executed by a computing device, causes the computing device to perform a method comprising: Via one or more processors: maintaining a plurality of communication channels for electronic communications via the network interface; maintaining a directory of a plurality of agents, each agent being a computer program that, when executed, implements a dialog-based service to provide information; identifying a conversation session between a first agent and a user device via a first communication channel of the plurality of communication channels, wherein the first communication channel is associated with a plurality of content formats; determining a first preferred content format to utilize during the conversation session based on the context of the conversation session and the plurality of content formats of the first communication channel; converting one or more messages from the first agent to the first preferred content format for communication to the user device via the first communication channel; maintaining the dialog session between the first agent and the user device; identifying an updated context of the dialog session; determining a second preferred content format to be utilized during the conversation session based on the updated context of the conversation session and the plurality of content formats of the first communication channel; as well as One or more messages from the first agent are converted to the second preferred content format for communication to the user device via the first communication channel.

10. The medium of claim 9, wherein the second preferred content format is different from the first preferred content format.

11. The medium of claim 9, wherein the first agent communicates using a first agent content format.

12. The medium of claim 11, wherein the first proxy content format is different from both the first preferred content format and the second preferred content format.

13. The medium of claim 11, wherein the second preferred content format is utilized while simultaneously continuing to utilize the first preferred content format.

14. The medium of claim 9, wherein the method further comprises switching from the first preferred content format to the second preferred content format.

15. The medium of claim 9, wherein the context is one or more of: a session context of a session, and a context of a user.

16. The medium of claim 9, wherein the determination of the first preferred content format to utilize during the conversation session is further based on one or more of: user device status, channel status, and user profile information.

17. A computerized method for processing a conversation, comprising: maintaining a plurality of communication channels for electronic communications via the network interface; maintaining a directory of a plurality of agents, each agent being a computer program that, when executed, implements a dialog-based service to provide information; identifying a conversation session between a first agent and a user device via a first communication channel of the plurality of communication channels, wherein the first communication channel is associated with a plurality of content formats; determining a first preferred content format to utilize during the conversation session based on the context of the conversation session and the plurality of content formats of the first communication channel; converting one or more messages from the first agent to the first preferred content format for communication to the user device via the first communication channel; maintaining the dialog session between the first agent and the user device; identifying an updated context of the dialog session; determining a second preferred content format to be utilized during the conversation session based on the updated context of the conversation session and the plurality of content formats of the first communication channel; as well as One or more messages from the first agent are converted to the second preferred content format for communication to the user device via the first communication channel.

18. The method of claim 17, wherein the second preferred content format is different from the first preferred content format.

19. The method of claim 17, wherein the first agent communicates using a first agent content format, and wherein the first agent content format is different from both the first preferred content format and the second preferred content format.

20. The method of claim 17, further comprising: identifying a second conversation session between the first agent and the user device via a second communication channel of the plurality of communication channels, wherein the second communication channel is associated with a second plurality of content formats, and wherein the plurality of content formats includes an additional content format that is different from any of the plurality of content formats associated with the first communication channel; as well as At least one message is transmitted from the first agent to the user device via the second communication channel using an additional content format different from any of the plurality of content formats associated with the first communication channel.

Images