Access authentication in ai systems

Abstract

Access authentication in an artificial intelligence system includes electronically perceiving, with an artificial intelligence voice response system (AIVRS) of a first user, a physical presence of a second user. A voice request is generated by the AIVRS of the first user and communicated to the second user, requesting access to a knowledge corpus stored by an AIVRS of the second user. Based on a voice response of the second user, the AIVRS of the first user instantiates an electronic communication session with the AIVRS of the second user. The session is initiated via an electronic communication connection with a portable device of the second user. A selected portion of the knowledge corpus is retrieved by the AIVRS of the first user from the AIVRS of the second user, the portion being selected based on the voice response. Responsive to a voice prompt interpreted by the AIVRS of the first user based on the selected portion, an action of one or more IoT devices is initiated.

Description

Background Technology

[0001] This disclosure relates to computer-based data exchange, and more specifically to secure data access and exchange between computer systems endowed with artificial intelligence.

[0002] It is estimated that over one million American households have automation systems, and the number of smart devices installed in American homes is expected to soon exceed 50 million. Smart devices are endowed with artificial intelligence (AI) and can operate based on specific user intentions and preferences. These user intentions and preferences are part of a knowledge corpus that smart devices can acquire using machine learning. Smart devices are increasingly communicating with each other and coordinating integrated functions via the internet or other data communication networks, thus forming the Internet of Things (IoT). Summary of the Invention

[0003] In one or more embodiments, a method includes electronically sensing the physical presence of a second user using an artificial intelligence (AI) voice response system of a first user. The method includes: transmitting a voice request generated by the first user's AI voice response system, the voice request requesting access to a knowledge base electronically stored by the second user's AI voice response system; and receiving a voice response from the second user. The method includes: instantiating an electronic communication session with the second user's AI voice response system based on the voice response, the electronic communication session being initiated by the first user's AI voice response system via an electronic communication connection with a portable device of the second user. The method includes retrieving a selected portion of the knowledge base from the second user's AI voice response system via a data communication network by the first user's AI voice response system, the selected portion of the knowledge base being selected based on the second user's voice response to the voice request transmitted by the first user's AI voice response system. The method includes initiating an action of one or more IoT devices in response to a voice prompt interpreted by the first user's AI voice response system based on the selected portion of the knowledge base.

[0004] In one or more embodiments, a system includes an artificial intelligence (AI) voice response system for a first user operably coupled to a processor configured to initiate operations. Operations include electronically sensing the physical presence of a second user. Operations include transmitting a voice request to access a knowledge base electronically stored by the second user's AI voice response system and receiving a voice response from the second user. The operation includes instantiating an electronic communication session based on the voice response between the first user's AI voice response system and the second user's AI voice response system, the electronic communication session being initiated via an electronic communication connection with a portable device of the second user. The operation includes retrieving a selected portion of the knowledge base from the second user's AI voice response system via a data communication network, the selected portion of the knowledge base being selected based on the second user's voice response to the request. The operation includes initiating action of one or more IoT devices in response to a voice prompt interpreted by the first user's AI voice response system based on the selected portion of the knowledge base.

[0005] In one or more embodiments, the computer program product includes one or more computer-readable storage media on which instructions are stored. The instructions are executable by a processor operatively coupled to a first user's artificial intelligence (AI) voice response system to initiate operations. Operations include electronically sensing the physical presence of a second user. Operations include transmitting a voice request to access a knowledge base electronically stored by the second user's AI voice response system and receiving a voice response from the second user. Operations include instantiating an electronic communication session based on the voice response between the first user's voice response system and the second user's AI voice response system, the electronic communication session being initiated via an electronic communication connection with a portable device of the second user. Operations include retrieving a selected portion of the knowledge base from the second user's AI voice response system via a data communication network, the selected portion of the knowledge base being selected based on the second user's voice response to the request. Operations include initiating actions of one or more IoT devices in response to a voice prompt interpreted by the first user's AI voice response system based on the selected portion of the knowledge base.

[0006] The summary portion of this invention is provided merely to introduce certain concepts and not to identify any key or essential features of the claimed subject matter. Other features of the inventive arrangement will become clear from the accompanying drawings and the following detailed description.

[0007] Brief description of the attached figures

[0008] The inventive arrangement is illustrated by way of example in the accompanying drawings. However, the drawings should not be construed as limiting the inventive arrangement to the specific implementation shown. The different aspects and advantages will become clear when referring to the following detailed description and the accompanying drawings.

[0009] Figure 1An exemplary computing environment operable with an AI voice response system according to an embodiment is described, wherein the intelligent virtual assistance is given the ability to securely access a remotely stored knowledge base.

[0010] Figure 2 This is a flowchart illustrating a method for securely accessing a knowledge base generated by another AI voice response system using an AI voice response system, according to an embodiment.

[0011] Figure 3 A cloud computing environment according to an embodiment is shown.

[0012] Figure 4 An abstract model layer according to an embodiment is shown.

[0013] Figure 5 A cloud computing node according to an embodiment is shown.

[0014] Figure 6 An example portable device according to an embodiment is shown. Detailed Implementation

[0015] Although this disclosure concludes with claims defining novel features, it is believed that the various features described herein will be better understood by considering the description in conjunction with the accompanying drawings. The processes, machines, manufactures, and any variations thereof described herein are provided for illustrative purposes. The specific structural and functional details described herein should not be construed as limiting, but rather serve only as the basis for the claims and as a representative basis for teaching those skilled in the art to employ the features described in virtually any appropriately detailed structure. Furthermore, the terminology and phrases used herein are not intended to be restrictive, but rather to provide an understandable description of the described features.

[0016] This disclosure relates to computer-based data exchange, and more specifically to secure data access and exchange between computer systems endowed with artificial intelligence.

[0017] AI is increasingly being applied to various types of automated systems and devices, ranging from individual devices to multiple devices integrated into the Internet of Things. For example, intelligent virtual assistants and chatbots are implementing Natural Language Understanding (NLU) to create human-machine interfaces that can machine-determine the structure and meaning of language from human speech.

[0018] AI voice response systems rely on a knowledge base to recognize and execute speech, text, and / or speech-to-text prompts and commands for controlling voice-enabled AI systems and responding to information queries. As defined herein, a “knowledge base” is a data structure comprising a collection of words, phrases, and sentences whose grammar and semantics express an individual’s intent and preferences. Intent and preference correspond, respectively, to the spoken (voice or text) command used to cause the device to perform an action and the preferred manner in which the device performs that action. For example, a spoken utterance such as “Make Bill a coffee the way he likes” includes the command “The automatic coffee machine makes a coffee” and the preference “Make coffee according to Bill’s preferences (e.g., strong, creamy, and without sugar)”. By “learning” from past use through machine learning to build a knowledge base to recognize the speaker and understand the grammar and semantics of the utterance, the AI ​​voice response system can respond to commands by setting parameters of the automatic coffee machine according to Bill’s preferences.

[0019] As defined herein, an “artificial intelligence voice response system (AIVRS)” is any system that provides a NLU computer-human interface through which a machine determines the structure and meaning of language based on human speech used to select, create, and / or modify data for controlling automated equipment or systems. This NLU computer-human interface uses machine learning to learn the intent and preferences of the device or system user from processing user speech, monitoring instances of user activity, and creating corresponding usage history or patterns of the device or system. For example, an AIVRS can be implemented in software running on a cloud-based server. For instance, an AIVRS running on a cloud-based server can interact with a user by connecting to a smart speaker via a data communication network (e.g., the Internet). The smart speaker may include a speaker and a virtual assistant or chatbot to facilitate verbal interaction with the user of the AIVRS.

[0020] Different users will have different voice characteristics, use different utterances, and have different activity patterns, and therefore different knowledge bases. A specific knowledge base typically corresponds to a specific individual. For privacy and / or data security reasons, access to a specific user's knowledge base may be restricted to that specific individual. While this protects privacy and provides security, this restriction can limit a user's ability to use another person's AIVRS, as it typically prevents other people's AIVRS from accessing the previous user's knowledge base. Without such access, another user's AIVRS is unaware of the user's intentions and preferences and cannot recognize the user's commands or prompts. For example, consider a scenario where an individual visits a friend's home or a colleague's office. The friend or colleague may invite the individual to use their AIVRS to control, for example, smart home or office IoT devices, based on the individual's personal choices regarding things like ambient temperature, lighting, background music, or the amount of sugar in coffee prepared by an automatic coffee machine. Without access to the individual's knowledge base, the individual cannot use their friend's or colleague's AIVRS to control any of these automated functions.

[0021] The systems, methods, and computer program products disclosed herein instantiate communication sessions between multiple AIVRSs to dynamically provide selected portions of a user's knowledge base to another user's AIVRS. Providing these selected portions of the knowledge base enables the first user's AIVRS to capture the intent and preferences learned by the second user's AIVRS and to respond to prompts and commands from the second user while maintaining the security of the second user's knowledge base.

[0022] In one arrangement, the first user's AIVRS identifies a command issued by or related to the second user, and the AIVRS does not possess a knowledge base about the second user to implement the command. Therefore, the first user's AIVRS attempts to obtain a knowledge base (or a relevant portion thereof) for implementing the command by engaging a nearby portable device (e.g., a smartphone, smartwatch) carried or worn by the second user. If the first user's AIVRS determines that permission is required and obtains permission from the second user, the first user's AIVRS instantiates a communication session with the second user's AIVRS to gain access to selected portions of the knowledge base generated by the second user's AIVRS. The first user's AIVRS may instantiate the communication session via a link on a data communication network.

[0023] The arrangement described herein relates to computer technology and provides improvements over existing computer technology. This arrangement improves computer technology by enabling AIVRS to function effectively on behalf of a broader user base. Further improvements to computer technology are made by reducing communication overhead (e.g., the required message exchange) to facilitate AIVRS's acquisition of knowledge bases to operate on behalf of a wider user base. A further improvement is the exchange of relevant portions of the knowledge base while maintaining the privacy of the broader user base and the security of the knowledge base corresponding to that broader user base.

[0024] Further aspects of the embodiments described in this disclosure will be described in more detail with reference to the following accompanying drawings. For purposes of simplicity and clarity, the elements shown in the drawings are not necessarily drawn to scale. For example, the dimensions of some elements may be enlarged relative to others for clarity. Furthermore, reference numerals are repeated in the drawings where deemed appropriate to indicate corresponding, similar, or analogous features.

[0025] Figure 1 An example computing environment 100 is illustrated. Computing environment 100 illustratively includes AIVRS 102 and AVIRS 104. AIVRS 102 and AVIRS 104 can each be implemented in a separate computer system and / or a separate computing node (not shown), such as computing node 500. Figure 5 Therefore, each computing node may include, for example, a cloud-based server. Both AIVRS 102 and AIVRS 104 are illustratively implemented in software, which may be stored in memory and executed on one or more processors of one or more computer systems (e.g., computer system 512 of computing node 500). Illustratively, AIVRS 102 is operatively coupled to voice interface 106. In some embodiments, voice interface 106 is a smart speaker that includes a speaker for handling human-computer interaction and for controlling IoT devices (not explicitly shown) coupled to voice interface 106 via wired or wireless connections (e.g., Wi-Fi, Bluetooth) and an integrated virtual assistant. Control of the IoT devices is achieved by capturing voice prompts and commands by voice interface 106 and based on different voice prompts and commands created by AIVRS 102 to correspond to the user's knowledge base 108. Knowledge base 108 is illustratively stored in the memory of a computer system that is identical to or operatively coupled to the computer system on which AIVRS 102 executes.

[0026] AIVRS 102 may reside on a computer node (e.g., a cloud-based server) located remotely from the voice interface 106, and IoT devices may be operatively coupled to the voice interface 106 and, through the voice interface 106, to the AIVRS 102. In other embodiments, the AIVRS 102, voice interface 106, and knowledge base 108 may be integrated into a single device, which has sufficient computing resources in terms of processing power, memory storage, etc., to support each resource. The voice interface 106 and the connected IoT devices may be located, for example, in a smart home or office corresponding to a user of the knowledge base 108. In some embodiments where the AIVRS 102 and voice interface 106 are located remotely from each other, the AIVRS 102 may be operatively coupled to the voice interface 106 via a data communication network 110. The data communication network 110 may be, and typically, the Internet. The data communication network 110 may be or include a wide area network (WAN), a local area network (LAN), or a combination of these and / or other types of networks, and may include wired, wireless, fiber optic, and / or other connections.

[0027] AIVRS 102 provides Natural Language Understanding (NLU), which enables the machine to determine the language structure and meaning from human speech received by the voice interface 106. Voice commands and prompts received by the voice interface 106 can be used to select, create, and modify data stored in electronic memory to control various automated systems and IoT devices (e.g., appliances, climate controllers, lighting systems, entertainment systems, energy monitors). The voice interface 106 can provide voice output in response to voice input, which is also generated based on the language structure and meaning of the voice input determined by the NLU provided by AIVRS 102.

[0028] AIVRS 102 uses machine learning to iteratively build a knowledge base 108 corresponding to a user, based on the user's usage history or patterns with different automation systems or IoT devices. AIVRS 102 stores the knowledge base as data stored in electronic storage. The electronic storage data corresponding to a user is identified by an account and / or other credentials associated with the user. Different knowledge bases can be created for different users based on their usage patterns with automation systems and IoT devices, and this knowledge base can be stored as data in cloud-based or other server-based electronic storage. AIVRS uses voice-based recognition and verification to determine the user's unique identifier. Simultaneously or before building the knowledge base 108, AIVRS 102 uses machine learning to learn to identify the user corresponding to the knowledge base 108 based on the user's unique voice characteristics (e.g., acoustic signal patterns). In response to commands or prompts issued as human speech, AIVRS 102 (as with other AI voice response systems) identifies the user issuing the command or prompt.

[0029] AIVRS 102 includes a knowledge corpus accessor module (KCAM) 112. KCAM 112 is illustratively implemented as software executing on one or more processors of a computer system in which the compute node of AIVRS 102 is implemented. In other embodiments, KCAM 112 may be implemented in a computer system remote from but operatively coupled to AIVRS 102. In other embodiments, KCAM 112 may be implemented in dedicated circuitry or a combination of software and circuitry.

[0030] Operationally, in conjunction with other features of AIVRS 102, KCAM 112 enables users to engage in voice interaction with AIVRS 102, even though AIVRS has not yet built a knowledge corpus corresponding to that user. While AIVRS 102 does not have a knowledge base corresponding to the user that enables AVIRS 102 to recognize and execute the user's voice commands, a knowledge corpus 114 corresponding to the user exists—although constructed by and operating with AIVRS 104, it is separate from AVIRS 102. AVIRS 104 can operate on different computer systems (e.g., cloud-based servers) located away from AIVRS 102, but even when operating on the same computer system (e.g., cloud-based servers), AIVRS 104 differs from AIVRS 102. The KCAM 112, which operates in conjunction with the AIVRS 102, enables users to engage in voice interaction with the AIVRS 102 by accessing selected portions of the knowledge base 114 via the AIVRS 104 and providing those selected portions to the AVIRS 102 while maintaining the security of the knowledge base 114.

[0031] KCAM 112 detects the physical presence of a user whose knowledge base is not stored in AIVRS 102. The physical presence of a user can be detected according to one or more of the procedures described in detail below. KCAM 112 responds to the detection of the user's physical presence by causing AIVRS 102 to generate a voice request to access a separately stored knowledge base corresponding to that user. AIVRS 102 conveys the voice request to the user via voice interface 106. If the user responds affirmatively to the voice prompt received by AIVRS 102 via voice interface 106, KCAM 112 causes AIVRS 102 to access the appropriate knowledge base by instantiating an electronic communication session with another AIVRS. For example, under the guidance of KCAM 112, AIVRS 102 instantiates an electronic communication session with AIVRS 104, which contains a knowledge corpus 114.

[0032] AVIRS 102 can initiate an electronic communication session via an electronic communication connection to a portable device 116 of a user for which AVIRS 102 does not store a knowledge base. The portable device 116 can be any device carried by the user (e.g., a smartphone) or worn (e.g., a smartwatch). The portable device 116 may include example portable device 600 (…). Figure 6 One or more components of the AIVRS 104. The electronic communication connection may include a wireless connection 118 between the voice interface 106 and the user's portable device 116, through which authorization dictated to the portable device for transmission to the AIVRS 104 is wirelessly transmitted to the portable device 116. The electronic communication connection does not need to be primarily peer-to-peer. The electronic communication connection may include authorization generated by the AIVRS 104 and, for example, in response to authorization dictated to the portable device 116 by the second user, transmitted to the AIVRS 102 via a data communication network 110 (e.g., the Internet). The authorization provides the AIVRS 102 with information (e.g., IP address) for instantiating the electronic communication session.

[0033] Based on the user's voice response granting permission to access knowledge base 114, received via voice interface 106, AVIRS 102 retrieves a selected portion of knowledge base 114 from AIVRS 104 via data communication network 110. Knowledge base 114, constructed by AIVRS 104, corresponds to the user, and the selected portion received by AIVRS 102 can be a portion selected based on the user's voice response to the request. KCAM 112 enables AIVRS 102 to generate and communicate this portion via voice interface 106.

[0034] By granting licenses to selected portions of the knowledge base 114, users can use utterances recognized by AIVRS 102 based on selected portions of the knowledge base 114 obtained from AIVRS 104 to command automated systems and IoT devices controlled by AIVRS 102.

[0035] In scenarios involving two individuals in the same environment (e.g., a smart home or office), including IoT devices controlled by voice commands recognizable by AIVRS 102, for example, a knowledge base 108 may have been machine-learned to recognize commands issued by only one user. AIVRS 102 may correspond to a first user, and the knowledge base 108 has been built using machine learning to recognize the first user's voice and prompts, but may not recognize the voice and prompts of a second user. The first user may be, for example, a regular (and legitimate) user, such as the owner (or family member) of a smart home or the occupant of a smart office, where AIVRS 102 controls IoT devices by recognizing voice commands from electronically stored data in the knowledge base 108.

[0036] The second user (e.g., Bill) could be a visitor or colleague of the first user (e.g., Abel) in an environment (e.g., a smart home or office) where IoT devices (e.g., an automatic coffee machine) are voice-controlled by AIVRS 102. If the first user (Abel) wakes up AIVRS 102 with a voice prompt and issues the command “Make Bill a coffee according to Bill’s preferences,” AIVRS 102 can only accept the command if it has access to a knowledge base that includes the second user’s (Bill’s) coffee preferences. If AIVRS 102 cannot accept the command, KCAM 112 can instruct AIVRS 102 to generate a voice output conveyed through voice interface 106 stating “Sorry, I don’t know Bill’s coffee preferences,” and initiate a process for accessing the appropriate knowledge base that can respond effectively to the command.

[0037] Operationally, KCAM 112 can instruct AIVRS 102 to sense the presence of a second user, and KCAM 112 can determine whether the knowledge base corresponding to the second user is electronically stored as data in an electronic memory accessible to AIVRS 102. If so, AIVRS 102 can accept commands based on the second user's intentions and preferences, according to the electronically stored data (including the knowledge base corresponding to the second user). Otherwise, KCAM 112 causes AIVRS 102 to generate a request to access the knowledge base generated and stored by a different AIVRS and corresponding to the second user.

[0038] In some embodiments, when accessing a selected portion of knowledge base 114, AIVRS 102 accesses data (e.g., intents and preferences) based on the context of permission given via a second user's voice response to create an ontology graph tree for performing a given action (e.g., automated coffee preparation). As indicated by KCAM 112, the ontology graph tree or other knowledge graph can be created by AIVRS 102 by initially obtaining contextual data for resource authentication and by prompting AIVRS 104 for access. AIVRS 102 can extract a selected portion of knowledge base 114 from AIVRS 104 and electronically store (at least temporarily) the selected portion. AIVRS 102 can create usage patterns corresponding to the second user based on the accessed portion of knowledge base 114. In some embodiments, AIVRS 102 implements a deep learning neural network for recognizing usage patterns and intents and preferences. For example, AIVRS 102 may implement a bidirectional LSTM (a combination of forward and backward recurrent neural networks that can be used for statistical learning of dependencies between discrete steps in a time series or data sequence (e.g., words in a sentence). Bidirectional LSTM can be used with reasoning hops to include pattern history and determine contextual data for a second user.

[0039] When a second user grants access to a separately stored knowledge base 114, they can issue commands with the appropriate access permissions. The AIVRS 104 that generates the knowledge base 114 identifies the second user based on speech recognition and analyzes their voice commands. Based on the voice commands, the AIVRS 104 associated with the knowledge base 114 identifies the selected portions for which access has been granted through natural language processing. Having gained access to the selected portions, the AIVRS 102 can construct a pattern history, intents, and preferences contained within those portions, enabling it to respond to one or more voice commands from the second user (e.g., “Prepare coffee for me the way I like my coffee”) and / or one or more voice commands from the first user regarding the second user (e.g., “Prepare coffee for Bill the way he likes his coffee”).

[0040] In different embodiments, KCAM 112 can sense the physical presence of a second user in different ways. For example, as described above, a first user for whom AIVRS 102 has created a knowledge base can issue a command referencing a second user for whom AIVRS 102 has not created a knowledge base. KCAM 112 can use natural language processing to determine that the "reference" is for a name (e.g., "Bill") not associated with the currently stored knowledge base and can initiate the described knowledge base access procedure.

[0041] In other embodiments, KCAM 112 can distinguish between the voice of a first user for whom AIVRS 102 has created a knowledge base and the voice of an individual (a second user) for whom AIVRS 102 has no knowledge base available. In other embodiments, AIVRS 102 is operatively coupled to one or more IoT cameras. Based on facial recognition applied to facial images captured by one or more IoT cameras, KCAM 112 can distinguish between a first user and a second user, where the first user's facial image corresponds to someone for whom AIVRS 102 has created a knowledge base, and the second user's facial image is of someone with whom AIVRS 102 has not previously interacted.

[0042] In any scenario across various contexts, KCAM 112 can initiate the described knowledge base access process in response to sensing the absence of a user on AIVRS 102 accessing an existing knowledge base. For example, in an embodiment where presence is sensed based on facial recognition, KCAM 112 can be configured to interact with one or more IoT cameras via a representative state transfer application program interface (REST API) using a machine-to-machine communication protocol to determine visual contextual data using a fast convolutional neural network (Fast CNN). KCAM 112 initially guides AIVRS 102 to create a knowledge graph based on contextual data obtained via Fast CNN for resource authentication and to generate a selection of a separately stored knowledge base (e.g., knowledge base 114) and store (at least temporarily) hints corresponding to the user's intent and preferences for the selection. KCAM 112 can guide AIVRS 102 to create a pattern history based on an iterative application of a bidirectional-LSTM model, which is used in conjunction with inference jumps to include the corresponding user's pattern history, and based on that, intent and preferences for responding to voice commands using the selected portion.

[0043] Optionally, once AIVRS 102 has been granted permission to access a remotely located knowledge base, the user-granted permission can additionally authorize AIVRS 102 to store (e.g., on a cloud-based server) selected portions for use in subsequent sessions via voice commands. The user granting permission via voice commands can specifically specify which portions can be stored and for how long. For example, a user can specify that AIVRS 102 has permanent access to the user's coffee brewing preferences, but discards certain other intentions and preferences after a specified time (e.g., two hours). In default mode, if no retention prompt (or other voice command) is received from the user granting initial access, AIVRS 102 discards the retrieved selected portions of the knowledge base after a predetermined time interval (e.g., 24 hours).

[0044] In some embodiments, KCAM 112 instructs AIVRS 102 to initiate the generation and transmission of a knowledge base access request only after first determining that the user sensed by it is not associated with a stored knowledge base. Thus, for users who have previously authorized AIVRS 102 to store selected portions of the knowledge base, it is not necessary to instantiate a new communication session between AIVRS 102 and another AIVRS to access the remotely stored knowledge base. In other embodiments, instead of authorizing the storage of selected portions of the knowledge corpus, users may allow AIVRS 102 to store active permissions for instantiating communication sessions and obtaining access to the knowledge base (but only on a selective basis). Each session can be initiated in response to a voice command from AIVRS 102 requesting access to a remotely located knowledge base. The voice command may specify the selection of different portions of the knowledge base in each individual session.

[0045] For example, in any of the above scenarios involving a second user, the second user who has already granted AIVRS 102 access to knowledge base 114 may additionally specify that in subsequent sessions, AIVRS 102 may access the knowledge base again, but only in response to new voice prompts and only regarding the portion of the selection specified in each new prompt. Thus, in a later session, the second user may issue a new prompt that grants AIVRS 102 access to the second user's coffee brewing preferences, enabling AIVRS 102 to respond to voice commands for controlling an automatic coffee machine. In a later session, the second user may voice-grant AIVRS 102 access to knowledge base 114 again, but only regarding the portion of the selection related to the second user's music preferences, enabling AIVRS 102 to respond to new voice commands for controlling the entertainment system. However, in either case, AIVRS 102 may have permission to store the corresponding preferences (coffee or music preferences) for more than a short period to facilitate real-time or near real-time interaction.

[0046] In some embodiments, KCAM 112 may also incorporate certain security features for a first user (i.e., a user who regularly (and legitimately) interacts with AIVRS 102 and for whom AIVRS creates knowledge base 108). In one arrangement, the first user may specify that AIVRS 102 never responds to certain specified voice commands from a second user, regardless of permission granted by the second user to access their own knowledge base. For example, AIVRS 102 may be prevented from altering or exiting the security system (e.g., via separate voice commands) in addition to responding to voice commands from the first user. Furthermore, KCAM 112 may enable the first user to create trust circles. Trust circles specify (e.g., based on voice and / or facial recognition) identities and specific commands that AIVRS 102 may respond to based on who issued the voice command. Trust circles may optionally be hierarchical, thereby allowing different levels of users to issue different types of voice commands. For example, a visiting friend granted access to their own knowledge base may have a wider command dimension (e.g., first level) than a mere acquaintance (e.g., second level).

[0047] In some embodiments, KCAM 112 can search a publicly accessible knowledge base or portions thereof before sensing the presence of a user on AIVRS 102 who does not have an existing knowledge base. KCAM 112 can initiate a search in response to data indicating the future presence of an individual on AIVRS 102 who does not have an existing knowledge base. For example, KCAM 112 can be configured to search an IoT calendar system intermittently (e.g., daily). By searching the IoT calendar system or other data sources associated with the first user, KCAM 112 can obtain data (e.g., date and time) about a second user on AIVRS 102 who is expected to be near the operation of the first user's IoT device (e.g., in a smart home or office). KCAM 112 can search for the second user's publicly accessible intent and preferences, such as one or more databases of the first user (e.g., computer annotations about previous encounters with the second user) or publicly accessible sites associated with the second user's name (e.g., social networking sites, associated professional or business organization websites).

[0048] A calendar system or other data source associated with the first user can indicate the nature of the second user's future presence. For example, the calendar can specify that the first user will meet the second user socially or business-wise. KCAM 112 can initiate a search of a publicly accessible knowledge base to identify the second user's intentions and preferences regarding coffee and / or ambient temperature, such as those intentions and preferences that KCAM 112 reasonably anticipates are among the voice commands that the second user can allow to be provided to AIVRS 102. When the first user, as the host, invites the second user to issue a voice command regarding coffee or temperature, the AIVRS 102 can store the intentions and preferences to respond to the second user's voice command, which expresses a preference for coffee made by the automatic coffee machine and / or temperature settings of the central air conditioning system.

[0049] Optionally, in the absence of information from a publicly available knowledge base, KCAM 112 may prompt AIVRS 102 to generate and convey to a second user (e.g., via a cellular phone or data communication network 116) a text or other electronic message, such as an email requesting prior access to the knowledge base of the second user's AIVRS (e.g., AIVRS 104). This prior access allows AIVRS 102 to store the second user's intentions and preferences for use during upcoming interactions between the first and second users in the vicinity of operation of the voice interface 106.

[0050] Figure 2 This is a flowchart of a method 200 for securely accessing a knowledge base generated by another AIVRS according to one embodiment. See also... Figure 1 As described, method 200 can be performed by a system including a KCAM operatively coupled to or integrated therein with the first user's AIVRS. At block 202, the system electronically senses the physical presence of the second user using the first user's AIVRS.

[0051] At box 204, the system transmits a voice request generated by the first user's AIVRS to access a knowledge base electronically stored by the second user's AIVRS. The system can receive a voice response from the second user. The second user's voice response grants access to the knowledge base electronically stored by the second user's AIVRS.

[0052] At box 206, based on voice response, the system instantiates an electronic communication session with a second user's AIVRS using the first user's AIVRS. The electronic communication session can be initiated by the first user's AIVRS via an electronic communication connection with the second user's portable device. The portable device can be, for example, a smartphone, smartwatch, or other such device carried or worn by the second user. The electronic communication connection between the first user's AIVRS and the portable device may include a wireless connection between the first user's AIVRS's voice interface and the portable device. Authorization spoken by the second user to the portable device for transmitting AIVRs to the second user can be wirelessly transmitted to the voice interface. The electronic communication connection may include, for example, authorization transmitted by the portable device to the second user's AIVRS and the first user's AIVRS via a data communication network (e.g., the Internet). The authorization provides the first user's AIVRS with network information (e.g., IP address) required for instantiating the electronic communication session.

[0053] At box 208, the system retrieves a selected portion of the knowledge base from the AIVRS of the second user via a data communication network using the AIVRS of the first user. The selected portion of the knowledge base is chosen based on the voice response of the second user to the voice request transmitted by the AIVRS of the first user.

[0054] At box 210, the system responds to a voice command (or other voice prompt) to initiate one or more actions by one or more IoT devices. The voice command may be issued by a second user or by the first user referring to (mentioning) the second user. The voice command is interpreted by the first user's AIVRS based on a selection from a knowledge base.

[0055] In some embodiments, the system transmits a voice request in response to a first user's AIVRS searching data stored in an electronic storage device and determining that the data stored in the electronic storage device lacks prior licenses for the electronic storage device and a selected portion of the knowledge base. In response to determining that the data stored in the electronic storage device includes prior licenses for the electronic storage device and receiving a response to a request to access the knowledge base, the system retrieves the selected portion of the knowledge base via a data communication network. In response to determining that the data stored in the electronic storage device already includes the selected portion of the knowledge base, the system avoids retrieving the selected portion via the data communication network.

[0056] In some embodiments, the system perceives a second user by recognizing the first user's voice prompts using the first user's AIVRS and by recognizing references to a second user using the first user's AIVRS, the identity of the second user being previously unknown to the first user's AIVRS. Additionally or alternatively, in some embodiments, the system perceives the second user by performing visual recognition of the second user based on images captured by an IoT camera operatively coupled to the first user's AIVRS.

[0057] In some embodiments, the system may use a first user's AIVRS to acquire second user information by capturing data from the first user's device, which is communicatively coupled to the first user's AIVRS. The data may correspond to the second user's intent and preferences (e.g., coffee brewing preferences) that the first user's AIVRS can use to respond to the second user's voice commands. For example, the first user's device may be a computer storing email exchanges between the first and second users, where the second user describes their preferred coffee preferences. Alternatively, the first user's device may be a smartphone storing text messages where the second user describes their favorite music genres. The first user's AIVRS can use this data to build its own knowledge base for the second user, and the first user's AIVRS can use this data to respond to voice commands requesting an automatic coffee machine to prepare coffee for the second user or voice commands from the second user requesting an entertainment system to play music.

[0058] The system can discard portions of the knowledge base selected by the second user and received by the first user's AIVRS. If a predetermined time interval elapses and the system does not receive a retention prompt from the second user, the system can automatically discard the selected portions. If the second user wishes to electronically store the selected portions in the first user's AIVRS, the second user can verbally specify the duration for which the selected portions are electronically stored.

[0059] It should be clearly noted that while this disclosure includes a detailed description of cloud computing, the implementation of the teachings set forth herein is not limited to cloud computing environments. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0060] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services), which can be rapidly provisioned and released with minimal management effort or interaction with the service provider. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0061] The features are as follows:

[0062] On-demand self-service: Cloud consumers can unilaterally and automatically provide computing power, such as server time and network storage, as needed, without requiring human interaction with the service provider.

[0063] Extensive network access: Capabilities are available through networks and accessed via standard mechanisms that facilitate the use of heterogeneous thin client or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0064] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically assigned and reassigned as needed. There is a sense of location independence because consumers typically do not have control or knowledge of the exact location of the resources provided, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0065] Rapid flexibility: The ability to provide capacity quickly and flexibly, automatically scaling down and up rapidly in some situations to scale up rapidly. For consumers, the available supply capacity often appears unlimited and can be purchased in any quantity at any time.

[0066] Measuring services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both service providers and consumers.

[0067] The service model is as follows:

[0068] Software as a Service (SaaS): This provides consumers with the ability to use the provider's applications running on cloud infrastructure. Applications can be accessed from different client devices via thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating system, storage, or even individual application capabilities, with possible exceptions such as limited user-specific application configuration settings.

[0069] Platform as a Service (PaaS): This provides consumers with the ability to deploy applications created or acquired by the consumer using programming languages ​​and tools supported by the provider onto cloud infrastructure. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they have control over the deployed applications and the configuration of any application hosting environment.

[0070] Infrastructure as a Service (IaaS): This provides consumers with the capability to deliver processing, storage, networking, and other basic computing resources that enable them to deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but rather have control over the operating system, storage, deployed applications, and potentially limited control over selected networking components (e.g., host firewalls).

[0071] The deployment model is as follows:

[0072] Private cloud: A cloud infrastructure that operates solely for an organization. It can be managed by the organization or a third party and can exist on-site or off-site.

[0073] Community cloud: A cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It can be managed by an organization or a third party and can exist on-site or off-site.

[0074] Public cloud: Makes cloud infrastructure available to the public or large industry groups and is owned by an organization that sells cloud services.

[0075] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardized or proprietary technologies that enable data and applications to be ported (e.g., cloud bursting for load balancing between clouds).

[0076] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure comprising a network of interconnected nodes.

[0077] See now Figure 3 This describes an illustrative cloud computing environment 300. As shown, the cloud computing environment 300 includes one or more cloud computing nodes 310 that can communicate with local computing devices used by cloud consumers, such as personal digital assistants (PDAs) or cellular phones 340a, desktop computers 340b, laptop computers 340c, and / or automotive computer systems 340n. The computing nodes 310 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 300 to provide infrastructure, platforms, and / or software as services that cloud consumers do not need to maintain for them on their local computing devices. It should be understood that... Figure 3The types of computing devices 340a-n shown are intended to be illustrative only, and computing node 310 and cloud computing environment 300 can communicate with any type of computerized device via any type of network and / or network-addressable connection (e.g., using a web browser).

[0078] Now for reference Figure 4 This demonstrates the 300 (cloud computing environment) Figure 3 This provides a set of functional abstractions. It should be understood beforehand. Figure 4 The components, layers, and functions shown are intended to be illustrative only, and embodiments of the invention are not limited thereto. As described, the following layers and corresponding functions are provided:

[0079] The hardware and software layer 460 includes hardware and software components. Examples of hardware components include a host 461; a server 462 based on a RISC (Reduced Instruction Set Computer) architecture; a server 463; a blade server 464; a storage device 465; and a network and network components 466. In some embodiments, software components include network application server software 467 and database software 468.

[0080] The virtualization layer 470 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 471; virtual storage 472; virtual network 473, including virtual private network; virtual application and operating system 474; and virtual client 475.

[0081] In one example, management layer 480 provides the following functionalities: Resource Provisioning 481 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and Pricing 482 provides cost tracking as resources are utilized within the cloud computing environment and bills or invoices for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User Portal 483 provides access to the cloud computing environment for consumers and system administrators. Service Level Management 484 provides cloud resource allocation and management to ensure that required service levels are met. Service Level Agreement (SLA) Planning and Fulfillment 485 provides pre-scheduling and procurement of cloud resources based on anticipated future needs according to the SLA.

[0082] Workload layer 490 provides examples of functionalities that can leverage a cloud computing environment. Examples of workloads and functionalities that can be provided from this layer include: mapping and navigation 491; software development and lifecycle management 492; virtual classroom education delivery 493; data analytics and processing 494; transaction processing 495; and AIVRS 496, which integrates with or is operationally coupled to KCMA.

[0083] Figure 5 A schematic diagram of an instance of compute node 500 is shown. In one or more embodiments, compute node 500 is an example of a suitable cloud computing node. Compute node 500 is not intended to impose any limitation on the use or scope of functionality of the embodiments of the invention described herein. Compute node 500 is capable of performing any of the functions described in this disclosure.

[0084] Computing node 500 includes computer system 512, which can operate with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with computer system 512 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices.

[0085] Computer system 512 can be described in the general context of computer system executable instructions (such as program modules) executed by the computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. Computer system 512 can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked via a communication network. In a distributed cloud computing environment, program modules can reside in local and remote computer system storage media, including memory storage devices.

[0086] like Figure 5 As shown, computer system 512 is illustrated as a general-purpose computing device. Components of computer system 512 may include, but are not limited to, one or more processors 516, memory 528, and a bus 518 coupling various system components, including memory 528, to processor 516. As defined herein, a “processor” means at least one piece of hardware circuitry configured to execute instructions. Hardware circuitry may be an integrated circuit. Examples of processors include, but are not limited to, central processing units (CPUs), array processors, vector processors, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), application-specific integrated circuits (ASICs), programmable logic circuits, and controllers.

[0087] Instructions by which a processor executes a computer program include executing or running the program. As defined herein, “running” and “executing” include a series of actions or events performed by the processor according to one or more machine-readable instructions. As defined herein, “running” and “executing” refer to actions or events actively performed by the processor. The terms run, running, execute, and executing are used synonymously herein.

[0088] Bus 518 represents one or more of several types of bus architectures, including memory buses or memory controllers, peripheral buses, accelerated graphics ports, and processor or local buses using any of the various bus architectures. As an example only and not a limitation, such architectures include the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, the Enhanced ISA (EISA) bus, the Video Electronics Standards Association (VESA) local bus, the Peripheral Component Interconnect (PCI) bus, and the PCI Express (PCIe) bus.

[0089] Computer system 512 typically includes various computer system readable media. Such media can be any available media accessible by computer system 512, and can include volatile and non-volatile media, removable and non-removable media.

[0090] Memory 528 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 530 and / or cache memory 532. Computer system 512 may further include other removable / non-removable, volatile / non-volatile computer system storage media. For example, storage system 534 may be provided for reading and writing from non-removable non-volatile magnetic media and / or solid-state drives (not shown, and generally referred to as "hard disk drives"). Although not shown, disk drives for reading from or writing to removable non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable non-volatile optical disks (such as CD-ROMs, DVD-ROMs, or other optical media) may be provided. In such cases, each may be connected to bus 518 via one or more data media interfaces. As will be further described and depicted below, memory 528 may include at least one program product having at least one set of program modules configured to perform embodiments of the invention.

[0091] A program / utility 540 having a set (at least one) of program modules 542, along with an operating system, one or more applications, other program modules, and program data, may be stored in memory 528 in an exemplary, and not limited, manner. Each or some combination of the operating system, one or more applications, other program modules, and program data may include an implementation of a network environment. Program modules 542 typically perform functions and / or methods as described herein in embodiments of the invention. For example, one or more of the program modules may include AIVRS 596, portions thereof, wherein KCMA is integrated into or operatively coupled to AIVRS 596.

[0092] Program / utility 540 may be executed by processor 516. Program / utility 540 and any data items used, generated, and / or manipulated by computer system 512 are functional data structures that are given functionality when used by computer system 512. As defined within this disclosure, a "data structure" is a physical implementation of the organization of data in a data model within physical memory. Accordingly, a data structure is formed by specific electrical or magnetic structural elements in memory. The data structure imposes a physical organization on data stored in memory, as used by an application program executed using the processor.

[0093] Computer system 512 can also communicate with one or more external devices 514, such as a keyboard, pointing device, display 524, etc.; one or more devices that enable a user to interact with computer system 512; and / or any device that enables computer system 512 to communicate with one or more other computing devices (e.g., network interface card, modem, etc.). Such communication can occur via input / output (I / O) interface 522. Furthermore, computer system 512 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via network adapter 520. As shown, network adapter 520 communicates with other components of computer system 512 via bus 518. It should be understood that, although not shown, other hardware and / or software components can be used in conjunction with computer system 512. Examples include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archiving storage systems.

[0094] Although compute node 500 is used to illustrate an instance of a cloud computing node, it should be understood that its use in conjunction with [other technologies] is not appropriate. Figure 5Computer systems with the same or similar architectures described herein can be used in non-cloud computing implementations to perform the different operations described herein. In this respect, the exemplary embodiments described herein are not intended to be limited to cloud computing environments. Compute node 500 is an example of a data processing system. As defined herein, a “data processing system” means one or more hardware systems configured for processing data, each hardware system including at least one processor and memory programmed to initiate operations.

[0095] Compute node 500 is an example of computer hardware. Depending on the specific type of device and / or system implemented, compute node 500 may include more than Figure 5 Fewer components are shown, or additional components are not shown. The specific operating system and / or application included may vary depending on the device and / or system type, and the type of I / O devices included may also vary. Furthermore, one or more of the illustrative components may be incorporated into another component or otherwise form part of another component. For example, a processor may include at least some memory.

[0096] Compute node 500 is also an example of a server. As defined herein, a "server" refers to a data processing system configured to share services with one or more other data processing systems. As defined herein, a "client device" refers to a data processing system that requests shared services from a server and with which a user interacts directly. Examples of client devices include, but are not limited to, workstations, desktop computers, computer terminals, mobile computers, laptop computers, netbook computers, tablet computers, smartphones, personal digital assistants, smartwatches, smart glasses, gaming devices, set-top boxes, smart TVs, etc. In one or more embodiments, the various user devices described herein may be client devices. Network infrastructure (such as routers, firewalls, switches, access points, etc.) is not a client device because the term "client device" is defined herein.

[0097] Figure 6 An example portable device 600 according to one or more embodiments described in this disclosure is shown. The portable device 600 may include a memory 602, one or more processors 604 (e.g., an image processor, a digital signal processor, a data processor), and interface circuitry 606.

[0098] In one aspect, the memory 602, processor(s) 604, and / or interface circuitry 606 are implemented as separate components. In another aspect, the memory 602, processor(s) 604, and / or interface circuitry 606 are integrated into one or more integrated circuits. The different components of the portable device 600 may be coupled, for example, via one or more communication buses or signal lines (e.g., interconnects and / or wires). In one aspect, the memory 602 may be coupled to the interface circuitry 606 via a memory interface (not shown).

[0099] Sensors, devices, subsystems, and / or input / output (I / O) devices may be coupled to interface circuitry 606 to facilitate the functions and / or operations described herein, including the generation of sensor data. Different sensors, devices, subsystems, and / or I / O devices may be coupled to interface circuitry 606 directly or via one or more intermediate I / O controllers (not shown).

[0100] For example, position sensor 610, light sensor 612, and proximity sensor 614 can be coupled to interface circuitry 606 to facilitate orientation, illumination, and proximity functions of portable device 600, respectively. Position sensor 610 (e.g., a GPS receiver and / or processor) can be connected to interface circuitry 606 to provide geolocation sensor data. Electronic magnetometer 618 (e.g., an integrated circuit chip) can be connected to interface circuitry 606 to provide sensor data that can be used to determine magnetic north for directional navigation purposes. Accelerometer 620 can be connected to interface circuitry 606 to provide sensor data that can be used to determine changes in the device's three-dimensional movement speed and direction. Altimeter 622 (e.g., an integrated circuit) can be connected to interface circuitry 606 to provide sensor data that can be used to determine altitude. Voice recorder 624 can be connected to interface circuitry 606 to store recorded speech.

[0101] Camera subsystem 626 may be coupled to optical sensor 628. Optical sensor 628 may be implemented using any of a variety of technologies. Examples of optical sensor 628 include charge-coupled device (CCD), complementary metal-oxide-semiconductor (CMOS) optical sensors, etc. Camera subsystem 626 and optical sensor 628 may be used to facilitate camera functions such as recording images and / or video clips (hereinafter referred to as "image data"). In one aspect, image data is a subset of sensor data.

[0102] Communication functionality can be facilitated by one or more wireless communication subsystems 630. The wireless communication subsystems 630 may include radio frequency receivers and transmitters, optical (e.g., infrared) receivers and transmitters, etc. The specific design and implementation of the wireless communication subsystem 630 may depend on the specific type of portable device 600 implemented and / or the communication network on which the portable device 600 is intended to operate.

[0103] For illustrative purposes, the wireless communication subsystem 630 may be designed to operate over one or more mobile networks (e.g., GSM, GPRS, EDGE), Wi-Fi networks including WiMax networks, short-range wireless networks (e.g., Bluetooth networks), and / or any combination of the foregoing. The wireless communication subsystem 630 may implement a managed protocol that allows the portable device 600 to be configured as a base station for other wireless devices.

[0104] The audio subsystem 632 can be coupled to the speaker 634 and the microphone 636 to facilitate voice-enabled functions such as speech recognition, speech duplication, digital recording, audio processing, and telephone functions. The audio subsystem 632 is capable of generating audio-type sensor data. In one or more embodiments, the microphone 636 can be used as a respirator sensor.

[0105] I / O device 638 may be coupled to interface circuitry 606. Examples of I / O devices 638 include, for example, display devices, touch-sensitive display devices, trackpads, keyboards, pointing devices, communication ports (e.g., USB ports), network adapters, buttons, or other physical controls. Touch-sensitive devices (such as displays and / or pads) are configured to detect contact, movement, contact interruption, etc., using any of a variety of touch-sensitive technologies. Example touch-sensitive technologies include, for example, capacitive, resistive, infrared, and surface acoustic wave technologies, other proximity sensor arrays, or other elements for determining one or more points of contact with the touch-sensitive device. One or more of I / O devices 638 may be adapted to control sensors, subsystems, and such functions of portable device 600.

[0106] Portable device 600 further includes a power source 640. Power source 640 is capable of providing power to various components of portable device 600. In one embodiment, power source 640 is implemented as one or more batteries. The batteries can be implemented using any of a variety of different battery technologies, whether disposable (e.g., replaceable) or rechargeable. In another embodiment, power source 640 is configured to draw power from an external source and provide power (e.g., DC power) to components of portable device 600. In the case of rechargeable batteries, power source 640 may also include circuitry capable of charging the battery or multiple batteries when coupled to an external power source.

[0107] Memory 602 may include random access memory (e.g., volatile memory) and / or non-volatile memory, such as one or more disk storage devices, one or more optical storage devices, flash memory, etc. Memory 602 may store operating system 652, such as LINUX, UNIX, mobile operating system, embedded operating system, etc. Operating system 652 may include instructions for processing system services and for performing hardware-related tasks.

[0108] Memory 602 may store additional program code 654. Examples of other program code 654 may include instructions for facilitating communication with one or more additional devices, one or more computers and / or one or more servers; for facilitating sensor-related functions; telephone-related functions; electronic messaging-related functions; web browsing-related functions; media processing-related functions; GPS and navigation-related functions; security functions; camera-related functions, including webcam and / or web video functions; and so on. The program code may contain program code for implementing an intelligent virtual assistant (IVA) in a portable device, the IVA being implemented in IVA program code 656 that executes on processor 604. The IVA may interact with, for example, an AIVRS that executes on a remote cloud-based server. Memory 602 may also store one or more other applications 662.

[0109] For illustrative and not limiting purposes, the different types of instructions and / or program code described are provided. The program code may be implemented as a separate software program, process, or module. Memory 602 may include additional instructions or fewer instructions. Furthermore, different functions of the portable device 600 may be implemented in hardware and / or software, including in one or more signal processing and / or application-specific integrated circuits.

[0110] The program code stored in memory 602 and any data used, generated, and / or manipulated by portable device 600 are functional data structures that give functionality to the device when used as part of the device. Other examples of functional data structures include, for example, sensor data, data obtained via user input, data obtained by querying external data sources, baseline information, etc. The term "data structure" refers to the physical implementation of the organization of data in a data model within physical memory. Accordingly, a data structure is formed by specific electrical or magnetic structural elements within memory. Data structures impose a physical organization on data stored in memory for use by the processor.

[0111] In some embodiments, one or more of the various sensors and / or subsystems described with reference to portable device 600 may be separate devices coupled to or communicatively linked to portable device 600 via wired or wireless connections. For example, one or more (or all) of position sensor 610, light sensor 612, proximity sensor 614, gyroscope 616, magnetometer 618, accelerometer 620, altimeter 622, recorder 624, camera subsystem 626, audio subsystem 632, etc., may be implemented as separate systems or subsystems operatively coupled to portable device 600 via I / O device 638 and / or wireless communication subsystem 630.

[0112] Depending on the specific type of system implemented, the portable device 600 may include more than Figure 6 The components shown are fewer or include, in addition to Figure 6 Additional components beyond those shown. Furthermore, the specific operating system and / or application and / or other program code included may vary depending on the system type. Additionally, one or more of the illustrative components may be incorporated into another component or otherwise formed part of another component. For example, a processor may include at least some memory.

[0113] Portable device 600 is provided for illustrative and not limiting purposes. Devices and / or systems configured to perform the operations described herein may have the same characteristics as... Figure 6 The different architectures shown herein. This architecture may be a simplified version of portable device 600 and may include a processor and a memory for storing instructions. The architecture may include one or more sensors as described herein. Portable device 600 or similar systems may use different sensors of the device or sensors coupled thereto to collect data. However, it should be understood that portable device 600 may include fewer sensors or other additional sensors. In this disclosure, data generated by sensors is referred to as “sensor data”.

[0114] Example implementations of portable device 600 include, for example, smartphones or other mobile devices or telephones capable of sensing and processing sensor-detected signals and data, wearable computing devices (e.g., smartwatches), specialized medical devices, or other suitable handheld, wearable, or comfortably portable electronic devices. It should be understood that the embodiments may be deployed as standalone devices or as multiple devices in a distributed client-server networking system. For example, in some embodiments, a smartwatch may be operatively coupled to a mobile device (e.g., a smartphone). The mobile device may or may not be configured to interact with remote servers and / or computer systems.

[0115] The terminology used herein is for the purpose of describing particular implementations only and is not intended to be limiting. Nevertheless, several definitions that apply throughout this document will now be presented.

[0116] As defined in this article, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” also include the plural forms.

[0117] As defined in this article, "another" means at least the second or more.

[0118] As defined herein, “at least one,” “one or more,” and “and / or” are open-ended expressions that are both combined and separate in operation unless otherwise expressly stated. For example, each of the expressions “at least one of A, B, and C,” “at least one of A, B, or C,” “one or more of A, B, and C,” “one or more of A, B, or C,” and “A, B, and / or C” represents a single A, a single B, a single C, A and B, A and C, B and C, or A, B, and C.

[0119] As defined in this article, “automatically” means without user intervention.

[0120] As defined herein, “includes,” “including,” “comprises,” and / or “comprising” specify the presence of the stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or combinations thereof.

[0121] As defined herein, depending on the context, “if” means “in response to” or “responsive to”. Therefore, depending on the context, the phrase “if determined” can be interpreted as “in response to determining” or “responsive to determining”. Similarly, depending on the context, the phrase “if [the stated condition or event] is detected” can be interpreted as “in response to detecting [the stated condition or event]” or “responsive to detecting [the stated condition or event]”.

[0122] As defined herein, “an embodiment,” “an embodiment,” “in one or more embodiments,” “in a particular embodiment,” or similar language means a specific feature, structure, or characteristic described in connection with that embodiment, which is included in at least one embodiment described in this disclosure. Therefore, the appearance of the above phrases and / or similar language throughout this disclosure may, but does not necessarily refer to the same embodiment.

[0123] As defined herein, the phrases “in response to” and “responsive to” refer to readily responding to or reacting to an action or event. Therefore, if a second action is performed in response to or in response to a first action, a causal relationship exists between the occurrence of the first action and the occurrence of the second action. The phrases “in response to” and “responsive to” indicate a causal relationship.

[0124] As defined in this article, "real-time" refers to a level of responsiveness that the user or system senses for a specific process to be performed or determines is sufficiently immediate, or that enables the processor to keep up with the responsiveness of certain external processes.

[0125] As defined herein, “substantially” means that the listed characteristics, parameters, or values ​​need not be achieved precisely, but rather that deviations or variations (including, for example, tolerances, measurement errors, measurement accuracy limitations, and other factors known to those skilled in the art) may occur in a quantity that does not preclude the effects that the characteristics are intended to provide.

[0126] As defined in this article, “user,” “individual,” and “visitor” each refer to a human being.

[0127] In this document, the terms first, second, etc., are used to describe various elements. These elements should not be limited by these terms, because unless otherwise stated or clearly indicated by the context, these terms are only used to distinguish one element from another.

[0128] This invention can be a system, method, and / or computer program product with any possible level of technical detail integration. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to execute aspects of the invention.

[0129] Computer-readable storage media can be tangible means for retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital universal disk (DVD), memory sticks, floppy disks, mechanical encoding devices such as punched cards or protrusions in slots having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0130] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network), or to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0131] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute computer-readable program instructions by utilizing state information from the computer-readable program instructions to personalize the electronic circuitry in order to perform aspects of this invention.

[0132] The present invention will now be described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0133] These computer-readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that the computer-readable storage medium storing the instructions includes an article of manufacture containing instructions that implement aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0134] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce computer-implemented processing, such that the instructions executed on the computer, other programmable apparatus, or other device perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the figures. For example, two blocks shown consecutively may actually be completed as a single step, executed simultaneously, substantially simultaneously, or with partial or complete temporal overlap, or the blocks may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0136] Various embodiments of the invention have been described for illustrative purposes, and these descriptions are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or technical improvements to technologies found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. A computer-implemented method comprising: electronically perceiving, using an artificial intelligence (Al) voice responsive system of a first user, a physical presence of a second user; communicating a voice request generated by the Al voice responsive system of the first user, the voice request requesting access to a knowledge base electronically stored by an Al voice responsive system of the second user, and receiving a voice response from the second user; based on the voice response, instantiating, by the Al voice responsive system of the first user, an electronic communication session with the Al voice responsive system of the second user, the electronic communication session initiated by the Al voice responsive system of the first user via an electronic communication connection with a portable device of the second user; retrieving, from the Al voice responsive system of the second user via a data communication network, a portion of the knowledge base selected by the Al voice responsive system of the first user, the selected portion of the knowledge base selected based on the voice response of the second user to the voice request communicated by the Al voice responsive system of the first user; and initiating an action of one or more IoT devices in response to a voice prompt interpreted by the Al voice responsive system of the first user based on the selected portion of the knowledge base.

2. The method of claim 1, wherein the communicating is performed in response to the Al voice responsive system of the first user retrieving electronically stored data and determining that the electronically stored data lacks an electronically stored prior permission and the selected portion of the knowledge base.

3. The method of claim 1, further comprising: in response to determining that the electronically stored data includes an electronically stored prior permission, performing retrieval of the selected portion of the knowledge base via the data communication network; and in response to determining that the electronically stored data includes the selected portion of the knowledge base, refraining from retrieval of the selected portion via the data communication network. the perceiving includes recognizing, by the Al voice responsive system of the first user, a voice prompt of the first user, and recognizing, by the Al voice responsive system of the first user, a reference to the second user.

4. The method of claim 1, wherein, the perceiving includes performing visual recognition of the second user based on images captured by an IoT camera operatively coupled to the Al voice responsive system of the first user.

5. The method of claim 1, wherein, 6. The method of claim 1, further comprising obtaining, by the Al voice responsive system of the first user, second user information based on data obtained by the Al voice responsive system of the first user from a device of the first user communicatively coupled to the Al voice responsive system of the first user. in the absence of receiving a hold prompt from the second user, the Al voice responsive system of the first user discarding the retrieved selected portion of the knowledge base after a predetermined time interval elapses.

7. The method of claim 1, wherein, 8. A system comprising: an artificial intelligence (Al) voice responsive system of a first user operatively coupled with a processor configured to initiate operations including: electronically perceiving a physical presence of a second user; ​ transmitting a voice request accessing a knowledge base electronically stored by an AI voice response system of the second user and receiving a voice response from the second user; instantiating, by the AI voice response system of the first user, an electronic communication session with the AI voice response system of the second user based on the voice response, the electronic communication session initiated via an electronic communication connection with a portable device of the second user; retrieving, from the AI voice response system of the second user via a data communication network, a selected portion of the knowledge base, the selected portion of the knowledge base selected based on the voice response of the second user to the request; and initiating an action of one or more IoT devices in response to a voice prompt interpreted by the AI voice response system of the first user based on the selected portion of the knowledge base.

9. The system of claim 8, wherein the transmitting is performed in response to searching data stored in electronic memory and determining that the data stored in electronic memory lacks a previously stored electronic permission and the selected portion of the knowledge base.

10. The system of claim 8, wherein the processor is configured to initiate further operations comprising: in response to determining that data stored in electronic memory includes a previously stored electronic permission, performing retrieving the selected portion of the knowledge base via the data communication network; and in response to determining that the data stored in electronic memory includes the selected portion of the knowledge base, refraining from retrieving the selected portion via the data communication network.

11. The system of claim 8, wherein the perceiving includes recognizing a first user voice prompt and recognizing a reference to the second user within the voice prompt.

12. The system of claim 8, wherein the perceiving includes performing visual recognition of the second user based on images acquired by a camera operatively coupled to the system.

13. The system of claim 8, wherein the processor is configured to initiate further operations comprising acquiring second user information based on data captured by a device of the first user communicatively coupled to the system.

14. A computer program product, the computer program product comprising: one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions executable by a processor to cause the processor to initiate operations comprising: electronically perceiving a physical presence of a second user; transmitting a voice request accessing a knowledge base electronically stored by an AI voice response system of the second user and receiving a voice response from the second user; instantiating, by the AI voice response system of the first user, an electronic communication session with the AI voice response system of the second user based on the voice response, the electronic communication session initiated via an electronic communication connection with a portable device of the second user; retrieving, from the AI voice response system of the second user via a data communication network, a selected portion of the knowledge base, the selected portion of the knowledge base selected based on the voice response of the second user to the request; and initiating an action of one or more IoT devices in response to a voice prompt interpreted by the AI voice response system of the first user based on the selected portion of the knowledge base. retrieving, via a data communication network, a selected portion of the knowledge base from an AI voice response system of the second user, the selected portion of the knowledge base being selected based on a voice response of the second user to the request; and initiating an action of one or more IoT devices in response to a voice prompt interpreted by the AI voice response system of the first user based on the selected portion of the knowledge base.

15. The computer program product of claim 14, wherein the transmitting is performed in response to searching the electronically stored data and determining that the electronically stored data lacks an electronically stored prior permission and the selected portion of the knowledge base.

16. The computer program product of claim 14, wherein the program instructions are executable by the processor to cause the processor to initiate operations further comprising: in response to determining that the electronically stored data includes an electronically stored prior permission, performing the retrieving the selected portion of the knowledge base via the data communication network; and in response to determining that the electronically stored data includes the selected portion of the knowledge base, refraining from retrieving the selected portion via the data communication network.

17. The computer program product of claim 14, wherein the perceiving includes recognizing a first user voice prompt and recognizing a reference to the second user within the voice prompt.

18. The computer program product of claim 14, wherein, the perceiving includes performing visual recognition of the second user based on images acquired by a camera operatively coupled to an AI voice response system of another user.

19. The computer program product of claim 14, wherein, the program instructions are executable by the processor to cause the processor to initiate operations further comprising: acquiring second user information based on data acquired by a first user device communicatively coupled to the AI voice response system of the first user.

20. The computer program product of claim 14, wherein the program instructions are executable by the processor to cause the processor to initiate operations further comprising: discarding the retrieved selected portion of the knowledge base after a predetermined time interval elapses without receiving a hold prompt from the second user.

Images