Dynamic function discovery and execution at separate computing devices

Abstract

Systems and methods are provided for implementing dynamic function discovery and execution using agentic artificial intelligence (AI) models. One system may include a processor of a first computing device. The processor may detect a second computing device separate from the first computing device, where the second computing device is configured to perform a function. The processor may register, at the first computing device, the function as available to the first computing device. The processor may receive, at the first computing device, a user query to perform an action. The processor may determine to execute the function for performance of the action. The processor may transmit, to the second computing device, a function call to invoke execution of the function at the second computing device.

Description

86362115DYNAMIC FUNCTION DISCOVERY AND EXECUTION AT SEPARATE COMPUTING DEVICES BACKGROUND

[0001] A virtual digital assistant (or a virtual entity) may be a software agent that can perform a range of tasks or services for a user. A virtual digital assistant may perform tasks related to, e.g., Internet of Things (loT) device control, scheduling, reminders, communication, entertainment, navigation, information, shopping, etc. The user may interact or engage with the virtual digital assistant by providing user input (e.g., commands, questions, etc.) and the virtual assistant may perform a corresponding task based on the user input.

[0002] The discussion above is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The following drawings are provided to help illustrate various features of examples of the disclosure and are not intended to limit the scope of the disclosure or exclude alternative implementations.

[0004] FIG. 1A schematically illustrates a system for implementing dynamic function discovery and execution according to some examples.

[0005] FIG. IB schematically illustrates another system for implementing dynamic function discovery and execution according to some examples.

[0006] FIG. 2 schematically illustrates an example computing device according to some examples.

[0007] FIG. 3 illustrates an example JSON payload that represents available functions for a given computing device according to some examples.

[0008] FIG. 4 schematically illustrates an example arrangement that includes a first computing device and a second computing device according to some examples.

[0009] FIG. 5 schematically illustrates an example scenario involving the first computing device and the second computing device of FIG. 4 according to some examples.

[0010] FIG. 6 is a flowchart illustrating a method for implementing dynamic function discovery' and execution according to some examples.

[0011] FIG. 7 is a sequence diagram illustrating a discovery , registration, and execution phase according to some examples.QB\95047131.5 186362115DETAILED DESCRIPTION OF THE PRESENT DISCLOSURE

[0012] As described above, a virtual digital assistant (or a virtual entity) may be a software agent that can perform a range of tasks or services for a user. A virtual digital assistant may perform tasks related to, e.g., Internet of Things (loT) device control, scheduling, reminders, communication, entertainment, navigation, information, shopping, etc. The user may interact or engage with the virtual digital assistant by providing user input (e.g.. commands, questions, etc.) and the virtual digital assistant may perform a corresponding task based on the user input. A virtual digital assistant (or a virtual entity) may also be referred to herein as an agent or an Al agent.

[0013] Virtual entities (or Al agents) may be implemented using (or otherwise including) a large language model (LLM). The development of large language models (LLMs) has unlocked a range of new and powerful applications. For instance, LLMs can be equipped with specialized tools, such as, e.g., code functions or APIs, to solve specific and complex problems more effectively. More complex workflows, sometimes known as “agentic workflows,” may¬ use multiple coordinated agents, each with distinct tools and capabilities, to handle more complex tasks effectively.

[0014] The technology disclosed herein provides an architecture to enable the discovery and execution of remote tools (or functions) for agents running on different computing devices. The technology disclosed herein may allow local agents running on a given computing device to be equipped with new tools (or functions) based on other computing devices (e.g., other nearby computing devices), which may enhance user experience.

[0015] Hardware constraints of a computing device, whether memory storage, processing pow er, and / or another constraint, limit an ability- to store and run a quantity- and / or ty pe of Al agents on-device that a particular user may desire. Further, the particular types of Al agents that a user desires can change over time, depending on a circumstance (e.g., time, location, activity ) of a user. The technology disclosed herein provides a technical solution to this technical problem by augmenting specific agent capabilities via tools offered by computing devices in proximity of the current computing device. Additionally, the technology disclosed herein provides an enhanced user experience when w orking across multiple computing devices, such as, e.g., in the context of hybrid work or Al applications. Further, the technology disclosed herein streamlines multi-device workflows by dynamically presenting relevant tools and features, reducing setup time, enabling users to focus on tasks without manual device coordination, etc.QB\95047131.5 286362115

[0016] Accordingly, in some examples, the technology disclosed herein implements a proximity-based framework for dynamic function (or tool) discovery and execution using agentic Al models (or Al agents). The technology disclosed herein provides various technical solutions and advantages, as described herein. As one example, the technology disclosed herein allows seamless integration between computing devices, including, e.g., computing devices across different manufacturers or providers. The technology disclosed herein creates a unified and collaborative ecosystem by enabling a computing device of a first manufacturer to offer functionalities to computing devices of second, different manufacturer.

[0017] In some configurations, the technology disclosed herein may implement an Al agent as a central hub to manage registered tools from nearby computing devices. As such, a user may no longer interact with each computing device individually, streamlining operations and making tasks more intuitive. Additionally, the technology disclosed herein may facilitate improved discovery and integration of new computing devices or tools with no or minimal additional setup, making the technology disclosed herein advantageously scalable. The technology disclosed herein may augment the user's experience by allowing the integration with additional devices and customizing the user’s experience.

[0018] FIG. 1A illustrates a system 100 for implementing dynamic function discovery and execution according to some examples. As illustrated in the example of FIG. 1A, the system 100 can include at least one computing device 110. In some examples, the system 100 can include fewer, additional, or different components in different configurations than illustrated in FIG. 1 A. For example, as illustrated, the system 100 includes three computing devices 110 (e.g., a first computing device 110A, a second computing device HOB, and aNthcomputing device 110Nth). However, in some examples, the system 100 can include fewer, different, or additional computing devices 110. As another example, components of the system 100 can be combined into a single device, divided among multiple devices, or a combination thereof.

[0019] The computing device(s) 110 can communicate over wired or wireless communication networks 130. Portions of the communication network(s) 130 can be implemented using a wide area network, such as the Internet, a local area network, such as a Bluetooth™ network or Wi-Fi. and combinations or derivatives thereof. In some examples, the communication network(s) 130 represents a direct wireless link between two components of the system 100 (e.g., via a Bluetooth™ or Wi-Fi link). Alternatively, or in addition, in some examples, two or more components of the system 100 can communicate through an intermediary device of the communication network 130 not illustrated in FIG. 1 A.QB\95047131.5 386362115

[0020] The computing device(s) 110 can include, e.g., a desktop computer, a laptop computer, a tablet computer, an all-in-one computer, a notebook computer, a terminal, a smart telephone, a smart television, a smart speaker, a smart imaging device, a smart wearable, or another suitable computing device that interfaces with a user. As described in greater detail herein, the computing device(s) 110 may be used by a user for interacting with a collaboration or communication platform, such as, e.g.. a collaboration or communication platform hosted or otherwise provided by a communication platform server (as described in greater detail herein). In some configurations, the computing device(s) 110 may include (or otherwise be) a collaboration device. A collaboration device may include, e.g., a video conferencing device, an interactive whiteboard, a smart projector, a conference phone, etc. In some examples, a collaboration device may be positioned (or implemented) within a designated collaboration area (e.g., within a meeting or conference room). As one example, a collaboration device may be a smart projector installed within a conference room (e.g., mounted to a ceiling of the conference room). In some instances, a collaboration device may be installed in a particular space or area (e.g., within a meeting or conference room). As described in greater detail herein, a collaboration device may be implemented to facilitate a collaboration or communication session via the collaboration or communication platform hosted or otherwise provided by a communication platform server (as described in greater detail herein). Alternatively, or in addition, as described in greater detail herein, the collaboration device (e.g., the computing device(s) 110) may perform a task, a function, or a service related to a collaboration or communication session provided via the collaboration or communication platform hosted or otherwise provided by a communication platform server.

[0021] FIG. IB illustrates an example of the system 100 for implementing dynamic function discovery and execution that includes a communication platform sen' er 120 according to some examples. As illustrated in the example of FIG. IB, the system 100 can include the computing device(s) 110 and the communication platform server 120. In some examples, the system 100 can include fewer, additional, or different components in different configurations than illustrated in FIG. IB. For example, as illustrated, the system 100 includes three computing devices 110 (e.g., the first computing device 110A. the second computing device HOB, and the Nthcomputing device 110Nth) and one communication platform server 120. However, in some examples, the system 100 can include fewer, different, or additional computing devices 110, communication platform servers 120, or a combination thereof. As another example, components of the system 100 can be combined into a single device, divided among multiple devices, or a combination thereof.QB\95047131.5 486362115

[0022] The computing device(s) 110 and the communication platform server 120 can communicate over wired or wireless communication networks 130. Portions of the communication network(s) 130 can be implemented using a wide area network, such as the Internet, a local area network, such as a Bluetooth™ network or Wi-Fi, and combinations or derivatives thereof. In some examples, the communication network(s) 130 represents a direct wireless link between two components of the system 100 (e.g., via a Bluetooth™ or Wi-Fi link). Alternatively, or in addition, in some examples, two or more components of the system 100 can communicate through an intermediary device of the communication network 130 not illustrated in FIG. IB.

[0023] The communication platform server 120 may host or otherwise provide a communication platform (or a collaboration platform). In some instances, the communication platform server 120 may host or otherwise provide the communication platform as a web-based service. A communication platform (or a collaboration platform) may facilitate a virtual or digital communication session (or collaboration session), such as, e.g., a virtual meeting. For instance, a communication session may allow participants to communicate or collaborate with each other. In some instances, the participants (or a portion thereof) of the communication session are not physically located at the same location (e.g., in the same collaboration or communication area, such as a meeting or conference room). Alternatively, or in addition, in some instances, the participants (or a portion thereof) of the communication session are physically located at the same location.

[0024] The communication platform server(s) 120 may be a computing device. Although not illustrated in FIG. IB, the communication platform server(s) 120 may include similar components as described herein with respect to the computing device(s) 110, such as an electronic processor (for example, a microprocessor, an application-specific integrated circuit (ASIC), or another suitable electronic device), a memory (for example, a non-transitory, computer-readable storage medium), a communication interface, such as a transceiver, for communicating over the communication network(s) 130 and, optionally, an additional communication network or connection, and an HMI.

[0025] As illustrated in FIG. 2, the computing device 110 may include an electronic processor 200, a memory 205, a communication interface 210, and a human-machine interface (“HMF’) 215. The electronic processor 200, the memory 205, the communication interface 210, and the HMI 215 can communicate wirelessly, over at least one communication line or bus, or a combination thereof. The computing device 110 can include additional, different, or fewer components than those illustrated in FIG. 2 in various configurations. The QB\95047131.5 586362115computing device 110 can perform additional functionality other than the functionality described herein. Also, the functionality (or a portion thereof) described herein as being performed by the computing device 110 can be performed by another component (e.g., a remote computing device, another computing device, or a combination thereof), distributed among multiple computing devices (e.g., as part of a cloud service or cloud-computing environment), combined with another component (e.g., a remote computing device, another computing device, another component of the system 100. or a combination thereof), or a combination thereof.

[0026] The communication interface 210 can include a transceiver that communicates with the communication platform server 120, another device of the system 100, another device external or remote to the system 100, or a combination thereof over the communication network(s) 130 and, optionally, at least one other communication network or connection. The electronic processor 200 may include a microprocessor, an ASIC, or another suitable electronic device for processing data, and the memory 205 may include a non-transitory. computer-readable storage medium. The electronic processor 200 is configured to retrieve instructions and data from the memory 205 and execute the instructions.

[0027] For example, as illustrated in FIG. 2, the memory 205 may store a communication application 230. The communication application 230 is a software application executable by the electronic processor 200 in the example illustrated and as specifically discussed herein, although a similarly purposed module can be implemented in other ways in other examples. In some configurations, the communication application 230 may be a dedicated software application locally stored in the memory' 205 of the computing device 110.

[0028] The communication application 230 (when executed by the electronic processor 200) may enable or facilitate interaction with a communication platform, as described in greater detail herein. For instance, the communication application 230 (when executed by the electronic processor 200) may allow a user to interact with a communication platform by, for example, hosting a communication session, participating in a communication session, preparing for a future communication session, viewing a previous communication session, and the like. A communication session may include, for example, a video conference, a group call, a webinar (including, a live webinar, a pre-recorded webinar, and the like), a collaboration session, a workspace, an instant messaging group, or the like. A communication session may also be referred to herein as a meeting or a collaboration session.

[0029] The communication application 230 may be associated with at least one communication platform (e.g., an electronic communication platform). As one example, a user QB\95047131.5 686362115may access and interact with a corresponding communication platform via the communication application 230. In some configurations, the memory 205 includes multiple communication applications 230. In such configurations, each communication application 230 is associated with a different communication platform. As one example, the memory 205 may include a first communication application associated with a first communication platform, a second communication application associated with a second communication platform, and an nthcommunication application associated with an nthcommunication platform.

[0030] The electronic processor 200 executes the communication application 230 to enable user interaction with a communication platform (e.g., a communication platform associated with the communication application 230), such as, e.g., a communication platform hosted or otherwise provided by the communication platform server 120 (as described in greater detail herein). The communication application 230 may be a web-browser application that enables access and interaction with a communication platform, such as, e.g., a communication platform associated with the communication platform server 120(e.g., where the communication platform is a web-based service). Alternatively, or in addition, the communication application 230 may be a dedicated software application that enables access and interaction with a communication platform, such as, e.g., a communication platform associated with (or hosted by) the communication platform server 120. Accordingly, in some configurations, the communication application 230 may function as a software application that enables access to a communication platform or service provided by the communication platform server 120.

[0031] The memory 205 may include a function repository 235. The function repository 235 may include a collection of functions of the computing device 110. In some instances, the function repository' 235 may include a collection of functions that are locally stored at the computing device 110 (e.g., in the memory’ 205). Alternatively, or in addition, in some instances, the function repository 235 may include a collection of functions that are available to the computing device 110 (e.g., whether locally stored at the computing device 110 or remotely accessible by the computing device 110 from another storage location or device, such as, e.g., a remote server).

[0032] The function repository 235 may include a listing of functions of the computing device 110. Alternatively, or in addition, the function repository' 235 may' include additional information related to the functions of the computing device 110. For instance, the function repository' 235 may include, e.g., for each function, a name of the function (e.g., "TumCameraOn"). a description of the function (e.g., "A method that turns the main camera of the meeting room on.”), a parameter of the function (e.g., ‘’name”: “meetingID”; ‘‘type”: QB\95047131.5 786362115“String”; “description”: “The id of the meeting to get the participants from.”; etc.), a return type of the function (e.g., “Boolean indicating if the camera was turned on.”), a protocol of the function (e.g., REST), an indication of whether the function is locally stored or remotely accessible, an instruction to call the function (e.g., a function call or call instruction), etc. For example, FIG. 3 provides an example JSON payload that represents available functions for a given computing device (e.g., as an example function repository’ 235).

[0033] As used herein, a function relates to a task or service that may be executed (e.g., by the computing device 110 or another device of the system), as described in greater detail herein. In some examples, a function may also be referred to herein as a tool. A tool (or function) may be a software application executable by the electronic processor 200 (or a corresponding agent). When executed by the electronic processor 200, the tool (or function) may provide (or otherwise perform) functionality’ as described herein. As one example, when executed by the electronic processor 200, the tool (or function) may facilitate performance of a task or sendee with respect to a collaboration or communication session (e.g., a meeting facilitated via a collaboration or communication platform), as described in greater detail herein. In some instances, a function may relate to controlling a hardware component, a software component, or a hardware and a software component of the computing device 110 (or another component of the system 100). For example, a function may include (or otherwise relate to) joining a meeting, muting or unmuting a microphone, obtaining a meeting summary, reserving a meeting room, starting a recording of a meeting, starting or stopping a video (or camera feed), obtaining a transcript of a meeting, obtaining a recording of a meeting, obtaining a list of participants of a meeting, starting or stopping screen sharing, obtaining a translation of a meeting, adjusting a setting or parameter of a hardware component, etc. Accordingly, in some instances, a function (or a tool) is (or otherwise include) an agent, a machine learning model, an Al model, a large language model (LLM), another type of model, or a combination thereof. As one example, a function (or tool) is (or otherwise includes) a machine learning model. As another example, a function (or tool) is (or otherwise includes) an agent. As another example, a function (or tool) is (or otherwise includes) an LLM. As yet another example, a function (or tool) is (or otherwise includes) a machine learning model and an LLM. As yet another example, a function (or tool) is (or otherwise includes) an agent and an LLM.

[0034] The memory 205 may store an agent 240. As described herein, the agent 240 may be a software agent. The agent 240 may be an artificial intelligence (Al) agent. For instance, the agent 240 may be a software application executable by the electronic processor 200 in the example illustrated and as specifically discussed herein, although a similarly purposed module QB\95047131.5 886362115can be implemented in other ways in other examples. In some examples, the agent 240 e is a computer program (e.g., a software application) that, when executed, acts on behalf of a user or another application or program to perform tasks autonomously (or semi-autonomously) (e.g., without human intervention or with reduced human intervention). In some instances, the agent 240 (when executed by the electronic processor 200) may make decisions and take actions based on predefined rules or learned patterns. The agent 240 may be, e g., a chatbot, a virtual assistant, or the like.

[0035] In some configurations, the agent 240 facilitates (or otherwise implements) dynamic function (or tool) discovery and execution, as described in greater detail herein. For instance, the agent 240 may be implemented as a software agent that can perform a range of tasks, functions, or services for a user (e.g.. when executed by the electronic processor 200). For example, the agent 240 may perform a range of tasks, functions, or services with respect to a collaboration or communication session (e.g., a meeting facilitated via a collaboration or communication platform). As described in greater detail herein, in some examples, the agent 240 (when executed by the electronic processor 200) facilitates execution of a function at one computing device responsive to a user query to perform an action that involves execution of the function, where the user query is received at another, different computing device.

[0036] In some configurations, the agent 240 includes (or otherwise utilizes) at least one LLM 245 (referred to herein collectively as ‘‘the LLMs 245” and individually as “the LLM 245”). Generally, the LLM 245 may include a deep Al or machine learning model that can comprehend and generate human language text. For instance, the LLM 245 may be configured to determine meanings (or context) from a sequence of words and understand relationships between those words and, ultimately, perform a task based on that understanding. For instance, the LLM 245 may perform a variety of natural language processing (NLP) related tasks to produce content based on input prompts in human language. Such tasks may generally include answ ering questions (e.g., responding to a user query), translating text, text generation, content summary, sentiment analysis, etc. The LLM 245 may be an artificial neural network that is trained using self-supervised learning, semi-supervised learning, or a combination thereof. In some instances, the LLM 245 may be multimodal such that the LLM 245 may receive various data types or formats, including, e g., text, images (or visual), audio, etc.

[0037] In some configurations, the memory' 205 may also include a learning engine 250. In some configurations, the learning engine 250 develops a model using an Al or machine learning function. Machine learning functions are generally functions that allow a computer application to leam without being explicitly programmed. In particular, the learning engine 250 QB\95047131.5 986362115is configured to develop a model based on training data. As one example, to perform supervised learning, the training data includes example inputs and corresponding desired (for example, actual) outputs, and the learning engine 250 progressively develops a model that maps inputs to the outputs included in the training data. As another example, to perform self-supervised learning (“SSL”), a model is trained on a task using the data itself to generate supervisory- signals (e.g., unlabeled training data), rather than relying on, e.g., external labels provided by a user (e.g.. labeled training data). As yet another example, to perform semi-supervised learning, the training data may include desired output values for a subset of the training data (e.g., labeled training data) while the remaining training data may be unlabeled or imprecisely labeled (e.g., unlabeled training data). Machine learning performed by the learning engine 365 may be performed using various types of methods and mechanisms including but not limited to decision tree learning, association rule learning, artificial neural networks, inductive logic programming, support vector machines, clustering, Bayesian networks, reinforcement learning, representation learning, similarity- and metric learning, sparse dictionary learning, and genetic algorithms. These approaches allow the learning engine 250 to ingest, parse, and understand data and progressively refine models. In some configurations, the learning engine 250 may develop the LLM 245, another model described herein, or the like.

[0038] The memory- 205 may store a user profile 255. The user profile 255 may be associated with (or otherwise relate to) a particular user. The user profile 255 may include information or data related to the associated user. For instance, the user profile 255 may include, e.g., a name, a role or classification (e.g., a manager, a customer service representative, an IT administrator, etc.), contact information (e.g., phone number, an email address, a mailing address, etc.), a schedule, a preference, an availability-, etc. As described in greater detail herein, in some instances, the agent(s) 240 (or the LLM(s) 245 thereof) may access the user profile 255 and utilize the user profile (or a portion of data included therein) when performing the functionality described herein. As one example, when providing a response to a user query, the agent(s) 240 (or the LLM(s) 245 thereof) may utilize at least a portion of data included in the user profile 255 to generate (or otherwise determine) a custom prompt such that a response provided to a user of the user profile 255 is tailored (or customized) to that user. Accordingly, in some instances, the user experience provided via the technology disclosed herein may be customized to a particular user (e.g., using the data included in the user profile 255). For example, when a user is a proj ect manager and the user asks for a meeting summary, the content of the meeting summary may be tailored according to the user profile of that user when theQB\95047131.5 1086362115agent 240 presents the final response. Such customization may be achieved through custom prompts for the LLM(s) 245.

[0039] The memory 205 may store communication session data 260. The communication session data 260 may include, e.g., a data stream (e.g., a live data stream) related to a communication session (e.g., a meeting conducted via a communication platform of the communication platform server 120). The communication session data 260 may include data or information related to a communication session, such as, e.g.. a present communication session (e.g., a communication session presently taking place), a previous communication session, a future communication session, etc. In some instances, the communication session data 260 may include, e.g.. a recording of a communication session, a transcription of a communication session, a summary of a communication session, a translation of a communication session, an action item of a communication session (e.g., an action item or task to be performed based on a communication session, such as after a communication session has been completed), a metric of a communication session (e.g., an attendance list, a duration, a location, a date, a time, an expected topic, an objective, etc.), etc.

[0040] The memory 205 may include additional, different, or fewer components in different configurations than illustrated in FIG. 2. Alternatively, or in addition, in some configurations, components of the computing device memory 205 may be combined into a single component, distributed among multiple components, or the like. Alternatively, or in addition, in some configurations, a component of the computing device memory 205 may be stored remotely from the computing device 110, or, in a remote database, another server, a remote user device, an external storage device, or the like.

[0041] For example, in some instances, the LLM(s) 245, the user profile(s) 255, or another component of the memory' 205 may be included as part of (or as a component of) the agent(s) 240. For instance, as one specific example, the agent 240 may include the user profile 255 and the LLM 245.

[0042] As noted herein, in some instances, the functionality' (or a portion thereof) described herein as being performed by the computing device 110 can be performed by another component (e.g.. a remote computing device, another computing device, or a combination thereof), distributed among multiple computing devices (e g., as part of a cloud service or cloud-computing environment), combined with another component (e.g., a remote computing device, another computing device, another component of the system 100, or a combination thereof), or a combination thereof. For example, in some instances, the communicationQB\95047131.5 1186362115application 230, the function repository 235, the agent(s) 240, the LLM(s) 245, the learning engine(s) 250, the user profile(s) 255. the communication session data 260, or a combination thereof may be stored and executed by another component, distributed among multiple computing devices, combined with another component, or a combination thereof.

[0043] As illustrated in FIG. 2, the computing device 110 can also include the HMI 215 for interacting with a user. The HMI 215 can include at least one input device, at least one output device, or a combination thereof. Accordingly, in some examples, the HMI 215 allows a user to interact with (e.g., provide input to and receive output from) the computing device 110. For example, the HMI 215 can include a keyboard, a cursor-control device (e.g., a mouse), a touch screen, a scroll ball, a mechanical button, a display device (e.g., a liquid crystal display (LCD), an organic light emitting diode (OLED) display, etc.), a printer, a speaker, a microphone, or a combination thereof.

[0044] In the illustrated example of FIG. 2, the HMI 215 includes at least one display device 265 (referred to herein collectively as “the display devices 265'’ and individually as “the display device 265”). The display device 265 can provide (or output) a media signal to a user. As one example, the display device 265 can display a user interface (e.g.. a graphical user interface (GUI)) associated with the communication application 230. The display device 265 can be included in the same housing as the computing device 110 or can communicate with the computing device 110 over a wired or wireless connection. As one example, the display device 265 can be a touchscreen included in a cellular phone, a smart wearable, a laptop computer, a tablet computer, a smart speaker, another type of portable smart device, etc. As another example, the display device 265 can be a monitor, a television, or a projector coupled to a terminal, desktop computer, or the like via a cable.

[0045] The HMI 215 can also include at least one imaging device 270 (referred to herein collectively as '‘the imaging devices 270” and individually as “the imaging device 270”). The imaging device 270 can be a component associated with the computing device 110 (e.g., included in the computing device 110 or otherwise communicatively coupled with the computing device 110). In some examples, the imaging device 270 can be internal to the computing device 110 (e.g.. a built-in webcam or camera). Alternatively, or in addition, the imaging device 270 can be external to the computing device 110 (e.g., an external webcam or camera positioned on the computing device 110 or proximate to the computing device 110, e.g., on a desk, table, shelf, wall, ceiling, etc.).

[0046] The imaging device 270 can electronically capture or detect a visual image (as an image data signal or data stream). In some examples, the imaging device 270 may capture QB\95047131.5 1286362115image data continuously in real-time (or near real-time). A visual image can include, e.g., a still image, a moving-image, a video stream, a live data stream, other data associated with providing a visual output, and the like. The imaging device 270 can include a camera, such as, e.g., a webcam, an image sensor, or the like. For example, the imaging device 270 can detect image data associated with a physical surrounding or environment of the computing device 110, such as, e.g., a meeting or conference room.

[0047] The HMI 215 can also include at least one microphone 275 (referred to herein collectively as ‘'the microphones 275” and individually as “the microphone 275”). The microphone 275 may capture (or otherwise record) audio data (also referred to herein as “voice data”). In some examples, the microphone 275 may capture audio data continuously in realtime (or near real-time). In some configurations, the audio data may be time series data or a data stream of audio data (e.g., an audio datastream). In some instances, the audio data captured by the microphone 275 may be user input provided by a user using the computing device 110.

[0048] The HMI 215 can also include at least one speaker 280 (referred to herein collectively as “the speakers 280” and individually as “the speaker 280”). The speaker 280 may output (or otherwise provide) audio data. As described in greater detail herein, in some configurations, the speaker 280 may provide audio data responsive to input received from a user using the computing device 110 (e.g., the audio data captured by the microphone 475, image data collected by the imaging device 270, text data received via an input device of the HMI 215, such as a keyboard, a touchscreen, etc.). In some examples, the agent(s) 240 may output a response to a user query using the speaker 280 (e.g., as an audio output signal).

[0049] In some configurations, the HMI 315 may include at least one sensor 285 ((referred to herein collectively as “the sensors 285” and individually as “the sensor 285”). In some configurations, the sensor(s) 285 may collect data related to the communication session. For example, the sensor(s) 285 may collect data or information related to a physical location of the communication session (e.g., a physical meeting room, a surrounding or environment of the communication session, etc.). As another example, the sensor(s) 285 may collect data or information related to a human entity of the communication session (e g., a physical attribute of the human entity, a location of the human entity within the physical location of the communication session, a presence of the human entity in the physical location of the communication session, etc.). In some configurations, the systems and methods disclosed herein may implement or utilize the data or information collected by the sensor(s) 285 when implementing the technology disclosed herein.QB\95047131.5 1386362115

[0050] In some configurations, the HMI 215 may be utilized by participants of a communication session (e.g., provide output to the entities or receive input from the entities). Alternatively, or in addition, in some configurations, the HMI 215 may be utilized by a user prior to a communication session (e.g., when organizing a meeting), during a communication session (e.g., providing output and receiving input from participants of a communication session), or after a communication session (e.g., provide a meeting transcript, recording, summary, translation, etc.). As one example, the display device 340 may provide a visual output to entities of the communication session. As another example, the speaker(s) 355 may provide an audio output to entities of the communication session. As yet another example, the microphone(s) 350 may collect an audio input of the entities of the communication session. As yet another example, the imaging device(s) 345 may collect a visual input of the entities of the communication session. Accordingly, the HMI 315 (or component(s) thereof) may collect information or data related to a communication session (e.g., the communication session data 335).

[0051] As noted herein, the computing device 110 (or component(s) thereof) can include additional, different, or fewer components than those illustrated in FIG. 2 in various configurations. Also, the functionality (or a portion thereof) described herein as being performed by the computing device 110 (or component(s) thereof) can be performed by another component (e.g.. a remote computing device, another computing device, or a combination thereof), distributed among multiple computing devices (e.g.. as part of a cloud service or cloud-computing environment), combined with another component (e g., a remote computing device, another computing device, another component of the system 100, or a combination thereof), or a combination thereof.

[0052] FIG. 4 schematically illustrates an example arrangement 400 that includes a first computing device 110A and a second computing device HOB according to some examples. The first computing device 110A and the second computing device HOB may be examples of the computing device 110 of FIG. 2. For instance, the first computing device 110A, the second computing device HOB, or a combination thereof may include similar components or perform similar functionality as described herein with respect to the computing device 110 of FIG. 2. In some examples, the first computing device 110A may be a personal computer, such as, e.g., alaptop, a tablet, etc. The second computing device 110B may be a collaboration device, such as, e.g., a smart projector, an interactive whiteboard, etc.

[0053] As illustrated in FIG. 4, the first computing device 110A and the second computing device HOB are communicatively coupled via a communication link 402 (e.g., over the QB\95047131.5 1486362115communication network(s) 130 of FIGS. 1A and IB). As described in greater detail herein, in some instances, the communication link 402 may be established responsive to the first computing device 110A and the second computing device HOB being within a particular distance range of each other (e.g., a distance between the first computing device 110A and the second computing device 11 OB satisfies a distance threshold). For instance, the communication link 402 may be established when the first computing device 110A is nearby the second computing device 110B (e.g., within the same meeting or conference room, etc.).

[0054] For instance, as illustrated in the example of FIG. 4, the first computing device 110A may include a first agent 240A (e.g., the agent 240 of FIG. 2). In some examples, the first agent 240A may be local to the first computing device 110A. Alternatively, or in addition, in some instances, the first agent 240A may be remotely accessible to the first computing device 110A, such as, e.g., from a remote server. In the illustrated example, the first agent 240A may be associated with (or otherwise include) a first LLM 245A (e.g., the LLM 245 of FIG. 2). The first computing device 110A may also include the user profile 255, as described in greater detail herein with respect to FIG. 2. The first computing device 110A may also include a first function repository 235A (e.g.. as similarly described herein with respect to the function repository 235 of FIG. 2). The first function repository 235A may include a collection of functions 405 of the first computing device 110A (e.g., functions available to the first computing device 110A). In the illustrated example, the first function repository 235 A may include a cooking assistant 410 (e.g., a function related to cooking), a shopping assistant 415 (e.g., a function related to shopping), a math tutor 420 (e g., a function related to math), and an entertainment assistant 425 (e.g., a function related to entertainment).

[0055] In some configurations, the cooking assistant 410, the shopping assistant 415, the math tutor 420. the entertainment assistant 425, or a combination thereof may be an agent (e.g., the agent(s) 240). For instance, the cooking assistant 410 may be an agent that assists with cooking tasks or sendees, the shopping assistant 415 may be an agent that assists with shopping tasks or services, the math tutor 420 may be an agent that assists with math tasks or services, the entertainment assistant 425 may be an agent that assists with entertainment tasks or services, etc. As such, in some instances, the first computing device 110A may include multiple agents.

[0056] The second computing device HOB may include a second agent 240B (e.g., the agent 240 of FIG. 2). In some examples, the second agent 240B may be local to the second computing device 110B. Alternatively, or in addition, in some instances, the second agent 240B may be remotely accessible to the second computing device 110B, such as, e.g., from a remote QB\95047131.5 1586362115server. In the illustrated example, the second agent 240B may be associated with (or otherwise include) a second LLM 245B (e.g., the LLM 245 of FIG. 2). The second computing device HOB may also include a second function repository 235B (e.g., as similarly described herein with respect to the function repository 235 of FIG. 2). The second function repository 235B may include a collection of functions of the second computing device HOB, such as, e.g., at least one post-session function 430, at least one device function 435, and at least one session function 440 (e.g., functions available to the second computing device HOB). In the illustrated example, the post-session function(s) 430 may include, e.g., a get meeting transcript function 445, a get meeting recording function 450, a get meeting participants function 455, and a get meeting summary' function 460. The device function(s) 435 may include, e.g., a start recording function 465, a mute / unmute microphone function 470, and a start / stop video function 480. The session function(s) 440 may include, e.g., a book this meeting space function 485, a join this meeting function 490, and a share screen function 495.

[0057] Accordingly, as illustrated in FIG. 4, the first function repository 235A and the second function repository 235B are different (e.g., include different functions). As described in greater detail herein, the technology disclosed herein may facilitate (or otherwise implement) a proximity -based framew ork for dynamic function discovery' and execution using agentic Al models (e.g., the agent(s) 240). For instance, with reference to FIG. 4, the technology' disclosed herein may facilitate the first computing device 110A (via the first agent 240 A) dynamically discovering the functions included in the second function repository 235B and the execution of those functions (or a portion thereof), as described in greater detail herein.

[0058] For example, FIG. 5 illustrates an example scenario 500 involving the first computing device 110A and the second computing device 110B of FIG. 4. As illustrated in FIG. 5, the first computing device 110A may receive the second function repository 235B from the second computing device HOB (represented in FIG. 5 by reference numeral 505). Responsive to receiving the second function repository 235B, the first computing device 110A may register the second function repository 235B (e.g., the functions included therein) such that the functions included in the second function repository 235B are added (or otherwise reflected) in the first function repository 235 A (represented in FIG. 5 by reference numeral 510). In some examples, after the functions of the second function repository' 235B are added (or otherwise reflected) in the first function repository' 235A, those functions may be available to the first computing device 110A. As one example, a user 515 of the first computing device 110A may provide, to the first computing device 110A, a user query requesting to join a meeting (e.g., ‘‘Please join on my meeting.’’) (reflected in FIG. 5 by reference numeral 520). QB\95047131.5 1686362115As described in greater detail herein, responsive to receiving the user query', the first computing device 110A may determine which function of the functions included in the first function repository 235A to execute in order to perform an action related to the user query7(e.g., to join a meeting) and facilitate execution of that function associated with joining a meeting (e.g., a join on meeting function 525). In the example of FIG. 5, the first computing device 110A may facilitate execution of the join on meeting function 525 by transmitting a function call 550 to the second computing device HOB. Upon receipt of the function call 550, the second computing device HOB may7execute the join on meeting function 525 such that the meeting is joined.

[0059] Accordingly, in some instances, when the user 515 enters a meeting room, the first computing device 110A and the second computing device 110B can discover each other using solutions such as Bluetooth Low Energy (BLE), mesh network technology7, etc. (e.g., as represented in FIG. 4 by the communication link 402). Upon discovery7, the technology disclosed herein may adapt dynamically to a proximity' of the user 515, with the first agent 240A querying nearby devices for available tools (or functions), such as, e.g., the second computing device HOB. The nearby devices (e.g.. the second computing device HOB) may respond with function information, including, e g., names, descriptions, parameters, etc. (e.g., represented in FIG. 5 by reference numeral 505). As noted herein, the first computing device 110A (e.g.. the first agent 240 A) may register function information (e.g., in the first function repository 235A). The first agent 240A may act as a central hub, offering general-purpose assistants and context-specific features, such as, e g., muting / unmuting the microphone, getting a meeting summary7, booking a roomjoining a meeting, etc. Accordingly, in some instances, functionality of a nearby device (e.g., the second computing device HOB) may be accessible when the corresponding device is in range (e.g., conditioned on a distance between the first computing device 110A and the second computing device HOB satisfying a distance threshold), enabling seamless, intuitive interactions tailored to a preference of a user, as generally illustrated in FIG. 5.

[0060] FIG. 6 is a flowchart illustrating a method 600 for implementing dynamic function discovery7and execution according to some examples. The method 600 is described as being performed by the first computing device 110A and, in particular, an electronic processor(s) of the first computing device 110A (e.g., the electronic processor(s) 200) executing at least one of the first agent 240A, the first LLM 245A, the learning engine 250, etc. However, as noted above, the functionality described with respect to the method 600 can be performed by other devices, such as the second computing device HOB, another remote sen7er or computing QB\95047131.5 1786362115device, another component of the system 100, or a combination thereof, or distributed among a plurality of devices, such as a plurality of servers included in a cloud service (e.g., a webbased service executing software or applications). Further, although generally described as begin performed by a processor (e.g., the electronic processor 200), this processor may include multiple processors (e.g., as a part of a distributed processor system or group of cooperating processors).

[0061] As illustrated in FIG. 6, the method 600 may include detecting, with the electronic processor 200, the second computing device HOB separate from the first computing device 110A (at block 605). As described herein, the second computing device HOB may be configured to perform a function (e.g., a function included in the second function repository 235B).

[0062] In some instances, the electronic processor 200 may detect the second computing device 110B (e.g., as a nearby device) when a distance between the first computing device 110A and the second computing device HOB satisfies a distance threshold. In some cases, the distance threshold may be based on size of a collaboration or meeting area (e.g., a size of a meeting or conference room). Alternatively, or in addition, the distance threhsold may be based on which communication technology or protocol is implemented (e.g., Wi-Fi, Bluetooth, Zigbee, etc.). For instance, when the technology' disclosed herein is implemented using Bluetooth, some examples of which may have a range of approximately up to 30 feet, the distance threshold may be 30 feet (or another threshold based on the range associated with a utilized Bluetooth protocol). In some examples, the electronic processor 200 may detect that the distance threshold is satisfied when the first computing device 110A and the second computing device 110B form a wireless connection or wirelessly communicate with one another. In some examples, the electronic processor 200 may detect that the distance threshold is satisfied in response to the electronic processor 200 determining that a strength of a wireless signal from the second computing device 110B is above a signal threshold. In some examples, the electronic processor 200 may estimate a distance between the first computing device 110A and the second computing device HOB (e.g., based on a strength of signal measurement) and the electronic processor 200 may detect that the distance threshold is satisfied in response the estimated distance being below the distance threshold. Although the technology disclosed herein is generally described in the context of “nearby devices,” the technology7disclosed herein may be implemented with respect to other contexts (e.g., with computing devices that are not considered “nearby” or within a short-range pf each other).QB\95047131.5 1886362115

[0063] In some configurations, responsive to detecting the second computing device 110B, the electronic processor 200 may facilitate a dynamic discovery of functionality provided by the second computing device HOB (e.g., the function(s) included in the second function repository 235B). In some examples, the electronic processor 200 may transmit, to the second computing device HOB a request for information related to the second function repository 235B. As noted herein, the information related to the second function repository 235B may include, e.g.. a name of a function, a description of a function, a parameter of a function, a return type of a function, a protocol of a function, an indication of whether a function is locally stored or remotely accessible, an instruction to call a function, etc. In some instances, responsive to receiving the request for information related to the second function repository 235B, the second computing device HOB (e.g., the second agent 240B) may generate and provide the information related to the second function repository 235B. As such, the electronic processor 200 may receive, from the second computing device 110B, the information related to the second function repository 235B. Responsive to receiving the information related to the second function repository 235B, the electronic processor 200 may add the information related to the second function repository 235B to the first function repository 235A of the first computing device 110A. That is, at block 610, the electronic processor 200 may register the function(s), of the second function repository 235B of the second computing device 110B, as available to the first computing device 110A. As one example, when the second function repository 235B identifies a function, the electronic processor 200 may register that function at the first computing device 110A such that the first function repository 235A indicates that function as available to the first computing device 110A. Accordingly, as illustrated in FIG. 7, the electronic processor 200 may register, at the first computing device 110A, the function(s) as available to the first computing device 110A (at block 610).

[0064] In some configurations, the electronic processor 200 may generate a notification that indicates that the function(s) is available to the first computing device 110A. The notification may include a listing of any newly registered (or available) functions. The electronic processor 200 may control an HMI (or a component thereof) (e.g., the HMI 215 of FIG. 215) of the first computing device 110A to output the notification to a user (e.g., the user 515) of the first computing device 110A. In some instances, the notification may be provided to the user 515 as a visual output (e.g., via a user interface or GUI) using a display device (e.g., the display device 265 of FIG. 2), as an audio output using a speaker (e.g., the speaker 280 of FIG. 2), etc. As such, a user may be notified of new functionality that is available to the userQB\95047131.5 1986362115via the proximity of the user (e.g., the first computing device 110A) to the nearby device (e.g., the second computing device HOB).

[0065] The electronic processor 200 may receive, at the first computing device 110A, a user query to perform an action (at block 615). In some instances, the user query may be a user command provided via the HMI 215 of the first computing device 110A (e.g., via a keyboard, a mouse, the microphone 275, or another component of the HMI 215 of the first computing device HOA). As one specific example, the user 515 may interact with a chatbot (e.g., as an example of the first agent 240A) of the first computing device 110A by providing an input to the chatbot (e.g., the first agent 240A), such as, e.g., a question, a command, etc. In some instances, the user query relates to performance of an action, where that action may be performed via execution of a function (e.g., a function included in the first function repository 235 A, including, in some instances, a function included in the second function repository 235B that was newly registered at the first computing device 110A). As noted herein, the user query may include, e.g., “Please join on my meeting.”, as described herein with respect to FIG. 5.

[0066] The electronic processor 200 may determine to execute the function for performance of the action (at block 620). In some configurations, the electronic processor 200 may determine to execute the function for performance of the action based on the first function repository 235A, the user query, etc. In some instances, the electronic processor 200 may invoke (or execute) the first LLM 245 A in order to determine to execute the function for performance of the action.

[0067] For example, in some configurations, the electronic processor 200 may provide the user uery to the first agent 240A of the first computing device 110A. The first agent 240A may generate a prompt based on the user query'. The first agent 240A may then provide the prompt to the first LLM 245A. The first LLM 245A may determine to execute the function for performance of the action, as described herein. For example, the first LLM 245A may access first function repository 235A. The first LLM 245A may determine which function included in the first function repository' 235 A should be executed for performance of the action. As such, in some configurations, the determination to execute the function for performance of the action is made in real-time (or near real-time) (as opposed to being made based on a lookup table). The first LLM 245A may generate a response to the prompt. The response may indicate (or otherwise identify) the function as the function to be executed for performance of the action. For example, the response may indicate the determination to execute the function for performance of the action. In some instances, the response may include additional information related to the function, such as, e.g., a function call for the function or other information related QB\95047131.5 2086362115to the function that is included in the first function repository 235 A. The first agent 240 A may receive, from the first LLM 245 A, the response to the prompt, which, in some instances, may include a function call to invoke execution of the function at the second computing device HOB.

[0068] The electronic processor 200 may transmit, to the second computing device HOB (e.g., the second agent 240B), a function call to invoke execution of the function at the second computing device HOB (at block 625). As noted herein, the function call may be included in the response received from the first LLM 245A. As such, in some instances, responsive to receiving the response from the first LLM 245A, the first agent 240A (e.g., the electronic processor 200) may transmit the function call to the second computing device 110B (e.g., the second agent 240B). In some configurations, the second computing device HOB may be implemented (or otherwise associated) with a predefined communication protocol. In some instances, the communication protocol of the second computing device 110B may be different than the communication protocol of the first computing device 110A. Alternatively, the communication protocols of the first computing device 110A and the second computing device 110B may be the same. In some instances, the electronic processor 200 (e.g.. the first agent 240A) may transmit the function call to the second computing device 110B (e g., the second agent 240B) using a communication protocol of the second computing device HOB. As such, in some instances, the technology disclosed herein may be implemented in agnostically across various types of computing devices (e.g., computing devices of various manufacturers or providers).

[0069] In response to receiving the function call, the second computing device HOB (e.g., the second agent 240B) may invoke the function such that the function is executed at the second computing device HOB. As one example, when the function call relates to turning on a camera at the second computing device HOB, the second computing device HOB (e.g., the second agent 240B) may, responsive to receiving the function call, turn the camera at the second computing device 110B on.

[0070] In some configurations, after the second computing device HOB executes the function, the second computing device HOB (e.g.. the second agent 240B) may generate a notification that includes information related to execution of the function at the second computing device 11 OB. In some instances, the notification may include a confirmation that the function was executed at the second computing device 110B, a status of the execution of the function at the second computing device HOB, etc. As one specific example, the notification may indicate that the function was successfully executed at the second computing QB\95047131.5 2186362115device HOB. As another specific example, the notification may indicate that the function was not successfully executed at the second computing device 110B (e.g., execution of the function failed or that an error occurred during execution of the function). Accordingly, in some instances, the second computing device 11 OB may generate a notification that indicates a result of the execution of the function at the second computing device HOB and transmit the notification to the first computing device 110A.

[0071] The second computing device HOB may transmit the notification (e.g., the information related to execution of the function at the second computing device 110B) to the first computing device 110A (e.g., the first agent 240A). Accordingly, in some examples, the electronic processor 200 (e.g., the first agent 240A) may receive, from the second computing device 110B. information related to execution of the function at the second computing device 110B. The electronic processor 200 (e.g., the first agent 240A) may generate, with the first LLM 245A of the first agent 240 A, a response based on the information related to execution of the function at the second computing device 11 OB and output the response at the first computing device 110A (e.g., via the HMI 215 of the first computing device 110A). In some examples, the first agent 240A may provide the notification (e.g.. the information related to execution of the function at the second computing device HOB) to the first LLM 245 A. The first LLM 245A may generate a response to the notification and provide the response to the first agent 240A such that the output may be provided to the user 515 of the first computing device 110A (e.g.. by controlling the HMI 215 of the first computing device 110A to output the response). As one example, the first computing device 1 10A may output a visual output that indicates the information related to execution of the function at the second computing device HOB (e.g., using a display device). As another example, the first computing device 11 A may output an audio output that indicates the information related to execution of the function at the second computing device 1 IB (e.g., using a speaker).

[0072] In some examples, the first agent 240A may access data from the user profile 255 (or a portion of data included therein) (e.g., the user profile 255 of the user 515). The firstagent 240A may provide the data accessed from the user profile 255 (or the portion of data included therein) to the first LLM 245A. In some instances, the first agent 240A may provide the data accessed form the user profile 255 in addition to the notification to the first LLM 245 A. In such configurations, the first LLM 245A may generate a response that is customized (or otherwise specific) to the user 515 (e.g., such that the response is generated in accordance with the user profile 255), as described in greater detail herein.QB\95047131.5 2286362115

[0073] FIG. 7 is a sequence diagram 700 illustrating a discovery, registration, and execution phase in accordance with configurations described herein. As illustrated in FIG. 7, the sequence diagram 700 includes the user 515, the first agent 240 A, the first LLM 245 A, the first function repository 235A, the second agent 240B, and the second function repository 235B.

[0074] As illustrated in FIG. 7, the first agent 240A may check for nearby devices, such as, e.g., the second computing device 110B (represented in FIG. 7 by reference numeral 705). In some instances, the first agent 240A may check for nearby devices using a predefined protocol (e.g., a communication protocol). In some configurations, when the first agent 240A detects a nearby device (e g., the second computing device HOB), the first agent 240A may generate and transmit a request for available functions (e.g., information related to the second function repository 235B) from the second computing device HOB (e.g., the second agent 240 A) (represented in FIG. 7 by reference numeral 710). Responsive to receiving the request the second agent 240B may access the second function repository 235B to determine available functions (represented in FIG. 7 by reference numeral 715). The second agent 240B may receive the available functions from the second function repository 235B (represented in FIG.7 by reference numeral 720). An example of a protocol for returning the functions from a function repository is illustrated in FIG. 3. The second agent 240B provides the available functions to the first agent 240A (represented in FIG. 7 by reference numeral 725). Responsive to receiving the available functions from the second agent 240B. the first agent 240A may register the available functions with the first function repository 235A (represented in FIG. 7 by reference numeral 730). The first agent 240A may notify the user 515 of new functions available (represented in FIG. 7 by reference numeral 735). The first agent 240A may notify the user 515 using, e.g., a dialog in a UI displayed to the user 515 via a display device. Thus, the user 515 is informed as to what extended functionalities the first computing device 110A (e.g., the first agent 240A) currently has.

[0075] The user 515 may provide a user query (e.g., “Turn Camera On.”) using the first computing device 110A. As one example, the user 515 may interact with a user interface associated with the first agent 240A (e.g.. a chatbot UI), such that the user 515 may type the action that the user 515 wants performance (e.g., turning a camera on at the second computing device HOB). As another example, the user 515 may provide the user query using voice or another input method. The user query may be provided to the first agent 240A (represented in FIG. 7 by reference numeral 740). The first agent 240A may provide a prompt (e.g., “Please turn on the camera”) to the first LLM 245A (represented in FIG. 7 by reference numeral 745). QB\95047131.5 2386362115The first LLM 245A may interact with the first function repository 235A (represented in FIG.7 by reference numeral 750) such that the first LLM 245A may access (or otherwise receive) the available functions from the first function repository 235A (represented in FIG. 7 by reference numeral 755). The first LLM 245A may generate a response that based on the interaction with the first function repository 235A (represented in FIG. 7 by reference numeral 760). As described herein, the response may indicate which function to execute for performance of the action as requested by the user 515 via the user query (e.g., the response of the first LLM 245A may indicate which function(s) should be called to address the user query. In some instances, the first LLM 245A provides a function call (e.g., “TumCameraOn()”) to the first agent 240A (represented in FIG. 7 by reference numeral 765). The first agent 240A may call the given function through a defined protocol specified by the second agent 240A (represented in FIG. 7 by reference numeral 770). The second agent 240A may receive the request and execute the function by, e.g., turning on the corresponding camera at the second computing device HOB (represented in FIG. 7 by reference numeral 780). The second agent 240A may provide a notification that includes information related to execution of the function at the second computing device HOB (e.g., “camera is on”) (represented in FIG. 7 by reference numeral 785). Responsive to receiving the notification from the second agent 240B, the first agent 240A may send a function response to the first LLM 245A (represented in FIG. 7 by reference numeral 790). The first LLM 245A may generate a final response based on the function response (e.g., whether the function was successfully executed at the second computing device HOB) (represented in FIG. 7 by reference numeral 792). The first LLM 245 A may provide the final response to the first agent 240A (represented in FIG. 7 by reference numeral 794). The first agent 240A may provide the final response (e.g., “Camera was successfully turned on.”) to the user 515, as described in greater detail herein (represented in FIG. 7 by reference numeral 796).

[0076] The disclosed technology is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the following drawings. Other examples of the disclosed technology are possible and examples described and / or illustrated here are capable of being practiced or of being earned out in various ways.

[0077] A plurality of hardware and software-based devices, as well as a plurality of different structural components can be used to implement the disclosed technology. In addition, examples of the disclosed technology can include hardware, software, and electronic components or modules that, for purposes of discussion, can be illustrated and described as if QB\95047131.5 2486362115the majority of the components were implemented solely in hardware. However, in at least one example, the electronic based aspects of the disclosed technology can be implemented in software (for example, stored on non-transitory computer-readable medium) executable by at least one processor. Although certain drawings illustrate hardware and software located within particular devices, these depictions are for illustrative purposes only. In some examples, the illustrated components can be combined or divided into separate software, firmware, hardware, or combinations thereof. As one example, instead of being located within and performed by a single electronic processor, logic and processing can be distributed among multiple electronic processors. Regardless of how they are combined or divided, hardware and software components can be located on the same computing device or can be distributed among different computing devices connected by at least one network or other suitable communication link.

[0078] In some examples, aspects of the technology, including computerized implementations of methods according to the technology, can be implemented as a system, method, apparatus, or article of manufacture using standard programming or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a processor device (e.g., a serial or parallel processor chip, a single- or multi-core chip, a microprocessor, afield programmable gate array, any variety of combinations of a control unit, arithmetic logic unit, and processor register, and so on), a computer (e.g., a processor device operatively coupled to a memory), or another electronically operated controller to implement aspects detailed herein. Accordingly, for example, examples of the technology can be implemented as a set of instructions, tangibly embodied on a non-transitory computer-readable media, such that a processor device can implement the instructions based upon reading the instructions from the computer-readable media. Some examples of the technology' can include (or utilize) a control device such as an automation device, a computer including various computer hardware, software, firmware, and so on, consistent with the discussion below. As specific examples, a control device can include a processor, a microcontroller, a field-programmable gate array, a programmable logic controller, logic gates etc., and other ty pical components that are known in the art for implementation of appropriate functionality (e.g., memory, communication systems, power sources, user interfaces and other inputs, etc.).

[0079] Certain operations of methods according to the technology7, or of systems executing those methods, can be represented schematically in the FIGS, or otherwise discussed herein. Unless otherwise specified or limited, representation in the FIGS, of particular operations in particular spatial order can not necessarily require those operations to be executed in a particular sequence corresponding to the particular spatial order. Correspondingly, certain QB\95047131.5 2586362115operations represented in the FIGS., or otherwise disclosed herein, can be executed in different orders than are expressly illustrated or described, as appropriate for particular examples of the technology. Further, in some examples, certain operations can be executed in parallel, including by dedicated parallel processing devices, or separate computing devices configured to interoperate as part of a large system.

[0080] As used herein in the context of computer implementation, unless otherwise specified or limited, the terms ‘"component,” “system.” “module,” “block,” and the like are intended to encompass part or all of computer-related systems that include hardware, software, a combination of hardware and software, or software in execution. For example, a component can be, but is not limited to being, a processor device, a process being executed (or executable) by a processor device, an object, an executable, a thread of execution, a computer program, or a computer. By way of illustration, both an application running on a computer and the computer can be a component. A component (or system, module, and so on) can reside within a process or thread of execution, can be localized on one computer, can be distributed between two or more computers or other processor devices, or can be included within another component (or system, module, and so on).

[0081] Also as used herein, unless otherwise limited or defined, “or” indicates a nonexclusive list of components or operations that can be present in any variety of combinations, rather than an exclusive list of components that can be present only as alternatives to each other. For example, a list of “A. B, or C” indicates options of: A; B; C; A and B; A and C; B and C; and A, B, and C. Correspondingly, the term “or” as used herein is intended to indicate exclusive alternatives only when preceded by terms of exclusivity7, such as “either,” “one of,” “only one of,” or “exactly one of.” Further, a list preceded by “one or more” (and variations thereon) and including “or” to separate listed elements indicates options of one or more of any or all of the listed elements. For example, the phrases “one or more of A, B, or C” and “at least one of A, B, or C” indicate options of: one or more A; one or more B; one or more C; one or more A and one or more B; one or more B and one or more C; one or more A and one or more C; and one or more of each of A, B. and C. Similarly, a list preceded by “a plurality of’ (and variations thereon) and including “or” to separate listed elements indicates options of multiple instances of any or all of the listed elements. For example, the phrases “a plurality of A, B, or C” and “two or more of A, B, or C” indicate options of: A and B; B and C; A and C; and A, B, and C. In general, the term “or” as used herein only indicates exclusive alternatives (e.g., “one or the other but not both”) when preceded by terms of exclusivity, such as “either.” “one of,” “only one of,” or “exactly one of.”QB\95047131.5 2686362115

[0082] Although the present technology has been described by referring to preferred examples, workers skilled in the art will recognize that changes can be made in form and detail without departing from the scope of the discussion.QB\95047131.5 27

Claims

86362115CLAIMSWhat is claimed is:

1. A system, comprising:a processor of a first computing device, the processor to:detect a second computing device separate from the first computing device, wherein the second computing device is configured to perform a function;register, at the first computing device, the function as available to the first computing device;receive, at the first computing device, a user query to perform an action; determine to execute the function for performance of the action; and transmit, to the second computing device, a function call to invoke execution of the function at the second computing device.

2. The system of claim 1, wherein the processor is to:provide the user query to an artificial intelligence (Al) agent of the first computing device;generate, with the Al agent, a prompt based on the user query;provide, with the Al agent, the prompt to a large language model (LLM) associated with the Al agent, the LLM configured to determine to execute the function for performance of the action; andreceive, from the LLM, a response to the prompt, the response including the function call.

3. The system of claim 1, wherein the processor is to:receive, from the second computing device, a notification that indicates a result of the execution of the function at the second computing device.

4. The system of claim 3. wherein the processor is to:provide, to an artificial intelligence (Al) agent, the notification related to execution of the function call at the second computing device, wherein the Al agent is configured to provide the notification to a large language model (LLM) associated with the agent, the LLM configured to generate a response to the notification;receive, from the LLM, the response; andcontrol a human machine interface (HMI) of the first computing device to output the response to a user of the first computing device.QB\95047131.5 28863621155. The system of claim 4, wherein the processor is to:provide, with the Al agent, a portion of data included in a user profile to the LLM, wherein the LLM is configured to generate the response to the notification based on the portion of data included in the user profile such that the response is generated in accordance with the user profile.

6. The system of claim 5, wherein the response is generated by the LLM using a custom prompt, wherein the custom prompt is based on the portion of data included in the user profile and the notification.

7. The system of claim 1, wherein the processor is to:transmit, to the second computing device, a request for information related to a function repository of the second computing device;receive, from the second computing device, information related to the function repository of the second computing device, wherein the information identifies the function; andadd the function to a function repository of the first computing device to register the function as available to the first computing device.

8. The system of claim 7. wherein the processor is to:generate a notification that indicates that the function is available to the first computing device; andcontrol a human machine interface (HMI) of the first computing device to output the notification to a user of the first computing device.

9. The system of claim 1, wherein execution of the function at the second computing device results in control of a hardware component of the second computing device.

10. The system of claim 1, wherein the processor is to:detect the second computing device when a distance between the first computing device and the second computing device satisfies a distance threshold.

11. A method, comprising:detecting, with a processor of a first computing device, a second computing device separate from the first computing device, wherein a distance between the first computing device and the second computing device satisfies a distance threshold, and wherein the second computing device is configured to perform a function;QB\95047131.5 2986362115registering, with the processor, with a function repository of the first computing device, the function as available to the first computing device such that the function repository of the first computing device includes the function;receiving, with the processor, at the first computing device, a user query to perform an action;determining, with the processor, using an artificial intelligence (Al) agent, to execute the function for performance of the action; andtransmitting, with the processor, to the second computing device over a communications network, a function call to invoke execution of the function at the second computing device.

12. The method of claim 11, wherein transmitting, with the processor, to the second computing device, the function call includes transmitting, with the processor, the function call to the second computing device using a communication protocol of the second computing device.

13. The method of claim 11, wherein the method includes:responsive to detecting the second computing device:transmitting, with the processor, to the second computing device over the communications network, a request for information related to a function repository of the second computing device;receiving, with the processor, from the second computing device over the communications network, information related to the function repository of the second computing device, wherein the information identifies the function and includes a name of the function, a description of the function, and a parameter of the function; andadding, with the processor, the function to the function repository of the first computing device to register the function as available to the first computing device.

14. A non-transitory computer-readable medium to store instructions that, when executed by a processor of a first computing device, cause the processor to:detect a second computing device separate from the first computing device, wherein a distance between the first computing device and the second computing device satisfies a distance threshold, and wherein the second computing device is configured to perform a function;QB\95047131.5 3086362115register, at the first computing device, the function as available to the first computing device by adding the function to a function repository of the first computing device;receive, at the first computing device, a user query to perform an action; determine to execute the function for performance of the action using a large language model (LLM) of an artificial intelligence (Al) agent; andtransmit, using the Al agent, to the second computing device using a communications protocol of the second computing device, a function call to invoke execution of the function at the second computing device.

15. The non-transitory computer-readable medium of claim 14, wherein the instructions, when executed by the processor, cause the processor to:receive, from the second computing device, information related to execution of the function at the second computing device;generate, with the LLM of the Al agent, a response based on the information related to execution of the function at the second computing device; andoutput, at the first computing device, the response.QB\95047131.5 31

Images