Abstraction of computer-based interactions for task automation

Abstract

The disclosed embodiments relate to protecting the semantic privacy of individuals while facilitating the automation of tasks across a population of individuals. [Solution] In various embodiments, data representing a set of observed interactions between a user and a computing device may be recorded and used to simulate multiple different composite sets of interactions between the user and the computing device. Each composite set may contain variations of the set of observed interactions at different levels of abstraction. User feedback may be obtained for each of the multiple different sets. Based on the user feedback, one of the multiple different composite sets of interactions may be selected and used to train a machine learning model.

Description

Technical Field

[0001] Individuals often operate computing devices to perform tasks that are likely to be replicated by others to varying degrees. For example, an individual can be involved in a series of operations using a first computer application to perform a given task, such as setting various application preferences, retrieving / viewing specific data accessible by a first computer application, performing a series of actions within a particular domain (e.g., 3D modeling, graphics editing, word processing). Another individual may be involved in a semantically similar series of actions to perform a semantically similar task in another context, such as while using another computer application later, or while using the same type of computer application but for a different purpose. Repeatedly performing actions that include these tasks can be cumbersome, error-prone, and can consume unnecessary computing resources and / or an individual's attention.

Summary of the Invention

[0002] This specification describes embodiments for protecting an individual's semantic privacy while facilitating the automation of tasks across a population of individuals. More specifically, but not by way of limitation, this specification describes embodiments for enabling an individual (often referred to as the "user") to adjust the level of abstraction associated with a captured sequence of computer-based interactions (e.g., user input, rendered output) before the captured sequence of computer-based interactions is utilized to automate the performance of tasks across the population.

[0003] In some embodiments, the method may be performed using one or more processors and include sampling a plurality of interactions between a user and a computer application, the interactions being collectively associated with a user performing a high-level task; encoding the plurality of interactions into one or more task embedding representations at a first abstraction level; processing one or more of the task embedding representations using a private machine learning model to simulate the execution of a high-level task at the first abstraction level for the user via one or more output devices; training the private machine learning model based on user input that rejects the first abstraction level, thereby generating a private machine learning model updated by training; and providing parameters for the updated private machine learning model for federated learning of a global machine learning model.

[0004] In various embodiments, the method may include: encoding multiple interactions into one or more second task embedding representations at a second abstraction level different from the first abstraction level in response to user input rejecting a first abstraction level; processing one or more of the second task embedding representations using a private machine learning model prior to training to simulate the execution of a high-level task at the second abstraction level for the user via one or more output devices.

[0005] In various embodiments, a simulated execution of a high-level task at the second level of abstraction may exclude one or more of the sampled dialogues. In various embodiments, a simulated execution of a high-level task at the second level of abstraction may exclude or obfuscate one or more pieces of information that were input or output by the user during sampling. In various embodiments, a different softmax layer temperature than that used to encode the second task embedding representation may be used to encode the first task embedding representation.

[0006] In various embodiments, providing may include providing data indicating local gradients to a remote computing system that maintains a global machine learning model. In various embodiments, the private machine learning model may be a transformer. In various embodiments, the private machine learning model may be a large-scale language model (LLM). In various embodiments, the tokens predicted based on the LLM may correspond to a first set of interactions.

[0007] In another related aspect, the method may be carried out using one or more processors and may include recording data representing an observed set of interactions between a user and a computing device, simulating a plurality of different composite sets of interactions between the user and the computing device based on the recorded data, each composite set comprising variations of the observed set of interactions at different levels of abstraction, obtaining user feedback on each of the plurality of different sets, selecting one of the plurality of different composite sets of interactions based on the user feedback, and training a machine learning model to produce an output representing the selected composite set of interactions.

[0008] In various embodiments, the simulation is performed based on a machine learning model. In various embodiments, the machine learning model can be trained to facilitate intelligent process automation.

[0009] In various embodiments, a machine learning model may be trained to generate probability distributions across an action space. In various embodiments, the machine learning model may include a private machine learning model, and the method may further include providing parameters for the trained private machine learning model for federated learning of a global machine learning model.

[0010] In addition, some embodiments include one or more processors in one or more computing devices, the one or more processors being operable to execute instructions stored in associated memory, the instructions being configured to perform any of the methods described above. Some embodiments include at least one non-temporary computer-readable storage medium storing computer instructions that can be executed by one or more processors to perform any of the methods described above. [Brief explanation of the drawing]

[0011] [Figure 1] This is a diagram of an exemplary environment in which embodiments disclosed herein may be implemented. [Figure 2] This diagram schematically illustrates an example of how the components in Figure 1 can interact and automate the execution of user tasks. [Figure 3] This flowchart shows an exemplary method for practicing selected aspects of the disclosure according to embodiments disclosed herein. [Figure 4] This flowchart shows another exemplary method for practicing selected aspects of the disclosure according to embodiments disclosed herein. [Figure 5] This diagram illustrates an exemplary architecture of a computing device. [Modes for carrying out the invention]

[0012] This specification describes embodiments for protecting the semantic privacy of individuals while facilitating the automation of tasks across a population of individuals. More specifically, but not limited to, embodiments described herein describe embodiments that allow individuals (often referred to as "users") to adjust the level of abstraction associated with a captured sequence of computer-based interactions (e.g., user input, rendered output) before the sequence of captured (e.g., recorded, observed) computer-based interactions is used to automate the performance of tasks across a population.

[0013] In various embodiments, a sequence of computer-based interactions between a user and a computing device may be recorded while the user performs high-level tasks such as writing letters, reading scientific papers, or editing digital images using the computing device. These computer-based interactions may include user inputs such as keystrokes, pointer device activity, engagement with graphical user interface (GUI) elements, and voice input, as well as outputs (e.g., auditory, visual, tactile, etc.) generated by the computing device.

[0014] To allow the user to observe and / or control how confidential and / or private information is shared and how it automates the execution of high-level tasks for other users, variations of these computer-based interactions at different levels of abstraction may then be simulated for the user. Each simulation may include a different composite set of computer-based interactions involving higher-level tasks previously performed by the user. In other words, each composite set may include variations of observed sets of interactions at different levels of abstraction. The user may respond to each simulation by providing feedback (e.g., acceptance, rejection, modification) in particular with respect to the respective levels of abstraction reflected in each simulation (e.g., level of detail of factual information).

[0015] As an example, suppose a sequence of computer-based interactions of a user interacting with a word processing application to write a letter is recorded. In an initial letter-writing simulation presented to the user, the recipient's physical address can be automatically entered. If the user does not want to share the recipient's address (for example, because it is private or not useful in other contexts), the user can provide feedback rejecting the current / applicable level of abstraction that would lead to the recipient's address being automatically entered (e.g., "No, do not include the recipient's address"). Subsequent letter-writing simulations may achieve a different level of abstraction by omitting the recipient's address, for example by including a generic address placeholder instead. If the user approves of subsequent letter-writing simulations, the data showing the computer-based interactions of those simulations can be used to automate the letter-writing task in general, for example, across a population of users.

[0016] High-level tasks can be automated in various ways. In some embodiments, “private” (e.g., local) embedded machine learning models (also referred to herein as “embedding models”) can be trained to generate embedding representations at various levels of abstraction. Additionally or alternatively, “private” (e.g., local) action machine learning models (also referred to herein as “action models”) can be trained to generate outputs that represent a synthetic sequence of computer-based interactions involved in the execution of high-level tasks. These private embedding and / or action models may be stored locally on a client device operated by the user, or in a “private cloud” whose access is controlled by the user. In either case, the learned parameters of private embedding and / or action models can be combined, for example, with the learned parameters of other private embedding and / or action models of other users, so that “global” or “public” embedding and / or action models can be trained as part of a federated learning framework.

[0017] A set of computer-based dialogues generated based on action models (private or global) as described herein is called a “synthetic” because its constituent computer-based dialogues are predicted rather than observed. Thus, a synthetic set of computer-based dialogues includes at least some predicted computer-based dialogues that are not identical to actual recorded computer-based dialogues. More generally, a synthetic set of computer-based dialogues may exhibit a different level of abstraction than a recorded set of computer-based dialogues. Recorded computer-based dialogues that could potentially expose sensitive information (e.g., social security numbers) to an untrusted agency may be “abstracted” so that the sensitive information is excluded or obfuscated. Similarly, recorded computer-based dialogues that do not necessarily expose sensitive information but are not widely applicable outside of a narrow context may be abstracted to be more widely applicable.

[0018] Private or global action models can take various forms. In some embodiments, the action model configured in selected aspects of this disclosure may take the form of a neural network trained to generate probability distributions across the action space. Based on these probability distributions, a composite set of computer-based interactions can be generated. The action space may be input to computer-based interactions (in particular, user inputs) that can be performed using a computing device, for example. To reduce and / or manage the size of the search space, in some embodiments, a domain-specific action model may be trained to generate probability distributions across the action space of a particular domain for a specific computer application, a specific context (e.g., various scientific disciplines, various positions within an organization), etc. As used herein, “domain” can mean a target area in which a computing component is intended to operate, for example, the scope of knowledge, influences, and / or activities around which the logic of the computing component revolves.

[0019] In other embodiments, the action model may take the form of a sequence-to-sequence model that generates a sequence of output tokens corresponding to and / or representing a composite set of computer-based interactions. The sequence-to-sequence action model may include various types of recurrent neural networks (RNNs), such as long-term short-term memory (LSTM) networks or gated recurrent unit (GRU) networks.

[0020] Alternatively, sequence-to-sequence action models may include various types of transformer networks, such as Bidirectional Encoder Representation (BERT) transformers or Generative Pre-trained Transformers (GPTs). In some embodiments, large-scale language models (LLMs) with numerous parameters, which can also take the form of transformers, may be used. In some such embodiments, beam search may be performed with various beam widths (which may be selected by the user, for example, as part of controlling the level of abstraction) to generate a composite set of computer-based interactions at different levels of abstraction. In some embodiments, a sequence of input tokens may represent observed computer-based interactions that actually occurred between the user and the computing device. In various embodiments, output tokens may correspond to and / or represent a composite set of computer-based interactions between the user and the computing device.

[0021] The level of abstraction associated with a set of computer-based interactions can be modified in other ways. In some embodiments, individual actions may be directly modified to change or exclude data based on commands received from the user, for example. For example, in the letter-writing example described above, the user provides the feedback "Do not include the recipient's address." As a result, natural language processing and / or pattern recognition can be performed on the created letter to identify the recipient's address, which can then be excluded, obfuscated, replaced with generic placeholders, etc., before being tokenized (e.g., converted into a domain-specific language (DSL) and / or embedding representation) and applied as input across the action model.

[0022] As another example, individual computer-based interactions can be modified to be more generally applicable, for example, symbolically and / or using various metaheuristics (e.g., simulated annealing, genetic algorithms). Suppose a physics expert is reading a digital paper on biology, and the expert receives explanations (e.g., definitions, acronyms) of various biological terms contained in the digital paper, either automatically or upon request, from a virtual assistant. These explanations may be presented to the physics expert in various ways, such as computer-generated voice and / or text from the virtual assistant, marginal annotations, or pop-up windows highlighting the terms being explained. Interactions between the physics expert, the application rendering the digital paper, the explained biological terms, the virtual assistant, and / or the entire computing device can be recorded. In some embodiments, contextual data, such as the fact that an expert in one domain (e.g., physics) is consuming media (a digital paper) from another domain (e.g., biology) that is not their area of ​​expertise, may also be recorded.

[0023] Some of these recorded computer-based dialogues can be abstracted so that they are applicable outside the specific context in which they were recorded. For example, explaining specific biological terms to an expert in physics may not be applicable in another context, such as when an expert in biology is reading a digital paper on computer science. Thus, one or more computer-based dialogues that collectively served to explain biological terms to a physics expert can be abstracted, for example, into a symbolic template that more generally explains appropriate terms or expressions in any domain to individuals who are not experts in that domain.

[0024] In some embodiments, a word or expression can be identified as suitable for an explanation based on metrics such as the length of the word, the frequency of the word, etc. (calculated, for example, using term frequency-inverse document frequency, i.e., "TF-IDF"). Suppose a biological term for which an expert in physics requests an explanation has a TF-IDF score that exceeds a certain threshold, is within a certain range, etc. The resulting symbolic template can be created to explain terms or expressions within any domain having a similar TF-IDF score. In some embodiments, the symbolic template can then be tokenized (e.g., converted into a DSL and / or an embedded representation) and applied as input across an action model.

[0025] Another way to change the level of abstraction of a set of computer-based interactions (observed or synthesized) is to reduce the dimensionality (e.g., vector / feature embeddings, DSLs, etc.) of the tokens encoding the set of computer-based interactions. In some embodiments, a recorded sequence of computer-based interactions may be tokenized / encoded into an x ​​(positive integer) dimensional embedding representation using one or more of the aforementioned private embedding models (e.g., derived from encoder-decoder machine learning models). If the user requests a higher level of abstraction, dimensionality reduction can be performed to encode the x-dimensional embedding representation into a new y-dimensional embedding representation, where y is a positive integer less than x. In some such embodiments, dimensionality reduction may be irreversible to prevent the reconstruction of information that the user intends to protect. By processing the y-dimensional embedding representation using action models configured in selected aspects of this disclosure, a synthetic set of computer-based interactions more abstract than that represented by the x-dimensional embedding representation may be obtained. In some embodiments, fuzzy semantic privacy can be used to make it difficult or impossible, at least probabilistically, to recreate the user's original scenario. For example, various types of transformers can be applied to data representing recorded user actions to create a new sequence of recorded actions. This new composite sequence of actions may be similar to the user's original actions, but may differ in details (e.g., values).

[0026] Another way to change the level of abstraction of a set of computer-based interactions is to adjust one or more hyperparameters of the action model itself. As an example, the temperature of the softmax layer of the action model can be adjusted to change the probability distribution generated by the action model. A "higher" temperature can result in a "softer", less confident, and / or more uniform probability distribution. On the other hand, a "lower" temperature can result in a "harder", more confident, and / or less uniform probability distribution. Probabilistic sampling from the former can result in more variations (and thus different levels of abstraction) than the latter.

[0027] As another example, assume that a user is editing a digital image using an image editing application. Further assume that the user is performing a specific action on the image, such as removing noise, trimming the image, converting the image to a specific resolution (e.g., a reduced resolution for viewing on a display as opposed to printing), etc. In various embodiments, the user may be presented with one or more simulations demonstrating variations of the action executed at different levels of abstraction, e.g., on the same image, on different images, or without any images at all. In the case of no images, the action may be presented to the user as a conceptual object having adjustable attributes corresponding to the parameters of the action the user performed on the actual image. The user may be able to specify the values of these parameters in order to create an appropriate abstraction for generalization. Additionally or alternatively, in some embodiments, the user may specify, e.g., using natural language, the quality of the image for which these parameters should be determined. For example, assume that the user manually trimmed the image to focus on a person depicted within the image while excluding an unwanted background. The user may be able to provide a natural language annotation such as "trim within 1 cm of the subject's face in all directions" as feedback in response to subsequent simulations.

[0028] Figure 1 schematically illustrates exemplary environments in which selected aspects of this disclosure may be implemented in various embodiments. Any computing device shown in Figure 1 or other figures may include logic such as one or more microprocessors (e.g., a central processing unit i. "CPU", a graphical processing unit i. "GPU", a tensor processing unit i. "TPU") that execute computer-readable instructions stored in memory, or other types of logic such as application-specific integrated circuits ("ASICs"), field-programmable gate arrays ("FPGAs"). Some of the systems shown in Figure 1, such as the task automation system 120, may be implemented using one or more server computing devices that form what is sometimes called a "cloud infrastructure," but this is not required.

[0029] The task automation system 120 can be operably coupled with one or more client computing devices (also referred to herein as “clients”), such as client computing device 110, via one or more computer networks 114. The task automation system 120 can be used to automatically determine a set of computer-based interactions and attempt to automate higher-level tasks performed by the user of a client device (e.g., 110).

[0030] An individual (which may also be called a “user” in the current context) may interact with other components shown in Figure 1 by operating a client device 110. Each client device 110 may be, for example, a desktop computing device, a laptop computing device, a tablet computing device, a mobile phone computing device, a computing device in a participant’s vehicle (such as an in-vehicle communication system, an in-vehicle entertainment system, or an in-vehicle navigation system), a standalone interactive speaker (with or without a display), or a wearable device including a computing device such as a head-mounted display (“HMD”) or a “smart” watch that provides an AR or VR immersive computing experience. Additional and / or alternative client devices may be provided.

[0031] The examples described herein generally relate to a user who operates a computing device, such as client device 110, to record a sequence of automated computer-based interactions. However, this is not intended to be limiting. A sequence of computer-based interactions may also include interactions with other types of computing devices. For example, when a driver operates a vehicle equipped with various in-vehicle circuits and / or logic configured in selected aspects of this disclosure, interactions between the driver and the vehicle may be recorded. These interactions may then be simulated at different levels of abstraction, for example, the driver or another individual (e.g., a person tasked with training one or more machine learning models). Based on these simulations, feedback may be provided that enables the automation of at least a portion of the driving interactions applicable outside the specific context in which the driver initially operated the vehicle. For example, by recording and abstracting the interactions in which the driver was involved during parallel parking between two vehicles, it may be possible to enable parallel parking between any two structures.

[0032] The client device 110 may include one or more applications, such as application 112, that interact with the task automation system 120. For example, application 112 may provide user input through it and render outputs generated by the task automation system 120 to the user, such as outputs that request user feedback (e.g., outputs that reflect the final state of a simulation of a set of candidate dialogues) and / or outputs that reflect a set of computer-based dialogues determined by the task automation system 120 (e.g., outputs for confirmation of set automation). In some embodiments, if the task involves controlling a computer application running on the client 110, application 112 (or another application) may be controlled using a composite set of computer-based dialogues determined by the task automation system 120.

[0033] In various embodiments, the task automation system 120 includes a global embedding engine 122, a global action engine 124, a global selection engine 126, a global simulation (SIM) engine 128, and / or a global evaluation engine 130. Although the task automation system 120 is shown in Figure 1 to be connected to the client 110 via a network 114, in various embodiments, one or more aspects of the task automation system 120 may be combined and / or implemented locally in the client 110. For example, one or more engines of the task automation system 120 may be additionally or alternatively implemented in the client 110. For example, the client 110 includes a private embedding engine 122A, a private action engine 124A, a private selection engine 126A, a private SIM engine 128A, and / or a private evaluation engine 130A. Additionally or alternatively, in some embodiments, all or part of the task automation system 120 may be implemented in a user-controlled private cloud (not shown), thereby enabling the utilization of substantially unlimited resources of the cloud without compromising the individual privacy or security of the users.

[0034] The public embedding engine 122 and / or the private embedding engine 122A may interface with one or more public and / or private embedding ML models 152, 152A when generating the embedding representations described herein. Which embedding ML models 152 / 152A the embedding engine 122 / 122A interfaces with, and / or the data processed when interfaceing with one or more of the embedding ML models 152 / 152A, may depend on the embedding technology used by the embedding engine 122 / 122A.

[0035] For example, given an NL input, the embedding engine 122 / 122A can process the NL input data using a domain-specific LLM model of the embedding model 152 / 152A and generate an embedding representation based on a first embedding technique that reflects the NL input. For various types of computer-based interactions, the embedding engine 122 / 122A can generate an embedding representation based on a second embedding technique that includes processing the computer-based interaction using some other domain-specific model.

[0036] The embedding techniques used in a given case by the embedding engine 122 / 122A may depend on various factors and, in some embodiments, may be indicated by the selection engine 126 / 126A. For example, the embedding techniques used may depend on the domain of the task, the computer-based dialogue in which the embedding representation is being generated, and / or the action models 154 / 154A used by the action engine 124 / 124A when generating a candidate composite set of computer-based dialogues. Various embedding ML models 152 / 152A can be provided. For example, embedding ML models 152 / 152A may include those specific to a particular domain, those specific to a particular set of domains, and / or those that are domain-independent. As another example, embedding ML models 152 / 152A may additionally or alternatively include those specific to a first type of data (e.g., natural language data), those specific to a second type of data (e.g., computer-based dialogues), and so on.

[0037] The global action engine 124 can interface with one or more global (or "public") action models 154 to facilitate the automation of a set of computer-based interactions. In some embodiments, one or more global action models 154 can be trained to facilitate robotic process automation and / or intelligent process automation. Similarly, the private action engine 124A can interface with one or more private action models 154A to facilitate the automation of a set of computer-based interactions while maintaining user privacy. In some embodiments, the action engines 124 / 124A also interface with one or more action rules 164 / 164A. Action rules 164 / 164A can be used to eliminate a candidate composite set of several generated computer-based interactions from further considerations (e.g., from further considerations by the evaluation engine 130 / 130A). Action rules 164 / 164A may be domain-specific and / or specific to a corresponding requesting entity such as a user or an organization associated with a user. For example, for a particular domain and organization, a given action rule can define that a given action is not permitted at all, or is not permitted if it occurs before or after another action.

[0038] Which action engine 124 / 124A interfaces with which action model 154 / 154A in a given case may depend on various factors and, in some embodiments, may be indicated by selection engine 126 / 126A. For example, the action model 154 / 154A used may depend on the task domain, the input from which the embedding representation is generated, and / or the embedding representation generated by embedding engine 122 / 122A. Also, for example, the action model 154 / 154A used for input in a given case may additionally or alternatively be based on action models used in previous cases when generating candidate composite sets of computer-based dialogues for input and / or evaluations of those candidate composite sets.

[0039] Various action models 154 / 154A can be provided. For example, an action model 154 / 154A may include machine learning models and / or heuristic models. As another example, an action model 154 / 154A may include those specific to a particular domain, those specific to a particular set of domains, and / or those that are domain-independent. As yet another example, an action model 154 / 154A may include those that represent a reinforcement learning (RL) policy and are used to generate candidate composite sets of computer-based dialogues by iteratively generating the corresponding next action of a candidate composite set based on applying updated state data in each iteration, those used to generate one or more candidate composite sets of computer-based dialogues in a single iteration, those that represent a value function and are used to generate a measure that reflects a set of computer-based dialogues, a value of a pair of current states, and / or other action models. For example, an action model 154 / 154A may include one or more of the following: an RL policy machine learning (ML) model, an action sequence ML model, a constraint satisfaction model, a SAT solver, and / or other models.

[0040] In some embodiments, the selection engine 126 / 126A can interact with the embedding engine 122 / 122A to indicate which embedding technique is being used by the embedding engine 122 / 122A in a given case, and / or can interact with the action engine 124 / 124A to indicate which action model is being used by the global action engine 124 / 124A in a given case. For example, the selection engine 126 / 126A can instruct the embedding engine 122 / 122A to initially utilize a first embedding technique. Then, if the evaluation engine 130 / 130A indicates, for example, based on user feedback, that the corresponding set of computer-based interactions generated based on the first embedding technique is inappropriately abstract, the selection engine 126 / 126A can instruct the embedding engine 122 / 122A to utilize a second embedding technique when generating additional embeddings. As another example, the selection engine 126 / 126A may instruct the embedding engine 122 / 122A to initially utilize the first and second embedding techniques. Then, only if the evaluation engine 130 / 130A indicates that the corresponding candidate synthesis set of computer-based dialogues generated based on the first and second embedding techniques is unsuitable, the selection engine 126 / 126A may instruct the embedding engine 122 / 122A to utilize the third and / or fourth embedding techniques when generating additional embedding representations.

[0041] In some embodiments, the selection engine 126 / 126A may optionally utilize one or more selection models 156 / 156A when determining which embedding technique and / or action to use in a given case. For example, the selection model 156 / 156A may include a selection ML model that can be used to process the domain of the task and / or NL input data requesting the task (e.g., an embedding representation of the NL input data) and generate an output showing the corresponding probability for each of a plurality of embedding techniques and / or action models. The selection engine 126 / 126A may utilize this generated output when selecting which embedding technique and / or action model to use. For example, the selection engine 126 / 126A may use this output to select the embedding technique and / or action model with the highest probability for initial use. Such a selection ML model may be trained on supervised training examples based on a past set of computer-based interactions that have been determined to be appropriately abstract (and optionally confirmed to be appropriate after their real-world implementation).

[0042] The SIM engine 128 / 128A can be used to simulate the execution of each set of computer-based dialogues generated by the action engine 124 in a simulated environment, such as a simulated environment that reflects the current state of the domain. Furthermore, the SIM engine 128 / 128A generates simulation data for each simulation. In some situations, action sets can be generated during simulation via the SIM engine 128. For example, some RL policy models can be used to generate a set of computer-based dialogues during simulation, which are executed during the simulation, and whose generation depends on the simulated state encountered during the simulation.

[0043] The evaluation engine 130 / 130A can determine whether a candidate composite set of computer-based interactions is appropriately abstract, and / or determine the most appropriate abstraction of a candidate composite set from among multiple candidate composite sets of computer-based interactions. In various embodiments that utilize simulation data when evaluating candidate composite sets of computer-based interactions, the evaluation engine 130 / 130A can request and / or utilize user feedback based on the simulation data. For example, the evaluation engine 130 / 130A can have a user who has recorded an observed set of computer-based interactions render simulation data from the simulation, and determine suitability based on feedback from the user in response to the rendering. For example, the evaluation engine 130 / 130A can have client 110 render screenshots, final states, videos, etc., of the simulated environment in its final state from the simulation. In response, the user can provide user interface inputs that reflect whether the output is sufficiently abstract to protect user privacy while maintaining a state suitable for overall automation of higher-level tasks. The evaluation engine 130 / 130A can use instances of negative feedback to eliminate a corresponding candidate composite set of computer-based dialogues or to negatively impact the suitability metric of a candidate composite set. Conversely, the evaluation engine 130 / 130A can use instances of positive feedback to select a candidate composite set of computer-based dialogues as the most appropriate or to positively impact the suitability metric of a corresponding candidate composite set. In various embodiments, when evaluating a composite set of computer-based dialogues, the evaluation engine 130 / 130A utilizes simulation data from the simulation of action sets by the SIM engine 128.

[0044] In some of these embodiments, the evaluation engine 130 / 130A can compare simulation data with one or more state rules 160 / 160A, which can be used to determine that a candidate action set is unsuitable and / or to negatively affect the suitability score for the candidate action set used when determining the suitability of the candidate action set. The state rules 160 / 160A can be domain-specific and / or specific to the corresponding request entity, such as an organization provided by or associated with the user. For example, for a particular domain and a particular organization, a given state rule (e.g., predefined or provided by the user as feedback) can define that a given state should never be encountered, or a particular sequence of states should never be encountered. If the simulation data from the simulation of the candidate action set indicates that a given state and / or a particular sequence of states was encountered, the evaluation engine 130 / 130A can determine that the candidate action set is unsuitable. As another example, for a particular domain and organization, a given state rule may define a given state or a particular sequence of states as undesirable but not prohibited. If simulation data from a simulation of a candidate action set indicates that a given state and / or a particular sequence of states was encountered, the evaluation engine 130 / 130A may negatively impact the suitability metric for the candidate action set.

[0045] The machine learning models described herein are of various architectures and can be trained in various ways. For example, one or more of the models may be graph-based neural networks (e.g., as graph neural networks (GNN), graph attention neural networks (GANN), or graph convolutional neural networks (GCN)), sequence-to-sequence neural networks such as transformers, encoder-decoders, or recurrent neural networks ("RNN", e.g., Long-Term Short-Term Memory i.e., "LSTM", Gate Recurrent Unit i.e., "GRU"), or BERT (a bidirectional encoder representation from a transformer). Furthermore, reinforcement learning, supervised learning, and / or imitation learning can be used, for example, when training one or more of the machine learning models. Further descriptions of several implementations of various machine learning models are provided herein.

[0046] To summarize Figure 1, the global (or "public") engines 122, 124, 126, 128, and / or 130 can be used, for example, in response to user requests to perform various high-level tasks, to select from multiple different candidate composite sets of computer-based interactions for execution in the real world. Furthermore, one or more of the global engines 122, 124, 126, 128, and / or 130 can facilitate federated learning of one or more models 152, 154, 156 based on local model parameters / local gradients received from the private (or "local") engines 122A, 124A, 126A, 128A, and / or 130A. For those parts, the private (or "local") engines 122A, 124A, 126A, 128A, and / or 130A can facilitate the generation and distribution of these local model parameters / local gradients based on user feedback provided for the simulation of composite computer-based interactions at various levels of abstraction.

[0047] Referring to Figure 2, examples of engines 122A, 124A, 126A, 128A, and 130A of the task automation system 120 implemented in the client device 110 for the purpose of federative learning are illustrated. Figure 2 also shows the possible interactions that may occur between these engines, models 152A, 154A, and 156A, and rules 160A and 164A (which may in some cases be the same as 160 and 164) that can be used by the client 110 to facilitate federative learning.

[0048] In Figure 2, the private embedding engine 122A processes various data, including data representing observed computer-based interactions ("interactions") 104 and, optionally, NL inputs 101. Based on this data, the private embedding engine 122A generates a task embedding representation 123. For example, domain-specific knowledge (DSK) 102 may include various reference documents or other supplementary data that can be provided as additional input to effectively "tune" or "prime" the model (e.g., 152A, 154A), similar to small-shot learning (except that the model weights may or may not be tuned). The context 103 may include a wide variety of data points, such as signals generated by the client 110 (e.g., location coordinates, time, foreground and / or background applications, other sensor signals, etc.), user preferences, and the user's area of ​​expertise (e.g., physics expert, biology expert, etc.).

[0049] Data 104 may include any data recorded to document computing interactions between the user and one or more computer applications running on the client device 110. In some embodiments, data 104 may include hardware inputs such as keystrokes, pointer device movements and / or actions (e.g., clicks, right-clicks, scrolls, etc.). In some embodiments, data 104 may include application-specific interactions such as interactions with graphical elements and / or menu items, sequences of input commands (including NL inputs 101 provided for interacting with one or more computer applications), and rendered auditory and / or visual outputs.

[0050] In some embodiments, data 104 may include application-specific computer code, such as code that may be generated in certain applications (e.g., word processing, spreadsheets, etc.) when a user chooses to record a “macro.” In some embodiments, data 104 may include user-entered information, such as information used to create documents (e.g., letters, emails, reports), information used to fill out spreadsheets, information used to fill out forms (e.g., web pages or parts of applications), information used to create graphic designs or drawings, and information exchanged with a virtual assistant.

[0051] In any case, the private embedding engine 122A may process data 104 (and other data 101-103, if applicable) to generate one or more task embedding representations 123. In some embodiments, a task embedding representation 123 may include a separate token / embedding representation that encodes each interaction (e.g., user input, output from a computer application). In some embodiments, multiple interactions may be combined (e.g., aggregated, concatenated) to form a semantically rich task embedding representation 123 representing multiple interactions.

[0052] NL input 101 (which is optional and / or may be recorded as part of or separately from data 104) may be provided by the user through interaction with the user interface input device of a client device (e.g., client 110 in Figure 1). For example, the user may provide NL input 101 such as, "I am now drafting a letter. Record my actions to automate this process and proceed." In some cases, NL input 101 may be the user's voice input detected via the microphone of client 110. The private embedding engine 122A may process recognized text generated based on voice input (e.g., using automatic speech recognition (ASR)) when generating the task embedding representation 123. As another example, NL input 101 may be typed text provided via a virtual or hardware keyboard of client 110, and the typed text may be processed by the private embedding engine 122A when generating the task embedding representation 123. For example, recognized text or typed text may be processed using the NL ML model of ML model 152A to generate the NL embedding representation. The NL ML model can be, for example, an LLM. The task embedding representation 123 may be this NL embedding representation, or it may be a function of this NL embedding representation and other NL embedding representations.

[0053] The private action engine 124A processes the task embedding representation 123 using one or more action ML models to generate one or more candidate composite sets 125 of computer-based interactions. For example, in a given case, the private action engine 124A may process the task embedding representation 123 using one of the following: the first RL policy model 154B, the second RL policy model 154C, the constraint satisfaction model 154D, the action sequence model 154N, or other models of the action model 154 (e.g., other models indicated by the vertical ellipsis in Figure 2). In some of these embodiments, the selection engine 126 may indicate which of the action models 154 is used by the private action engine 124A in a given case. In some embodiments, the action sequence model 154N is a sequence-to-sequence model, such as a transformer model (e.g., BERT, GPT, etc.), which can be applied to predict a sequence of tokens representing a sequence of computer-based interactions, for example, iteratively or one at a time.

[0054] The private SIM engine 128A can be used to simulate the execution of an action set in a simulated environment, such as a simulated environment reflecting the current state of a domain, for each of the candidate composite sets 125 of computer-based dialogues. Furthermore, the private SIM engine 128A generates simulation (SIM) data 127 for each simulation. In some situations, the candidate composite sets of computer-based dialogues for set 125 may be generated independently of their simulation, and the private SIM engine 128A may be used to simulate the action set after the candidate composite sets of computer-based dialogues have been generated. In some other situations, the candidate composite sets of computer-based dialogues may be generated by the private action engine 124A during the simulation via the private SIM engine 128A. This is reflected by the dashed double arrow between the private action engine 124A and the private SIM engine 128A.

[0055] The SIM data 127 can be presented to the user (e.g., rendered) by, for example, a private SIM engine 128A or a private evaluation engine 130A, so that the user can provide feedback 129. For example, when the SIM data 127 is presented to the user, for example visually (e.g., as an animation, a snapshot of the current or final state of the domain, a final document, etc.) and / or audibly, the user may be prompted to provide feedback 129 to accept, reject, and / or modify one or more levels of abstraction of computer-based interactions from a candidate composite set of computer-based interactions 125. This may allow the user to control how much personal and / or sensitive information is provided to untrusted and / or public entities.

[0056] The private evaluation engine 130A can determine, based on user feedback 129, whether the corresponding candidate action set of the candidate composite set 125 of the computer-based dialogue is appropriately abstract to protect user privacy, and / or determine the most appropriately abstract candidate composite set of the computer-based dialogue from among multiple candidate composite sets of the computer-based dialogue. In some of these embodiments, the private evaluation engine 130A can compare simulation data with one or more state rules 160A, which can be used to determine that the candidate composite set of the computer-based dialogue is inappropriately abstract (e.g., too specific, contains personal data, or is not broadly applicable), and / or negatively impacts the suitability score used when determining the suitability of the candidate composite set of the computer-based dialogue.

[0057] If the private evaluation engine 130A determines, for example, based on user feedback 129, that none of the candidate composite sets 125 of computer-based dialogues are suitable, it can output an instruction 131 indicating that they are unsuitable to the private selection engine 126A. In response, the private selection engine 126A can adapt the embedding techniques used by the private embedding engine 122A and / or the private action model 154A used by the private action engine 124A. Furthermore, the candidate composite sets 125 of computer-based dialogues may then be generated based on different task embedding representations 123 (e.g., generated using alternative embedding techniques) and / or different action models of the private action model 154A.

[0058] For example, the private selection engine 126A can adapt the embedding technique being used (e.g., by reducing the dimensionality of the embedding), but it cannot adapt the action model 154 being used. In contrast, the private embedding engine 122A can generate different task embedding representations 123 using different adapted embedding techniques, and the private action engine 124A processes the different task embedding representations 123 using the same action model as before. This may result in the generation of different candidate composite sets 125 for the computer-based dialogue due to the different task embedding representations 123. The different action sets 125 can be simulated by the private SIM engine 128A, and the resulting SIM data 127 can be used by the private evaluation engine 130A to enable the user to provide new feedback 129 regarding the different candidate composite sets 125 for the computer-based dialogue. This can be repeated multiple times, for example, until the private evaluation engine 130A and / or the user determine that the evaluated candidate composite sets for the computer-based dialogue are suitable for propagation to global and / or public entities.

[0059] The private evaluation engine 130A (and / or private selection engine 126A) can use other techniques to control the level of abstraction used to automate tasks. In some embodiments, based on user feedback 129, the private evaluation engine 130A can use various rules and / or heuristics to modify certain information recorded as part of the interaction data 104. One example mentioned above involved the user indicating (as part of the feedback 129) that they should not use the address of a particular recipient when attempting to automate the task of writing a letter. Another example is that during the automation of filling out a webpage form to make a purchase, the user may reject a simulation that presents a candidate composite set (125) of computer-based interactions that include a form field in which a particular credit card number is entered. This could result in, for example, the user's credit card number being excluded, obfuscated, and replaced with a generic placeholder before the private embedding engine 122A generates a new task embedding representation 122A.

[0060] In some embodiments, the private evaluation engine 130A, the private selection engine 126A, and / or the private action engine 124A may select different private action models 154A and / or modify one or more parameters of a particular action model 154A to control / modify the level of abstraction. As an example, the temperature of the softmax layer of the action model may be adjusted to change the probability distribution generated by the action model. A "higher" temperature may result in a "softer," less reliable, and / or more uniform probability distribution. On the other hand, a "lower" temperature may result in a "harder," more reliable, and / or less uniform probability distribution. Probabilistic sampling from the former may result in more variation (and therefore different levels of abstraction) than from the latter.

[0061] Returning to Figure 2, if the private evaluation engine 130A determines in a given iteration that one of the candidate composite sets 125 of computer-based dialogues is appropriate (for example, if user feedback 129 indicates that dialogue 125 is appropriately abstract), the private evaluation engine 130A may generate an appropriate instruction 132. In various embodiments, based on the inappropriate instruction 131 and / or appropriate instruction 132, a local gradient 133 may be calculated using techniques such as backpropagation, gradient descent, or cross-entropy. As indicated by the arrows, the local gradient 133 may then be used to update the private embedded ML model 152A and / or the private action model 154A. The updated parameters of the local gradient 133 and / or the private embedded ML model 152A and / or the private action model 154A may then be provided to the global embedding engine 122 and / or the global action engine 124. The global embedding engine 122 can update the global embedding model 152 as part of a federated learning framework using local gradients 133 received from multiple different clients 110. Similarly, the global action engine 124 can update the global action model 154 as part of a federated learning framework using local gradients 133 received from multiple different clients 110.

[0062] Figure 3 is a flowchart illustrating an exemplary method 300 that implements a selected aspect of the disclosure according to embodiments disclosed herein. For convenience, the operations in the flowchart are described with reference to a system that performs the operations. This system may include various components of various computer systems, such as one or more components of the task automation system 120. Furthermore, the operations of method 300 are shown in a particular order, but this is not intended to be limiting. One or more operations can be rearranged, omitted, or added.

[0063] In block 302, the system may record data (e.g., 104) indicating a set of observed interactions between the user and the computing device, for example, by application 112, the operating system, and / or private embedded engine 122A. In some embodiments, this recording may be triggered by a command from the user, which may be issued by the user using various types of input (e.g., keyboard input, pointer device input, voice input, etc.). In other embodiments, this recording may be triggered automatically, for example, in response to the detection that the user has repeatedly performed several actions. For example, if the user repeatedly performs several sequences of mostly similar actions, an agent such as a virtual assistant or application assistant configured in a selected aspect of this disclosure may issue a prompt such as, "Okay, you want me to [insert task name] again? Would you like me to generate an automated routine to perform those steps for you in the future?"

[0064] Based on this recorded data, in block 304, the system may simulate multiple different composite sets of interactions between the user and the computing device, for example, by a private SIM engine 128A. Each composite set of computer-based interactions may be a variation of an observed set of interactions at different levels of abstraction. In some embodiments, the simulation in block 304 may be based on a machine learning model. The machine learning model may be a private action model 154A that is trained to generate probability distributions across action spaces, such as the action space of the domain in which the user is operating (e.g., word processing, graphic design, web browsing, spreadsheet manipulation, etc.).

[0065] For example, RL policies 154B and / or 154C may generate a probability distribution across the domain's action space in each iteration based on the domain's current state. Based on that probability distribution, one or more subsequent actions may be selected and simulated, and the process may be repeated. Similarly, the action sequence model 154N may be a sequence-to-sequence model that generates, for example, a sequence of tokens at once, each representing one or more actions in the domain's action space. In some such embodiments, each token may contain a probability distribution across several different actions. In other such embodiments, the entire sequence may be assembled based on the probability distribution and then simulated.

[0066] Assume the iterative task is to compose a letter. The user may be presented with simulations such as one where the recipient's street address is omitted or replaced with a placeholder, but the recipient's city and / or state is preserved; another simulation where the recipient's address (including city and state) is completely omitted or replaced with a placeholder; and yet another simulation where all or part of the letter's body, sender's address, etc., are abstracted.

[0067] Suppose a repetitive task prepares and / or organizes a spreadsheet to input an output range of cells by performing some calculations based on data contained in the input range of cells. One simulation might include specific values ​​in the input range of cells and the formula used to input the output range of cells. Another simulation might not include specific values ​​in the input range of cells, or it might include obfuscated or random values, but still be able to input the same formula used by the user into the output range of cells. Yet another simulation might include only the general formatting adopted by the user in the original spreadsheet, without including any data used or calculated for the user.

[0068] In block 306, the system may obtain user feedback on each of several different sets, for example, via the private evaluation engine 130A. In some embodiments, this feedback may be obtained after each simulation (e.g., the user may be prompted to provide feedback), in which case, once the user accepts the latest simulation, no further simulations need to be performed. In other embodiments, the user may be presented with several simulations before feedback is obtained. In the latter case, the user may identify one or more simulations that were satisfactory and / or one or more simulations that were unsatisfactory (e.g., because they would potentially leak confidential or personal information).

[0069] Based on user feedback obtained in block 306, the system may, in block 308, for example, by the private evaluation engine 130A, select one of several different composite sets of interactions. For example, if the user is presented with a satisfactory simulation that is comfortable and does not result in the inadvertent disclosure of confidential (or more generally, not widely applicable) information, the user may accept the simulation.

[0070] In block 310, the system may train a machine learning model to produce an output representing a composite set of selected dialogues. For example, in block 310A, the private evaluation engine 130A, the private embedding engine 122A, and / or the private action engine 124A may train the private embedding ML model 152A and / or the private action model 154A. In block 310B, the private evaluation engine 130A, the private embedding engine 122A, and / or the private action engine 124A may provide one or more parameters (e.g., local gradients) of the private ML models 152A and 154A to the global embedding engine 122 and / or the global action ML model 124 for federated learning of the global embedding ML model 152 and / or the global action ML model 154. Combined with other local gradients, the resulting global embedding ML model 152 and / or global action ML model 154 can be distributed to other clients to facilitate task automation.

[0071] Figure 4 is a flowchart of another exemplary method 400 that puts into practice a selected aspect of the disclosure according to embodiments disclosed herein. For convenience, the operations in the flowchart are described with reference to a system that performs the operations. This system may include various components of various computer systems, such as one or more components of the task automation system 120. Furthermore, although the operations of method 400 are shown in a particular order, this is not intended to be limiting. One or more operations can be rearranged, omitted, or added.

[0072] In block 402, the system can sample (e.g., record) multiple interactions between a user and a computer application, for example, by a private embedded engine 122A. These interactions may be collectively associated with a user performing high-level tasks such as writing a letter, reading an academic paper (e.g., within or outside the user's area of ​​expertise), or manipulating a spreadsheet or other document.

[0073] In block 404, the system may encode multiple interactions into a first task embedding representation at a first abstraction level, for example, by a private embedding engine 122A. In some such embodiments, the private embedding engine 122A may utilize one or more private embedding ML models 152A to perform the encoding in block 404. These private embedding ML models 152A may be selected by a private selection engine 126A based on, for example, context 103, interaction data 104, NL input 101 (if any), DSK 102, etc.

[0074] In block 406, the system may process a first task embedding representation using a private machine learning model (e.g., 154A) by, for example, a private action engine 124A and / or a private SIM engine 128A (which may be combined into a single unit in some cases) to simulate the execution of a high-level task at a first abstraction level for the user via one or more output devices.

[0075] In block 408, the system may receive prompted or unsolicited user feedback, for example, by the private SIM engine 128A and / or the private evaluation engine 130A. If the user feedback includes a rejection at the first level of abstraction, in block 410, the private embedded ML model 152A and / or the private action ML model 154A may be trained, for example, by the private evaluation engine 130A and / or the private action engine 124A, resulting in the generation of a first updated private ML model.

[0076] In block 412, the system may, for example, provide the parameters of the first updated private machine learning model to the global embedding ML model 152 and / or global action ML model 154, for federated learning of the global embedding ML model 152 and / or global action ML model 154, for example, by the private action engine 124A and / or private evaluation engine 130A.

[0077] In block 414, the system may determine, for example, by a private evaluation engine 130A, whether the user has accepted or rejected a simulated execution of a high-level task at the first abstraction level. If the answer is "yes", method 400 may terminate. However, if the answer in block 414 is "no", method 400 may return to block 402 and repeat the process. For example, in response to user input rejecting the first abstraction level, multiple identical or different sampled dialogues (e.g., if the user rejects a particular dialogue) may be encoded, for example, by a private embedding engine 122A, into a second task embedding representation at a second abstraction level different from the first abstraction level. The second task embedding representation may be used for the user via one or more output devices to simulate the execution of a high-level task at the second abstraction level. The process may be repeated as described above for N iterations, as shown in Figure 4.

[0078] Figure 5 is a block diagram of an exemplary computing device 510 that can be optionally used to implement one or more aspects of the technology described herein. In some embodiments, a client device 110, a task automation system 120, and / or other components may comprise one or more components of the exemplary computing device 510.

[0079] The computing device 510 typically includes at least one processor 514 that communicates with several peripheral devices via a bus subsystem 512. These peripheral devices may include, for example, a storage subsystem 524 including a memory subsystem 525 and a file storage subsystem 526, a user interface output device 520, a user interface input device 522, and a network interface subsystem 516. The input and output devices enable the computing device 510 to interact with a user. The network interface subsystem 516 provides an interface to an external network and is coupled to a corresponding interface device in another computing device.

[0080] The user interface input device 522 may include pointing devices such as keyboards, mice, trackballs, touchpads, or graphics tablets, scanners, touchscreens integrated into displays, voice recognition systems, microphones, and / or other types of input devices. Generally, the use of the term “input device” is intended to include all possible types of devices and methods for inputting information into the computing device 510 or a communication network.

[0081] The user interface output device 520 may include a non-visual display such as a display subsystem, printer, fax machine, or audio output device. The display subsystem may include a flat panel device such as a cathode ray tube (CRT), liquid crystal display (LCD), projection device, or any other mechanism for creating a visible image. The display subsystem may also provide a non-visual display via an audio output device, etc. In general, the use of the term “output device” is intended to include all possible types of devices and methods for outputting information from the computing device 510 to a user or another machine or computing device.

[0082] The storage subsystem 524 stores programming and data structures that provide some or all of the functionality of the modules described herein. For example, the storage subsystem 524 may include logic for performing selected aspects of Method 300 in Figure 3, Method 400 in Figure 4, and / or other methods described herein.

[0083] These software modules are generally executed by processor 514 alone or in combination with other processors. The memory 525 used within the storage subsystem 524 may include several memories, including main random access memory (RAM) 530 for storing instructions and data during program execution, and read-only memory (ROM) 532 for storing fixed instructions. The file storage subsystem 526 can provide persistent storage for program and data files and may include a hard disk drive, a floppy disk drive with associated removable media, a CD-ROM drive, an optical drive, or a removable media cartridge. Modules performing the functions of a particular embodiment may be stored by the file storage subsystem 526 within the storage subsystem 524 or on other machines accessible by processor 514.

[0084] The bus subsystem 512 provides a mechanism for various components and subsystems of the computing device 510 to communicate with each other as intended. Although the bus subsystem 512 is schematically shown as a single bus, alternative embodiments of the bus subsystem may use multiple buses.

[0085] The computing device 510 can be of various types, including workstations, servers, computing clusters, blade servers, server farms, or any other data processing systems or computing devices. Due to the constantly changing nature of computers and networks, the description of the computing device 510 shown in Figure 5 is intended only as a specific example to illustrate several embodiments. Many other configurations of the computing device 510 are possible, having more or fewer components than the computing device shown in Figure 5.

[0086] While several embodiments have been described and illustrated herein, various other means and / or structures can be utilized to perform the function and / or to obtain one or more of the results and / or benefits described herein, and each of such variations and / or modifications is considered to be within the scope of the embodiments described herein. More generally, all parameters, dimensions, materials and configurations described herein are intended to be illustrative, and actual parameters, dimensions, materials and / or configurations depend on one or more specific uses in which this teaching is used. Those skilled in the art will be able to recognize or investigate many equivalents to the specific embodiments described herein simply by using routine experiments. Thus, it will be understood that the embodiments described herein are presented only as examples, and that within the scope of the appended claims and their equivalents, embodiments may be practiced in ways other than those specifically described and claimed. Embodiments of this disclosure cover each individual feature, system, article, material, kit and / or method described herein. Furthermore, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the scope of this disclosure, provided that such features, systems, articles, materials, kits, and / or methods are not inconsistent with each other.

Claims

1. A method that is carried out using one or more processors, Sampling multiple interactions between a user and a computer application, wherein the interactions are collectively associated with the user performing a high-level task. Encoding the aforementioned multiple dialogues into one or more first task embedding representations at the first abstraction level, Using a private machine learning model to process one or more of the first task embedding representations, to simulate the execution of the high-level task at the first abstraction level for the user via one or more output devices, Training the private machine learning model based on user input rejecting the first level of abstraction, thereby generating an updated private machine learning model, and To provide the parameters of the updated private machine learning model for federated learning of the global machine learning model, Methods that include...

2. In response to the user input rejecting the first level of abstraction, the plurality of dialogues are encoded into one or more second task embedding representations at a second level of abstraction different from the first level of abstraction. The method according to claim 1, further comprising processing one or more of the second task embedding representations using the private machine learning model before the training, to simulate the execution of the high-level task at the second abstraction level for the user via one or more of the output devices.

3. The method according to claim 2, wherein the simulated execution of the high-level task at the second level of abstraction excludes one or more of the sampled dialogues.

4. The method according to claim 2 or 3, wherein the simulated execution of the high-level task at the second level of abstraction excludes or obfuscates one or more pieces of information input by the user or output to the user during the sampling.

5. The method according to any one of claims 2 to 4, wherein a softmax layer temperature different from that used to encode the second task embedding representation is used to encode the first task embedding representation.

6. The method according to any one of claims 1 to 5, wherein the provision includes providing data indicating local gradients to a remote computing system that maintains the global machine learning model.

7. The method according to any one of claims 1 to 6, wherein the private machine learning model includes a transformer.

8. The method according to any one of claims 1 to 7, wherein the private machine learning model includes a large-scale language model (LLM).

9. The method according to claim 8, wherein the tokens predicted based on the LLM correspond to the first plurality of dialogues.

10. A method that is carried out using one or more processors, Recording data that shows an observed set of interactions between the user and the computing device. Based on the recorded data, simulate a plurality of different composite sets of interactions between the user and the computing device, each composite set including variations of the observed set of interactions at different levels of abstraction. Obtaining user feedback on each of the aforementioned multiple different composite sets of dialogues, Based on the user feedback, select one of the multiple different composite sets of dialogues, and To train a machine learning model to produce an output that represents the composite set of the selected dialogues, Methods that include...

11. The method according to claim 10, wherein the simulation is performed based on the machine learning model.

12. The method according to claim 11, wherein the machine learning model is trained to facilitate intelligent process automation.

13. The method according to claim 11 or 12, wherein the machine learning model is trained to generate a probability distribution across the action space.

14. The method according to any one of claims 11 to 13, wherein the machine learning model includes a private machine learning model, and the method further comprises providing parameters of the trained private machine learning model for federated learning of a global machine learning model.

15. A system comprising one or more processors and a memory for storing instructions, wherein the instructions, in response to execution by the one or more processors, are to be performed by the one or more processors. Recording data that shows a set of observed interactions between the user and the computing device. Based on the recorded data, simulate a plurality of different composite sets of interactions between the user and the computing device, each composite set including variations of the observed set of interactions at different levels of abstraction. Obtaining user feedback for each of the aforementioned multiple different sets, Based on the user feedback, select one of the multiple different composite sets of dialogues, and To train a machine learning model to produce an output that represents the composite set of the selected dialogues, A system that enables this to happen.

16. The system according to claim 15, wherein the machine learning model is used to simulate the plurality of different composite sets of interactions between the user and the computing device.

17. The system according to claim 15 or 16, wherein the machine learning model is trained to facilitate intelligent process automation.

18. The system according to any one of claims 15 to 17, wherein the machine learning model is trained to generate a probability distribution across the action space.

19. The system according to any one of claims 15 to 18, wherein the machine learning model includes a private machine learning model.

20. The system according to claim 19, further comprising instructions for providing parameters for the trained private machine learning model for federated learning of a global machine learning model.

Images