System and method for generating unified target representations across task generalization in robotic navigation

Abstract

Systems and methods for generating unified target representations for cross-task generalization in robotic navigation. Systems and methods described herein may include one or more processors configured to receive commands from a user related to a topic; accessing a representation space associated with the command; receiving a first data set related to the command, a second data set related to the topic, and a third data set including the topic related to the command; updating the representation space based on at least one of the first data set, the second data set, and the third data set; generating a target representation based on the representation space; receiving sensor data of the current environment from the plurality of sensors; generating a first series of steps and a second series of steps based on the target representation and the current environment; annotating the sensor data based on execution of the first series of steps to generate annotated sensor data; and updating a second series of steps based on the annotated sensor data.

Description

Technical Field

[0001] The present disclosure relates to image processing utilizing machine learning models for navigation. Background Art

[0002] Machine learning (ML) has been used in a variety of critical applications, including autonomous driving, medical imaging, industrial fire detection, and credit scoring. Such applications require thorough evaluation before deployment in order to assess model capabilities and limitations. Unforeseen model errors can cause serious consequences in the real world: for example, a false sense of security in ML models can cause safety issues in driver assistance and industrial systems, misdiagnosis in medical or therapeutic analysis, and bias against individuals and groups. Summary of the invention

[0003] A system of one or more computers may be configured to perform specific operations or actions by having software, firmware, hardware, or a combination thereof installed on the system, which when operated causes the system to perform the actions. One or more computer programs may be configured to perform specific operations or actions by including instructions, which when executed by a data processing device causes the device to perform the actions.

[0004] In a general aspect, a computer-implemented method may include receiving, by a device, a command from a user associated with a topic. The computer-implemented method may also include accessing a representation space associated with the command, wherein similar topics and commands in the representation space are clustered together. The method may further include receiving a first data set associated with the command, a second data set associated with the topic, and a third data set including the topic associated with the command. The method may additionally include updating the representation space based on at least one of the first data set, the second data set, and the third data set. The method may further include generating a target representation based on the representation space through a target description machine learning model. The method may also include receiving sensor data of a current environment from a plurality of sensors. The method may further include generating a first series of steps and a second series of steps based on the target representation and the current environment. The method may additionally include annotating the sensor data based on the execution of the first series of steps through a progress description machine learning model to generate annotated sensor data. The method may further include updating the second series of steps based on the annotated sensor data through a policy machine learning model.

[0005] Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0006] Implementations may include one or more of the following features. A computer-implemented method wherein the representation space is updated comprises the following steps: analyzing a first data set and a second data set in view of a target representation to determine an inter-task score for at least one subject represented in the representation space associated with the subject of the command; regularizing the position of at least one subject in the target representation based on the inter-task score. A computer-implemented method wherein the representation space is updated comprises the following steps: analyzing a third data set in view of the target representation to determine an intra-task score for at least one subject represented in the representation space not associated with the subject of the command; regularizing the position of at least one subject in the target representation based on the intra-task score. A computer-implemented method wherein the first data set may include target-related sensor data organized as tuples, wherein each sensor data is positively correlated with the command, wherein each tuple may include subject-related sensor data, instruction-related sensor data, and audio-related sensor data; wherein the second data set may include target-related sensor data organized as tuples, wherein one of the sensor data is negatively correlated with the command; and wherein the third data set may include target-related sensor data organized as tuples, wherein the sensor data is either positively correlated with the command or negatively correlated with the command. A computer-implemented method wherein the policy machine learning model is further trained based on the annotated sensor data. A computer-implemented method wherein the training of a goal description machine learning model, a progress description machine learning model, and a policy machine learning model is frozen. A computer-implemented method wherein the training of a goal description machine learning model, a progress description machine learning model, and a policy machine learning model is trained at a server and operated locally at a device.

[0007] Implementations of the described techniques may include hardware, a method or process, or computer-tangible media. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A system for training a neural network is shown.

[0009] Figure 2 A computer-implemented method for training a neural network is shown.

[0010] Figure 3 An embodiment of a system workflow for identifying data slices and associated attributes is illustrated.

[0011] Figure 4 An embodiment of a workflow model of the overall system is illustrated.

[0012] Figure 5 Diagram showing an example of a data slicing workflow.

[0013] Figure 6An embodiment of an interface having the ability to output attributes associated with various slices of input data is illustrated.

[0014] Figure 7 An embodiment of a flow diagram of an algorithm for estimation model optimization is illustrated.

[0015] Figure 8 Depicted is a schematic diagram of a control system configured to control a power tool, such as a drill or driver, having an at least partially autonomous mode.

[0016] Fig. 9 A schematic diagram of a control system configured to control an automated personal assistant is depicted.

[0017] Fig.10 Depicted is a schematic diagram of a control system configured to control a monitoring system.

[0018] Fig.11 Depicted is a schematic diagram of a control system configured to control an imaging system, such as an MRI apparatus, an x-ray imaging apparatus, or an ultrasound apparatus.

[0019] Fig.12 Describes a goal description network that can be used to encode a rich representation of the task being performed. DETAILED DESCRIPTION

[0020] Embodiments of the present disclosure are described herein. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various and alternative forms. The figures are not necessarily drawn to scale; some features may be exaggerated or minimized to show the details of a particular component. Therefore, the specific structural and functional details disclosed herein should not be interpreted as limiting, but merely as a representative basis for teaching those skilled in the art to adopt the embodiments in various ways. As will be understood by those of ordinary skill in the art, the various features illustrated and described with reference to any one of the figures may be combined with the features illustrated in one or more other figures to produce embodiments that are not explicitly described or described. The combination of the illustrated features provides representative embodiments of typical applications. However, for specific applications or embodiments, it may be desirable to carry out various combinations and modifications of features consistent with the teachings of the present disclosure.

[0021] As used herein, "a", "an", and "the" refer to both the singular and the plural, unless the context clearly dictates otherwise. As an example, a "processor" programmed to perform various functions refers to one processor programmed to perform each function or more than one processor programmed together to perform each of the various functions.

[0022] In the present disclosure, the systems and methods described herein can utilize multimodal base models in machine learning training and reasoning pipelines. The systems and methods described herein can be configured to use base models, which are models with large data representations (e.g., through a large number of layer sizes and internal weights and bias parameters, such as in large language models or "LLMs") that have been additionally pre-trained on multiple large-scale datasets. These datasets can consist of millions of paired data samples (e.g., images with their captions), and the LLMs can be trained with one or more purposes. In some embodiments, the purpose can be to learn to score the alignment (i.e., similarity) between inputs (e.g., arbitrary images and arbitrary text captions). Another purpose can include reconstructing images when random patches of data are missing or deleted, taking into account the natural language text caption and the corresponding image. In addition to these training purposes, some intermediate continuous-valued vector representations from the base model can be used to perform (i.e., pretext) tasks (e.g., image classification, image captioning, object segmentation, semantic segmentation, object recognition, or any other appropriate mathematical concept).

[0023] In some embodiments, the systems and methods described herein may be configured to use a base model to perform one of the tasks in its pretext task set. Through such extensive pre-training (e.g., using large datasets with challenging training purposes, on a variety of pretext tasks), the LLM may have accumulated enough training in multiple domains to serve as the basis for a task-specific architecture that can be built on top of the base model. In some embodiments, the systems and methods described herein may be configured to configure the base models after pre-training of the base models, so that these base models can be untrainable (i.e., frozen) and used only in "inference mode" for various downstream tasks. In this way, the base model can enable cross-domain generalization capabilities of downstream task-specific frameworks through experience in modeling several tasks and domains.

[0024] For example, in the context of robotic navigation, an agent can implicitly perform goal description (i.e., encode a rich representation of the task it needs to perform), progress representation learning and monitoring (i.e., inspect current information and compare it to the goal to inform action selection), and multimodal alignment (i.e., learn to capture complementarities between different modalities or “views” of a novel scene).

[0025] The systems and methods described herein can be configured to extract, refine, and use a general representation of a task goal derived in part from a base model in the context of multimodal goal-oriented robotic navigation. The systems and methods described herein can be configured to obtain cross-domain generalization capabilities (preserved from the base model) as well as competitive intra-domain performance (from task-specific components).

[0026] The systems and methods described herein may be directed to goal-directed multimodal robotic navigation, a task within the artificial intelligence community. Several robotic navigation task variants have specific modalities in which the goal is specified. For example, in the INSTRUCTIONGOAL task, the goal is specified as a natural language command; in the OBJECTGOAL task, the goal is specified via an RGB image of the object; in the AUDIOGOAL task, the goal is specified via the sound of the object that the agent needs to locate. In each task, the agent may be able to monitor progress toward the goal via some other progress modality, typically visual signals (RGB images, videos, LiDAR frames, RADAR frames, depth frames, etc.) or explicit state-action trajectories.

[0027] The systems and methods described herein may include functionality that is best conveyed as an equation. The equation may allow M G and M P denote the sets of inputs from the goal and progress modalities, respectively. At each time step t, the agent becomes aware of the state s of the environment t , which can be defined in terms of the goal and progress inputs up to the current time step, such that in, and At each time step, the agent can perform an action α t ∈A (from the action space A) in order to transform the environment between physical states into the next state s t+1 For goal-driven problems GDN , there is an acceptable solution This includes reaching s goal The initial state s0, that is, for a set length T, ψ = {s0, a0, s1, a1, ..., a T ,s goal The agent aims to generate predicted solution trajectories that are closely aligned with the true acceptable solution ψ

[0028] The systems and methods described herein can provide a novel framework for generalizing across multiple goal-oriented robot navigation tasks using a base model (e.g., CLIP). The difference across these tasks is the input modality used to specify the target (e.g., natural language text in the case of INSTRUCTIONGOAL tasks, images in the case of OBJECTGOAL tasks, acoustic signals in the case of AUDIOGOAL tasks, etc.). The systems and methods described herein can be configured to enable agents to generalize across various goal-oriented navigation tasks, and the agents have a unified encoding algorithm (i.e., any subset of the target modalities in the supported *-GOAL task set can be processed into a semantically consistent multimodal target representation). The encoder can be obtained via the base model architecture; it is observed that the base model may not have all the input interfaces of the desired *-GOAL robot navigation task set, and the systems and methods described herein can be configured to generate a data set from a robot simulation environment to establish a (ground) base model in an additional target modality. Once the system and method described herein obtains the base model we have established, the base model can support an appropriate input target modality set, and the system and method described herein trains the target decoder on the established base model. This goal decoder can be used to further align goal representations from different modalities while also contextualizing goal representations for robotic navigation tasks.

[0029] The systems and methods described herein may consist of an established base model, a target decoder, a progress encoder module, a policy encoder / decoder module, and a policy network.

[0030] The systems and methods described herein can be configured to build a CLIP-like base model with an additional modality (e.g., audio), for which we leverage CLIP4X. CLIP4X is a general framework we have developed for building new modalities "X" into CLIP or CLIP-like base models (e.g., Align and LiT). This is done to avoid duplication of development work when integrating a new modality "X" (such as audio for the AudioGoal task) into an existing base model. This also facilitates understanding of the relationships between modalities, including existing modalities (i.e., image and natural language) and new modalities (such as audio, radar, and time series).

[0031] The systems and methods described herein can be configured to utilize CLIP4X, which implements the envisioned functionality and can be reused by different projects (e.g., for comparative learning purposes, multi-GPU distributed training, common model components, comprehensive test suites, and experiment logs). To make CLIP4X general and extensible, the code base is designed to be modular and configurable, utilizing the Hydra framework. To validate its use, the systems and methods described herein can use CLIP4X in several projects, where X can be audio, radar, and time series. CLIP4X users can get classes from CLIP4X and add custom modules for specific tasks.

[0032] Target Decoder: Refining Target Representation via Contrast Regularization

[0033] The goal of the target decoder is to project the output of the built base model into a representation space that can be used by downstream parts of the overall framework (e.g., the policy decoder). At the same time, we want the projected output of the built base model to be used as a generalizable and reusable target representation for various embedded AI tasks, rather than a specific downstream navigation policy architecture.

[0034] In some embodiments, the systems and methods described herein can be configured so that regardless of the input target modality used, as long as samples from the same or different *-GOAL tasks are semantically similar (e.g., they involve the same object, location, task, action, or any other concept related to the task), these samples should be "close" together in the latent space of the target decoder. Conversely, semantically dissimilar samples should be well separated in this space. For example, the target decoder should ensure that target descriptions in the form of an image of a phone (i.e., as the ObjectGoal task goal, using the visual interface of the established base model), an instruction to "find a phone" (i.e., INSTRUCTIONGOAL, using the language interface), and a ringing phone (i.e., AUDIOGOAL, using the newly established audio interface) should all be mapped to similar target representations. The systems and methods described herein can be configured to call entries that represent versatility.

[0035] As a basis for representing versatility, the systems and methods described herein can be configured to construct three datasets. First, the systems and methods described herein construct a dataset Dinter+ with observations from various *-GOAL tasks, where each sample consists of a target observation tuple, a strong internal semantic alignment (i.e., a positive example; "+"), such as in the above-mentioned phone example, i.e., The superscripts {"OG", "IG", and "AG"} refer to the tasks {OBJECTGOAL, INSTRUCTIONGOAL, and AUDIOGOAL}, respectively. Next, the system and method described in this paper construct the dataset Dinter-, where for the target description observation For each sample of , there may be at least one observation that is semantically inconsistent with other (one or more) observations (negative example; "-"). Finally, the systems and methods described herein can construct a third dataset Dintera, where each sample consists of a target description observation pair from the same task and a predetermined number of dataset samples N, the target description observation pair can be semantically aligned ("+") or semantically dissimilar ("-"), that is: i≠j. For AUDIOGOAL task observations, the systems and methods described herein can use the dataset provided by Tatiya et al. (2022). For INSTRUCTIONGOAL task observations, the systems and methods described herein start with the dataset provided by Ku et al. (2020), however, the systems and methods described herein can extract the last natural language sub-instruction from each sample, which provides a short text description of the object / location that the agent needs to find / assume. For OBJECTGOAL task observations, the systems and methods described herein can use the "OBJECTGOAL" task from Anderson et al. in 2018, which is refined for the Habitat environment (Savva et al., 2019).

[0036] Equipped with these datasets, the systems and methods described herein can use the datasets to regularize the target decoder representation to enforce representation versatility. The systems and methods described herein can achieve this by implementing intra-task comparisons and inter-task comparisons.

[0037] The systems and methods described herein may be configured to implement intra-task comparison. Here, the systems and methods described herein may want to regularize the output of the target decoder for similarity / comparison between samples from the same target task. The systems and methods described herein may be configured to implement intra-task comparison. Here, the systems and methods described herein may want to regularize the output of the target decoder for similarity / comparison between samples from the same target task. intra Get a random batch sample.

[0038] The systems and methods described herein may be configured to implement inter-task comparison. Here, the systems and methods described herein may want to regularize the output of the target decoder for similarity / comparison between samples from different target tasks. The systems and methods described herein may be configured to perform task-to-task comparison. Get a random batch sample.

[0039] The systems and methods described herein can be configured to use the established base model in goal-directed robotic navigation. When the base model is established with additional modalities to support new *-GOAL tasks (e.g., audio for AUDIOGOAL) and when the extracted target representations are regularized according to intra-task and inter-task comparison goals, the systems and methods disclosed herein can be ready to start using the established and regularized target description representations in downstream goal-directed navigation tasks. Target inputs that express semantically similar tasks (such as in the above-mentioned phone example) can be mapped to similar or identical target representations. The systems and methods disclosed herein can be configured to also want similar target representations for performing similar tasks. The systems and methods disclosed herein can be said to be modality agnostic.

[0040] As further described below, the systems and methods described herein can utilize our goal representation from the "Goal Description Network" to encode a rich representation of the task that needs to be performed, whether it is an AUDIOGOAL task, an OBJECTGOAL task, or an INSTRUCTIONGOAL task. At the same time, the systems and methods described herein can encode the progress modality (which happens to be the visual context in all *-GOAL robot tasks) by means of a "Progress Monitoring Network". Both the goal representation and the progress representation can be fed to the downstream policy network. These component models can be implemented as trainable neural networks or any other type of model with learnable / adjustable internal function parameters. The goal description network can remain frozen after contrast regularization, or can be further fine-tuned for additional task specialization. The progress network and the policy network can be trained by imitating the purpose, in the event that an expert demonstration is provided, by updating by means of a policy gradient-based purpose (pg) or by any other means after providing data and supervision to (one or more) models.

[0041] The systems and methods described herein may be configured to include task-specific agent components. enc It can be a visual encoder that maps observations to a visual vector representation space. Module f clfObjects in the agent's visual context can be classified and the visual representations are implicitly combined with the language embeddings of the detected object labels as scene graph vertices. Those scene graph vertices can be fed to a graph encoder network module GEN to produce spatially and semantically aware contextual representations. This GEN can facilitate pre-training and reasoning including based on scene priors. The scene memory transformer M keeps track of the prior contextual representations and automatically re-prioritizes them for use by the "policy encoder", which feeds the output to the similarity memory module M of the policy network e At the same time, the target embedding can be fed into a “strategy decoder” that implicitly converts M e The provided context is compared with the embedding — the target at each time step. This generates an appropriate state-context vector to be used by the further downstream policy network, which performs action value estimation and action decoding.

[0042] Figure 1 A system 100 for training a neural network is shown. The system 100 may include an input interface for accessing training data 102 for the neural network. For example, Figure 1 As illustrated in , the input interface can be constituted by a data storage interface 104, which can access the training data 102 from a data storage device 106. For example, the data storage interface 104 can be a memory interface or a permanent storage interface, such as a hard disk or SSD interface, but can also be a personal, local or wide area network interface, such as a Bluetooth, Zigbee or Wi-Fi interface or an Ethernet or fiber optic interface. The data storage device 106 can be an internal data storage device of the system 100, such as a hard disk drive or SSD, but can also be an external data storage device, for example, a network accessible data storage device.

[0043] In some embodiments, data storage 106 may further include a data representation 108 of an untrained version of the neural network, which may be accessed by system 100 from data storage 106. However, it will be appreciated that training data 102 and data representation 108 of the untrained neural network may also each be accessed from a different data storage, e.g., via a different subsystem of data storage interface 104. Each subsystem may be of a type as described above with respect to data storage interface 104.

[0044] In some embodiments, the data representation 108 of the untrained neural network may be generated internally by the system 100 based on the parameters of the neural network, and thus may not be explicitly stored on the data storage device 106. The system 100 may further include a processor subsystem 110 that may be configured to provide an iteration function as a replacement of a layer stack of the neural network to be trained during operation of the system 100. Here, the corresponding layers in the layer stack that are replaced may have mutually shared weights and may receive the output of the previous layer as input, or, for the first layer of the layer stack, receive an initial activation and a portion of the input of the layer stack.

[0045] The processor subsystem 110 may further be configured to iteratively train the neural network using the training data 102. Here, the iterations of the training performed by the processor subsystem 110 may include a forward propagation portion and a back propagation portion. The processor subsystem 110 may be configured to, in addition to defining other operations that may be performed for the forward propagation portion, perform the forward propagation portion by, inter alia, determining an equilibrium point of the iterative function at which the iterative function converges to a fixed point, wherein determining the equilibrium point includes finding a root solution of the iterative function minus its input using a numerical root finding algorithm, and by providing the equilibrium point as a replacement for the output of the layer stack in the neural network.

[0046] The system 100 may further include an output interface for outputting a data representation 112 of the trained neural network, which data may also be referred to as trained model data 112. Figure 1 , the output interface may be comprised of a data storage interface 104, which in these embodiments is an input / output (“IO”) interface, via which trained model data 112 may be stored in a data storage device 106. For example, during or after training, a data representation 108 defining an “untrained” neural network may be at least partially replaced by a data representation 112 of a trained neural network, as parameters of the neural network, such as weights, hyperparameters, and other types of parameters of the neural network, may be adapted to reflect the training on the training data 102. Figure 1 This is also illustrated in FIG. 1 by reference numerals 108, 112 referring to the same data records on the data storage device 106. In some embodiments, the data representation 112 may be stored separately from the data representation 108 defining the "untrained" neural network. In some embodiments, the output interface may be separate from the data storage interface 104, but may generally be of the type described above for the data storage interface 104.

[0047] Figure 2A computing system 200 implementing a system for annotating data is depicted. The computing system 200 may include at least one computing system 202. The computing system 202 may include a processor 204 operatively connected to a memory unit 208. The processor 204 may include one or more integrated circuits that implement the functionality of a central processing unit (CPU) 206. The CPU 206 may be a processing unit that implements an instruction set, such as one of the x86, ARM, Power, or MIPS instruction set families. During operation, the CPU 206 may execute stored program instructions that are retrieved from the memory unit 208. The stored program instructions may include software that controls the operation of the CPU 206 to perform the operations described herein. In some examples, the processor 204 may be a system on chip (SoC) that integrates the functionality of the CPU 206, the memory unit 208, a network interface, and an input / output interface into a single integrated device. The computing system 202 may implement an operating system for managing various aspects of the operation.

[0048] The memory unit 208 may include volatile memory and non-volatile memory for storing instructions and data. The non-volatile memory may include solid-state memory, such as NAND flash memory, magnetic and optical storage media, or any other suitable data storage device that retains data when the computing system 202 is disabled or loses power. The volatile memory may include static and dynamic random access memory (RAM) that stores program instructions and data. For example, the memory unit 208 may store a machine learning algorithm 210 or algorithms, a training data set 212 for the machine learning algorithm 210, and raw source data 215.

[0049] The computing system 202 may include a network interface device 222 configured to provide communications with external systems and devices. For example, the network interface device 222 may include a wired and / or wireless Ethernet interface as defined by the Institute of Electrical and Electronics Engineers (IEEE) 802.11 series of standards. The network interface device 222 may include a cellular communication interface for communicating with a cellular network (e.g., 3G, 4G, 5G). The network interface device 222 may further be configured to provide a communication interface to an external network 224 or a cloud.

[0050] The external network 224 may be referred to as the World Wide Web or the Internet. The external network 224 may establish a standard communication protocol between computing devices. The external network 224 may allow information and data to be easily exchanged between computing devices and the network. One or more servers 330 may communicate with the external network 224.

[0051] Computing system 202 may include input / output (I / O) interface 220, which may be configured to provide digital and / or analog input and output. I / O interface 220 may include an additional serial interface for communicating with external devices (e.g., a universal serial bus (USB) interface).

[0052] The computing system 202 may include a human-machine interface (HMI) device 218, which may include any device that enables the computing system 200 to receive control inputs. Examples of input devices may include human-machine interface inputs, such as keyboards, mice, touch screens, voice input devices, and other similar devices. The computing system 202 may include a display device 232. The computing system 202 may include hardware and software for outputting graphics and text information to the display device 232. The display device 232 may include an electronic display screen, a projector, a printer, or other suitable device for displaying information to a user or operator. The computing system 202 may be further configured to allow interaction with a remote HMI and a remote display device via a network interface device 222.

[0053] The computing system 200 may be implemented using one or more computing systems. Although the example depicts a single computing system 202 that implements all of the described features, it is intended that the various features and functions may be separated and implemented by multiple computing units that communicate with each other. The specific system architecture selected may depend on various factors.

[0054] The computing system 200 may implement a machine learning algorithm 210 configured to analyze raw source data 215. The raw source data 215 may include raw or unprocessed sensor data, which may represent an input data set for the machine learning system. The raw source data 215 may include videos, video clips, images, text-based information, and raw or partially processed sensor data (e.g., a radar map of an object). In some examples, the machine learning algorithm 210 may be a neural network algorithm designed to perform a predetermined function. For example, a neural network algorithm may be configured in an automotive application to identify pedestrians in a video image.

[0055] The computer system 200 may store a training data set 212 for the machine learning algorithm 210. The training data set 212 may represent a set of previously constructed data for training the machine learning algorithm 210. The training data set 212 may be used by the machine learning algorithm 210 to learn weighting factors associated with the neural network algorithm. The training data set 212 may include a set of source data having corresponding outcomes or results that the machine learning algorithm 210 attempts to replicate via the learning process. In this example, the training data set 212 may include source videos with and without pedestrians and corresponding presence and location information. The source videos may include various scenes in which pedestrians are identified.

[0056] The machine learning algorithm 210 can operate in a learning mode using the training data set 212 as input. The machine learning algorithm 210 can be executed in multiple iterations using data from the training data set 212. With each iteration, the machine learning algorithm 210 can update the internal weighting factors based on the achieved results. For example, the machine learning algorithm 210 can compare the output results (e.g., annotations) with the output results included in the training data set 212. Since the training data set 212 includes the expected results, the machine learning algorithm 210 can determine when the performance is acceptable. After the machine learning algorithm 210 reaches a predetermined performance level (e.g., 100% consistent with the outcome associated with the training data set 212), the machine learning algorithm 210 can be executed using data that is not in the training data set 212. The trained machine learning algorithm 210 can be applied to a new data set to generate annotated data.

[0057] The machine learning algorithm 210 may be configured to identify specific features in the raw source data 215. The raw source data 215 may include multiple instances or input data sets for which annotation results are desired. For example, the machine learning algorithm 210 may be configured to identify the presence of pedestrians in a video image and annotate the presence. The machine learning algorithm 210 may be programmed to process the raw source data 215 to identify the presence of specific features. The machine learning algorithm 210 may be configured to identify the features in the raw source data 215 as predetermined features (e.g., pedestrians). The raw source data 215 may be derived from various sources. For example, the raw source data 215 may be actual input data collected by the machine learning system. The raw source data 215 may be machine generated for testing the system. As an example, the raw source data 215 may include raw video images from a camera.

[0058] In an example, the machine learning algorithm 210 can process the raw source data 215 and output an indication of a representation of an image. The output can also include an enhanced representation of the image. The machine learning algorithm 210 can generate a confidence level or factor for each output generated. For example, a confidence value exceeding a predetermined high confidence threshold can indicate that the machine learning algorithm 210 is confident that the identified feature corresponds to a particular feature. A confidence value less than a low confidence threshold can indicate that the machine learning algorithm 210 has a certain degree of uncertainty about the particular feature.

[0059] Figure 3 is a flow chart of an example process 300. In some implementations, Figure 3 One or more process blocks of may be performed by machine learning algorithm 210. In some implementations, process 300 may be performed by processor 204.

[0060] like Figure 3 As shown in , process 300 may include receiving a command from a user related to a topic via a device (block 302). In some implementations, the device may be computing system 200. For example, a machine learning network may receive a command from a user related to a topic via a device, such as a phone. Figure 3 As shown in , process 300 may include accessing a representation space associated with commands, wherein similar topics and commands in the representation space are clustered together (block 304). For example, a machine learning network may access a representation space associated with a ringing phone, wherein similar devices, similar audio, and similar tasks associated with those devices are clustered in the representation space in view of their similarity. In some embodiments, the representation space is pre-populated with data objects representing topics, tasks, and audio, wherein the space between data objects in the representation space indicates a level of similarity. For example, a smart phone and a flip phone will be closer together than a rotary phone.

[0061] Further Figure 3 As shown in , process 300 may include receiving a first data set related to commands, a second data set related to topics, and a third data set including topics related to commands (box 306). For example, process 300 may receive a first data set having observations from various previously performed tasks, wherein each observation consists of a target observation tuple, a strong internal semantic alignment (i.e., a positive example). Process 300 may receive a second data set, wherein for each sample of target description observations, there may be at least one observation that is semantically inconsistent with other (one or more) observations (i.e., a negative example). Process 300 may receive a third data set, wherein each sample consists of a target description observation pair from the same task, which target description observation pair may be either semantically aligned (i.e., a positive example) or semantically dissimilar (i.e., a negative example).

[0062] Also like Figure 3 As shown in , process 300 may include updating the representation space based on at least one of the first data set, the second data set, and the third data set (block 308). For example, the machine learning network may update the representation space based on at least one of the first data set, the second data set, and the third data set to implement representation versatility. Further as Figure 3 As shown in , process 300 may include generating a target representation based on a representation space by describing a machine learning model through a target (block 310). For example, a machine learning network may generate a target representation based on a representation space by describing a machine learning model through a target, as described above.

[0063] Also like Figure 3 As shown in FIG. 3 , process 300 may include receiving sensor data of the current environment from a plurality of sensors (block 312). For example, such sensors may include, but are not limited to, temperature sensors, proximity sensors, IR sensors, accelerometers, gyroscopes, compasses, light sensors, moisture sensors, ultrasonic sensors, alcohol sensors, humidity sensors, smoke sensors, gas sensors, heartbeat sensors, or any other suitable sensors. Further, as shown in FIG. Figure 3 As shown in , process 300 may include generating a first series of steps and a second series of steps based on the target representation and the current environment (block 314). For example, the machine learning network may generate a first series of steps in which the device moves toward the subject of the command based on the best prediction of the subject's location. Figure 3 As shown in , process 300 may include annotating sensor data based on the execution of the first series of steps by progress description machine learning model to generate annotated sensor data (block 316). For example, when executing the steps, the machine learning network may annotate sensor data based on the execution of the first series of steps by progress description machine learning model to generate annotated sensor data.

[0064] Further Figure 3 As shown in , process 300 may include updating the second series of steps based on the annotated sensor data through the policy machine learning model (box 318). For example, the machine learning network may update the second series of steps based on the annotated sensor data through the policy machine learning model, as described above.

[0065] although Figure 3 An example block diagram of process 300 is shown, but in some implementations, Figure 3 Compared to the blocks depicted in , process 300 may include additional blocks, fewer blocks, different blocks, or blocks arranged in a different manner. Additionally or alternatively, two or more of the blocks in process 300 may be executed in parallel.

[0066] Figure 4 is a flow chart of an example process 400 that focuses on the comparison and contrast steps when updating a representation space. In some embodiments, Figure 4 One or more process blocks of can be performed by a machine learning network. In some implementations, process 400 can be performed by processor 204.

[0067] Further Figure 4 As shown in , process 400 may include receiving a first data set related to commands, a second data set related to topics, and a third data set including topics related to commands (box 402). For example, process 400 may receive a first data set having observations from various previously performed tasks, wherein each observation consists of a target observation tuple, a strong internal semantic alignment (i.e., a positive example). Process 400 may receive a second data set, wherein for each sample of target description observations, there is at least one observation that is semantically inconsistent with other (one or more) observations (i.e., a negative example). Process 400 may receive a third data set, wherein each sample consists of a target description observation pair from the same task, which target description observation pair may be either semantically aligned (i.e., a positive example) or semantically dissimilar (i.e., a negative example).

[0068] Process 400 may include additional implementations, such as any single implementation or any combination of implementations described below and / or in conjunction with one or more other process descriptions described elsewhere herein. Figure 4 As shown in , process 400 may include updating the representation space, which includes the following steps: analyzing the first data set and the second data set in view of the target representation to determine an inter-task score of at least one topic represented in the representation space associated with the topic of the command; regularizing the position of at least one topic in the target representation based on the inter-task score (box 404). Process 400 can be configured to implement intra-task comparison. For example, process 400 may want to regularize the output of the target decoder for similarity / comparison between samples from the same target task.

[0069] Further Figure 4 As shown in , process 400 may include updating the representation space, which further includes the following steps: analyzing the third data set in view of the target representation to determine the intra-task score of at least one topic represented in the representation space that is not associated with the topic of the command; regularizing the position of at least one topic in the target representation based on the intra-task score (box 406). Process 400 can be configured to implement inter-task comparison. For example, process 400 may want to regularize the output of the target decoder for similarity / comparison between samples from different target tasks.

[0070] Further Figure 4 As shown in , process 400 may include generating a target representation based on a representation space by describing the machine learning model through a target (block 408). For example, a machine learning network may generate a target representation based on a representation space by describing the machine learning model through a target, as described above.

[0071] Figure 5 A schematic diagram of the interaction between a computer-controlled machine 500 and a control system 502 is depicted. The computer-controlled machine 500 includes an actuator 504 and a sensor 506. The actuator 504 may include one or more actuators, and the sensor 506 may include one or more sensors. The sensor 506 is configured to sense a condition of the computer-controlled machine 500. The sensor 506 may be configured to encode the sensed condition into a sensor signal 508 and transmit the sensor signal 508 to the control system 502. Non-limiting examples of the sensor 506 include video, radar, LiDAR, ultrasonic, and motion sensors. In some embodiments, the sensor 506 is an optical sensor configured to sense an optical image of an environment close to the computer-controlled machine 500.

[0072] The control system 502 is configured to receive sensor signals 508 from the computer controlled machine 500. As explained below, the control system 502 may be further configured to calculate actuator control commands 510 depending on the sensor signals and transmit the actuator control commands 510 to the actuators 504 of the computer controlled machine 500.

[0073] like Figure 5 As shown in , the control system 502 includes a receiving unit 512. The receiving unit 512 can be configured to receive a sensor signal 508 from a sensor 506 and transform the sensor signal 508 into an input signal x. In an alternative embodiment, the sensor signal 508 is directly received as an input signal x without the need for the receiving unit 512. Each input signal x can be a portion of each sensor signal 508. The receiving unit 512 can be configured to process each sensor signal 508 to generate each input signal x. The input signal x can include data corresponding to the image recorded by the sensor 506.

[0074] The control system 502 includes a classifier 514. The classifier 514 can be configured to classify the input signal x into one or more labels using a machine learning (ML) algorithm, such as the neural network described above. The classifier 514 can be configured to be parameterized by parameters, such as the parameters described above (e.g., parameter θ). The parameter θ can be stored in and provided by the non-volatile storage device 516. The classifier 514 is configured to determine the output signal y from the input signal x. Each output signal y includes information that assigns one or more labels to each input signal x. The classifier 514 can transmit the output signal y to a conversion unit 518. The conversion unit 518 is configured to convert the output signal y into an actuator control command 510. The control system 502 is configured to transmit the actuator control command 510 to the actuator 504, which is configured to actuate the computer-controlled machine 500 in response to the actuator control command 510. In some embodiments, the actuator 504 is configured to actuate the computer-controlled machine 500 directly based on the output signal y.

[0075] After the actuator 504 receives the actuator control command 510, the actuator 504 is configured to perform an action corresponding to the associated actuator control command 510. The actuator 504 may include control logic configured to transform the actuator control command 510 into a second actuator control command for controlling the actuator 504. In one or more embodiments, the actuator control command 510 may be used to control a display instead of or in addition to the actuator.

[0076] In some embodiments, the control system 502 includes the sensor 506 instead of or in addition to the computer-controlled machine 500 including the sensor 506. The control system 502 may also include the actuator 504 instead of or in addition to the computer-controlled machine 500 including the actuator 504.

[0077] like Figure 5 As shown in , the control system 502 also includes a processor 520 and a memory 522. The processor 520 may include one or more processors. The memory 522 may include one or more memory devices. The classifier 514 (e.g., ML algorithm) of one or more embodiments may be implemented by the control system 502, which includes a non-volatile storage device 516, a processor 520, and a memory 522.

[0078] The non-volatile storage 516 may include one or more permanent data storage devices, such as a hard drive, an optical drive, a tape drive, a non-volatile solid-state device, a cloud storage device, or any other device capable of permanently storing information. The processor 520 may include one or more devices selected from a high-performance computing (HPC) system including a high-performance core, a microprocessor, a microcontroller, a digital signal processor, a microcomputer, a central processing unit, a field programmable gate array, a programmable logic device, a state machine, a logic circuit, an analog circuit, a digital circuit, or any other device that manipulates signals (analog or digital) based on computer-executable instructions residing in the memory 522. The memory 522 may include a single memory device or multiple memory devices, including (but not limited to) random access memory (RAM), volatile memory, non-volatile memory, static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, cache memory, or any other device capable of storing information.

[0079] The processor 520 may be configured to read into the memory 522 and execute computer executable instructions that reside in the non-volatile storage 516 and embody one or more ML algorithms and / or methods of one or more embodiments. The non-volatile storage 516 may include one or more operating systems and applications. The non-volatile storage 516 may store data compiled and / or interpreted from computer programs created using a variety of programming languages ​​and / or technologies, including (but not limited to) Java, C, C++, C#, Objective C, Fortran, Pascal, JavaScript, Python, Perl, and PL / SQL, alone or in combination.

[0080] When executed by the processor 520, the computer executable instructions of the non-volatile storage device 516 may cause the control system 502 to implement one or more of the ML algorithms and / or methods as disclosed herein. The non-volatile storage device 516 may also include ML data (including data parameters) that supports the functions, features, and processes of one or more embodiments described herein.

[0081] The program code embodying the algorithms and / or methods described herein can be distributed as a program product in various forms, either individually or collectively. The program code can be distributed using a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to implement aspects of one or more embodiments. The computer-readable storage medium, which is non-temporary in nature, can include volatile and non-volatile and removable and non-removable tangible media implemented in any method or technology for storing information such as computer-readable instructions, data interfaces, program modules, or other data. The computer-readable storage medium can further include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state memory technology, portable compact disc read-only memory (CD-ROM) or other optical storage device, cassette, tape, disk storage device or other magnetic storage device, or any other medium that can be used to store the desired information and can be read by a computer. The computer-readable program instructions can be downloaded from the computer-readable storage medium to a computer, another type of programmable data processing device, or another device, or downloaded to an external computer or external storage device via a network.

[0082] The computer-readable program instructions stored in the computer-readable medium can be used to guide a computer, other types of programmable data processing devices, or other devices that operate in a specific manner, so that the instructions stored in the computer-readable medium produce an article of manufacture, which includes instructions for implementing the functions, actions, and / or operations specified in the flowchart or diagram. In certain alternative embodiments, the functions, actions, and / or operations specified in the flowchart and diagram can be reordered, processed serially, and / or processed simultaneously, consistent with one or more embodiments. In addition, any flowchart and / or diagram can include more or fewer nodes or boxes than the illustrated nodes or boxes, consistent with one or more embodiments.

[0083] The processes, methods or algorithms may be embodied in whole or in part using suitable hardware components, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a state machine, a controller or other hardware components or devices or a combination of hardware, software and firmware components.

[0084] Figure 6A schematic diagram of a control system 502 configured to control a vehicle 600, which may be an at least partially autonomous vehicle or an at least partially autonomous robot, is depicted. The vehicle 600 includes an actuator 504 and a sensor 506. The sensor 506 may include one or more video sensors, cameras, radar sensors, ultrasonic sensors, LiDAR sensors, and / or location sensors (e.g., GPS). One or more of the one or more specific sensors may be integrated into the vehicle 600. Alternatively or in addition to the one or more specific sensors identified above, the sensor 506 may include a software module that is configured to determine the state of the actuator 504 when executed. A non-limiting example of a software module includes a weather information software module that is configured to determine the current or future state of the weather near the vehicle 600 or other location.

[0085] The classifier 514 of the control system 502 of the vehicle 600 may be configured to detect an object near the vehicle 600 depending on the input signal x. In such an embodiment, the output signal y may include information characterizing the object near the vehicle 600. The actuator control command 510 may be determined based on such information. The actuator control command 510 may be used to avoid a collision with the detected object.

[0086] In some embodiments, the vehicle 600 is an at least partially autonomous vehicle, and the actuator 504 can be embodied in the brakes, propulsion system, engine, transmission, or steering wheel of the vehicle 600. Actuator control commands 510 can be determined so that the actuator 504 is controlled so that the vehicle 600 avoids a collision with a detected object. The detected objects can also be classified according to what the classifier 514 thinks they are most likely to be (such as a pedestrian or a tree). The actuator control commands 510 can be determined depending on the classification. In scenarios where adversarial attacks may occur, the system described above can be further trained to better detect objects or identify changes in lighting conditions or the angle of the sensors or cameras on the vehicle 600.

[0087] In some embodiments where the vehicle 600 is an at least partially autonomous robot, the vehicle 600 may be a mobile robot configured to perform one or more functions, such as flying, swimming, diving, and walking. The mobile robot may be an at least partially autonomous lawn mower or an at least partially autonomous cleaning robot. In such embodiments, the actuator control commands 510 may be determined such that a propulsion unit, a steering unit, and / or a braking unit of the mobile robot may be controlled such that the mobile robot may avoid a collision with the identified object.

[0088] In some embodiments, vehicle 600 is an at least partially autonomous robot in the form of a gardening robot. In such embodiments, vehicle 600 may use an optical sensor as sensor 506 to determine the state of plants in the environment proximate to vehicle 600. Actuator 504 may be a nozzle configured to spray a chemical. Depending on the identified species and / or identified state of the plant, actuator control commands 510 may be determined to cause actuator 504 to spray the plant with an appropriate amount of an appropriate chemical.

[0089] The vehicle 600 may be an at least partially autonomous robot in the form of a household appliance. Non-limiting examples of household appliances include a washing machine, a stove, an oven, a microwave, or a dishwasher. In such a vehicle 600, the sensor 506 may be an optical sensor configured to detect the state of an object to be processed by the household appliance. For example, in the case where the household appliance is a washing machine, the sensor 506 may detect the state of the laundry in the washing machine. The actuator control command 510 may be determined based on the detected state of the laundry.

[0090] Figure 7 A schematic diagram of a control system 502 is depicted, the control system 502 being configured to control a system 700 (e.g., a manufacturing machine), such as a punch tool, a tool, or a gun drill, of a manufacturing system 702 (such as part of a production line). The control system 502 can be configured to control an actuator 504, which is configured to control the system 700 (e.g., a manufacturing machine).

[0091] The sensor 506 of the control system 700 (e.g., a manufacturing machine) may be an optical sensor configured to capture one or more properties of the manufactured product 704. The classifier 514 may be configured to determine the state of the manufactured product 704 from one or more of the captured properties. The actuator 504 may be configured to control the system 700 (e.g., a manufacturing machine) depending on the determined state of the manufactured product 704 to perform subsequent manufacturing steps of the manufactured product 704. The actuator 504 may be configured to control the function of the control system 700 (e.g., a manufacturing machine) on the subsequent manufactured product 706 of the control system 700 (e.g., a manufacturing machine) depending on the determined state of the manufactured product 704.

[0092] Figure 8 A schematic diagram of a control system 502 configured to control a power tool 800 (such as a drill or driver) having an at least partially autonomous mode is depicted. The control system 502 may be configured to control an actuator 504 that is configured to control the power tool 800.

[0093] The sensor 506 controlling the power tool 800 may be an optical sensor configured to capture one or more properties of the work surface 802 and / or the fastener 804 driven into the work surface 802. The classifier 514 may be configured to determine the state of the work surface 802 and / or the fastener 804 relative to the work surface 802 from one or more captured properties. The state may be that the fastener 804 is flush with the work surface 802. The state may alternatively be the hardness of the work surface 802. The actuator 504 may be configured to control the power tool 800 so that the drive function of the power tool 800 is adjusted depending on the determined state of the fastener 804 relative to the work surface 802 or one or more captured properties of the work surface 802. For example, if the state of the fastener 804 is flush with the work surface 802, the actuator 504 may stop the drive function. As another non-limiting example, the actuator 504 may apply additional or less torque depending on the hardness of the work surface 802.

[0094] Fig. 9 Depicted is a schematic diagram of a control system 502 configured to control an automated personal assistant 900. The control system 502 may be configured to control an actuator 504 configured to control the automated personal assistant 900. The automated personal assistant 900 may be configured to control a household appliance, such as a washing machine, stove, oven, microwave, or dishwasher.

[0095] Sensor 506 may be an optical sensor and / or an audio sensor. The optical sensor may be configured to receive a video image of user 902's gesture 904. The audio sensor may be configured to receive a voice command of user 902.

[0096] The control system 502 of the automated personal assistant 900 may be configured to determine an actuator control command 510 that is configured to control the system 502. The control system 502 may be configured to determine the actuator control command 510 based on the sensor signal 508 of the sensor 506. The automated personal assistant 900 is configured to transmit the sensor signal 508 to the control system 502. The classifier 514 of the control system 502 may be configured to execute a gesture recognition algorithm to identify the gesture 904 performed by the user 902, determine the actuator control command 510, and transmit the actuator control command 510 to the actuator 504. The classifier 514 may be configured to retrieve information from the non-volatile storage device in response to the gesture 904 and output the retrieved information in a form suitable for receipt by the user 902.

[0097] Fig.10A schematic diagram of a control system 502 is depicted, which is configured to control a monitoring system 1000. The monitoring system 1000 can be configured to physically control access through a door 1002. A sensor 506 can be configured to detect scenes relevant to deciding whether access is authorized. The sensor 506 can be an optical sensor configured to generate and transmit image and / or video data. Such data can be used by the control system 502 to detect a person's face.

[0098] The classifier 514 of the control system 502 of the monitoring system 1000 can be configured to interpret the image and / or video data by matching the identities of known persons stored in the non-volatile storage device 516 to determine the identity of the person. The classifier 514 can be configured to generate an actuator control command 510 in response to the interpretation of the image and / or video data. The control system 502 is configured to transmit the actuator control command 510 to the actuator 504. In this embodiment, the actuator 504 can be configured to lock or unlock the door 1002 in response to the actuator control command 510. In some embodiments, non-physical logical access control is also possible.

[0099] The monitoring system 1000 may also be a surveillance system. In such an embodiment, the sensor 506 may be an optical sensor configured to detect a scene under surveillance, and the control system 502 is configured to control the display 1004. The classifier 514 is configured to determine a classification of the scene, for example, whether the scene detected by the sensor 506 is suspicious. The control system 502 is configured to transmit the actuator control command 510 to the display 1004 in response to the classification. The display 1004 may be configured to adjust the displayed content in response to the actuator control command 510. For example, the display 1004 may highlight an object that the classifier 514 considers suspicious. Using embodiments of the disclosed system, the surveillance system can predict the appearance of an object at some time in the future.

[0100] Fig.11 A schematic diagram of a control system 502 is depicted, which is configured to control an imaging system, such as an MRI device, an x-ray imaging device, or an ultrasound device. For example, the sensor 506 can be an imaging sensor. The classifier 514 can be configured to determine a classification of all or part of the sensed image. The classifier 514 can be configured to determine or select an actuator control command 510 in response to the classification obtained by the trained neural network. For example, the classifier 514 can interpret an area of ​​the sensed image as potentially abnormal. In this case, the actuator control command 510 can be determined or selected so that the display 1102 displays the imaging and highlights the area of ​​the potential abnormality.

[0101] In some embodiments, a method for labeling audio data includes receiving video stream data associated with a data capture environment from at least one image capture device. The method also includes receiving audio stream data corresponding to at least a portion of the video stream data from at least one audio capture array. The method also includes labeling at least some objects of the video stream data using output from at least a first machine learning model, the first machine learning model being configured to provide output including one or more object detection predictions. The method also includes calculating at least one offset value for at least a portion of the audio stream data corresponding to at least one labeled object of the video stream data based on at least one data capture characteristic, and synchronizing at least a portion of the video stream data with a portion of the audio stream data corresponding to at least one labeled object of the video stream data using at least the at least one offset value. The method also includes labeling at least a portion of the audio stream data corresponding to at least one labeled object of the video stream data using one or more tags and at least one offset value for the labeled objects of the video stream data. The method also includes generating training data using at least some of the labeled portions of the audio stream data and training a second machine learning model using the training data.

[0102] In some embodiments, at least one audio capture array includes a plurality of audio capture devices. In some embodiments, at least one audio capture array is remotely located from at least one image capture device. In some embodiments, marking at least some objects of the video stream data using outputs from at least a first machine learning model includes marking at least some objects of the video stream data with at least an event type, an event start indicator, and an event end indicator. In some embodiments, at least one data capture characteristic includes one or more characteristics of at least one image capture device. In some embodiments, at least one data capture characteristic includes one or more characteristics of at least one audio capture array. In some embodiments, at least one data capture characteristic includes one or more characteristics corresponding to a position of at least one image capture device relative to at least one audio capture array. In some embodiments, at least one data capture characteristic includes one or more characteristics corresponding to movement of an object in the video stream data. In some embodiments, calculating at least one offset value for at least a portion of the audio stream data corresponding to at least one marked object of the video stream data based on at least one data capture characteristic includes using at least one probability-based function.

[0103] Fig.12 Depicted is a goal description network 1200 that can be used to encode a rich representation of the task being performed. The goal description network 1200 utilizes the established base model to extract a unified multimodal goal embedding and can be executed by the processor 204.

[0104] At 1202, the target description network 1200 directs the target input to the appropriate base model based on the modality of the target input. For example, the target input can be an image, a spoken command, a written command, or any appropriate modality. At 1204, the target description network 1200 processes each modality with its appropriate interface. For example, vision-based commands will be routed to the visual interface, language-based input signals are routed to the language interface, and audio-based input targets are routed to the audio signal interface (also known as the signal "x" interface).

[0105] At 1206, the target description network 1200 can route the output of the base model to a target decoder, which can refine the target representation by contrast regularization. For example, the target decoder can project the output of the established base model into a representation space that can be used by the downstream partial processes described herein. At 1208, the target description network 1200 generates an update or generates a target embedding based on the output of the target decoder. The target embedding includes all inputs related to the target, regardless of the modality of the input.

[0106] Although exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms covered by the claims. The words used in the specification are descriptive words, not limiting words, and it is understood that various changes can be made without departing from the spirit and scope of the present disclosure. As previously described, the features of each embodiment can be combined to form further embodiments of the present invention that may not be explicitly described or illustrated. Although each embodiment may have been described as providing advantages or being superior to other embodiments or prior art implementations in terms of one or more desired characteristics, it is recognized by those of ordinary skill in the art that one or more features or characteristics can be compromised to achieve the desired overall system properties, depending on the specific application and implementation. These properties may include (but are not limited to) cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, applicability, weight, manufacturability, ease of assembly, etc. As such, to the extent that any embodiment is described as being less desirable than other embodiments or prior art implementations in terms of one or more characteristics, these embodiments are not outside the scope of the present disclosure and may be desirable for a specific application.

Claims

1. A computer-implemented method for a machine learning network, comprising: receiving, via the device, a command from a user associated with the subject matter; accessing a representation space associated with the command, wherein similar topics and commands in the representation space are clustered together; receiving a first data set associated with the command, a second data set associated with the subject matter, and a third data set including the subject matter associated with the command; updating the representation space based on at least one of the first data set, the second data set, and the third data set; Generate target representation based on representation space through target description machine learning model; receiving sensor data of the current environment from a plurality of sensors; generating a first series of steps and a second series of steps based on the target representation and the current environment; annotating the sensor data based on the execution of the first series of steps by progress describing the machine learning model to generate annotated sensor data; as well as A second series of steps is updated based on the annotated sensor data via a strategic machine learning model.

2. The computer-implemented method of claim 1 , wherein updating the representation space comprises the steps of: analyzing the first data set and the second data set in view of the target representation to determine an inter-task score for at least one subject represented in a representation space associated with a subject of the command; as well as The position of the at least one subject in the target representation is regularized based on the inter-task score.

3. The computer-implemented method of claim 1 , wherein updating the representation space comprises the steps of: analyzing the third data set in view of the target representation to determine a within-task score for at least one subject represented in the representation space that is not associated with the subject of the command; as well as The position of the at least one subject in the target representation is regularized based on the within-task score.

4. The computer-implemented method of claim 1 , wherein the first data set comprises target-related sensor data organized as tuples, wherein each sensor data is positively correlated to a command, wherein each tuple comprises subject-related sensor data, instruction-related sensor data, and audio-related sensor data; wherein the second data set comprises target-related sensor data organized as tuples, wherein one of the sensor data is negatively correlated with the command; and The third data set includes target-related sensor data organized as tuples, wherein the sensor data is either negatively correlated with the command or positively correlated with the command.

5. The computer-implemented method of claim 1 , wherein the policy machine learning model is further trained based on the annotated sensor data.

6. A computer-implemented method according to claim 1, wherein the training of the goal description machine learning model, the progress description machine learning model and the strategy machine learning model is frozen.

7. A computer-implemented method according to claim 1, wherein the training of the goal description machine learning model, the progress description machine learning model and the strategy machine learning model is performed at a server and performed locally at a device.

8. A system for a machine learning network, comprising: One or more processors configured to: receiving, via the device, a command from a user associated with the subject matter; accessing a representation space associated with the command, wherein similar topics and commands in the representation space are clustered together; receiving a first data set associated with the command, a second data set associated with the subject matter, and a third data set including the subject matter associated with the command; updating the representation space based on at least one of the first data set, the second data set, and the third data set; Generate target representation based on representation space through target description machine learning model; receiving sensor data of the current environment from a plurality of sensors; generating a first series of steps and a second series of steps based on the target representation and the current environment; annotating the sensor data based on the execution of the first series of steps by progress describing the machine learning model to generate annotated sensor data; as well as A second series of steps is updated based on the annotated sensor data via a strategic machine learning model.

9. The system of claim 8, wherein updating the representation space comprises the following steps: analyzing the first data set and the second data set in view of the target representation to determine an inter-task score for at least one subject represented in a representation space associated with a subject of the command; The position of the at least one subject in the target representation is regularized based on the inter-task score.

10. The system of claim 8, wherein updating the representation space comprises the following steps: analyzing the third data set in view of the target representation to determine a within-task score for at least one subject represented in the representation space that is not associated with the subject of the command; The position of the at least one subject in the target representation is regularized based on the within-task score.

11. The system of claim 8, wherein the first data set comprises target-related sensor data organized as tuples, wherein each sensor data is positively correlated with a command, wherein each tuple comprises subject-related sensor data, instruction-related sensor data, and audio-related sensor data; wherein the second data set comprises target-related sensor data organized as tuples, wherein one of the sensor data is negatively correlated with the command; and The third data set includes target-related sensor data organized as tuples, wherein the sensor data is either negatively correlated with the command or positively correlated with the command.

12. The system of claim 8, wherein the policy machine learning model is further trained based on the annotated sensor data.

13. A system according to claim 8, wherein the training of the goal description machine learning model, the progress description machine learning model and the strategy machine learning model is frozen.

14. A system according to claim 8, wherein the training of the goal description machine learning model, the progress description machine learning model and the strategy machine learning model is performed at the server and performed locally at the device.

15. A machine learning network for a machine learning network, comprising: One or more processors configured to: receiving, via the device, a command from a user associated with the subject matter; accessing a representation space associated with the command, wherein similar topics and commands in the representation space are clustered together; receiving a first data set associated with the command, a second data set associated with the subject matter, and a third data set including the subject matter associated with the command; updating the representation space based on at least one of the first data set, the second data set, and the third data set; Generate target representation based on representation space through target description machine learning model; receiving sensor data of the current environment from a plurality of sensors; generating a first series of steps and a second series of steps based on the target representation and the current environment; annotating the sensor data based on the execution of the first series of steps by progress describing the machine learning model to generate annotated sensor data; as well as A second series of steps is updated based on the annotated sensor data via a strategic machine learning model.

16. The machine learning network of claim 15, wherein updating the representation space comprises the following steps: analyzing the first data set and the second data set in view of the target representation to determine an inter-task score for at least one subject represented in a representation space associated with a subject of the command; as well as The position of the at least one subject in the target representation is regularized based on the inter-task score.

17. The machine learning network of claim 15, wherein the first data set comprises target-related sensor data organized as tuples, each sensor data being positively correlated with a command, each tuple comprising subject-related sensor data, instruction-related sensor data, and audio-related sensor data; wherein the second data set comprises target-related sensor data organized as tuples, wherein one of the sensor data is negatively correlated with the command; and The third data set includes target-related sensor data organized as tuples, wherein the sensor data is either negatively correlated with the command or positively correlated with the command.

18. The machine learning network of claim 15, wherein the policy machine learning model is further trained based on the annotated sensor data.

19. The machine learning network of claim 15, wherein the training of the goal description machine learning model, the progress description machine learning model, and the strategy machine learning model are frozen.

20. The machine learning network of claim 15, wherein the training of the goal description machine learning model, the progress description machine learning model, and the strategy machine learning model is performed at a server and performed locally at a device.

Images