System and method for providing preceding temporal ensemble technique for enhancing imitation learning-based robot task speed

Abstract

This system for providing a predictive temporal ensemble technique for enhancing imitation learning-based robot task speed comprises a memory storing instructions and a processor, wherein the instructions, when executed by the processor, cause the system to: receive, as inputs, ambient environment data and robot movement data collected through teleoperation; learn a policy of a robot through an imitation learning algorithm by using the collected data, wherein the policy is a function that receives the current state as an input and outputs a future action sequence; generate a control input for the robot by ensembling, in a predictive temporal manner, actions after the current time point in a future action sequence predicted from the policy; and control the robot by using the generated control input.

Description

System and method providing a prior temporal ensemble technique for improving robot task speed based on imitation learning

[0001] The present invention relates to a method for improving the work speed of a robot moving through imitation learning. In particular, it provides a precedence-temporal ensemble technique that enables the robot to perform tasks faster without being limited by the work speed of the demonstrator.

[0002] Traditional robot control methods present the difficulty of requiring operators to manually program every movement of the robot. To address this problem, a method called imitation learning was introduced. Imitation learning is a method in which a robot learns a task by mimicking the actions of a demonstrator. However, existing imitation learning algorithms have a limitation in that the robot's task speed is influenced by the demonstrator's speed.

[0003] The present invention aims to improve the working speed of a robot in imitation learning-based robot control. Specifically, it aims to provide a method that enables a robot to perform tasks at a faster speed compared to conventional imitation learning.

[0004] The system can generate the robot's next movement in an end-to-end manner by receiving only images obtained from a camera and robot joint information as input. By utilizing a CVAE-based transformer structure, it can predict future robot movements from the current state and improve work speed through PTE.

[0005] A system providing a leading-temporal ensemble technique for improving robot task speed based on imitation learning includes a memory and a processor for storing instructions, wherein when the instructions are executed by the processor, the system receives surrounding environment data and robot movement data collected through teleoperation, and learns a robot policy through an imitation learning algorithm using the collected data, wherein the policy is a function that takes a current state as input and outputs a future action sequence, generates a robot control input by leading-temporally ensembling actions after the current point in time in the future action sequence predicted from the policy, and can control the robot using the generated control input.

[0006] The present invention utilizes a precedence-temporal ensemble technique to solve the above problem. This technique increases the robot's operation speed by reflecting the future action sequence predicted by the policy into the control input at the current time point. Specifically, the present invention includes the following steps: During the process of a demonstrator operating the robot using a teleoperation device, ambient environment data and robot movement data are collected. Using the collected data, the robot's policy is learned through an imitation learning algorithm. The policy is a function that takes the current state as input and outputs a future action sequence. Actions following the current time point in the future action sequence predicted by the policy are ensembled and used as the robot's control input.

[0007] The prior time ensemble technique of the present invention can provide the following effects.

[0008] The prior temporal ensemble technique of the present invention can accelerate the work speed of a robot moving by a network policy as desired. The prior temporal ensemble technique of the present invention can improve work speed without additional data collection or policy learning. The prior temporal ensemble technique of the present invention can be simply applied to existing imitation learning algorithms.

[0009] FIG. 1 is a diagram illustrating a system that provides a leading temporal ensemble technique for improving robot task speed based on imitation learning of artificial intelligence according to one embodiment.

[0010] FIG. 2 is a diagram illustrating the learning of a neural network according to one embodiment.

[0011] FIG. 3 is a diagram illustrating the configuration of an artificial intelligence model according to one embodiment.

[0012] FIG. 4 is a block diagram showing the configuration of a system providing a leading temporal ensemble technique for improving robot task speed based on imitation learning according to one embodiment.

[0013] FIG. 5 is a flowchart of a method for providing a leading temporal ensemble technique for improving robot task speed based on imitation learning according to one embodiment.

[0014] FIG. 6 is a flowchart illustrating a method for providing a leading temporal ensemble technique for improving robot task speed based on imitation learning according to one embodiment.

[0015] FIG. 7 is a flowchart illustrating a method for providing a leading temporal ensemble technique for improving robot task speed based on imitation learning according to one embodiment.

[0016] FIG. 8 illustrates the prediction and ensemble process of robot motion over time according to one embodiment.

[0017] Figure 9 shows the overall configuration of a dual robot system for block color classification tasks.

[0018] Figure 10 illustrates an experimental scene showing the continuous process of block classification using a priori temporal ensemble algorithm in chronological order.

[0019] Hereinafter, embodiments are described in detail with reference to the attached drawings. However, various modifications may be made to the embodiments, and thus the scope of the patent application is not limited or restricted by these embodiments. It should be understood that all modifications, equivalents, and substitutions to the embodiments are included within the scope of the rights.

[0020] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, the embodiments are not limited to the specific disclosed forms, and the scope of this specification includes modifications, equivalents, or substitutions that fall within the technical concept.

[0021] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0022] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or joined to that other component, or that there may be other components in between.

[0023] The terms used in the embodiments are for illustrative purposes only and should not be interpreted as intended to be limiting. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0024] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the embodiments pertain. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0025] In addition, when describing with reference to the attached drawings, identical components are assigned the same reference numeral regardless of drawing symbols, and redundant descriptions thereof are omitted. In describing the embodiments, if it is determined that a detailed description of related prior art could unnecessarily obscure the essence of the embodiments, such detailed description is omitted.

[0026] The embodiments can be implemented in various forms of products such as personal computers, laptop computers, tablet computers, smartphones, televisions, smart home appliances, intelligent automobiles, kiosks, and wearable devices.

[0027] Artificial Intelligence (AI) systems are computer systems that implement human-level intelligence; unlike existing rule-based smart systems, they are systems in which machines learn and make decisions autonomously. As AI systems improve in recognition accuracy and gain a more accurate understanding of user preferences with continued use, existing rule-based smart systems are gradually being replaced by deep learning-based AI systems.

[0028] Artificial intelligence technology consists of machine learning and component technologies utilizing machine learning. Machine learning is an algorithmic technology that autonomously classifies and learns the characteristics of input data, while component technologies are technologies that mimic the cognitive and judgmental functions of the human brain by utilizing machine learning algorithms such as deep learning, and are comprised of technological fields such as linguistic understanding, visual understanding, reasoning / prediction, knowledge representation, and motion control.

[0029] The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding refers to technologies that recognize, apply, and process human language and text, including natural language processing, machine translation, dialogue systems, question answering, and speech recognition / synthesis. Visual understanding refers to technologies that perceive and process objects like human vision, including object recognition, object tracking, image search, people recognition, scene understanding, spatial understanding, and image enhancement. Inference and prediction refers to technologies that logically reason and predict by judging information, including knowledge / probability-based inference, optimization prediction, preference-based planning, and recommendation. Knowledge representation refers to technologies that automatically process human experiential information into knowledge data, including knowledge construction (data generation / classification) and knowledge management (data utilization). Motion control refers to technologies that control the autonomous driving of vehicles and the movement of robots, including motion control (navigation, collision, driving) and manipulation control (behavior control).

[0030] Generally, to apply machine learning algorithms to real-world situations, training is performed using a trial-and-error method due to the inherent characteristics of the fundamental methodologies. In particular, deep learning requires hundreds of thousands of iterations. Since it is impossible to execute this in a real physical external environment, training is instead performed through simulations that virtually recreate the actual physical environment on a computer.

[0031] In the present invention, Artificial Intelligence (AI) refers to a technology that imitates human learning ability, reasoning ability, and perceptual ability, and implements them on a computer, and may include concepts such as machine learning and symbolic logic. Machine Learning (ML) is an algorithmic technology that classifies or learns the characteristics of input data on its own. AI technology can analyze input data as a machine learning algorithm, learn from the results of the analysis, and make judgments or predictions based on the results of the learning. Furthermore, technologies that mimic the functions of the human brain, such as cognition and judgment, by utilizing machine learning algorithms can also be understood as falling within the category of AI. For example, technological fields such as linguistic understanding, visual understanding, reasoning / prediction, knowledge representation, and motion control may be included.

[0032] Machine learning can refer to the process of training neural network models using experience in processing data. It implies that through machine learning, computer software improves its own data processing capabilities. A neural network model is constructed by modeling the correlations between data, and these correlations can be expressed by multiple parameters. A neural network model extracts and analyzes features from given data to derive correlations between them; machine learning can be defined as the process of optimizing the model's parameters by repeating this process. For example, a neural network model can learn the mapping (correlation) between inputs and outputs for data given as input-output pairs. Alternatively, even when only input data is provided, a neural network model can derive regularities between the given data and learn those relationships.

[0033] An artificial intelligence learning model or neural network model can be designed to implement the structure of the human brain on a computer and may include multiple network nodes that have weights and simulate neurons of a human neural network. The multiple network nodes may have interconnected relationships by simulating the synaptic activity of neurons, where neurons exchange signals through synapses. In an artificial intelligence learning model, multiple network nodes may be located in layers of different depths and exchange data according to convolutional connections. The artificial intelligence learning model may be, for example, an Artificial Neural Network (ANN) or a Convolutional Neural Network (CNN). As an embodiment, the artificial intelligence learning model may be machine learned according to methods such as supervised learning, unsupervised learning, and reinforcement learning. Machine learning algorithms for performing machine learning may include Decision Tree, Bayesian Network, Support Vector Machine, Artificial Neural Network, Ada-boost, Perceptron, Genetic Programming, and Clustering.

[0034] Among these, CNNs are a type of multilayer perceptron designed to use minimal preprocessing. CNNs consist of one or more convolutional layers and standard artificial neural network layers stacked on top, additionally utilizing weights and pooling layers. Thanks to this structure, CNNs can fully utilize two-dimensional input data. Compared to other deep learning architectures, CNNs demonstrate good performance in both image and audio fields. CNNs can also be trained using standard backpropagation. CNNs have the advantage of being easier to train than other feedforward artificial neural network techniques and using a small number of parameters.

[0035] Convolutional networks are neural networks comprising sets of nodes with bounded parameters. Many computer vision tasks have been significantly improved, driven by the increased size of available training data and the availability of computational power, combined with advancements in algorithms such as discriminative linear units and dropout training. In the case of massive datasets, such as those available for many tasks today, outfitting is not critical, and increasing the network size improves test accuracy. Optimal utilization of computing resources becomes a limiting factor. To address this, distributed, scalable implementations of deep neural networks can be used.

[0036]

[0037] FIG. 1 is a diagram illustrating a system that provides a leading temporal ensemble technique for improving robot task speed based on imitation learning of artificial intelligence according to one embodiment.

[0038] As illustrated in FIG. 1, a system (100) providing a prior temporal ensemble technique for improving robot task speed based on imitation learning of artificial intelligence may include a plurality of user terminals (110-1,…), a server (120), and a database (130). According to one embodiment, the database (130) is shown as being configured separately from the server (120), but is not limited thereto, and the database (130) may be provided within the server (120). For example, the server (120) may include a plurality of artificial intelligences for performing machine learning algorithms. According to one embodiment, the plurality of user terminals (110-1, …), the server (120), and the database (130) may be connected to communicate with each other through a network (N).

[0039] A network (N) can perform wireless or wired communication between multiple user terminals (110-1, …), a server (120), a database (130), etc. For example, the network can perform wireless communication according to methods such as LTE (long-term evolution), LTE-A (LTE Advanced), CDMA (code division multiple access), WCDMA (wideband CDMA), WiBro (Wireless BroadBand), WiFi (wireless fidelity), Bluetooth, NFC (near field communication), GPS (Global Positioning System), or GNSS (global navigation satellite system). For example, the network (N) can perform wired communication according to methods such as USB (universal serial bus), HDMI (high definition multimedia interface), RS-232 (recommended standard 232), or POTS (plain old telephone service).

[0040] The database (130) can store various data. The data stored in the database (130) may include software (e.g., programs) as data acquired, processed, or used by at least one component of a plurality of user terminals (110-1, server (120). The database (130) may include volatile and / or non-volatile memory.

[0041] In the present invention, Artificial Intelligence (AI) refers to a technology that imitates human learning ability, reasoning ability, and perceptual ability, and implements them on a computer, and may include concepts such as machine learning and symbolic logic. Machine Learning (ML) is an algorithmic technology that classifies or learns the characteristics of input data on its own. AI technology can analyze input data as a machine learning algorithm, learn from the results of the analysis, and make judgments or predictions based on the results of the learning. Furthermore, technologies that mimic the functions of the human brain, such as cognition and judgment, by utilizing machine learning algorithms can also be understood as falling within the category of AI. For example, technological fields such as linguistic understanding, visual understanding, reasoning / prediction, knowledge representation, and motion control may be included.

[0042] Machine learning can refer to the process of training neural network models using experience in processing data. Through machine learning, computer software can be said to improve its own data processing capabilities. A neural network model is constructed by modeling the correlations between data, and these correlations can be expressed by multiple parameters. A neural network model extracts and analyzes features from given data to derive correlations between them; machine learning can be defined as the process of optimizing the model's parameters by repeating this process. For example, a neural network model can learn the mapping (correlation) between inputs and outputs for data given as input-output pairs. Alternatively, even when only input data is provided, a neural network model can derive regularities between the given data and learn those relationships.

[0043] An artificial intelligence learning model or neural network model can be designed to implement the structure of the human brain on a computer and may include multiple network nodes that have weights and simulate neurons of a human neural network. The multiple network nodes may have interconnected relationships by simulating the synaptic activity of neurons, where neurons exchange signals through synapses. In an artificial intelligence learning model, multiple network nodes may be located in layers of different depths and exchange data according to convolutional connections. The artificial intelligence learning model may be, for example, an Artificial Neural Network (ANN) or a Convolutional Neural Network (CNN). As an embodiment, the artificial intelligence learning model may be machine learned according to methods such as supervised learning, unsupervised learning, and reinforcement learning. Machine learning algorithms for performing machine learning may include Decision Tree, Bayesian Network, Support Vector Machine, Artificial Neural Network, Ada-boost, Perceptron, Genetic Programming, and Clustering.

[0044] Among these, CNNs are a type of multilayer perceptron designed to use minimal preprocessing. CNNs consist of one or more convolutional layers and standard artificial neural network layers stacked on top, additionally utilizing weights and pooling layers. Thanks to this structure, CNNs can fully utilize two-dimensional input data. Compared to other deep learning architectures, CNNs demonstrate good performance in both image and audio fields. CNNs can also be trained using standard backpropagation. CNNs have the advantage of being easier to train than other feedforward artificial neural network techniques and using a small number of parameters.

[0045] Convolutional networks are neural networks comprising sets of nodes with bounded parameters. Many computer vision tasks have been significantly improved, thanks to the increased size of available training data and the availability of computational power, combined with advancements in algorithms such as discriminative linear units and dropout training. In the case of massive datasets, such as those available for many tasks today, outfitting is not critical, and increasing the network size improves test accuracy. Optimal utilization of computing resources becomes a limiting factor. To address this, distributed, scalable implementations of deep neural networks can be used.

[0046]

[0047] FIG. 2 is a diagram illustrating the learning of a neural network according to one embodiment.

[0048] As illustrated in FIG. 2, the learning device can train a neural network (123) to process review responses received from a plurality of user terminals (110-1, ���) by item. Additionally, the learning device can train a neural network (123) to extract user stay history from user movement path information. According to one embodiment, the learning device may be a separate entity from the server (120), but is not limited thereto.

[0049] The neural network (123) includes an input layer (121) into which training samples are input and an output layer (125) that outputs training outputs, and can be trained based on the difference between the training outputs and the labels. Here, the labels are defined based on items corresponding to review responses and can be defined based on user dwell history corresponding to movement path information. The neural network (123) is connected as a group of multiple nodes and is defined by weights between the connected nodes and an activation function that activates the nodes.

[0050] The learning device can train the neural network (123) using the GD (Gradient Descent) technique or the SGD (Stochastic Gradient Descent) technique. The learning device can use a loss function designed by the outputs and labels of the neural network.

[0051] The learning device can calculate the training error using a predefined loss function. The loss function can be predefined with labels, outputs, and parameters as input variables, where the parameters can be set by weights within the neural network (123). For example, the loss function can be designed in the form of Mean Square Error (MSE), entropy, etc., and various techniques or methods may be employed in the embodiments in which the loss function is designed.

[0052] The learning device can find weights that affect the training error using the backpropagation technique. Here, the weights are relationships between nodes within the neural network (123). The learning device can use the SGD technique with labels and outputs to optimize the weights found through the backpropagation technique. For example, the learning device can update the weights of a loss function defined based on labels, outputs, and weights using the SGD technique.

[0053] According to one embodiment, the learning device extracts first objects from review responses, obtains first labels which are items corresponding to the first objects, applies the first objects to a first neural network to generate first training outputs corresponding to the first objects, and can train the first neural network based on the first training outputs and the first labels.

[0054] The learning device extracts second objects from movement path information, obtains second labels which are user dwell history corresponding to the second objects, applies the second objects to a second neural network to generate second training outputs corresponding to the second objects, and can train the second neural network based on the second training outputs and second labels.

[0055] According to one embodiment, the learning device can generate first training feature vectors based on the constituent features, location features, and pattern features of the review response. Various methods may be employed for extracting features.

[0056] According to one embodiment, the learning device can generate second training feature vectors based on the constituent features, length features, and pattern features of the movement path information. Various methods may be employed for extracting features.

[0057] According to one embodiment, the learning device may obtain training outputs by applying first training feature vectors to the neural network (123). The learning device may train a review item extraction algorithm of the neural network (123) based on the training outputs and first labels. The learning device may train a review item extraction algorithm of the neural network (123) by calculating training errors corresponding to the training outputs and optimizing the connection relationships of nodes within the neural network (123) to minimize the training errors. The server (120) may extract items from review responses using the first neural network that has been trained. For example, the extracted items may include, but are not limited to, friendliness, store cleanliness, service satisfaction, etc.

[0058] According to one embodiment, the learning device can obtain training outputs by applying second training feature vectors to the neural network (123). The learning device can train the user stay history acquisition algorithm of the neural network (123) based on the training outputs and second labels. The learning device can train the user stay history acquisition algorithm of the neural network (123) by calculating training errors corresponding to the training outputs and optimizing the connection relationships of nodes within the neural network (123) to minimize the training errors. The server (120) can obtain user stay history from movement path information using the second neural network that has been trained.

[0059]

[0060] FIG. 3 is a diagram illustrating the configuration of an artificial intelligence model according to one embodiment.

[0061] An artificial intelligence model according to one embodiment may include an input layer, a hidden layer, and an output layer.

[0062] The input layer is the layer associated with the input values ​​fed into the artificial intelligence model.

[0063] In the hidden layer, a feature map can be output by performing MAC (multiply-accumulate) and activation operations on the input values.

[0064] A MAC operation can be an operation that multiplies each input value by its corresponding weight and sums the multiplied values.

[0065] The activation operation may be an operation that inputs the result of the MAC operation into an activation function and outputs a result value. The activation function may be of various types. For example, the activation function may include a sigmoid function, a tangent function, a ReLU function, a Leaky ReLU function, a Max Out function, and / or an ELU function, but there are no restrictions on the types.

[0066] A hidden layer may consist of at least one layer. For example, if the hidden layer consists of a first hidden layer and a second hidden layer, the first hidden layer performs MAC operations and activation operations based on input values ​​of an input system to output a feature map, and the feature map, which is the result value of the first hidden layer, may become the input value of the second hidden layer. The second hidden layer may perform MAC operations and activation operations based on the feature map, which is the result value of the first hidden layer.

[0067] The output layer may be a layer associated with the result of an operation performed in the hidden layer.

[0068] In one embodiment, the learning model learns syllable (character) patterns that are frequently combined and used in a given corpus to automatically learn the boundaries of compound words and named entities, integrates object information from a first UI source with object information rendered in a browser to create a learning object information file, uses the learning object information file to create learning data for the learning of a deep learning network, receives data from various domains of a support system, standardizes the data from the various domains into an integrated format based on at least one standardization method corresponding to each of the various domains, learns and infers data from a specific domain, determines information to be transmitted for standardization from the data of the specific domain, and can perform post-processing on the data from the various domains. The first UI source includes an XML file, and the learning object information file includes an input JSON file for learning features and an output JSON file that is the correct answer (label) data during learning, and the output JSON file includes a file containing HTML DOM Tree information implemented in compliance with web standards, and the various domains include at least one of a RAN (radio access network), a transport, or a core, and the post-processing may include a correlation function.

[0069]

[0070] FIG. 4 is a block diagram showing the configuration of a system providing a leading temporal ensemble technique for improving robot task speed based on imitation learning according to one embodiment.

[0071] A system (400) according to one embodiment may include a processor (420) and a memory (430), and some of the illustrated components may be omitted or substituted. A system (400) according to one embodiment may be a server or a terminal. According to one embodiment, the processor (420) is a component capable of performing operations or data processing regarding the control and / or communication of each component of the system (400), and may be composed of one or more processors. The memory (430) may store information related to the method described above or store a program in which the method described above is implemented. The memory (430) may be volatile memory or non-volatile memory. The memory (430) may store various file data, and the stored file data may be updated according to the operation of the processor (120).

[0072] According to one embodiment, the processor (420) can execute a program and control the device (400). The code of the program executed by the processor (120) can be stored in memory (430). Operations of the processor (420) can be performed by loading instructions stored in memory (430). The system (400) can be connected to an external device (e.g., a personal computer or a network) through an input / output device (not shown in the drawing) and exchange data.

[0073] The present invention relates to an advanced action generation system for imitation learning, and in particular, describes in detail a robot control method utilizing the ACT (Action Chunking Transformer) algorithm.

[0074] ACT is based on a Conditional Variational Autoencoder (CVAE) structure and utilizes a Transformer architecture to configure the encoder and decoder. Specifically, the CVAE encoder receives the robot's continuous action data and current pose information as input and maps them into a latent space called a style vector. For example, when the robot performs a picking motion of an object, characteristics such as gripping strength or approach angle are encoded in this style vector.

[0075] The CVAE decoder receives RGB image data acquired from a stereo camera, along with the style vector generated in this way, and time-series position information of the robot end effector as input to predict a series of actions to be performed by the robot in the future. At this time, to clarify the time relationship, a sequence containing observation information from a past time point (t-ks) to a current time point (t) is represented as st-ks:t, and a predicted action sequence from a current time point (t) to a future time point (t+ka) is defined as at:t+ka. The system (400) collects image data using an RGB stereo camera with a resolution of 640x480 and can use the robot's joint angle information as an additional input. The system (400) can learn an End-to-End policy to generate the robot's next action from the image and joint information using a Transformer-based CVAE (Conditional Variational Autoencoder) structure.

[0076] The Action Chunking technique proposed in this invention predicts a continuous sequence of actions all at once, rather than actions at a single point in time. This effectively resolves the compounding error problem that occurs in conventional step-by-step prediction methods. For example, when a robot performs the task of picking up a cup and moving it to another location, the conventional method can make it difficult to achieve the final goal as small errors occurring at each step accumulate. In contrast, Action Chunking enables more natural and accurate execution of actions by predicting the entire motion as a single continuous sequence.

[0077] In actual robot control, an ensemble method is adopted that considers past predicted values ​​together. To this end, exponential function-based weights are applied, and a parameter m representing the slope of the weight function plays an important role. The larger the value of m, the greater the weight given to the latest prediction, enabling immediate response to environmental changes; conversely, the smaller the value, the more consistent operation robust against noise becomes possible. In the embodiment of the present invention, 0.05, which was identified as the optimal balance point through various experiments, was adopted as the value of m.

[0078] Through this method, the present invention enables more stable and precise object manipulation compared to existing robot control systems, and demonstrates superior performance, particularly in complex tasks requiring continuous motion.

[0079]

[0080] FIG. 5 is a flowchart of a method for providing a leading temporal ensemble technique for improving robot task speed based on imitation learning according to one embodiment.

[0081] Although process steps, method steps, algorithms, etc. are described in a sequential order in the flowchart of FIG. 5, such processes, methods, and algorithms may be configured to operate in any suitable order. In other words, the steps of the processes, methods, and algorithms described in various embodiments of the present invention do not need to be performed in the order described in the present invention. Furthermore, even if some steps are described as being performed asynchronously, in other embodiments, such steps may be performed simultaneously. Also, the example of a process by the depiction in the drawings does not mean that the exampled process excludes other variations and modifications therefrom, does not mean that any of the exampled process or its steps is essential to one or more of the various embodiments of the present invention, and does not mean that the exampled process is desirable.

[0082] In operation 510, the system (e.g., the system (400) of FIG. 4) can receive ambient environment data and robot movement data under the control of a processor (e.g., the processor (420) of FIG. 4). The system (400) can receive ambient environment data and robot movement data collected through teleoperation.

[0083] The system (400) collects surrounding environment data through various sensors under the control of the processor (420). For example, the system (400) recognizes the type, location, size, etc. of objects from image data obtained through a camera and generates a 3D map of the surrounding environment through a LiDAR sensor. In addition, the system (400) collects movement data such as the angle, speed, and acceleration of the robot arm through a sensor attached to the robot arm.

[0084] The teleoperation system receives input data on the surrounding environment and robot movement collected while a human directly operates the robot. The system transmits the operator's movements to the robot in real time and provides the robot's sensor data to the operator, enabling remote control. For example, it shares the robot's field of view through VR equipment worn by the operator and transmits the force applied to the robot arm to the operator, providing the sensation of directly controlling the robot.

[0085] In operation 520, the system (400) can learn the robot's policy through an imitation learning algorithm. The system (400) can learn the robot's policy through an imitation learning algorithm using collected data. A policy may refer to a function that takes a current state as input and outputs a future action sequence.

[0086] The system (400) learns the robot's policy through an imitation learning algorithm using collected surrounding environment data and robot movement data. Imitation learning is an algorithm that learns how to perform a task by imitating the actions of an expert.

[0087] A policy is a function that takes the current state as input and outputs a sequence of future actions. For example, when a robot arm learns to grasp an object, the current position of the robot arm, the position of the object, and the type of the object become the input values, while the robot arm's movements—namely, joint angles, velocity, and acceleration—become the output values.

[0088] The system (400) uses the "Behavioral Cloning" algorithm, which is one of various imitation learning algorithms. Behavioral cloning is a method of learning policies in a supervised learning manner using behavioral data from experts.

[0089] Operations 530 and 540 operate as a real-time feedback loop and can constitute the actual operation phase of the system (400).

[0090] In operation 530, the system (400) predicts the robot's next movement through a policy. Specifically, it receives current environmental information (images and robot joint data) and generates a robot posture sequence for the next 1 to 2 seconds. At this time, the policy predicts not just the posture at the next point in time, but also postures at multiple future points in time. For example, by presenting a posture 1 second in time as a target instead of a posture 0.1 seconds in time, the robot is enabled to move at a faster speed. The system (400) generates a final control input by ensembling these predicted future postures in a leading time.

[0091] The system (400) generates control inputs for the robot by ensembling actions after the current point in time in a predicted future action sequence in a leading time. Ensemble is a technique that combines multiple models to improve performance. For example, when the policy predicts 10 action sequences, the final control input is generated by averaging these 10 sequences.

[0092] Precedential ensemble is a technique that reflects the behavior of a future point in time into the control input of the present point in time. For example, when a robot arm moves to grasp an object, the policy can predict not only the motion of grasping the object but also the motion of moving the object after grasping it. In this case, the motion of moving the object is ensembled precedentially and reflected in the robot arm control input of the present point in time.

[0093] In operation 540, the system (400) controls the robot according to the generated control input and feeds the result back to 530. The control input is transmitted to the motors of each robot joint to achieve a target posture. The current state of the robot is monitored in real time through sensors, and this information is transmitted to 530 as an input for the next prediction. This feedback loop operates in real time at a cycle of 20 Hz, allowing the robot to move continuously and stably. The system (400) receives the current state input every 20 Hz cycle to predict future movements and can improve the work speed through the Predictive Task Execution (PTE) technique.

[0094] Through this real-time feedback structure, the system (400) can respond immediately to environmental changes while achieving a fast work speed through a leading time ensemble. The system (400) can terminate the operations of FIG. 5 when the robot's control operation is completed and user input to stop the feedback is detected.

[0095] The system (400) can control the robot using control inputs. The control inputs are transmitted to each joint of the robot to control the robot's movement. For example, in the case of a robot arm, the angle, speed, acceleration, etc., of each joint are controlled to perform desired actions. The robot's movement is monitored in real time through sensors, and the control inputs are dynamically adjusted according to the robot's state. For example, the system (400) transmits the generated control inputs to the robot's motors and joints to move it to a desired position and perform specific tasks. For example, the robot can perform actions such as moving to a target point while avoiding obstacles or picking up an object and moving it to another location. The system (400) adjusts these control inputs in real time to enable the robot to perform tasks accurately and efficiently.

[0096] The system (400) collects data regarding the robot's surrounding environment and movement through a processor (420). Specifically, it acquires environmental data in real time from various sensors (e.g., camera, LiDAR, IMU, etc.) mounted on the robot, and also collects movement data such as the robot's joint angles, speed, and acceleration. In particular, during the teleoperation process, all data generated while a professional operator remotely controls the robot is stored and used as an important dataset for future learning. For example, when the robot performs the task of picking up and moving an object, detailed data such as the object's position and shape information, the approach angle of the robot gripper, and the gripping force are all recorded.

[0097] The data collected in this way is utilized for learning the robot's policy through an imitation learning algorithm. A policy is a function that takes the robot's current state as input and determines the optimal sequence of actions. For example, it plans all actions to be performed over the next few seconds by receiving the robot's current position, attitude, and surrounding environment information. During the learning process, a deep learning-based neural network is used to extract effective motion patterns from expert demonstration data and generalize them to generate policies applicable in various situations.

[0098] The learned policy is used to generate control inputs for the robot in real time. Specifically, the optimal control input is calculated by temporally ensembling actions following the current point in time within the future action sequence predicted by the policy. For example, the most appropriate control command at the current point in time is determined by considering all joint angles and torque values ​​for the next 1 second. This process enables stable and efficient control by simultaneously considering the robot's dynamic constraints and task objectives.

[0099] Finally, the generated control inputs are transmitted to each joint and actuator of the robot to implement actual motion. The control inputs primarily consist of the joint's target angle, velocity, and torque, and the robot's low-level controller converts these into actual motor commands for execution. This process is repeated at a very high rate, creating continuous and smooth robot movement.

[0100]

[0101]

[0102] According to one embodiment, the policy may be characterized by being learned using an Action Chunking with Transformer (ACT) algorithm based on Conditional Variational Autoencoder (CVAE).

[0103] The system (400) performs robot control through an advanced policy that combines the Conditional Variational Autoencoder (CVAE) and Action Chunking with Transformer (ACT) algorithms. CVAE is a probabilistic generative model capable of generating various behavioral patterns based on conditions such as input states or goals. For example, in a situation where a robot needs to pick up an object, it can generate different grasping actions depending on the shape or position of the object. CVAE maps input data into a latent space through an encoder and generates actions that meet the conditions through a decoder. By learning the probability distribution of latent variables, it can present various solutions even in similar situations. The ACT algorithm can generate robot action sequences (chunks) for the next 1 to 2 seconds based on environmental information (e.g., images and robot joint data) during each control cycle. Since the future prediction values ​​generated at each point in time are accumulated at the current point in time, the ACT algorithm can create stable movement by acting as a kind of filter by combining them to generate the current robot's action. For example, a series of actions such as 'object approach - gripper opening - object grasp - lifting' is recognized as a single chunk. Each chunk has an independent meaning while being organically connected for the execution of the overall task. Through the Transformer's self-attention mechanism, the temporal and logical relationships between chunks are learned to establish long-term action plans.

[0104] The system (400) efficiently links current and future actions through a preceding temporal ensemble technique. This is a method of integrating multiple future actions predicted after a currently performed action into a single control signal. For example, in a task where a robot picks up and moves an object, the next actions, such as gripper control and lifting, can already be prepared while the robot is approaching the object. These future actions represent the part of the entire action sequence generated by the policy that follows the present.

[0105] In the integration process, the average or weighted average of the predicted future actions is calculated. The weights can be determined based on the importance or temporal proximity of each future action. For example, higher weights can be assigned to actions closer to the future to improve immediate responsiveness. Finally, the system (400) utilizes these integrated control inputs to perform the current action while simultaneously preparing the next action, thereby increasing the overall speed and efficiency of the task execution. For example, parallel processing is possible, such as moving an object while simultaneously rotating the gripper at an appropriate angle.

[0106] According to one embodiment, the preceding temporal ensemble may be characterized by integrating a plurality of future actions predicted after an action at a current point in time into a single control input, thereby enabling the robot to prepare for the next actions while performing the current action.

[0107]

[0108]

[0109] According to one embodiment, CVAE may be characterized by learning the conditional probability distribution of input data to generate various action sequences according to given conditions.

[0110] According to one embodiment, ACT may be characterized by using a Transformer model to divide a sequence of actions into chunks and sequentially predict each chunk to establish a long-term action plan.

[0111] According to one embodiment, future actions may be characterized as meaning actions after the present point in time in a sequence of actions predicted by the policy. According to one embodiment, integration may be characterized as including calculating the average or weighted average of the future actions.

[0112] According to one embodiment, the robot may be characterized by improving work speed by simultaneously performing the current action and the next action using the integrated control input.

[0113] The system (400) can control the robot to move quickly from the current position to the next position by integrating the predicted future positions after the current robot position into a single control input. For example, the system (400) can control the robot to move at a faster speed by presenting the robot with a position 1 second later as a target instead of a position 0.1 seconds later.

[0114]

[0115]

[0116] According to one embodiment, the system (400) can improve the working speed of the robot without additional data collection or policy learning. The surrounding environment data may include location and state information of the target object that the robot must manipulate. The robot movement data may be characterized by including the joint angles of the robot and the position information of the end effector.

[0117] The system (400) improves the robot's work speed without additional data collection or policy learning. That is, the robot's work speed can be increased using only a prior temporal ensemble technique by utilizing already learned policies. The system (400) can improve the robot's work speed without collecting new data or relearning policies after policy learning. This helps increase the efficiency of the system and improve adaptability to new environments or tasks.

[0118] According to one embodiment, the work speed improvement may be characterized as being achieved only through the aforementioned pre-temporal ensemble technique. The work speed improvement is achieved only through the pre-temporal ensemble technique. As previously explained, the pre-temporal ensemble reflects the action at a future point in time into the control input at the present point in time, enabling the robot to prepare for the next action in advance.

[0119] According to one embodiment, the system (400) can improve the working speed of the robot without collecting new data or relearning the policy after policy learning.

[0120] According to one embodiment, the position information of the target object may include the three-dimensional coordinates, direction, size, etc. of the target object, and the state information of the target object may include the color, shape, material, weight, etc. of the target object.

[0121] According to one embodiment, the joint angle of the robot is characterized as meaning the rotation angle or displacement of each joint, and the position information of the end effector may be characterized as including the three-dimensional coordinates and direction of the end effector.

[0122] Surrounding environment data includes location and state information of the target object that the robot must manipulate. The location information of the target object includes the object's 3D coordinates, orientation, size, etc. For example, if the robot needs to grasp a cup, it determines the cup's 3D coordinates, the cup's orientation (whether it is standing upright or lying down), and the cup's size (diameter, height), etc.

[0123] State information of the target object includes its color, shape, material, weight, etc. It identifies the cup's color, shape (cylindrical, square, etc.), material (glass, plastic, etc.), and weight (whether it is empty or contains a beverage). Robot motion data includes the robot's joint angles and end effector position information. The robot's joint angles refer to the rotation angle or displacement of each joint. In the case of a robot arm, the current posture of the robot arm is determined by measuring the rotation angle of each joint.

[0124] The position information of the end effector includes the 3D coordinates and orientation of the end effector. An end effector is a part attached to the end of a robot arm to perform a task, such as a gripper for grasping a cup or a screwdriver for assembling objects. The 3D coordinates and orientation of the end effector are measured to improve the accuracy of the task.

[0125]

[0126] The system (400) can efficiently improve the work speed of the robot without additional data collection or policy learning. This is based on a previously learned policy and is realized through a prior temporal ensemble technique. A prior temporal ensemble technique refers to a method of optimizing the movement of the robot by considering data from multiple points in time simultaneously.

[0127]

[0128] The robot movement data processed by the system (400) may include the following information.

[0129] Rotation angle of each joint (angle value in the range of 0~360°)

[0130] Joint displacement information (distance traveled by a joint undergoing linear motion)

[0131] 3D coordinates of the end effector (position of the end device within the workspace)

[0132] End effector direction vector (direction the terminal device is facing)

[0133]

[0134] The system (400) processes these data in real time to optimize the robot's movements. For example, in the task of picking up and moving an object, the system identifies the position and state of the object and analyzes the current state of the robot joints to calculate the most efficient path. At this time, by utilizing a previously learned policy, fast and accurate task execution is possible without additional learning.

[0135] All of these processes are carried out through a leading-time ensemble technique, which is a key feature of the system (400). Through this technique, the system can generate optimal operations in real time and continuously improve the speed of operations.

[0136]

[0137] According to one embodiment, the system (400) can improve the working speed of a robot without additional data collection or policy learning. This is a method that enables the robot to perform tasks more quickly and efficiently based on previously learned policies and collected data. For example, the system (400) enables the robot to quickly perform tasks such as picking up or moving objects by utilizing already learned behavioral patterns. This approach enables rapid decision-making without the need for data collection and retraining processes.

[0138] The surrounding environment data may include location and state information of the target object that the robot needs to manipulate.

[0139] Robot motion data is characterized by including information on the robot's joint angles and end-effector positions. Joint angles refer to the rotation angle or displacement of each joint, which is essential for the precise control of the robot's movements. For example, when a robot arm needs to rotate at a specific angle to grasp an object, the exact angle of each joint must be known. End-effector position information includes the 3D coordinates and orientation where the end-effector is located, enabling precise position adjustment when grasping or releasing an object. For instance, when a robot needs to pick up an object, the end-effector's position must match the object's position for successful manipulation.

[0140] According to one embodiment, work speed improvement can be achieved only through a leading temporal ensemble technique. This technique integrates multiple predicted future actions following a current action to enable the robot to prepare for the next action in advance. For example, by calculating the direction of the next movement in advance while the robot picks up an object, it enables the two tasks to be performed sequentially. This maximizes the robot's work speed and increases work efficiency.

[0141] According to one embodiment, the system (400) can improve the working speed of the robot without collecting new data or relearning the policy after policy learning. This means the ability to respond to immediate changes in the situation by utilizing previously learned information. For example, when the robot needs to manipulate a new object in an environment it has previously learned, it can perform the task quickly based on existing data without an additional learning process.

[0142]

[0143] A robot's joint angle refers to the rotation angle or displacement of each joint. This serves as a crucial factor in precisely controlling the robot's motion, and the robot's overall movement is determined by the rotation angle of each joint. The position information of an end effector includes the 3D coordinates and orientation where the end effector is located, providing the information necessary to accurately grasp or release objects. This information is essential when a robot performs complex tasks and contributes to increasing the accuracy of the robot's movements.

[0144]

[0145]

[0146] According to one embodiment, the system (400) may receive ambient environment data and robot movement data collected through the demonstrator's teleoperation. The ambient environment data may be obtained through one or more of an RGB camera, a depth camera, a LiDAR, an ultrasonic sensor, and a contact sensor.

[0147] It receives surrounding environment data and robot movement data collected through the demonstrator's teleoperation. Teleoperation is a technology that allows the demonstrator to remotely control the robot. The demonstrator uses teleoperation equipment to control the robot's movements and perceives the surrounding environment through the robot's sensor data.

[0148] Environment data is acquired through one or more of RGB cameras, depth cameras, LiDAR, ultrasonic sensors, and contact sensors. RGB cameras acquire images containing color information, similar to the human eye. Depth cameras acquire depth information for each pixel of an image. LiDAR uses lasers to generate a 3D map of the surrounding environment. Ultrasonic sensors use ultrasound to measure the distance to obstacles. Contact sensors detect whether the robot has come into contact with an object. Environment data includes one or more of the following: the 3D coordinates, orientation, size, color expressed as RGB values, shape expressed as polygons or curves, material information, and weight of the target object. For example, when a robot performs the task of grasping a cup on a table, it acquires information such as the 3D coordinates of the cup, the orientation of the cup, the size of the cup, the color of the cup, the shape of the cup, the material of the cup (glass, plastic, etc.), and the weight of the cup.

[0149] Robot motion data is acquired through one or more of a robot controller, an encoder, and an inertial measurement unit, and may include one or more of the rotation angle of each joint, angular velocity, torque, three-dimensional coordinates of the end effector, direction of the end effector expressed as an Euler angle or quaternion, velocity of the end effector, and acceleration.

[0150] A robot controller is a device that controls the movement of a robot by transmitting commands to each joint of the robot. An encoder is a sensor that measures the rotation angle of a robot joint. An inertial measurement unit is a sensor that measures the acceleration and rotational speed of a robot. Robot motion data includes one or more of the following: the rotation angle of each joint, angular velocity, torque, the 3D coordinates of the end effector, the direction of the end effector expressed as Euler angles or quaternions, and the velocity and acceleration of the end effector. For example, while a robot arm is moving, information such as the rotation angle of each joint, angular velocity, torque, the 3D coordinates of the end effector, the direction of the end effector, and the velocity and acceleration of the end effector is acquired.

[0151] According to one embodiment, the system (400) can learn a robot policy through an imitation learning algorithm using collected data. The imitation learning algorithm includes one or more of a CVAE-based ACT algorithm and a recurrent neural network, and the CVAE can learn the conditional probability distribution of input data to generate various action sequences according to given conditions.

[0152] The robot's policy is learned using an imitation learning algorithm based on collected data. Imitation learning is an algorithm that learns how to perform tasks by mimicking the behavior of experts. The imitation learning algorithm includes one or more of the CVAE-based ACT algorithm and recurrent neural networks. As previously explained, the CVAE-based ACT algorithm is an algorithm that learns policies to generate efficient action sequences from a diverse and long-term perspective. Recurrent neural networks are algorithms suitable for learning data changes over time and can be used to learn the robot's continuous movements. CVAE can generate various action sequences based on given conditions by learning the conditional probability distribution of input data.

[0153] In this way, the system (400) collects surrounding environment data and robot movement data through various sensors and learns the robot's policy through an imitation learning algorithm. The learned policy guides the robot to perform a given task efficiently.

[0154]

[0155] The system (400) collects information about the surrounding environment and the robot's movement through various sensors while a demonstrator, who is a robot operation expert, directly remotely controls the robot. For example, when demonstrating the task of picking up and moving an object, an RGB camera can collect images. A depth camera and LiDAR can collect information on the distance to the object and 3D position information, an ultrasonic sensor can detect proximity, and a contact sensor can determine whether the object is in contact. In this case, if a red apple is being picked up, information such as the apple's position coordinates (x = 0.5m, y = 0.3m, z = 0.8m), direction (roll = 0°, pitch = 0°, yaw = 45°), spherical shape with a diameter of 10cm, red color with RGB values ​​(255,0,0), smooth surface texture, and weight of about 200g can be collected.

[0156] Simultaneously, the robot's movements are recorded in detail; the robot controller measures how much each joint motor has rotated, encoders determine precise joint angles, and the Inertial Measurement Unit (IMU) measures the robot's overall posture and movement. For example, when a 6-axis robot arm picks up an apple, the data includes a 30° rotation of the shoulder joint, a 45° rotation of the elbow joint, a wrist rotation speed of 0.5 rad / s, the 3D coordinates of the endpiece (gripper) (x = 0.4m, y = 0.2m, z = 0.7m), a quaternion value indicating the gripper's orientation (w = 0.707, x = 0, y = 0, z = 0.707), an approach speed of 0.1 m / s, and an acceleration of -0.05 m / s for deceleration. 2 Detailed operation data of the back is recorded.

[0157] The system (400) utilizes the vast amount of data collected in this way to train the robot so that it can mimic the movements of a demonstrator. The main learning method uses the Adversarial Curiosity Transformer (ACT) algorithm based on Conditional Variational AutoEncoder (CVAE), which can generate various robot movements depending on the input environmental conditions. Additionally, it learns the patterns of time-series data using a recurrent neural network (RNN) and finds the optimal movements using reinforcement learning. For example, in the task of picking up an apple, CVAE can generate various approach paths and gripper angles that can stably grasp the apple when given conditions such as the position and size of the apple.

[0158]

[0159]

[0160] According to one embodiment, the system (400) uses a Transformer model to divide a sequence of actions into chunks and sequentially predicts each chunk to establish a long-term action plan, and the recurrent neural network can learn the sequence of actions of a robot by processing sequential data.

[0161] The system (400) uses a Transformer model to divide a sequence of actions into chunks and predicts each chunk sequentially to establish a long-term action plan. The Transformer model is a deep learning model that exhibits excellent performance in the field of natural language processing. By applying this model to robot control, a long sequence of actions is divided into meaningful units, and actions are predicted for each unit to establish a more efficient action plan. The Transformer model can perform the role of a generative artificial intelligence (AI) that generates robot actions when an image is input. For example, a generative AI model (e.g., GPT) outputs a sentence or an image when a sentence is input, and in this algorithm, the Transformer can similarly have a configuration that outputs a robot action when an image is input.

[0162] Recurrent Neural Networks (RNNs) learn sequences of robot actions by processing sequential data. RNNs are suitable for learning continuous robot movements because they possess the characteristic of remembering information from previous inputs. For example, when a robotic arm learns to draw, the RNN remembers the previous brush stroke to predict the next movement.

[0163] According to one embodiment, the system (400) can generate a control input for a robot by ensembling actions after the current point in time in a future action sequence predicted from a policy in a leading time. The leading time ensemble is performed using one or more methods among an average ensemble, a weighted average ensemble, a mode ensemble, a median ensemble, and a variance-based ensemble, and the weights in the weighted average ensemble may be determined based on the importance and probability of future actions.

[0164] According to one embodiment, the system (400) generates a control input for the robot by ensembling actions after the current point in time in a future action sequence predicted from a policy in a leading time. The leading time ensemble is performed using one or more methods among average ensemble, weighted average ensemble, mode ensemble, median ensemble, and variance-based ensemble. The average ensemble calculates the average value of the predicted future actions and reflects it in the control input. The weighted average ensemble calculates the average value by assigning weights based on the importance and probability of the future actions. For example, when the robot performs a motion of grasping an object, a higher weight is assigned to the "motion of grasping an object" than to the "motion of extending fingers."

[0165] According to one embodiment, the mode ensemble incorporates the most frequently occurring value among the predicted future actions into the control input. The median ensemble incorporates the value located in the center when the predicted future actions are sorted into the control input. The variance-based ensemble incorporates the variance of the predicted future actions into the control input. Since high variance of the predicted actions indicates high uncertainty, a conservative control input is generated. The weights in the weighted average ensemble are determined based on the importance and probability of the future actions.

[0166] According to one embodiment, the system (400) controls a robot using a control input, and controls the robot to perform a preparatory action for the next action while the robot is performing a current action, and the preparatory action includes pre-positioning of the robot joints and pre-directioning of the end effector for the next action, thereby enabling the robot to perform the work at a faster speed than the demonstrator's work speed.

[0167] The system (400) controls the robot using control inputs, and controls the robot to perform preparatory actions for the next action while the robot is performing the current action. For example, when the robot performs the "task of grabbing and moving an object," it simultaneously performs a preparatory action of moving to a position for moving while the action of grabbing the object is being performed. The preparatory action includes pre-positioning of the robot joints and pre-direction adjustment of the end effector for the next action. Through this, the robot can be controlled to perform the task at a faster speed than the demonstrator's task speed. In this way, the system (400) predicts and controls the robot's actions through various methods, thereby enabling the efficient performance of complex tasks.

[0168] The system (400) integrates the following advanced artificial intelligence technologies. The system (400) uses a Transformer model to divide complex robot movements into smaller, manageable chunks. For example, the 'picking up an object' movement is subdivided into 'extending the arm' - 'spreading fingers' - 'grabbing the object' - 'lifting'. These divided chunks are predicted sequentially to establish an overall motion plan. A recurrent neural network learns this continuous motion data, and just as a person learns movements through repetitive learning, the robot also learns motion patterns. Through reinforcement learning, the robot performs movements in a real environment and finds the optimal method of movement through trial and error. For example, when grabbing an object, it learns the appropriate force strength or grabbing angle through interaction with the environment.

[0169] The system (400) combines future actions predicted through policies in chronological order to create input values ​​necessary for robot control. At this time, various ensemble methods are used, such as an average ensemble that simply averages multiple predicted values, a weighted average ensemble that assigns weights according to importance to each predicted value, a mode ensemble that selects the most frequently predicted value, a median ensemble that uses the median of the predicted values, and a variance-based ensemble that considers the variance of the predicted values. In particular, in the case of a weighted average ensemble, weights are applied differently depending on the importance or probability of occurrence of each future action.

[0170] The system (400) begins preparing for the next action in advance before the action currently being performed by the robot is completed. For example, when performing the task of picking up an object and placing it in another location, the joints begin to rotate toward the next location in advance while the object is being held. Additionally, the direction of the end effector is also adjusted in advance, such as by adjusting the wrist angle required to put the object down. Through this preemptive action preparation, the robot can perform tasks at a faster speed than a human.

[0171]

[0172] FIG. 6 is a flowchart illustrating a method for providing a leading temporal ensemble technique for improving robot task speed based on imitation learning according to one embodiment.

[0173] Although process steps, method steps, algorithms, etc. are described in a sequential order in the flowchart of FIG. 6, such processes, methods, and algorithms may be configured to operate in any suitable order. In other words, the steps of the processes, methods, and algorithms described in various embodiments of the present invention do not need to be performed in the order described in the present invention. Furthermore, even if some steps are described as being performed asynchronously, in other embodiments, such steps may be performed simultaneously. Also, the example of a process by the depiction in the drawings does not mean that the exampled process excludes other variations and modifications thereof, nor does it mean that any of the exampled process or any of its steps is essential to one or more of the various embodiments of the present invention, nor does it mean that the exampled process is desirable.

[0174] In operation 610, the system (e.g., the system (400) of FIG. 4) can collect work environment data acquired in real time through a plurality of sensors under the control of a processor (e.g., the processor (420) of FIG. 4). The system (400) collects work environment data acquired in real time through a plurality of sensors, wherein the work environment data may include three-dimensional object information collected from an RGB-D camera, LiDAR, and ultrasonic sensor, obstacle information collected from a depth camera and laser scanner, and environmental condition information collected from an illuminance sensor, a temperature sensor, and a humidity sensor. The system (400) can collect robot movement data from a robot controller, an encoder, an IMU, and a force / torque sensor through teleoperation by a demonstrator. The robot movement data may include joint angles, angular velocity, three-dimensional position coordinates of an end effector, direction vectors, velocity, acceleration, and torque values.

[0175] The system (400) utilizes various sensors to comprehensively assess the state of the workspace. An RGB-D camera recognizes the shape and distance of objects by providing pixel-wise depth values ​​along with color information, and a LiDAR collects wide-range, precise 3D scan data using a laser. An ultrasonic sensor detects nearby obstacles through the reflection of sound waves. A depth camera and a laser scanner provide location and shape information of fixed / moving obstacles within the workspace. Illumination, temperature, and humidity sensors monitor the physical conditions of the work environment to measure environmental variables that may affect robot operation. When a demonstrator remotely operates the robot, the robot controller records control commands for each joint, the encoder measures the amount of rotation of the joint, the IMU detects the direction and acceleration of each part of the robot, and the force / torque sensor detects physical interactions with the outside. Through this, complete state information regarding the robot's 6-degree-of-freedom movement can be obtained.

[0176] In operation 620, the system (400) can input collected work environment data and robot movement data into a deep reinforcement learning model. The system (400) can learn a policy for performing tasks by the robot by inputting the collected work environment data and robot movement data into a deep reinforcement learning model. The deep reinforcement learning model may include one or more of a soft actor-critic network, a proximity policy optimization network, an LSTM-based recurrent neural network, and a transformer network.

[0177] The system (400) uses collected multidimensional sensor data as input to a deep reinforcement learning model to learn a robot control policy. The soft actor-critic network enables stable learning in a continuous action space, and proximity policy optimization prevents abrupt policy changes, thereby ensuring safe learning. The LSTM-based recurrent neural network captures the temporal dependencies of time-series data, and the Transformer effectively learns long-term dependencies through a self-attention mechanism.

[0178] In operation 630, the system (400) can predict a future action sequence from the current state of the robot using a learned policy. The future action sequence can be represented as time series data of control variables including the angle, angular velocity, torque of the robot joints and the position, direction, and velocity of the end effector.

[0179] The "current state" of a robot can include various types of information. For example, the current angles of each joint of the robot arm, the position and orientation of the end effector at the end of the arm, and information about the surrounding environment (images obtained through a camera, distance information measured by a LiDAR sensor, etc.) can all be elements that constitute the current state.

[0180] The "future action sequence" predicted by the policy represents a series of actions that the robot will perform in the future, in chronological order. The future action sequence is expressed as time-series data of control variables, including the angles, angular velocities, and torques of the robot's joints, as well as the position, orientation, and velocity of the end effectors. In other words, it predicts the robot's movements as they change over time.

[0181] In operation 630, the system (400) analyzes the current state of the robot using the learned policy and predicts how the robot should move in the future (future action sequence) accordingly. This prediction result is used to generate control inputs for the robot in the next operation, 640.

[0182] In operation 640, the system (400) can generate a robot control input at the current time from a predicted future action sequence. The system (400) can generate a robot control input at the current time from a predicted future action sequence through an ensemble technique that considers temporal precedence. The ensemble technique may include calculating a time-weighted average, calculating a weighted average based on action importance, or selecting based on the mode.

[0183] The system (400) predicts robot behavior for a certain period of time based on the learned policy model and the current state of the robot and environment. This behavior sequence is represented as a time series of state variables that fully define the robot's posture and movement at each point in time. The system determines the optimal control input at the current point in time by considering the temporal causal relationships in the predicted behavior sequence. The calculation of a time-weighted average gives greater weight to behaviors in the near future, the behavior importance-based weighted average prioritizes behaviors important for task success, and the mode-based selection selects the most frequently predicted stable behavior.

[0184]

[0185] The system (400) can collect robot motion data from a robot controller, encoder, IMU (Inertial Measurement Unit), and force / torque sensor through teleoperation by a demonstrator. This data is necessary to precisely analyze and control the robot's motion. The robot motion data includes joint angles, angular velocity, three-dimensional position coordinates of end effectors, direction vectors, velocity, acceleration, and torque values. Joint angles represent the rotational state of each joint, and angular velocity represents the rotational speed of the joint. The three-dimensional position coordinates and direction vectors of end effectors clarify the robot's working position and direction, providing information necessary for grasping or moving objects. Velocity and acceleration represent the rate of change of the robot's motion, and torque values ​​represent the magnitude of the force required for the robot to move each joint.

[0186] The system (400) can input collected work environment data and robot movement data into a deep reinforcement learning model. This process plays an important role in learning policies for the robot's task performance. The deep reinforcement learning model may include one or more of a soft actor-critic network, a near policy optimization network, an LSTM-based recurrent neural network, and a transformer network. The soft actor-critic network learns policies and value functions simultaneously, which is advantageous for providing stable learning. The near policy optimization network helps to enable stable learning even when there are large changes in the policy. The LSTM-based recurrent neural network has strengths in processing time-series data and is useful for predicting future behavior by remembering past behavior patterns. The transformer network is suitable for processing complex sequence data because it can effectively model the relationships between data.

[0187] The system (400) can predict a future action sequence from the robot's current state using a learned policy. The future action sequence can be represented as time-series data of control variables including the angle, angular velocity, and torque of the robot joints, and the position, direction, and velocity of the end effector. This prediction helps to plan in advance all movements required for the robot to perform a specific task. For example, the angle and velocity of each joint are calculated in advance during the process of the robot picking up and moving an object, allowing the task to be performed smoothly while maintaining consistency between movements.

[0188] The system (400) can generate robot control inputs at the current time from a predicted future action sequence. This process is necessary to adjust the robot's movements in real time. The system (400) can generate robot control inputs at the current time from a predicted future action sequence through an ensemble technique that considers temporal precedence. The ensemble technique may include calculating an average value with time weights, calculating a weighted average based on action importance, or selecting based on the mode. Calculating an average value with time weights is a method of generating control inputs suitable for the current situation by giving higher weight to recent actions. Calculating a weighted average based on action importance is a method of evaluating the importance of each action and giving greater weight to more important actions. Selecting based on the mode is a method of determining the action most frequently selected among several predictions as the current control input, which can ensure stable operation. In this way, the system (400) can optimize the robot's movements and increase the efficiency of the operation.

[0189]

[0190]

[0191] According to one embodiment, the system (400) can optimize the generated control input using an information-theoretic model predictive control technique. Optimization can be performed to minimize a cost function that takes into account work time, energy efficiency, and safety.

[0192] The system (400) optimizes control inputs using the Information Theoretic Model Predictive Control (IT-MPC) technique. Here, control inputs refer to physical control values ​​such as the torque, speed, and acceleration of each joint of the robot, and various factors are considered during the optimization process. In terms of work time, the total task completion time is minimized; in terms of energy efficiency, power consumption and mechanical wear of the motor are minimized; and in terms of safety, sudden movements or the use of excessive force are limited. These factors are integrated into a mathematically defined cost function to derive an optimal control strategy. For example, during welding operations, the movement speed and orientation of the torch directly affect the welding quality, so these are reflected in the cost function to generate optimal movements.

[0193] The system (400) can control the robot using optimized control inputs. By comparing the cost of the control trajectory of the previous time step and the cost of the newly generated control trajectory, the more efficient trajectory can be selectively applied.

[0194] The system (400) can control the robot to perform tasks at a faster speed than the demonstrator's work speed. The system (400) can perform tasks including one or more of object manipulation, welding, painting, inspection, and assembly tasks, and can control the robot's movements by considering collision avoidance and safety constraints in real time when performing tasks.

[0195] The system (400) is optimized to perform tasks at a faster speed by utilizing the mechanical characteristics of the robot, while based on the movements of a human demonstrator. It can perform various industrial tasks, such as calculating the optimal approach angle and gripping force of the gripper in object manipulation tasks, controlling the welding speed and angle in welding tasks, and optimizing paint uniformity and coverage amount in painting tasks. Inspection tasks, it calculates the optimal position of the camera or sensor, and assembly tasks, it ensures accurate interlocking between parts. During all tasks, it monitors the surrounding environment in real time to avoid collisions with workers or obstacles, and continuously checks for safety constraints such as vibration or overload to maintain the stability of the robot system.

[0196] The system (400) can optimize control inputs generated using an information-theoretic model predictive control technique. This technique operates by predicting the behavior of the system and determining the optimal control input accordingly. The information-theoretic model predictive control predicts future behavior based on the robot's current state and environmental information, thereby helping to establish an efficient control strategy. The optimization process is performed to minimize a cost function that considers work time, energy efficiency, and safety. For example, when the robot performs the task of picking up and moving a specific object, the cost function evaluates the travel time, energy consumption, and the probability of collision to select the optimal path.

[0197] The system (400) can control the robot using optimized control inputs. In this process, the cost of the control trajectory from the previous time step and the cost of the newly generated control trajectory can be compared to selectively apply a more efficient trajectory. For example, when the robot picks up and moves an object, the path set in the previous step is compared with the new path to select a path that takes less time or uses less energy. Through this, the system (400) increases the work efficiency of the robot and enables it to perform tasks along the optimal path while reducing unnecessary movements.

[0198] The system (400) can control the robot to perform tasks at a faster speed than the demonstrator's work speed. This allows the robot to perceive the environment in real time and rapidly adjust its movements based on a predicted sequence of actions. The system (400) performs tasks including one or more of object manipulation, welding, painting, inspection, and assembly, each task having unique characteristics and requirements. For example, in object manipulation tasks, precise positioning is required as the robot picks up and moves an object, while in welding tasks, continuous movements are required. When performing these tasks, the system (400) controls the robot's movements by considering collision avoidance and safety constraints in real time. For example, when the robot works in a confined space, it adjusts its path in real time to prevent collisions with surrounding obstacles, thereby ensuring safe operation. Through this, the system (400) can simultaneously guarantee the robot's work speed and safety.

[0199]

[0200]

[0201] According to one embodiment, the system (400) can collect environmental data and robot control data obtained through a multi-sensor system. The environmental data includes object recognition information collected from an RGB-D camera, LiDAR, and an ultrasonic sensor, and the object recognition information includes position information, attitude information, shape information, and material information in a three-dimensional coordinate system, and the robot control data may include position and orientation information of a 6-degree-of-freedom end effector, joint angle information, and gripper opening / closing information.

[0202] According to one embodiment, the system (400) extracts work features from the collected data using a hierarchical deep neural network, wherein the hierarchical deep neural network is composed of a convolutional layer, a pooling layer, and a fully connected layer, and the extracted features may include geometric characteristics of an object, operation patterns, and work sequence information.

[0203] The hierarchical deep neural network consists of five convolutional layers, three pooling layers, and two fully connected layers. The convolutional layers use 3x3 filters to extract features stepwise, ranging from low-level features such as object boundaries, edges, and surface textures to high-level features such as shape and local relationships. The max pooling layers compress feature maps using a 2x2 window to reduce computational load and ensure positional invariance of features. The fully connected layers synthesize the features extracted through 1024 and 512 neurons to transform them into geometric characteristics such as the object's shape, size, and center of mass, motion patterns such as translation, rotation, and deformation, and task sequence information indicating the order and dependencies of operations.

[0204] According to one embodiment, the system (400) can determine the working speed of the robot through a working speed optimization module. The system (400) can determine the speed within a range of 1 to 3 times that of the demonstrator, taking into account the difficulty of the current work, precision requirements, and safety constraints.

[0205] According to one embodiment, the system (400) plans actions for N future work steps in advance through a predictive action generation module, wherein the predictive action generation module is implemented based on a transformer network and can predict detailed actions for future work by considering the work context up to that point.

[0206] The system (400) plans actions for N future work steps in advance through a predictive action generation module based on a transformer network. The transformer network is a deep learning model widely used in the field of natural language processing, and it demonstrates excellent performance in understanding the relationships between words by grasping the context of an entire sentence. The system (400) utilizes this transformer network to predict detailed actions for future work by considering the context of work up to that point. In other words, the robot does not simply react to the current state, but plans future actions by comprehensively considering the previous work process and the tasks to be performed in the future. Through this, the robot can perform tasks more efficiently and accurately.

[0207] According to one embodiment, the system (400) monitors the status of work execution through a real-time monitoring system, wherein the monitoring system evaluates work accuracy, safety indicators, and energy efficiency in real time and can perform immediate speed adjustment or work interruption if it deviates from a set threshold. Each module of the system operates with a control cycle of 10ms or less, all operations ensure real-time performance through CUDA acceleration, and all data generated during work execution is stored in a time-series database and can be utilized for future performance improvement.

[0208] The system (400) continuously monitors the status of work execution through a real-time monitoring system. The monitoring system evaluates various indicators in real time, such as work accuracy, safety indicators, and energy efficiency, and performs immediate speed adjustment or work stoppage if they exceed a set threshold. In other words, it detects problems that may occur while the robot is performing work in real time and adjusts the robot's movements as needed to ensure safe and efficient work execution.

[0209] The system (400) can collect environmental data and robot control data obtained through a multi-sensor system. In this process, the environmental data includes object recognition information collected from an RGB-D camera, LiDAR, and an ultrasonic sensor. The RGB-D camera provides color information and depth information simultaneously, and is used to precisely measure the position and shape of an object. LiDAR uses lasers to analyze the structure of a three-dimensional space, and the ultrasonic sensor helps detect the distance and presence of an object. The collected object recognition information includes position information, attitude information, shape information, and material information in a three-dimensional coordinate system, providing details of the environment necessary for the robot to perform tasks. For example, when the robot needs to pick up a specific object, this information is essential for identifying the exact position and shape of the object. Additionally, the robot control data includes position and orientation information of a 6-degrees-of-freedom end effector, joint angle information, and gripper opening / closing information. The 6-degrees-of-freedom end effector is designed to move in all directions necessary for the robot to grasp or manipulate an object, and this information is necessary to precisely adjust the robot's movements.

[0210] The system (400) extracts work features from collected data using a hierarchical deep neural network. This hierarchical deep neural network consists of a convolutional layer, a pooling layer, and a fully connected layer, each layer performing different levels of data extraction. The convolutional layer is effective for extracting features from input images, and the pooling layer increases computational efficiency by reducing the dimensionality of the data. The fully connected layer combines the finally extracted features to provide insights into the tasks that the robot must perform. The extracted features include geometric characteristics of objects, motion patterns, and work sequence information, providing information necessary for the robot to perform complex tasks. For example, when the robot performs an assembly task, this information is crucial for understanding the shape of each part and the assembly sequence.

[0211] The system (400) determines the working speed of the robot through a working speed optimization module. Considering the difficulty of the current task, precision requirements, and safety constraints, the system (400) can determine the speed within a range of 1 to 3 times that of the demonstrator. For example, when the robot performs a simple object manipulation task, it can be set to perform the task faster than the demonstrator, while conversely, in high-precision assembly tasks, the speed can be adjusted to simultaneously increase safety and accuracy. Through this optimization process, the system (400) supports the robot in performing tasks effectively in various work environments.

[0212] The system (400) generates robot movements through a reinforcement learning-based adaptive controller. This adaptive controller has an actor-critic network structure, which is a form of reinforcement learning used to learn optimal behaviors as the robot interacts with the environment. The actor-critic structure consists of two main components. The actor learns a policy function that determines the action to be performed in the current state, and the critic learns a state value function that evaluates the value of each state (i.e., predicts long-term rewards). By learning these two functions simultaneously, the system (400) can generate optimal control inputs. For example, when the robot performs a grasping motion, the actor determines the appropriate opening angle of the gripper, and the critic evaluates the probability of success of the motion and provides a reward, thereby allowing the robot to gradually learn how to grasp the object more efficiently.

[0213] The system (400) plans actions for N future work steps in advance through a predictive action generation module. This module is implemented based on a transformer network, and a transformer is a model with powerful capabilities for processing sequence data, which is widely used, particularly in the field of natural language processing. The system (400) can predict detailed actions for future work by considering the context of the work done so far. For example, if a robot performs an assembly task, it can plan in advance the actions required for the next step (e.g., moving a gripper in a specific direction or rotating a part) based on the state and position of the part currently being assembled. This allows the robot to perform tasks more smoothly and minimize the transition time between tasks.

[0214] The system (400) monitors the status of work execution through a real-time monitoring system. This monitoring system evaluates work accuracy, safety indicators, and energy efficiency in real time to continuously verify whether the robot's work meets set criteria. If it deviates from the set threshold, the system (400) can perform immediate speed adjustment or stop the work. For example, if parts are misaligned during the process of assembling an object by the robot, the system detects this and immediately reduces speed or stops the work to prevent an accident. Each module of the system operates with a control cycle of 10ms or less, which ensures a fast response speed. All operations are guaranteed to be real-time through CUDA acceleration, which maximizes computation speed by utilizing parallel processing technology. All data generated during work execution is stored in a time-series database, and this data can be used for future performance improvement. For example, error data from a specific task can be analyzed and used to improve the robot's control algorithm.

[0215]

[0216] FIG. 7 is a flowchart illustrating a method for providing a leading temporal ensemble technique for improving robot task speed based on imitation learning according to one embodiment.

[0217] Although process steps, method steps, algorithms, etc. are described in a sequential order in the flowchart of FIG. 7, such processes, methods, and algorithms may be configured to operate in any suitable order. In other words, the steps of the processes, methods, and algorithms described in various embodiments of the present invention do not need to be performed in the order described in the present invention. Furthermore, even if some steps are described as being performed asynchronously, in other embodiments, such steps may be performed simultaneously. Also, the example of a process by the depiction in the drawings does not mean that the exampled process excludes other variations and modifications therefrom, nor does it mean that any of the exampled process or any of its steps is essential to one or more of the various embodiments of the present invention, nor does it mean that the exampled process is desirable.

[0218] In operation 710, the system (e.g., the system (400) of FIG. 4) can perform preprocessing on the collected data under the control of a processor (e.g., the processor (420) of FIG. 4). The system (400) may include a data collection unit that collects ambient environment data and robot movement data through teleoperation of a demonstrator. The ambient environment data includes location, size, and shape information of static obstacles and location, velocity, direction of movement, size, and shape information of dynamic obstacles, and the robot movement data may include the robot's joint angle, velocity, acceleration, end effector position, force / torque information, and status information of the work tool. The system (400) collects ambient environment data by integrating data from an RGB-D camera, LiDAR, radar, ultrasonic sensor, and proximity sensor through a multi-sensor fusion technique using the data collection unit, and the collected data can perform Kalman filter-based noise removal and outlier removal preprocessing.

[0219] The system (400) first collects and processes all data generated in the robot work environment. At this time, the demonstrator directly controls the robot remotely to acquire environmental data and data related to the robot's movement. In the case of environmental data, for fixed obstacles, it includes information such as where the location is, how big it is, and what shape it has. For moving obstacles, it identifies not only the current location but also detailed information such as how fast it is moving, which direction it is going, and what its size and shape are. Regarding the movement of the robot itself, it collects all detailed information such as at what angle each joint is bent, the speed and acceleration when moving, the exact location of the robot arm end (end effector), the amount of force and rotational force applied to each part, and the current state of the tool used by the robot.

[0220] In operation 720, the system (400) can learn the long-term movement pattern of a dynamic obstacle. The system (400) may include a policy learning unit that learns the robot's policy through a hierarchical imitation learning algorithm using the preprocessed data. The policy may refer to a function that takes a current state and a target state as inputs and outputs an optimal future action sequence.

[0221] The system (400) applies multi-sensor fusion technology that utilizes various types of sensors simultaneously to collect such data. It obtains color information and depth information simultaneously with an RGB-D camera, performs precise distance measurement with a LiDAR, obtains speed information of moving objects with a radar, detects obstacles at close range with an ultrasonic sensor, and detects objects at very close range with a proximity sensor. The various data collected in this way undergoes a preprocessing process to filter out abnormal values.

[0222] The system (400) includes a dynamic obstacle learning module that performs online cooperative learning of the multi-scale movement distribution of dynamic obstacles, and the online cooperative learning can share movement data of dynamic obstacles observed by multiple robots in real time through an edge computing-based distributed learning structure. The dynamic obstacle learning module automatically optimizes model parameters through a Bayesian nonparametric method and can learn long-term movement patterns of dynamic obstacles through a deep recurrent neural network-based long-term prediction model.

[0223] The system (400) deeply analyzes and learns the patterns of moving obstacles, in particular. In this process, more accurate learning is possible by utilizing edge computing technology to share and integrate information observed by multiple robots in real time. The patterns of moving obstacles can be represented by a multi-layered Gaussian distribution that considers both time and space. Additionally, a deep recurrent neural network is used to perform long-term predictions on how the obstacles will move in the future. Based on all this data and learning results, the system (400) learns a policy that determines how the robot should act through a hierarchical imitation learning algorithm. This policy serves as a guideline that determines the order and manner in which the robot should act in the future, taking into account the robot's current situation and the goal it intends to achieve.

[0224] The system (400) can adaptively adjust the speed and acceleration of the robot according to the work situation. The system (400) includes a model predictive control module that constitutes a distributed motion control model based on adaptive model predictive control, defines a multi-objective function including a term for minimizing work completion time, an energy efficiency optimization term, and a term for maximizing safety margin, defines extended constraints including dynamic constraints, workspace constraints, speed constraints, acceleration constraints, jerk constraints, and torque constraints of the robot, and can apply a robust control strategy that considers model uncertainty. The robust control strategy ensures that stable control performance is maintained even if prediction errors occur by considering model uncertainty. Model uncertainty can occur due to differences between the actual system and the model, sensor measurement errors, changes in the external environment, etc. By generating control inputs considering this uncertainty, the robust control strategy ensures that the robot operates stably even if prediction errors occur.

[0225] The system (400) adjusts the speed and acceleration of the robot in real time according to the work situation.

[0226] Constraints considered during control include the dynamic limits of the robot joints, working range, maximum and minimum velocities, acceleration limits, jerk (rate of change of acceleration) limits, and the maximum torque that each joint can generate. In addition, robust control strategies are applied to prepare for model uncertainty. For example, if the robot's end section deviates from the expected path, real-time correction is performed to ensure stable task execution.

[0227] The system (400) includes a control input generation unit that generates optimal control inputs for a robot by applying a time-frequency domain ensemble technique to a future action sequence predicted from a policy, and may include an adaptive robot control unit that adaptively adjusts the speed and acceleration of the robot according to the work situation based on the generated control inputs. The time-frequency domain ensemble technique is a technique that improves control performance by combining information in the time domain and the frequency domain. In the time domain, changes in the system over time are analyzed, and in the frequency domain, the frequency characteristics of the system are analyzed. By combining these two pieces of information, the dynamic characteristics of the system can be identified more accurately, and effective control inputs can be generated.

[0228] The system (400) performs ensemble analysis in the time-frequency domain on predicted future actions to generate optimal control inputs. This causes the robot's various joints to move in harmony with one another, much like how various instruments in an orchestra play together in harmony. The generated control signals are adjusted and applied in real time according to the work situation.

[0229] The system (400) can implement a hierarchical safety guarantee mechanism for obstacle avoidance. The system (400) implements a hierarchical safety guarantee mechanism for obstacle avoidance, but sets a conservative safety area by applying the extended convex set separation theorem for static obstacles, applies a risk-based avoidance strategy considering probabilistic uncertainty for dynamic obstacles, and applies a priority-based dynamic space allocation protocol in a multi-robot environment. The method of using a conservative safety area is to secure a safety margin by setting a safety area that is wider than the actual obstacle. For example, if the size of the actual obstacle is 1m x 1m, the safety area is set to 1.5m x 1.5m to reduce the risk of the robot colliding with the obstacle.

[0230] To avoid obstacles, a three-stage hierarchical safety mechanism is implemented. First, for fixed obstacles, the extended convex set separation theorem is applied to establish a safe zone with sufficient margin. For example, a virtual safety boundary is formed around fixed obstacles such as workplace pillars or walls. Second, for moving obstacles, a probabilistic avoidance strategy is used that accounts for the uncertainty of their movement. This involves predicting the worker's movement patterns when they are working near the robot to maintain a sufficient safety distance. Third, in environments where multiple robots work simultaneously, workspaces are dynamically allocated based on priority.

[0231] In operation 730, the system (400) can generate an optimal speed profile for each task by analyzing the demonstrator's work pattern. The system (400) generates an optimal speed profile for each task by analyzing the demonstrator's work pattern and dynamically adjusts the speed according to the real-time work situation, but can control the task to be performed at a speed faster than the demonstrator's work speed within a range where safety is guaranteed. The system (400) monitors the status of the system in real time, executes a safe recovery strategy in the event of an exception, and can fine-tune the policy online by providing feedback on errors that occur during the execution of the task.

[0232] The system (400) learns human work patterns and creates an optimal speed profile for each task. Within a range where safety is guaranteed, it can perform tasks at a faster speed than the demonstrator. For example, in assembly work, the action of picking up and moving parts is performed quickly, while the action requiring precise fitting is performed slowly. The system continuously monitors the work status and, if a problem occurs, immediately switches to safety mode and executes a recovery strategy. It also improves control policies in real time by utilizing error information generated during the work.

[0233] The system (400) performs preprocessing on collected data under the control of the processor (420). This preprocessing process is essential to improve the quality of the data and to ensure better performance in subsequent analysis and learning stages. The system (400) includes a data collection unit that collects ambient environment data and robot movement data through the teleoperation of a demonstrator. The ambient environment data includes location, size, and shape information of static obstacles and location, speed, direction of movement, size, and shape information of dynamic obstacles. For example, in an environment where a robot works, a fixed wall or table is considered a static obstacle, and a person or other robot is considered a dynamic obstacle. The robot movement data includes the robot's joint angle, speed, acceleration, end effector position, force / torque information, and status information of the work tool. This data is necessary to precisely control the robot's movements and increase work efficiency. The system (400) collects ambient environment data by integrating data from an RGB-D camera, LiDAR, radar, ultrasonic sensor, and proximity sensor through a multi-sensor fusion technique using the data collection unit. In this process, data from various sensors can be combined to obtain more accurate and complete environmental information. The collected data undergoes Kalman filter-based noise and outlier removal preprocessing to enhance data reliability. The Kalman filter is an algorithm that estimates the state of a dynamic system changing over time and is effective in reducing noise in sensor data.

[0234] The system (400) can learn the long-term movement patterns of dynamic obstacles. To this end, the system (400) includes a policy learning unit that learns the robot's policy through a hierarchical imitation learning algorithm using preprocessed data. A policy refers to a function that takes a current state and a target state as inputs and outputs an optimal future action sequence, which plays an important role in determining the actions the robot must perform in the future. For example, in the process of the robot picking up and moving an object, the most efficient path is calculated by considering the current position and the target position.

[0235] The system (400) includes a dynamic obstacle learning module that uses a hierarchical Dirichlet process blending model to learn the multi-scale movement distribution of dynamic obstacles online collaboratively. This model helps to effectively learn and predict various movement patterns of dynamic obstacles. Online collaborative learning shares movement data of dynamic obstacles observed by multiple robots in real time through an edge computing-based distributed learning structure. By doing so, information collected by each robot contributes to the learning of other robots, thereby improving the performance of the entire system. The hierarchical Dirichlet process blending model can be expressed as a blend of multi-level Gaussian distributions that consider spatiotemporal characteristics, which is useful for more precisely modeling the movement patterns of dynamic obstacles. The dynamic obstacle learning module automatically optimizes model parameters through Bayesian nonparametric methods, which enables rapid adaptation to changes in data. Long-term movement patterns of dynamic obstacles can be learned through a deep recurrent neural network-based long-term prediction model, which enables the robot to make more accurate predictions about approaching obstacles. For example, applications such as predicting the movement path of a person the robot encounters during work and adjusting the path to allow the work to continue safely are possible.

[0236] The system (400) can adaptively adjust the speed and acceleration of the robot according to the work situation. To this end, the system (400) includes a model predictive control module that constitutes a distributed motion control model based on adaptive model predictive control. This module helps determine the optimal speed and acceleration by reflecting the requirements of the task the robot must perform. The multi-objective function includes three goals: minimizing the task completion time, optimizing energy efficiency, and maximizing the safety margin, and ensures that the optimal result is derived by considering each goal simultaneously. For example, when the robot performs a task of assembling an object, the speed and acceleration are adjusted in a direction that minimizes the task completion time while saving energy and increasing safety. The system (400) defines extended constraints including dynamic constraints, workspace constraints, speed constraints, acceleration constraints, jerk constraints, and torque constraints of the robot, thereby enabling the robot to perform optimal movements within a physically possible range. The jerk constraint limits the rate of change of the robot's acceleration, contributing to maintaining smooth movements. Additionally, the system (400) applies a robust control strategy that considers model uncertainty, enabling stable operation even in unexpected situations. For example, when a robot works on an unstable surface, it can adjust its movements to reflect this uncertainty.

[0237] The system (400) includes a control input generation unit that generates optimal control inputs for a robot by applying a time-frequency domain ensemble technique to a future behavior sequence predicted from a policy. This ensemble technique is a method for determining optimal control inputs by synthesizing various predicted behaviors. It may include an adaptive robot control unit that adaptively adjusts the speed and acceleration of the robot according to the work situation based on the generated control inputs. For example, when the robot works in a complex environment, it adjusts the speed and acceleration in real time to match changes in the surroundings, thereby enabling the robot to perform tasks safely and efficiently.

[0238] The system (400) can implement a hierarchical safety guarantee mechanism for obstacle avoidance. This mechanism establishes a conservative safety zone by applying the extended convex set separation theorem to static obstacles. Through this, the robot secures a safety zone to avoid collisions with fixed obstacles. For dynamic obstacles, a risk-based avoidance strategy considering probabilistic uncertainty is applied so that the robot can recognize surrounding moving objects and adjust its avoidance behavior accordingly. In a multi-robot environment, a priority-based dynamic space allocation protocol is applied to prevent collisions and efficiently utilize space when multiple robots work simultaneously. For example, when two robots perform different tasks, the system (400) maintains a safe working environment by adjusting the path according to the task priority of each robot.

[0239] The system (400) can generate an optimal speed profile for each task by analyzing the demonstrator's work pattern. In this process, the system (400) analyzes the demonstrator's work speed, method of operation, and efficiency to define a speed profile suitable for each task. The speed is dynamically adjusted according to real-time work conditions, but the system can control the work to be performed at a speed faster than the demonstrator's work speed within a range where safety is guaranteed. For example, when the robot performs an assembly task, it is enabled to perform the task quickly within a safe range even if it exceeds the demonstrator's speed. The system (400) monitors the status of the system in real time and executes a safe recovery strategy in the event of an exception. Errors occurring during task execution can be fed back to fine-tune the policy online, thereby continuously improving the robot's performance. For example, if the robot causes repetitive errors in a specific task, the system (400) analyzes this to adjust the policy and reduce errors in future tasks.

[0240]

[0241] FIG. 8 illustrates the prediction and ensemble process of robot motion over time according to one embodiment.

[0242] This invention describes a leading temporal ensemble technique for improving the speed of motion generated in a robot's imitation learning algorithm. The leading temporal ensemble technique presented in this invention determines the robot's input by weighted averaging control inputs from future points in time that precede the current point in time, thereby significantly improving the robot's work speed.

[0243] The vertical axis represents the time from a past point in time to the present point in time, ranging from -5 to the present point in time t, and the horizontal axis represents the predicted time at each point in time, including the interval from -5 to +4.

[0244] In the drawing, each block represents an individual movement of the robot predicted at that point in time, and the blocks are arranged diagonally according to the flow of time. Five consecutive movements are predicted at one point in time, which are represented by five consecutive blocks arranged horizontally in the drawing.

[0245] The blocks (α, β, γ, δ, ε) shown in different colors in the diagram represent the prediction actions used in the ensemble construction at the current time t.

[0246] Specifically, the blocks marked α represent the current actions that have been predicted from a past point in time to the present.

[0247] The blocks labeled β represent the predicted actions for time t+1.

[0248] The blocks marked with γ represent the predicted actions at time t+2.

[0249] The blocks marked with δ represent the predicted actions at time t+3.

[0250] The block marked with ε represents the prediction operation at time t+4.

[0251] When applying the pre-temporal ensemble technique, different sets of blocks are used depending on the value of the pre-temporal variable f. For example, when f=2, the blocks labeled γ become the main components of the ensemble, representing prediction actions at a time point two steps ahead of the current time point.

[0252] Through these visual representations, one can intuitively understand the motion prediction and ensemble processes that take place at each point in time, and clearly grasp how the prior temporal ensemble technique utilizes future predicted motions to improve the motion speed of the robot.

[0253] Specifically, multiple actions are predicted at each control step. For example, assuming that 5 actions are predicted at one step, the actions predicted during the last 6 steps are stored as blocks in chronological order. At this time, among the actions predicted from the past to the present, the actions actually used at the current point in time are defined as 'α' blocks, which become elements constituting the existing ensemble matrix Bt.

[0254] The variable action chunking technique proposed in the present invention uses a future ensemble matrix Bt+f instead of the existing ensemble matrix Bt when ensembling robot movements. Here, f is a leading time variable that has a value smaller than the total number of predicted actions ka. For example, if f is 2, Bt+2 consists of actions 2 steps after the current time, which are indicated by 'γ' blocks in the figure.

[0255] The prior time ensemble technique of the present invention sets a future action point as a target point for the robot, thereby enabling the robot to operate at a faster speed to track the target point. However, in contact tasks where objects must be manipulated precisely, such rapid operation may cause problems that impair accuracy.

[0256] To solve these problems, the present invention proposes a method for dynamically adjusting a lead time variable f according to the characteristics of the task. Specifically, for contact tasks requiring precise manipulation, a low value of the lead variable f is applied to ensure accuracy, while for non-contact movements such as free movement in the air, a high value of the lead variable f is applied to maximize the speed of the operation. Through such a variable lead time ensemble technique, the task speed of a robot generated by imitation learning can be optimized.

[0257]

[0258]

[0259] Figure 9 shows the overall configuration of a dual robot system for block color classification tasks.

[0260] The workspace is organized around a white table, and two high-precision 6-degree-of-freedom robot manipulators (Manipulator R, Manipulator L) are installed symmetrically on the top of the table. A 1-degree-of-freedom gripper (Gripper R, Gripper L) for precise gripping movements is mounted at the end of each manipulator. The grippers are designed to stably grip blocks of various sizes and shapes, and the maximum opening and closing width is set to approximately 1.5 times the size of the block.

[0261] Orange (Block R) and sky blue (Block L) blocks, which are the subjects of classification, are randomly placed in the central area of ​​the table. Each block has the same size (40mm x 40mm x 40mm) and shape, and its surface is matte-finished to facilitate camera recognition. The placement of the blocks is random for each task, reflecting the uncertainty of the actual industrial environment.

[0262] Boxes for storing the sorted blocks are located at both ends of the table. A yellow box (Box R) for orange blocks is placed on the right, and a gray box (Box L) for sky-blue blocks is placed on the left. Each box has a grid structure, allowing the internal stacking status to be easily checked from the outside.

[0263] A stereo camera is installed on a vertical support in the center of the table to monitor the workspace. This camera provides a global view of the entire work area and captures stereo images with a resolution of 640x480 in real time. The area marked with a red dotted line indicates the main work area of ​​each robot, which is set up to prevent collisions between robots and for efficient work distribution.

[0264] This experimental environment is configured to simulate object classification tasks in actual industrial settings while enabling systematic data collection for robot learning and performance evaluation. In particular, through the collaboration of a dual-robot system, classification tasks in large work areas that are difficult to perform with a single robot can be efficiently carried out.

[0265] This system is designed for automating object classification in industrial settings and performs the task of classifying blocks of different colors using two 6-degree-of-freedom robotic arms and 1-degree-of-freedom grippers mounted on each.

[0266] The work environment consists of orange and blue blocks randomly placed on a table; these blocks have the same size and shape but differ only in color. Storage boxes for sorting are located at both ends of the table and are placed freely without separate fixing devices. This setup takes into account the various work environments that may occur in actual industrial settings.

[0267] The master control device, specially designed for data collection, is designed at a 2:1 scale of the actual robot arm and can control a total of 7 degrees of freedom, including 6 degrees of freedom of the robot arm and 1 degree of freedom of the gripper. The control data consists of position (p), orientation (o), and gripper grip degree (θ), which enables precise manipulation in three-dimensional space.

[0268] Three stereo cameras were strategically positioned for environmental perception. Two are mounted on the wrists of each robot to secure a close-up view of the workspace, while the remaining one is fixedly installed in a third position to provide a view of the entire workspace. Each camera provides images with a resolution of 640x480, generating a total of six high-quality image streams.

[0269] When performing a task, if collaboration between robots is required, a block transfer process is included. For example, if the right robot (R) picks up a blue block but needs to place it in a blue box located on the left, the right robot transfers the block to the left robot (L), and the left robot ultimately places the block in the box.

[0270] To ensure data diversity, experiments were conducted under various environmental conditions, including different lighting conditions and changes in box placement. Through a total of 1,000 demonstrations—250 for each environment—20 data points were collected per second, ultimately constructing a massive dataset of 1.1 million data points.

[0271] The neural network structure for policy learning adopts a transformer-based encoder-decoder structure. Input images are processed by integrating stereo pairs at a size of 1280x480, and the robot's state information includes position (p), direction vector (i,j,k), and gripper state. The direction information is represented by each column vector of the rotation matrix to precisely represent the 3D direction of the robot end.

[0272] Training was conducted using distributed training with 8 GTX4090 GPU servers and optimized using the NVIDIA Apex library. Approximately 100,000 steps were trained over one week with a batch size of 256, and parameters were updated using the AdamW optimizer. The policy network receives the two most recent state information as input to predict 24 consecutive future movements, and the robot's control cycle is set to 20Hz to implement smooth movements.

[0273]

[0274] Figure 10 illustrates an experimental scene showing the continuous process of block classification using a priori temporal ensemble algorithm in chronological order.

[0275] It consists of a total of 6 frames, and each frame represents the operating state of the robot at a specific time (t).

[0276] In the initial state (a) t=0, two collaborative robots share a workspace, with yellow and blue boxes placed on the left and right sides, respectively. Blocks of various colors (green, purple, sky blue, and light green), which are the work objects, are located in the center. This shows the beginning stage of the classification task.

[0277] (b) At time t=8, the robot on the left is performing the action of grasping the first block. At this time, the robot accurately recognizes the position and color of the block through the stereo camera system and determines the optimal grasping posture.

[0278] (c) At t=17, the process of moving the first block to the box of the corresponding color is shown, and (d) at t=22, the sorting of the second block is in progress. In this process, the robot plans the next action in advance through a precedence-temporal ensemble algorithm to increase work efficiency.

[0279] (e) At time t=34, the classification of the remaining blocks is performed, and (f) at t=37, the point in time when the classification of all blocks is completed is shown. Since the entire task is completed in 37 seconds, the proposed algorithm demonstrates efficient task performance.

[0280] The robot movements observed in each frame are highly sophisticated, and it can be confirmed that efficient collaboration is taking place while avoiding collisions between the two robots. Furthermore, it demonstrates that accurate environmental perception through the stereo camera system enabled correct classification based on the color of each block.

[0281] These experimental results demonstrate that the pre-temporal ensemble algorithm operates stably in real-world environments and exhibits excellent performance in terms of task speed and accuracy. In particular, the completion of classification for all blocks within a relatively short time of 37 seconds shows that a significant performance improvement has been achieved compared to existing imitation learning methods.

[0282] In relation to the evaluation and analysis of prior temporal ensemble performance, experiments were conducted to verify the performance of an ACT (Action-Conditional Transformation)-based imitation learning algorithm. The experiments were performed under the condition f=0, divided into single block classification and four-block multi-classification. For the single block classification experiment, a total of 120 experiments were conducted by varying the position of the boxes, lighting conditions, and block colors to ensure diversity in the experimental environment, achieving a success rate of 83.33%.

[0283] Failure cases that occurred during the experiment are broadly classified into two types. The first is color classification error, with six instances where blocks were placed in boxes of the wrong color. The second is covariate bias, which occurred a total of 14 times. Covariate bias refers to errors that occur in a state space that was not sufficiently covered in the training data; for example, this includes phenomena such as two robot arms attempting to grasp the same block located in the center simultaneously and stopping to avoid a collision, or failing to return to their original state from a specific posture.

[0284] In the multi-block classification experiment, the task of classifying 4 blocks was performed 50 times under the same environmental conditions, and a success rate of 65% was recorded. Failure cases included 6 cases of color classification errors, 9 cases related to covariate bias, and 3 cases of blocks falling out during the block retention process.

[0285] To precisely analyze the performance of the precedence temporal ensemble algorithm, experiments were conducted to place a single block in a nearby box by applying various f values. The experiment was repeated 20 times for each f value to measure the average time taken and the success rate, with block grasping failure or box placement failure considered as failure cases. As a result of the experiment, it was confirmed that the work speed improved by up to three times while maintaining a success rate of 75%, and that a twofold increase in speed was possible without a decrease in the success rate.

[0286] To verify the scalability of the experiment, classification experiments were also conducted on various blocks not included in the training data (4 blue cells, 3 green cells, 2 purple cells, and light green blocks with different shapes). In this experiment, the blue and purple blocks were accurately classified as blue boxes, and the green and yellow blocks were accurately classified as yellow boxes, demonstrating the excellent generalization ability of the imitation learning algorithm.

[0287] A total of three stereo cameras were utilized for the system's environmental perception. The camera mounted on the wrist monitors the object gripping status of the gripper, while the camera installed on the head is used to recognize color information of the boxes on both sides. For example, there may be cases where it is difficult to distinguish whether the block is moving to the left or to the right, which can lead to a phenomenon where the two robots endlessly pass the block back and forth.

[0288] The prior-temporal ensemble algorithm proposed in this study has demonstrated that it can dramatically improve task speed in imitation learning-based robot control systems. In particular, it holds great significance in that performance improvement is possible without additional data collection or policy retraining. By achieving a three-fold speed improvement in block classification tasks using the CVAE-based ACT algorithm, the limitations of existing imitation learning-based policies, which were restricted by the speed of the demonstrator, have been overcome. This is expected to contribute significantly to productivity enhancement through autonomous object manipulation technology in the future.

Claims

1. In a system providing a precedent temporal ensemble technique for improving robot task speed based on imitation learning, Memory for storing instructions, and Includes a processor, When the above instructions are executed by the processor, the system It receives surrounding environment data and robot movement data collected through teleoperation, and Using the collected data above, the robot's policy is learned through an imitation learning algorithm, wherein the policy is a function that takes the current state as input and outputs a future action sequence, and Generates control inputs for the robot by ensembling actions after the current point in time in the future action sequence predicted from the above policy in a leading time, and A system for controlling a robot using the control input generated above.

2. In Paragraph 1, The above policy is characterized by being trained using the CVAE (Conditional Variational Autoencoder)-based ACT (Action Chunking with Transformer) algorithm, and A system characterized by the above-mentioned prior temporal ensemble integrating multiple future actions predicted after an action at a current point in time into a single control input, thereby enabling the robot to prepare for subsequent actions while performing the current action.

3. In Paragraph 2, The above CVAE is characterized by learning the conditional probability distribution of input data to enable the generation of various behavioral sequences according to given conditions, and The above ACT is characterized by using a Transformer model to divide a sequence of actions into chunks and sequentially predicting each chunk to enable the establishment of a long-term action plan. The above future behaviors are characterized by meaning behaviors after the current point in time in the behavior sequence predicted by the above policy, and The above integration is characterized by including calculating the average or weighted average of the above future actions, and A system characterized by the robot above performing the current action and the next action simultaneously using the integrated control input to improve work speed.

4. In Paragraph 1, When the above instructions are executed by the processor, the system Improving the work speed of the above robot without additional data collection or policy learning, and The above surrounding environment data Includes location and state information of the target object that the robot must manipulate, A system characterized in that the above robot movement data includes the joint angles of the robot and the position information of the end effector.

5. In Paragraph 4, The above work speed improvement is characterized by being achieved only through the above-mentioned preceding temporal ensemble technique, and The above system is characterized by being able to improve the working speed of the robot without collecting new data or relearning the policy after policy learning. The position information of the above-mentioned target object is characterized by being able to include the 3D coordinates, direction, size, etc. of the target object, and The state information of the above-mentioned object may include the color, shape, material, weight, etc. of the object, and is characterized by the fact that The joint angle of the above-mentioned robot is characterized by meaning the rotation angle or displacement of each joint, and A system characterized by the position information of the end effector including the three-dimensional coordinates and direction of the end effector.

6. In Paragraph 1, When the above instructions are executed by the processor, the system Receive surrounding environment data and robot movement data collected through the demonstrator's teleoperation, The above surrounding environment data is acquired through one or more of an RGB camera, a depth camera, LiDAR, an ultrasonic sensor, and a contact sensor, and includes one or more of the 3D coordinates, orientation, size, color expressed as RGB values, shape expressed as a polygon or curve, material information, and weight of the target object. The above robot motion data is acquired through one or more of a robot controller, an encoder, and an inertial measurement unit, and includes one or more of the rotation angle of each joint, angular velocity, torque, 3D coordinates of the end effector, the direction of the end effector expressed as Euler angles or quaternions, the velocity of the end effector, and acceleration. Using the collected data above, learn the robot's policy through an imitation learning algorithm, but The above imitation learning algorithm includes one or more of a CVAE-based ACT algorithm and a cycle, wherein the CVAE learns the conditional probability distribution of input data to generate various behavioral sequences according to given conditions, and The above ACT uses a Transformer model to divide the behavior sequence into chunks and sequentially predicts each chunk to establish a long-term action plan, and the above recurrent neural network processes sequential data to learn the robot's behavior sequence, and Control inputs for a robot are generated by ensembling actions after the current point in time in a future action sequence predicted from the above policy in a leading-time manner, wherein the leading-time ensemble is performed using one or more methods among mean ensemble, weighted mean ensemble, mode ensemble, median ensemble, and variance-based ensemble, and the weights in the weighted mean ensemble are determined based on the importance and probability of future actions. A system that controls a robot using the control input generated above, wherein the robot is controlled to perform a preparatory action for the next action while performing the current action, and wherein the preparatory action includes pre-positioning of the robot joints and pre-directioning of the end effector for the next action, thereby controlling the robot to perform the work at a speed faster than the demonstrator's work speed.

7. In Paragraph 1, When the above instructions are executed by the processor, the system Collecting work environment data acquired in real time through multiple sensors, wherein the work environment data includes 3D object information collected from an RGB-D camera, LiDAR, and ultrasonic sensor, obstacle information collected from a depth camera and laser scanner, and environmental condition information collected from an illuminance sensor, temperature sensor, and humidity sensor. Robot motion data is collected from a robot controller, encoder, IMU, and force / torque sensor through teleoperation by a demonstrator, wherein the robot motion data includes joint angles, angular velocity, 3D position coordinates of an end effector, direction vector, velocity, acceleration, and torque values. Collected work environment data and robot movement data are input into a deep reinforcement learning model to learn a policy for performing tasks of the robot, wherein the deep reinforcement learning model includes one or more of a soft actor-critic network, a proximity policy optimization network, an LSTM-based recurrent neural network, and a transformer network. Predict a future action sequence from the current state of the robot using a learned policy, wherein the future action sequence is expressed as time series data of control variables including the angle, angular velocity, torque of the robot joints and the position, direction, and velocity of the end effector. A system that generates robot control inputs at the current time point through an ensemble technique considering temporal precedence from a predicted future action sequence, wherein the ensemble technique includes calculating a time-weighted average, calculating a weighted average based on action importance, or selecting based on the mode.

8. In Paragraph 7, When the above instructions are executed by the processor, the system The control input generated using an information-theoretic model predictive control technique is optimized, wherein the optimization is performed to minimize a cost function considering operation time, energy efficiency, and safety. Control the robot using optimized control inputs, but selectively apply the more efficient trajectory by comparing the cost of the control trajectory from the previous time step with the cost of the newly generated control trajectory, and A system that controls the robot to perform tasks at a speed faster than the demonstrator's working speed, performs tasks including one or more of object manipulation, welding, painting, inspection, and assembly, and controls the robot's movements by considering collision avoidance and safety constraints in real time during task execution.

9. In Paragraph 1, When the above instructions are executed by the processor, the system Environmental data and robot control data acquired through a multi-sensor system are collected, wherein the environmental data includes object recognition information collected from an RGB-D camera, LiDAR, and ultrasonic sensor, and the object recognition information includes position information, attitude information, shape information, and material information in a 3D coordinate system, and the robot control data includes position and orientation information of a 6-DOF end effector, joint angle information, and gripper opening / closing information. Task features are extracted from the collected data using a hierarchical deep neural network, wherein the hierarchical deep neural network is composed of convolutional layers, pooling layers, and fully connected layers, and the extracted features include geometric characteristics of an object, motion patterns, and task sequence information. The working speed of the robot is determined through a working speed optimization module, wherein the working speed optimization module is implemented by combining a fuzzy logic controller and a genetic algorithm, and determines the speed within a range of 1 to 3 times that of the demonstrator by considering the difficulty of the current task, precision requirements, and safety constraints. A robot's motion is generated through a reinforcement learning-based adaptive controller, wherein the adaptive controller has an actor-critic network structure and simultaneously learns a state value function and a policy function to generate optimal control inputs, and Plan actions for N future work steps in advance through a predictive action generation module, wherein the predictive action generation module is implemented based on a transformer network and predicts detailed actions for future work by considering the work context up to the present, and The status of work execution is monitored through a real-time monitoring system, wherein the monitoring system evaluates work accuracy, safety indicators, and energy efficiency in real time, and performs immediate speed adjustment or work stoppage if they exceed a set threshold. A system characterized in that each module of the above system operates with a control cycle of 10ms or less, all operations ensure real-time performance through CUDA acceleration, and all data generated during operation is stored in a time-series database and utilized for future performance improvement.

10. In Paragraph 1, When the above instructions are executed by the processor, the system It includes a data collection unit that collects surrounding environment data and robot movement data through teleoperation by a demonstrator, wherein the surrounding environment data includes location, size, and shape information of static obstacles and location, velocity, direction of movement, size, and shape information of dynamic obstacles, and wherein the robot movement data includes the robot's joint angles, velocity, acceleration, end effector position, force / torque information, and status information of a work tool, and wherein the data collection unit collects surrounding environment data by integrating data from an RGB-D camera, LiDAR, radar, ultrasonic sensor, and proximity sensor through a multi-sensor fusion technique, and performs Kalman filter-based noise removal and outlier removal preprocessing on the collected data. It includes a policy learning unit that learns a robot policy through a hierarchical imitation learning algorithm using the above-mentioned preprocessed data, wherein the policy is a function that takes a current state and a target state as inputs and outputs an optimal future action sequence. It includes a dynamic obstacle learning module that performs online cooperative learning of multi-scale movement distributions of dynamic obstacles, wherein the online cooperative learning shares movement data of dynamic obstacles observed by multiple robots in real time through an edge computing-based distributed learning structure, wherein the hierarchical Dirichlet process mixture model is expressed as a mixture of multi-level Gaussian distributions considering spatiotemporal characteristics, and wherein the dynamic obstacle learning module automatically optimizes model parameters through a Bayesian nonparametric method and learns long-term movement patterns of dynamic obstacles through a deep recurrent neural network-based long-term prediction model. It includes a model predictive control module that constitutes a distributed motion control model based on adaptive model predictive control, defines multiple objective functions including a task completion time minimization term, an energy efficiency optimization term, and a safety margin maximization term, defines extended constraints including robot dynamic constraints, workspace constraints, velocity constraints, acceleration constraints, jerk constraints, and torque constraints, and applies a robust control strategy considering model uncertainty, It includes a control input generation unit that generates optimal control inputs for a robot by applying a time-frequency domain ensemble technique to a future action sequence predicted from the above policy, and an adaptive robot control unit that adaptively adjusts the speed and acceleration of the robot according to the work situation based on the generated control inputs. For static obstacles, a conservative safe region is established by applying the extended convex set separation theorem, and for dynamic obstacles, a risk-based avoidance strategy considering probabilistic uncertainty is applied, and in a multi-robot environment, a priority-based dynamic space allocation protocol is applied, It analyzes the demonstrator's work patterns to generate an optimal speed profile for each task, dynamically adjusts the speed according to real-time work conditions, and controls the task to be performed at a speed faster than the demonstrator's work speed within a range where safety is guaranteed, and A system that monitors the status of the system in real time, executes safe recovery strategies in the event of exceptions, and fine-tunes policies online by providing feedback on errors that occur during operation.

Images