Generate digital twins of the interactions between parts of a physical system

Abstract

A method includes receiving, via a first component in a production environment, sensor measurements corresponding to a second component in a production environment. A first digital twin corresponding to the first component is identified, and a perception algorithm is applied to identify a component type associated with the second component. A second digital twin is selected based on the component type, and a third digital twin is selected that models an interaction between the first digital twin and the second digital twin. The third digital twin is used to generate instructions for the first component that allow the first component to interact with the second component. The instructions can then be sent to the first component.

Description

Technical Field

[0001] The present disclosure relates generally to methods, systems and devices for creating and using digital twins to model interactions between system components. For example, the disclosed technology can be applied to manage interactions in automatic or semi-automatic systems such as factories or self-driving vehicles. Background Art

[0002] For complex physical systems, it is important to understand how different components interact to accomplish a task, especially in unknown situations. Most physical systems, such as robots, are designed to accomplish a predetermined task. As robots become more powerful and intelligent, they can also operate in unknown situations. For example, a person may try to lift an object that has never been seen before. He or she may first use one hand to lift the object. When the object is too heavy, both hands are used, or even the shoulder, waist, or legs are used. The machine has no perception. It has to evaluate the new situation by understanding and then simulating it, and plan the interactions between its parts, much like a person would.

[0003] For individual components, digital twins provide a way to understand how real-world components react under different scenarios. In short, a digital twin is a digital version of a machine. Once created, the digital twin can be used to represent the machine in a digital representation of the real-world system. The digital twin is created to be identical in form and behavior to the corresponding machine. Additionally, in larger systems, the digital twin can reflect the state of the machine. For example, sensors can be placed on the machine to collect real-time (or near real-time) data from the physical object to feed it back to the remote digital twin. The digital twin can then make any necessary changes to maintain its correspondence with the physical twin.

[0004] While digital twins provide a wealth of information about individual devices, there is currently no way to view, manage, and use data related to interactions between devices such as the tasks described above. Summary of the invention

[0005] Embodiments of the present invention address and overcome one or more of the above-mentioned shortcomings and deficiencies by providing methods, systems and devices related to creating and using digital twins to model interactions between system components.

[0006] According to some embodiments, a method includes receiving sensor measurements corresponding to a second component in a production environment via a first component in a production environment. A first digital twin corresponding to the first component is identified, and a perception algorithm is applied to identify a component type associated with the second component. A second digital twin is selected based on the component type, and a third digital twin is selected to model the interaction between the first digital twin and the second digital twin. The third digital twin is used to generate an instruction to the first component that allows the first component to interact with the second component. The instruction can then be sent to the first component.

[0007] According to other embodiments, a system includes three digital twins. A first digital twin corresponds to a first component in a production environment, and a second digital twin corresponds to a second component in the production environment. A third digital twin uses the first digital twin and the second digital twin to model an interaction between the first component and the second component.

[0008] According to other embodiments, a system for modeling the interaction between a first component and a second component in a production environment includes: a perception module, a digital twin selection module, an interactive digital twin, and an optimization module. The perception module receives sensor data from the first component and identifies the second component based on the sensor data. The digital twin selection module selects a first digital twin corresponding to the first component and a second digital twin corresponding to the second component. The interactive digital twin uses the first digital twin and the second digital twin to model the interaction between the first component and the second component. The optimization module identifies the optimal interaction between the first component and the second component by using the interactive digital twin.

[0009] Other functions and advantages of the present invention will become apparent from the following detailed description of some exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The foregoing and other aspects of the present invention will be better understood from the following detailed description when read in conjunction with the accompanying drawings. For the purpose of illustrating the present invention, the accompanying drawings show presently preferred embodiments, but it should be understood that the present invention is not limited to the specific means disclosed. The accompanying drawings include the following figures:

[0011] Figure 1A Simple examples are provided where the robot’s task is to pick up boxes from a conveyor belt;

[0012] Figure 1B An overview of the interactions of system components that may be modeled using digital twins according to some embodiments of the present invention is provided;

[0013] Figure 2 An example of an interactive digital twin according to some embodiments of the present invention is shown;

[0014] Figure 3 illustrates an example method for interactive modeling according to some embodiments of the present invention; and

[0015] Figure 4 An exemplary computing environment within which a mission planning computer may be implemented is shown. DETAILED DESCRIPTION

[0016] This article describes systems, methods, and devices for the creation of digital twins that primarily involve interactions between parts of a physical system. The technology described in this article designs and utilizes machine perception systems to understand the context of the parts of a physical system. Innovative computer vision techniques are used for semantic understanding of physical 3D scenes. Every object involved in the system is identified. In addition, the dynamics of any moving object can be predicted. Based on the information collected, a simulated environment within a digital world (i.e., a digital twin) can be created. The physical modeling of all objects is much like it is done in a computer game, except that the parameters are calculated and estimated from the physical world. However, the same components in the scene with the same individual dynamics will behave differently when interacting together.

[0017] In short, the technique described in this article uses data obtained from the same multi-component physical system under different circumstances (e.g., component 1 continuously pushes component 2 in a certain position, component 1 and component 3 separate from each other without contact, etc.) to learn the nature of the interaction between the components. Once the interaction is learned, such as in the form of a rule-based system, simulations using the learned interaction model can be used to make advance predictions about the physical world, and then the simulation is calibrated with the measured physical interactions between parts or objects to predict the next instance.

[0018] To illustrate the concepts described herein, Figure 1A A simple example is provided where the robot 105 is tasked with picking up a box 110 from a conveyor belt 115. Each component has a digital twin associated with it. The exact design and structure of the digital twin can vary, but Figure 1A In the example of , the digital twin includes four modules. The electronic simulation module simulates the electrical equipment in the component. For example, for robot 105, the electronic simulation can simulate various data associated with the motor. Similarly, the software simulation simulates any onboard software executed by the device. The digital twin also includes a structural model module for representing the physical structure of the component, and a motion simulation module for simulating any motion of the device. It should be noted that not every component needs to implement all modules. For example, box body 110 may have only one structural model. It should be noted that for the sake of illustration, Figure 1A Oversimplification of digital twins. For example, each digital twin may include additional interfaces for collecting and processing data.

[0019] Although each digital twin is conceptually shown to be located in Figure 1A , but in a real-world scenario, the digital twin can be collected at one or more computer systems, which can be local or remote to the production environment. For example, in one embodiment, data is collected from the robot 105 and the conveyor belt 115 during their respective operations. The data can then be transmitted to a cloud-based computing server via a network to update the digital twins for the robot 105 and the conveyor belt 115. For components that do not generate data, such as the box body 110, the digital twin may not only be updated with the operational data, or other data sources outside the component may be used to monitor its status. For example, relative to the box body 110, one or more cameras or other sensors in the production environment can collect data about the status of the box body 110 (such as positioning, location, etc.) and transmit it to the computing system that hosts the digital twin of the box body 110.

[0020] Figure 1B An overview of the interaction of system components that can be modeled using digital twins according to some embodiments is provided. The robot 105 uses an onboard computer to capture images of the box 110 and the conveyor 115. In other embodiments, other types of sensors can be used to collect information about the box 110 and the conveyor 115 in addition to or instead of the visual camera.

[0021] The captured images are sent to a modeling computer 125 via a network 120. The network can generally be any network known in the art, including a local intranet or the Internet. Figure 4 One example of a modeling computer 125 is shown.

[0022] The modeling computer 125 uses the captured image 130 as input to the perception module 135. The perception module 135 applies one or more perception algorithms to detect objects in the captured image 130. In general, any perception algorithm known in the art can be used. For example, for visual images, only one machine learning model can be used, such as the Google Cloud Vision API. Given an image, Google Cloud Vision will identify the objects present in the image and provide some other contextual information. For example, given the image of the production environment shown in Figure 1, Google Cloud Vision can return "box" and "conveyor belt". It should be understood that Google Cloud Vision is just an example of a perception algorithm, and other similar algorithms can be used interchangeably.

[0023] Like in Figure 1BAs in, when there are multiple objects in the captured image 130, the perception module 135 can perform additional analysis on the image. In some cases, objects that cannot be directly interacted with are excluded. For example, the robot 105 may not be able to interact with the conveyor belt 115 in any meaningful way. Therefore, the conveyor belt 115 is excluded from consideration and only the box body 110 can be used for further processing. In some embodiments, this knowledge can be encoded into the machine learning model so that the knowledge of the requesting component (i.e., the robot 105) can be used to decide which objects are relevant.

[0024] The digital twin selection module 140 identifies the digital twin associated with the requesting component (i.e., robot 105) and the output of the perception module 135 (i.e., box 110). In embodiments where the digital twin is stored in the modeling computer 125, the digital twin itself can be copied into the memory, or its respective file location can be identified. In other embodiments where the digital twin is remote from the modeling computer 125, the component's application programming interface (such as a REST interface) can be identified. The interactive digital twin 145 uses two component digital twins to simulate the interaction between real-world components.

[0025] Figure 2 Examples of interactive digital twins according to some embodiments are shown. As shown, the structural model of the robot digital twin includes models of two grippers and models of the shoulder, elbow, and wrist portions of the robot arm. For simplicity, it is assumed that the digital twin of the box includes only one structural model; however, it should be understood that more complex structural models may be used in other embodiments, particularly in embodiments where the physical components themselves are more complex. The gripper structural model and the box structural model are connected to a box gripping interaction model that simulates the interaction of the gripper squeezing the box. The shoulder, elbow, and wrist models are connected to a lifting interaction model that simulates the effect of lifting the box by the gripper. For example, the lifting interaction model can simulate the stresses to which the robot arm is subjected when lifting a box of a given weight at different arm positions.

[0026] Back to Figure 1B , the optimization module 150 determines the optimal interaction by simulating multiple interaction scenarios with variable parameters (such as arm position, gripping force, etc.). In general, any known technique in the art can be used to determine the optimal interaction. For example, in one embodiment, reinforcement learning is used with a reward system defined based on a target state that minimizes one or more characteristics (such as stress on component parts, time, cost, etc.). Reinforcement learning is defined in more detail below. Once the optimal interaction is determined, the instruction module 155 generates instructions 160 to the robot 105 to allow it to perform its part of the interaction.

[0027] Although Figure 1A, Figure 1B and Figure 2 represents a relatively simple case, but the general concept of an interacting digital twin can be extended by hierarchically building more complex interactions. For example, a car consists of various subsystems, including the engine, fuel system, exhaust system, cooling system, lubrication system, electrical system, transmission, and chassis. Within each subsystem are various subcomponents that interact to enable the operation of the vehicle. One approach to using an interacting digital twin is to model component-to-component interactions with an interacting digital twin at the lowest level of the architecture. As the design progresses to higher levels, interacting digital twins may be combined. So, for example, once at the master system level, the interaction between the engine and the fuel system can be modeled using an interacting digital twin model based on the interaction of the various subcomponents. In addition, the driver can also be modeled by a digital twin, and the interaction between the driver and the vehicle can be modeled using an interacting twin designed according to the techniques described in this article.

[0028] Figure 3 An exemplary method 300 for modeling an interaction according to some embodiments is shown. For example, the method can be performed by a component in a production environment or another computer connected to the component via a network (such as the modeling computer 125). The interaction of a first component and a second component is modeled in this example. Starting from 305, the computer receives a sensor measurement corresponding to the second component. For example, the sensor measurement can be received by one of the components or another device in the production environment (such as an overhead camera). As explained below, the sensor measurement is used to identify the second component. To continue the example discussed above, in some embodiments, the first component is a robot and the second component is a box or other workpiece. In general, any type of sensor measurement that provides sufficient information for identifying the second component can be used. For example, in one embodiment, the sensor measurement includes an image captured by a camera mounted on the first component. As an alternative to a visual image, in other embodiments, the sensor measurement includes a point cloud captured by a camera mounted on the first component. Other types of sensor measurements can also be used, such as auditory measurements, thermal measurements, force measurements, etc.

[0029] At step 310, the computer system identifies a first digital twin corresponding to the first component. For example, this identification can be performed as an identifier received from the first component (e.g., a field in a header of a data packet transmitting sensor data). Based on this identification, the first digital twin can be retrieved (e.g., from a local database).

[0030] Next, at step 315, a perception algorithm is applied to identify the component type associated with the second component (as described above with respect to Figure 1BOnce the component type is known, the component type is used to select the second digital twin in step 320. Then, in step 325, a third digital twin is selected to model the interaction between the first digital twin and the second digital twin. After identifying the first component and the second component, the selection of the third digital twin can actually be a simple search. For example, it is known that the first component is a robot and the second component is a box body. The computer in step 325 only needs to select the "robot-box body" interaction digital twin. In some embodiments, additional details about the interaction can be used to provide further specificity to the interactive digital twin. For example, if the robot indicates that it wants to lift the box body, the specific robot box body interaction digital twin can be selected to lift.

[0031] At step 330, the computer generates instructions for the first component using the third digital twin that allow the first component to interact with the second component. In some embodiments, the third digital twin uses a machine learning model trained by multiple interactions between the first component and the second component to model the interaction. The machine learning model can be trained with a library of real-world interactions between the first component and the second component. If there is not enough real-world data to support such training, synthetic data can be used.

[0032] For example, in one embodiment, the interactions include multiple real-world interactions and multiple synthetic interactions generated by a generative adversarial network, where the generative adversarial network is trained with real-world interactions. Generative adversarial networks generally represent a class of artificial intelligence algorithms that fall into the category of unsupervised learning. In its simplest form, a generative adversarial network is a combination of two neural networks: one network learns how to generate examples (e.g., synthetic interactions) from a training data set (e.g., real-world data describing the interactions), while the other network attempts to distinguish the generated examples from the training data set. If the examples generated by the generative network converge with the actual data so that the discriminative network can never distinguish the two, then the training process is successful.

[0033] In a generative adversarial network, the training examples consist of two datasets, X and Y. The datasets are unpaired, meaning that there is no one-to-one correspondence between the training images in X and Y. The generator network learns how to generate an image y' for any image x in the dataset X. More specifically, the generator network learns a mapping G:X→Y that produces an image y' (y'=G(x)). The generator network trains the mapping G:X→Y such that y' is indistinguishable from y by a discriminator network that is trained to distinguish y' from y. In other words, the generator network continues to generate examples until the discriminator network can no longer reliably classify an example as being generated by the generator network (y') or as being provided as an actual example (y).

[0034] Once the training data is generated, the machine model can be trained. The exact method of training will depend on the model used. For example, in other embodiments, the machine learning model may be one or more recurrent neural networks (RNNs). For example, in some embodiments, the third digital twin models the interaction as a sequential sequence of interaction states, and each interaction state includes a first configuration corresponding to the first component and a second configuration corresponding to the second component. Each state contains data from the first and second digital twins, which describes their respective positions, forces applied or applied, etc. For example, for a robot component, the state information may include the position of each component of the arm, the gripper, the force applied to the arm due to the gripper, etc. In this way, the interaction state can be directly estimated using RNN based on data from the first digital twin and the second digital twin. For example, in one embodiment, the RNN model is designed with two layers. The first layer is a long short-term memory (LSTM) model that receives data from the first digital twin and the second digital twin and generates internal output data. The second layer is an LSTM model that receives internal output data and estimates the interaction state.

[0035] In other embodiments, the machine learning model is a deep reinforcement learning model. General deep learning techniques are often applied to solve a variety of problems, from image classification, object detection and segmentation, speech recognition to transfer learning. Deep learning is the automatic learning of hierarchical data representations that describe underlying phenomena. In other words, deep learning proposes an automated feature function that extracts and decomposes data description attributes directly from the original input, rather than handcrafting features. The hierarchical structure encoded by the neural network is used to model the learning method.

[0036] Some embodiments use deep learning in conjunction with reinforcement learning (RL). RL facilitates learning as an end-to-end cognitive process for an artificial agent rather than a predefined method. An RL setting consists of an artificial agent that can interact with an uncertain environment (e.g., a request to obtain image data with limited or no parameters) with the goal of achieving a predetermined goal (e.g., obtaining an image with optimal parameters). The agent can observe the state of the environment and choose to act on it, similar to a trial-and-error search, to maximize the future reward signal received as a response to the environment. The environment can be modeled by simulation or operators that give positive and negative rewards to the current state.

[0037] In the reward function r t In the context of the optimal action value function approximator Q*, the agent is given by the state space s t The response of the parameterized image acquisition is estimated. This reward-based decision process is modeled in RL theory as a Markov decision process (MDP) defined by a tuple M: ​​= S, A, T, R, γ, where S is a finite state set and st ∈S is the state of the agent at time t. A is a finite set of actions that allow the agent to interact with the environment, a t ∈A is the action performed by the agent at time t. T∶S×A×S→[0;1] is a random transfer function, where is the probability that the agent reaches state s′ after executing action a in state s. R: is a scalar reward function, where is the expected reward after the state transition, and γ is the discount factor that controls the importance of future rewards relative to immediate rewards.

[0038] Intelligent agent in The discounted future reward can be written as Where T indicates the end of the learning phase, r t Define the immediate reward that the agent receives at time t. In model-free reinforcement learning, the goal can be used to find the optimal so-called "action value function", which expresses the maximum expected future discounted reward when starting and executing action a in state s as:

[0039]

[0040] Where π is the action policy. In other words, the action policy is the probability distribution of possible actions in each given state. Once the optimal action value function is estimated, the optimal action policy that determines the behavior of the agent can be directly calculated in each state. The formula is as follows:

[0041]

[0042] The optimal action-value function approximator Q* is the Bellman optimality equation, which represents the recursive formulation of Q * (s, a) = , is defined as:

[0043]

[0044] Among them, s′ defines the states that may be visited after the action corresponding to s,a′, represents the current instant reward in a compact notation. The Bellman equation defines the contraction map, which is viewed as an operator τ. Applying Q i+1 =τ(Q i ), Function Q iconverges to Q* at infinity. However, this standard, model-based policy iteration approach may not be feasible in practice. Another approach in practice is to use model-free temporal difference methods, typically Q-Learning, which exploits the correlation of consecutive states. Approximating the function Q with a parameter function further achieves the goal of higher computational efficiency. Given the expected nonlinear structure of the action-value function, neural networks represent a sufficiently powerful approximation.

[0045] In the context of the present invention, as implemented in some embodiments, the artificial intelligence agent is part of an interactive digital twin, and can learn an optimal action-value function approximator based on the interaction state and synthetic interaction data observed over time. The data can include successful interactions as well as unsuccessful ones. Continuing with the example of a robot moving a box, interactions can be used when the robot moves the box at different speeds, arm angles, etc. In addition, interactions can also be used in "unsuccessful" situations where the box is damaged or dropped by the robot. Moreover, in some embodiments, the stress level of the interaction can be detected (e.g., by a designer or operator), and interactions that over-stress the component can be considered "unsuccessful." Automation or semi-automation of the process can be achieved by defining thresholds for the various parts of each component and marking the interaction as "unsuccessful" when any threshold is exceeded.

[0046] Back to Figure 3 , in step 335, the computer passes the instruction to at least one of the first component and the second component. With full knowledge of the interaction state, generating instructions only requires converting the state into instructions that the component can execute. The exact conversion method may vary depending on the performance of the component and how the component requires the specified instructions. For example, in some embodiments, a series of explicit instructions are generated (e.g., move the arm 10 degrees, engage the gripper with a force of 90 to 110 Newtons, etc.). This conversion can be performed on the computer executing method 300, or the instructions can be generated by another computer in the system. Since the interaction is actually performed by the first component and the second component, their respective digital twins can be continuously monitored to collect further real-world information, which can be used to further train the machine learning model of the interactive digital twin.

[0047] Figure 4 An exemplary computing environment 400 is shown in which a computer 125 (eg, Figure 1B 4 ). Computing environment 400 includes computer system 410, which is an example of a computing system on which embodiments of the present invention may be implemented. Computers and computing environments, such as computer system 410 and computing environment 400, are well known to those skilled in the art and are therefore only briefly described herein.

[0048] like Figure 4 As shown, computer system 410 may include a communication mechanism, such as bus 421 or other communication mechanism for transmitting information in computer system 410. Computer system 410 further includes one or more processors 420 coupled to bus 421 for processing information. Processor 420 may include one or more central processing units (CPUs), graphics processing units (GPUs), or any other processor known in the art.

[0049] The computer system 410 also includes a system memory 430 coupled to the bus 421 to store information and instructions to be executed by the processor 420. The system memory 430 may include computer-readable storage media in the form of volatile and / or non-volatile memory, such as read-only memory (ROM) 431 and / or random access memory (RAM) 432. The system memory RAM 432 may include other dynamic storage devices (such as dynamic RAM, static RAM, and synchronous DRAM). The system memory ROM 431 may include other static storage devices (such as programmable ROM, erasable PROM, and electrically erasable PROM). In addition, the system memory 430 may be used to store temporary variables or other intermediate information during the execution of instructions by the processor 420. The basic input / output system (BIOS) 433 contains basic programs that help transfer information between components within the computer system 410, such as during startup, and may be stored in the ROM 431. The RAM 432 may contain data and / or program modules that the processor 420 can immediately access and / or is currently operating on. System memory 430 may also include, for example, an operating system 434, application programs 435, task-specific modules 436, and program data 437. Application programs 435 may include, for example, one or more executable applications that may retrieve one or more task-specific modules 436 in response to a request received from robotic device 480.

[0050] The computer system 410 also includes a disk controller 440 coupled to the bus 421 to control one or more storage devices for storing information and instructions, such as a hard disk 441 and a removable media drive 442 (e.g., an optical drive, a solid-state drive, etc.). Storage devices can be added to the computer system 410 using a suitable device interface (e.g., small computer system interface (SCSI), integrated device electronics (IDE), universal serial bus (USB), or FireWire).

[0051] The computer system 410 can respond to the processor 420 executing one or more sequences of one or more instructions contained in a memory (such as system memory 430) to perform some or all of the processing steps of the embodiments of the present invention. These instructions can be read into the system main memory 430 from another computer-readable medium, such as a hard disk 441 or a removable media drive 442. The hard disk 441 may contain one or more data storage and data files used by the embodiments of the present invention. For example, in some embodiments, the hard disk 441 can be used as an alternative or supplement to the RAM 432 to store task-specific modules. The data storage content and data files can be encrypted to improve security. The processor 420 can also be used in a multi-processing arrangement to execute one or more sequences of instructions contained in the system memory 430. In an alternative embodiment, a hard-wired circuit can be used instead of a software instruction, or a hard-wired circuit can be used in combination with a software instruction. Therefore, the embodiments are not limited to any specific combination of hardware circuits and software.

[0052] As described above, the computer system 410 may include at least one computer-readable medium or memory for storing instructions programmed according to embodiments of the present invention and for accommodating data structures, tables, records, or other data described herein. The term "computer-readable medium" as used herein refers to any medium that participates in providing execution instructions to the processor 420. Computer-readable media can take many forms, including but not limited to non-volatile media, volatile media, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid-state drives, magnetic disks, and magneto-optical disks, such as hard disks 441 or removable media drives 442. Non-limiting examples of volatile media include dynamic memory, such as system memory 430. Non-limiting examples of transmission media include coaxial cables, copper wires, and optical fibers, including wires that make up bus 421. Transmission media can also take the form of sound waves or light waves, such as waves generated during radio wave and infrared data communications.

[0053] When used in a network environment, the computer system 410 may include a modem 472 for establishing communications with a robotic device 480 or other remote computing system via a network 471, such as the Internet. The modem 472 may be connected to the bus 421 via a user network interface 470 or another appropriate mechanism. Figure 4 In the illustrated example, the robotic device 480 is shown as being connected to the computer system 410 via the network 471, but in other embodiments of the invention, the computer system 410 may be directly connected to the robotic device 480. For example, in one embodiment, the computer system 410 and the robotic device 480 are located in the same or adjacent rooms, and the devices are connected using any transmission medium generally known in the art.

[0054] The network 471 can be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or a series of connections, a cellular telephone network, or any other network or medium that can facilitate the communication between the computer system 410 and other computers (such as the robot device 480). The network 471 can be wired, wireless, or a combination of wired and wireless. A wired connection can be implemented with Ethernet, a universal serial bus (USB), RJ-11, or any other wired connection generally known in the art. A wireless connection can be implemented with Wi-Fi, WiMAX, Bluetooth, infrared, cellular networks, satellites, or any other wireless connection methods generally known in the art. In addition, several networks can work alone or communicate with each other to facilitate communication in the network 471.

[0055] The general architecture of computer system 410 can be used to implement the internal computing system of robotic device 480. In some embodiments, the various components of computer system 410 described above can be used in a simplified form. For example, robotic device 480 can use a single processor and relatively little system memory 430. In addition, components such as hard disk 441 and removable media drive 442 can be omitted. In addition, robotic device 480 can store additional data, such as machine-specific modules, to enable it to perform the techniques described herein. It should be understood that the components need not be robotic devices, and in other embodiments, other types of computing devices can be similarly connected via network 471.

[0056] Embodiments of the present invention may be implemented by any combination of hardware and software. In addition, embodiments of the present invention may be included in an article (such as one or more computer program products) having a computer-readable non-transitory medium. The medium has been implemented herein, for example, computer-readable program code for providing and facilitating the mechanisms of embodiments of the present invention. The article may be included as part of a computer system or may be sold separately.

[0057] Although various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are intended to be illustrative and not intended to be limiting, with the true scope and spirit being indicated by the following claims.

[0058] Unless otherwise explicitly stated in the following discussion, it should be understood that terms such as "apply", "generate", "identify", "determine", "process", "compute", "select", etc. can refer to the actions and processes of a computer system or similar electronic computing device, which operates on data represented by physical (or electronic) quantities in the registers and memory of the computer system and converts it into other data represented by physical quantities in the memory or registers of the computer system or other such information storage, transmission or display devices. The embodiments of the methods described herein can be implemented using computer software. If the instruction sequence designed to implement the described method is written in a programming language that meets the recognized standards, the instruction sequence can be compiled so as to be executed on a variety of hardware platforms and used for interfaces of various operating systems. In addition, the embodiments of the present invention are not described with reference to any particular programming language. It should be understood that various programming languages ​​can be used to implement the embodiments of the present invention.

[0059] As used herein, executable applications include code or machine-readable instructions for regulating a processor to implement a predetermined function, such as a predetermined function of an operating system, a contextual data acquisition system, or other information processing system, in response to a user instruction or input. An executable program is a fragment, subroutine, other different fragments of code, or part of an executable application that performs one or more specific processes. These processes may include receiving input data and / or parameters, operating on the received input data, and / or performing functions based on the received input parameters, and providing final output data and / or parameters.

[0060] Here, functions and process steps may be automatically or fully or partially executed according to user instructions. Automatically executed activities (including steps) are performed according to one or more executable instructions or device operations, and the user does not need to directly initiate the activity.

[0061] The systems and processes of the accompanying drawings are not exclusive. Other systems, processes and menus can be derived according to the principles of the present invention to achieve the same purpose. Although the present invention is described with reference to specific embodiments, it should be understood that the embodiments and variations shown and described herein are for illustrative purposes only. Those skilled in the art may implement modifications to the current design without departing from the scope of the present invention. As described herein, various systems, subsystems, agents, managers and processes may be implemented using hardware components, software components and / or combinations thereof. None of the claim elements herein should be interpreted according to the provisions of 35 USC 112 (f), and the elements are explicitly stated with the phrase "device for..."

Claims

1. A method for modeling interactions between system components, comprising: receiving, via a first component in a production environment, a sensor measurement corresponding to a second component in the production environment; identifying a first digital twin corresponding to the first component; applying a perception algorithm to identify a component type associated with the second component; selecting a second digital twin based on the component type; selecting a third digital twin that models an interaction between the first digital twin and the second digital twin; generating, using the third digital twin, instructions for the first component that allow the first component to interact with the second component; as well as sending the instruction to the first component, wherein the third digital twin uses a machine learning model to model the interaction between the first component and the second component, the machine learning model is trained using multiple interactions between the first component and the second component, the interactions include multiple real interactions and multiple synthetic interactions generated by a generative adversarial network, and the generative adversarial network is trained using the multiple real interactions, wherein the third digital twin models the interactions as a sequential sequence of interaction states, and each interaction state includes a first configuration corresponding to the first component and a second configuration corresponding to the second component, wherein the machine learning model is used to directly estimate the interaction state based on data from the first digital twin and the second digital twin.

2. The method according to claim 1, wherein: The sensor measurements include images acquired by a camera mounted on the first component.

3. The method according to claim 1, wherein: The sensor measurements include a point cloud acquired by a camera mounted on the first component.

4. The method according to claim 1, wherein: The first component is a robot and the second component is a workpiece.

5. The method according to claim 1, wherein: The machine learning model is a deep reinforcement learning model that uses a reward system that provides positive reinforcement for interactions that produce one or more target states, wherein one or more stress levels are associated with the first component and are below a predetermined limit.

6. The method according to claim 1, wherein: The machine learning model includes one or more recursive neural network (RNN) models, which directly estimate the interaction state based on data from the first digital twin and the second digital twin.

7. The method according to claim 6, wherein: The one or more recursive neural network (RNN) models include a first-layer long short-term memory (LSTM) model that receives data from the first digital twin and the second digital twin and generates internal output data, and a second-layer LSTM model that receives the internal output data and estimates the interaction state.

8. A system for modeling interactions between system components, comprising: a first digital twin corresponding to the first component in the production environment; a second digital twin corresponding to a second component in the production environment; a third digital twin that models an interaction between the first component and the second component using the first digital twin and the second digital twin, wherein the third digital twin uses a machine learning model to model the interaction between the first component and the second component, the machine learning model is trained using multiple interactions between the first component and the second component, the interactions include multiple real interactions and multiple synthetic interactions generated by a generative adversarial network, and the generative adversarial network is trained using the multiple real interactions, wherein the third digital twin models the interactions as a sequential sequence of interaction states, and each interaction state includes a first configuration corresponding to the first component and a second configuration corresponding to the second component, wherein the machine learning model is used to directly estimate the interaction state based on data from the first digital twin and the second digital twin.

9. The system according to claim 8, wherein: The first component is a robot and the second component is a workpiece.

10. The system according to claim 8, wherein: The machine learning model is a deep reinforcement learning model that uses a reward system that provides positive reinforcement for interactions that produce one or more target states, wherein one or more stress levels are associated with the first component and are below a predetermined limit.

11. The system according to claim 8, wherein: The machine learning model includes one or more recursive neural network (RNN) models, which directly estimate the interaction state based on data from the first digital twin and the second digital twin.

12. The system according to claim 11, wherein: The one or more recursive neural network (RNN) models include a first-layer long short-term memory (LSTM) model that receives data from the first digital twin and the second digital twin and generates internal output data, and a second-layer LSTM model that receives the internal output data and estimates the interaction state.

13. A system for modeling an interaction between a first component and a second component in a production environment, the system comprising: a perception module, receiving sensor data from the first component and identifying the second component based on the sensor data; A digital twin selection module, which selects a first digital twin corresponding to the first component and a second digital twin corresponding to the second component; an interactive digital twin, using the first digital twin and the second digital twin to model the interaction between the first component and the second component, wherein the interactive digital twin uses a machine learning model to model the interaction between the first component and the second component, the machine learning model is trained using multiple interactions between the first component and the second component, the interactions include multiple real interactions and multiple synthetic interactions generated by a generative adversarial network, the generative adversarial network is trained using the multiple real interactions, wherein the interactive digital twin models the interactions as a sequential sequence of interaction states, and each interaction state includes a first configuration corresponding to the first component and a second configuration corresponding to the second component, wherein the machine learning model is used to directly estimate the interaction state based on data from the first digital twin and the second digital twin; and An optimization module uses the interactive digital twin to identify an optimal interaction between the first component and the second component.

Images