System and method for feeding constraints in autonomous skill execution back into design

Abstract

The present invention relates to systems and methods for feeding constraints in the execution of autonomous skills back into the design. A computer-implemented method for designing a process to be performed by a robotic unit, the method comprising: obtaining a process goal and one or more process constraints. The method comprises: accessing a configuration library and a skill library. Each configuration comprises a digital representation of a component of the robotic unit or a geometric transformation of the robotic unit. Each skill comprises a functional description of using a robot of the robotic unit to interact with a physical environment to perform the skill goal. The method uses a simulation engine to simulate multiple designs, wherein each design is represented as a combination of a configuration and a skill that achieves the process goal, and determines a set of feasible designs that satisfy one or more process constraints. The method comprises: outputting a recommended design from the set of feasible designs.

Description

Technical Field

[0001] The present invention generally relates to autonomous systems, and more particularly, to a method and apparatus for designing processes performed by a robotic unit based on constraints in the performance of autonomous skills. Background Art

[0002] In a global and highly competitive landscape, the need to manage rapid innovation cycles, complex customization requirements, and growing cost pressures is increasingly challenging traditional industrial automation systems. This challenge is driving manufacturers to shift from automation to autonomy. In contrast to automation, autonomy empowers individuals within the factory with decision-making and self-control capabilities, allowing them to independently act in the event of local problems.

[0003] In the context of autonomous systems, machine instructions can be abstracted into skills, such as object grasping or placement, and these skills can be combined to generate appropriately context-aware applications.

[0004] The industrial use of autonomous systems in factories is expected to be widespread and cover a wide range of application scenarios. In some cases, human involvement will need to be reduced or even eliminated. In other scenarios, autonomous machines can augment the physical and intellectual capabilities of factory workers. This development is at the heart of the technologies that enable the flexible manufacturing operations envisioned in the context of Industry 4.0.

[0005] Flexibility often comes at the expense of speed. It is desirable to be able to make informed decisions in the design and commissioning of autonomous systems in factories to optimize performance while adding constraints (e.g., speed / throughput). Summary of the Invention

[0006] Briefly stated, aspects of the present invention are directed to a technique for feeding back constraints in the execution of an autonomous skill into the design of a process performed by a robotic cell.

[0007] A first aspect of the present invention provides a computer-implemented method for designing a process to be performed by a robotic cell. The method includes obtaining a process objective and one or more process constraints. The method includes accessing a construction library and a skill library. Each construction in the construction library includes a digital representation of a component of the robotic cell or a geometric transformation of the robotic cell. Each skill in the skill library includes a functional description of using a robot of the robotic cell to interact with a physical environment to perform the skill objective. The method includes simulating a plurality of designs using a simulation engine, wherein each design is characterized as a combination of a construction and a skill that achieves the process objective, and thereby obtaining a set of feasible designs that satisfy one or more process constraints. The method further includes outputting a recommended design from the set of feasible designs.

[0008] Other aspects of the present invention implement the features of the above-described method in a computing system and a computer program product.

[0009] Another aspect of the present invention provides a system for designing a process to be performed by a robotic cell. The system includes: a library module, including a construction library and a skill library. Each construction in the construction library includes a digital representation of a component of the robotic cell or a geometric transformation of the robotic cell. Each skill in the skill library includes a functional description of using the robot of the robotic cell to interact with the physical environment to perform a skill goal. The system includes: a simulation module, including a simulation engine configured to simulate multiple designs, wherein each design is represented as a combination of a construction and a skill that achieves a specified process goal, and thereby obtains a set of feasible designs that meet one or more specified process constraints. The system further includes: a recommendation module, configured to output a recommended design from the set of feasible designs.

[0010] Additional technical features and benefits can be achieved through the technology of the present invention. The embodiments and aspects of the present invention are described in detail herein and are considered to be part of the subject matter claimed. For a better understanding, reference is made to the detailed description and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other aspects of the present invention will be best understood from the following detailed description when read in conjunction with the accompanying drawings.To easily identify the discussion of any element or activity, one or more most significant digits in a reference numeral refer to the figure number in which the element or activity is first introduced.

[0012] Figure 1 A schematic block diagram of an elucidation system according to an exemplary embodiment is shown.

[0013] Figure 2 An illustrative embodiment is shown demonstrating a baseline design of a robotic cell that performs the process of packaging bottles.

[0014] Figure 3 Shown by the system based on Figure 2 Example of a design spectrum in which process objectives are evaluated.

[0015] Figure 4 Shown for execution with Figure 2 One embodiment of a recommended design of a robotic cell with the same process goal.

[0016] Figure 5 A computing environment is shown according to an exemplary embodiment. DETAILED DESCRIPTION

[0017] Various techniques related to systems and methods are now described with reference to the accompanying drawings, in which similar reference numerals represent similar elements throughout. The drawings discussed below and the various embodiments used to describe the principles of the present invention in this patent document are merely illustrative and should not be interpreted in any way as limiting the scope of the invention. It will be understood by those skilled in the art that the principles of the present invention can be implemented in any suitable arrangement of devices. It will be understood that the functions described as being performed by a specific system element can be performed by multiple elements. Similarly, for example, an element can be configured to perform the functions described as being performed by multiple elements. Multiple innovative teachings of the present application are described with reference to exemplary, non-limiting embodiments.

[0018] Aspects of the present invention relate to an autonomous system. Autonomy empowers individuals within a factory with the decision-making and self-control capabilities to independently act in the event of a local problem. In conventional automation, automated robots are typically programmed to perform a single, repetitive task, such as positioning an automotive panel in the exact same position on every vehicle. In these cases, engineers typically program the entire task from start to finish, typically generating individual commands using low-level code. In contrast, in an autonomous system, skills are used instead of individual commands to program the autonomous machine or robot at a higher level of abstraction.

[0019] In the context of programming autonomous machines, skills are represented by individual programming blocks (also called skill code), which include functional descriptions of how the robot interacts with the physical environment to perform specified skill goals. Skills can have functional components as well as structural components. Skills are derived for higher-level abstract behaviors, which are centered around how the robot is programmed to modify the environment. Illustrative examples of skills that can be implemented using the techniques described herein include a skill for opening a door, a skill for detecting an object, a skill for grasping and picking up an object, a skill for placing an object, and the like. Activated skills can be designed as functions within the programming environment. For example, this can be done by calling a skill function that is part of a device service. Once activated, the skill function reads structural information from the physical environment to determine its operation.

[0020] The mechanical design and configuration of a robotic cell has a profound impact on the speed and type of autonomous skills it can support. In other words, software-defined autonomous skills can dramatically increase the success rate and throughput of a cell if leveraged by the mechanical configuration. These configurations can take the form of equipment / components of the robotic cell, such as fixtures, feeders, jig design, etc., as well as the geometric layout of the cell. The two dimensions of the design execution process—the skills used (software) and the mechanical design—are highly interrelated, and optimal design cannot be achieved based on only one of these dimensions.

[0021] The current state of the art lacks the ability to connect autonomous skills with other aspects of production cell or product design. For example, if a skill is "picking," existing systems provide the necessary programming or configuration steps to ensure the robot can actually pick up an object. However, there's no connection between the robot, camera, and three-arm gripper. If throughput needs to be increased or if the system experiences too many errors, the system integrator is forced to improve the overall cell design. Alternatively, the product design team needs to be involved and the design revised. Alternatively, the company that programmed the skill needs to reprogram it again, retraining its algorithm using a different set of images. In all these cases, delays and potential downtime are introduced, and decisions are made in silos. Even if humans are the ultimate designers of a product or cell, today's information is stored in silos. Design teams have no foresight into the capabilities of autonomous machine skills. System integrators have some constraints on how materials are obtained and how they are input and output, but they don't necessarily know the consequences of their design decisions. For example, if the system only needs to place objects in different boxes or bins, the "pick from box" skill can perform well, but if the picked object needs to be inserted precisely, it may not perform optimally. As another example: if positioning is required, two robot arms can perform the task if the "handover skill" is available, but at the expense of speed and cost. A fixture that can help the robot rotate the object can reduce costs (no need for a second arm). If throughput is constrained, different feeders can be a solution. All these constraints make the work of the system integrator challenging.

[0022] Embodiments of the present invention address the problem of using autonomous skill performance as feedback in the mechanical design of robotic cells, and in some embodiments, in product design and / or other engineering steps (e.g., training of machine learning models).

[0023] Currently, efforts are underway to create digital twins of production and operations for traditional automation development, closing the loop between design and operations. The disclosed embodiments take this a step further and add a dimension of autonomy, increasing system flexibility but also increasing unpredictability and the need for tighter coupling with the physical equipment in the robotic cell. The disclosed embodiments leverage high-fidelity simulation to achieve a better robotic cell design, given the constraints imposed by the robotic cell commissioning process, thereby providing a more rational virtual commissioning through collaborative simulation.

[0024] Turning now to the accompanying drawings, Figure 1A system 100 for designing a process to be performed by a robotic cell according to an embodiment of the present invention is shown. Broadly speaking, the system 100 includes a library module 102 including a skills library 104 and a construction library 106, a simulation module 108, and a recommendation module 110. The disclosed modules of the system 100 can be incorporated into one or more computing systems. The computing system includes a processor coupled to a memory. Optionally, the system can include a display for displaying a graphical user interface (GUI), a memory having recorded thereon a computer program including instructions for performing the method. The memory can also store a data set including the library described herein. Memory refers to any hardware adapted for such storage and can include several physically distinct parts (e.g., one part for a program and one part for a database). Reference Figure 5 An exemplary embodiment of a computing environment is shown.

[0025] The method disclosed in conjunction with system 100 is computer-implemented. This means that the steps (or substantially all of the steps) of the disclosed method are performed by at least one computer or computing system. Thus, the disclosed steps can be performed fully or semi-automatically by a computer. In embodiments, at least some of the steps of the disclosed method can be triggered through user-computer interaction. The degree of user-computer interaction required can depend on the desired degree of automation and be balanced against the need to achieve user desires. In embodiments, the degree can be user-defined and / or predefined.

[0026] Continue to refer Figure 1 , the skill library 104 can include a set of skills that can be performed by one or more robots available in the robotic cell. Each skill in the skill library 104 includes a functional description of using a robot of the robotic cell to interact with the physical environment to perform a skill goal. In one implementation, the skill library 104 can include a skill code for each individual skill. In various embodiments, the skill code can include a machine learning model and / or a machine vision algorithm. In another implementation, instead of an actual skill code, the skill library 104 can provide a clear characterization of each skill. In addition to the functional description of the skill, the characterization can include one or more skill performance parameters for each skill, such as execution time, error rate, cost, etc.

[0027] The build library 106 includes digital representations of available components of a robotic cell and / or different geometric transformations of the robotic cell. For example, the components of a robotic cell can include one or more types of robots, fixtures, grippers, feeders, etc. Geometric transformations can include different geometric layouts of the robotic cell, such as camera placement, input object orientation, etc. For example, the digital representation of a build can include a CAD model containing specifications from which the geometric structure is generated. In other embodiments, instead of or in addition to using a CAD model, the digital representation of the build can be generated using sensor data such as imaging data.

[0028] For example, the simulation module 108 is available as a digital twin of a robotic cell, which provides a high-fidelity digital model of the robotic cell by utilizing digital representations of the cell's physical components and geometric layout (e.g., a CAD model). The simulation module 108 includes a physics-based simulation engine (also referred to as a "physics engine") configured to simulate a process performed by the robotic cell. The execution of a process by the robotic cell includes at least one robot performing one or more skills (typically a combination of skills) in a physical environment defined by a combination of configurations representing the mechanical design of the cell to achieve a process goal.

[0029] The disclosed embodiment utilizes the simulation module 108 to generate a set of designs based on one or more specified process constraints to achieve specified process goals. The process goals define the overall goals or objectives of the process (in Figure 2 In the illustrated embodiment, such as packaging bottles in boxes. Process constraints can define acceptable performance parameters (e.g., execution time, error rate, cost, etc.) for the overall process. In one embodiment, the process goal and one or more process constraints are obtained based on inputs 112 via a user interface (e.g., a GUI). In other embodiments, the same inputs 112 can be automatically obtained based on information already stored in the memory of the computing system.

[0030] Based on the information accessed from the library module 102, the simulation module 108 uses a simulation engine to simulate a plurality of designs, wherein each design is characterized as a combination of a construction and a skill that achieves a process goal. The simulation of each design involves measuring the overall performance of the design, i.e., measuring one or more performance parameters of the overall process performed using the design. In a first implementation, the simulation engine is capable of executing the skill code (i.e., activating or calling a function) of each individual skill included in the design to measure one or more skill performance parameters of each skill, and thereby measuring the overall design performance. In a second implementation, the simulation engine is capable of measuring the overall design performance based on one or more skill performance parameters of each individual skill included in the design, wherein the skill performance parameters are explicitly specified in the skill library. Based on the measured overall design performance, a set of feasible designs that meet the specified process constraints are obtained.

[0031] In one embodiment, the simulation module 108 can be configured to generate a design for simulation based on a process that generates designs. To this end, the simulation module 108 can include a machine learning-based generative model, for example, including a neural network, that generates designs based on specified process constraints. An initial training phase can include an iterative process of generating designs using a generative model using a combination of skills and configurations (the skills library 104 and the configuration library 106, respectively) that achieve the process goals simulated by the simulation engine to measure overall design performance; and adjusting model parameters (e.g., weights, biases of the neural network) so that the measured overall design performance meets the specified constraints. In a later phase, the generative model can use available skills from the skills library 104 and available configurations from the configuration library 106 to generate multiple feasible designs that achieve the process goals that meet the process constraints.

[0032] like Figure 1 As shown in , in the disclosed embodiment, a design for simulation is generated by generating an optimal transformation of a baseline design 114 (by generative design or other techniques). Typically, the baseline design can include the current system design of the robotic cell, or another reference design. Transformations of the baseline design can be generated by identifying at least one bottleneck skill in the baseline design. The bottleneck skill is identified as the skill whose execution introduces the greatest constraint in the design, thereby making the design unsuitable for meeting the specified process constraints. The bottleneck skill of the baseline design can be determined by measuring skill performance parameters at runtime, for example, by simulation or using real-world data (in the case of existing system designs). Based on the identified bottleneck skills, transformations can be generated by shuffling constructions related to executing the bottleneck skill and / or generating other combinations of skills that eliminate the bottleneck skill.

[0033] In one embodiment, the simulation module 108 can generate designs based on decomposing the process goals into technical problems and evaluating the design parameter spectrum (configuration and / or skills) associated with each problem (e.g., see Figure 3 ). A spectrum of design parameters relevant to each problem can be identified based on domain expertise. In one embodiment, artificial intelligence based on an expert system can be used for this purpose. For example, one identified technical problem can involve robotic transport of an object. If it is determined (e.g., through simulation or real-world data) that the object is too heavy and the gripper cannot apply sufficient suction force, resulting in the object being dropped and requiring re-grasping, the simulation module 108 can evaluate alternative suction cup diameters or shapes that maximize suction force.

[0034] The recommendation module 110 generates an output 116 derived from a set of feasible designs for evaluation by an engineer or designer. The output 116 can be provided via a GUI that allows the user to view and, in some cases, control the execution of the simulation process based on the recommended design. The output 116 of the recommendation module 110 typically includes a large number of design options that a human designer can review. Each design option is characterized by a combination of configuration (the mechanical hardware of the unit) and skill.

[0035] In some embodiments, a very large number of feasible designs can be generated by the simulation module 108. In this case, the recommendation module 110 can selectively output only a small subset of the feasible designs. In one exemplary embodiment, the recommendation module 110 can include an algorithm that fine-tunes the feasible region by, for example, selecting a specific design or changing input values ​​(e.g., constraints). In another exemplary embodiment, the recommendation module 110 can include a clustering algorithm that clusters designs that are substantially similar to each other and outputs only one representative design of the cluster.

[0036] If there is no device in the construction library 106 that is capable of performing the transformation, the recommendation module 110 can indicate this in the output 116. The designer can then design the component and add it to the cell.

[0037] In various embodiments, other feedback loops can be established for different stages of production that can simplify or accelerate the work of the robotic cell. One embodiment provides automatic feedback for product design. Another embodiment provides automatic feedback for engineering / machine learning pipelines.

[0038] In one embodiment, the output 116 of the recommendation module 110 can include specific recommendations based on executing the skill code for the identified bottleneck skill. In a first exemplary embodiment, the skill code for the bottleneck skill can be based on a machine vision algorithm. A bottleneck can arise due to the skill, for example, due to false detections that reduce average throughput. In this case, executing the skill code includes simulating images captured by a camera. In this case, the recommendation module 110 can output recommendations for changing the design of the product being handled by the robotic cell. For example, the recommendation can include distinguishing features in the product design to make it easier for the machine vision algorithm to recognize. In a second exemplary embodiment, the skill code for the bottleneck skill can be based on a machine learning model. For example, the skill code for the "Pick" skill can be based on a deep learning method. A bottleneck can arise due to the skill, for example, due to difficulty in predicting the successful pick rate of a given object, which in turn delays the overall operation. In this case, the recommendation module 110 can output recommendations for retraining the machine learning model. For example, the recommendation can be to retrain the machine learning model for the "Pick" skill using a different set of images or objects.

[0039] Based on reference Figures 2 to 4 The illustrative embodiments of the present invention can better understand the method disclosed above.

[0040] Figure 2 A baseline design 200 of a robotic cell for performing the process of packaging bottles is shown. The top portion of the figure depicts the skills used in the baseline design 200 to achieve the process goal, i.e., packaging the bottles into boxes, and the corresponding execution time for each skill. The bottom portion of the figure shows the mechanical design of the robotic cell according to the baseline design 200. Since the final orientation of the bottle is important, the autonomous system first needs to identify the orientation of the bottle, then needs to pick up the bottle, calculate the final orientation, perform a repositioning action with the help of a second robot, and finally, insert the bottle into the box. Figure 2 As shown in FIG, the first robot 202 performs Skill 1 (Pose Evaluation) to identify the input orientation of the bottle 204 and Skill 2 (Pickup) to pick up the bottle 204. The first robot 202 then performs Skill 3 (Orientation) in conjunction with the second robot 206 to achieve the desired final orientation of the bottle 204. Finally, the second robot 206 performs Skill 4 (Insertion) to insert the bottle 204 into the box 208 at the final orientation.

[0041] Since the bottle can have any input orientation, the autonomous system has maximum flexibility at the expense of speed. We assume that the introduction of process constraints limits the overall process execution time to 10ms. The baseline design 200 has a process execution time of 20ms and therefore no longer meets the specified process constraints. The disclosed method can use the skill library and the configuration library (including potential equipment and geometric transformations) to find the optimal transformation of the baseline design 200 to increase speed by reducing flexibility. Figure 2 In one embodiment, it can be determined that posture assessment and orientation skills slow down the operation and are therefore identified as bottleneck skills. After identifying the bottleneck skills, the disclosed method geometrically searches for ways to facilitate the task or even eliminate the need for these bottleneck skills in the first place by simulating all library components. The method can involve generatively simulating thousands or millions of combinations and returning a large number of experimental designs to the designer for evaluation.

[0042] Figure 3 Shown is the system based on Figure 2 3. The items in the leftmost column 302 refer to specific problems into which the overall process goal can be decomposed. The items in column 304 include design parameters related to the corresponding problems.

[0043] Figure 4 shows the method for executing Figure 2An embodiment of a recommended design 400 for a robotic cell with the same process goal. For example, Figure 3 Design 400 is generated using the design spectrum shown in [ 400 ] to shuffle configurations around and / or eliminate identified bottleneck skills. According to this recommended design 400, a feeder 402 is introduced at the input to automatically rotate bottles 204 to the desired orientation to achieve the desired final orientation. This eliminates the need for a second robot in design 400 and also eliminates the bottleneck skills identified in baseline design 200. The combination of configurations and skills in design 400 achieves higher speed or throughput by sacrificing flexibility to meet the specified process constraints.

[0044] The disclosed embodiments provide system integrators and end users of robotic cells with greater transparency into the constraints imposed by autonomous skills and provide a link to the mechanical design of the cell. The disclosed embodiments result in several technical benefits, including a modular architecture, optimal mechanical-software design, and cost reduction. This is expected to significantly impact current trends such as skill-based programming of autonomous systems.

[0045] Figure 5 An exemplary computing environment including computer system 502 is shown within which aspects of the present invention can be implemented. In one embodiment, the computing environment can include a cloud computing environment that includes one or more computing systems described herein. Computers and computing environments such as computer system 502 and computing environment 500 are known to those skilled in the art and are therefore briefly described herein.

[0046] like Figure 5 As shown in FIG, computer system 502 can include a communication mechanism such as a system bus 504 or other communication mechanism for communicating information with computer system 502. Computer system 502 further includes one or more processors 506 coupled to system bus 504 for processing information. Processor 506 can include one or more central processing units (CPUs), graphics processing units (GPUs), or any other processors known in the art.

[0047] Computer system 502 also includes system memory 508 coupled to system bus 504 for storing information and instructions executed by processor 506. System memory 508 can include computer-readable storage media in the form of volatile and / or non-volatile memory, such as read-only memory (ROM) 510 and / or random access memory (RAM). System memory RAM 512 can include other dynamic storage devices (e.g., dynamic RAM, static RAM, and synchronous DRAM). System memory ROM 510 can include other static storage devices (e.g., programmable ROM, erasable PROM, and electrically erasable PROM). In addition, system memory 508 can be used to store temporary variables or other intermediate information while processor 506 executes instructions. A basic input / output system 514 (BIOS), containing basic routines that help transfer information between elements within computer system 502, such as during startup, can be stored in system memory ROM 510. System memory RAM 512 can contain data and / or program modules that are immediately accessible to and / or currently being operated on by processor 506. Furthermore, for example, system memory 508 can include operating system 516 , application programs 518 , other program modules 520 , and program data 522 .

[0048] The computer system 502 also includes a disk controller 524 coupled to the system bus 504 to control one or more storage devices for storing information and instructions, such as a magnetic hard disk 526 and a removable media drive 528 (e.g., a floppy disk drive, a compact disk drive, a tape drive, and / or a solid-state drive). Storage devices can be added to the computer system 502 using an appropriate device interface (e.g., small computer system interface (SCSI), integrated device electronics (IDE), universal serial bus (USB), or FireWire).

[0049] The computer system 502 can also include a display controller 530 coupled to the system bus 504 to control a display 532, such as a cathode ray tube (CRT) or liquid crystal display (LCD), to display information to a computer user. The computer system 502 includes a user input interface 534 and one or more input devices, such as a keyboard 536 and a pointing device 538, to interact with the computer user and provide information to the one or more processors 506. For example, the pointing device 538 can be a mouse, light pen, trackball, or pointer for communicating directional information and command selections to the one or more processors 506 and for controlling cursor movement on the display 532. The display 532 can provide a touch screen interface that allows input to supplement or replace the communication of directional information and command selections by the pointing device 538.

[0050] The computing system 502 also includes an I / O adapter 546 coupled to the system bus 504 to connect the computing system 502 to a controllable physical device, such as a robot. Figure 5 In the illustrated embodiment, the I / O adapter 546 is connected to a robot controller 548, which includes, for example, one or more motors for controlling the linear and / or angular position of various parts of the physical device or robot (e.g., arm, base, etc.).

[0051] Computer system 502 can perform some or all of the processing steps of embodiments of the present invention in response to one or more processors 506 executing one or more sequences of one or more instructions contained in a memory, such as system memory 508. The instructions can be read into system memory 508 from another computer-readable storage medium, such as a magnetic hard disk 526 or a removable media drive 528. Magnetic hard disk 526 can contain one or more databases and data files used by embodiments of the present invention. The database contents and data files can be encrypted to enhance holographic security. Processors 506 can also be employed in a multi-processing arrangement to execute one or more sequences of instructions contained in system memory 508. In alternative embodiments, hard-wired circuitry can be used in place of or in combination with software instructions. Thus, embodiments do not specify any particular combination of hardware circuitry and software.

[0052] The computer system 502 can include at least one computer-readable storage medium or memory for storing instructions programmed according to embodiments of the present invention and for containing data structures, charts, records, or other data described herein. The term "computer-readable storage medium" as used herein refers to any medium that participates in providing instructions to one or more processors 506 for execution. Computer-readable storage media can take a variety of forms, including but not limited to non-transitory, non-volatile media, volatile media, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid-state drives, magnetic disks, magneto-optical disks, such as magnetic hard drives 526 or removable media drives 528. Non-limiting examples of volatile media include dynamic memory, such as system memory 508. Non-limiting examples of transmission media include coaxial cables, copper wire, and optical fibers, including the wires that make up the system bus 504. Transmission media can also take the form of acoustic or light waves, such as the waves generated during radio wave and infrared data communication.

[0053] The computing environment 500 can further include a computer system 502 operating in a network environment using logical connections to one or more remote computers, such as a remote computing device 544. The remote computing device 544 can be a personal computer (laptop or desktop), a mobile device, a server, a router, a network PC, a peer device, or other common network node, and typically includes many or all of the elements described above with respect to the computer system 502. When used in a network environment, the computer system 502 can include a modem 542 for establishing communications over a network 540, such as the Internet. The modem 542 can be connected to the system bus 504 via a network interface 546 or via another appropriate mechanism.

[0054] The network 540 can be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct or serial connection, a cellular telephone network, or any other network or medium that can facilitate communication between the computer system 502 and other computers (e.g., remote computing device 544). The network 540 can be wired, wireless, or a combination thereof. A wired connection can be implemented using Ethernet, a universal serial bus (USB), RJ-6, or any other wired connection generally known in the art. A wireless connection can be implemented using Wi-Fi, WiMAX, as well as Bluetooth, infrared, cellular networks, satellite, or any other wireless connection method generally known in the art. In addition, several networks can operate individually or in combination with each other to facilitate communication within the network 540.

[0055] Embodiments of the present invention can be implemented through a combination of hardware and software. In addition, embodiments of the present invention can include an article of manufacture (e.g., one or more computer program products) having, for example, a non-volatile computer-readable storage medium. For example, the computer-readable storage medium can contain computer-readable program instructions for providing and facilitating the mechanisms of embodiments of the present invention. The article of manufacture can be included as part of a computer system or sold separately.

[0056] Computer-readable storage media can include tangible devices that hold and store instructions for use by instruction execution devices. For example, computer-readable storage media can be, but are not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. The computer-readable program instructions described herein can be downloaded from the computer-readable storage medium to a corresponding computing / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network.

[0057] The systems and processes shown in the figures are not exclusive. Other systems, processes, and menus can be derived based on the principles of the present invention to achieve the same goals. Although the present invention has been described with reference to specific embodiments, it should be understood that the embodiments and variations shown and described herein are for illustrative purposes only. Those skilled in the art will be able to modify the current design without departing from the scope of the present invention.

Claims

1. A computer-implemented method for designing a process to be performed by a robotic cell, the method comprising: obtaining a process objective and one or more process constraints; accessing a library of configurations, each configuration comprising a digital representation of a component of the robotic cell or a geometric transformation of the robotic cell; accessing a library of skills, each skill represented by a respective programming block and each skill comprising a functional description of performing a skill objective using a robot of the robotic cell interacting with a physical environment; simulating a plurality of designs using a simulation engine, wherein each design is represented by a combination of configurations and skills to achieve a process goal, and thereby obtaining a set of feasible designs that satisfy the one or more process constraints; Applying a clustering algorithm to cluster designs that are generally similar to each other through a recommendation module; and A recommended design is output from the set of feasible designs, wherein only one representative design of the cluster design is output.

2. The method according to claim 1, comprising: Based on the one or more process constraints, a design for simulation is generated using a machine learning-based generative model.

3. The method according to any one of claims 1 and 2, comprising: A design for simulation is generated by generating a transformation of a baseline design based on identifying at least one bottleneck skill in the baseline design, the baseline design comprising a current design of the robotic cell or another reference design, the bottleneck skill being a skill that introduces a maximum constraint when executed in the design.

4. The method according to claim 3, comprising: Outputs recommendations based on the programming blocks that execute the bottleneck skill.

5. The method according to claim 4, wherein The programming block for the bottleneck skill is based on a machine vision algorithm and, among other things, the recommendations include design changes for the product being handled by the robotic cell.

6. The method according to claim 4, wherein: The programming block of the bottleneck skill is based on a machine learning model, and wherein the recommendation includes retraining the machine learning model.

7. The method according to claim 1, wherein The simulation engine's simulation of each design includes measuring the overall performance of the design based on one or more skill performance parameters for each individual skill included in the design, wherein the skill performance parameters are explicitly specified in the skill library.

8. The method according to claim 1, wherein The simulation engine simulates each design by executing a programming block for each individual skill included in the design to measure one or more skill performance parameters of the individual skills and thereby measure the overall performance of the design.

9. The method according to claim 7, wherein: The one or more skill performance parameters are selected from the group consisting of execution time, error rate, and cost.

10. A non-transitory computer-readable storage medium comprising instructions which, when processed by a computer, configure the computer to perform the method according to any one of claims 1 to 9.

11. A computing device comprising: processor; and A memory storing instructions which, when executed by a processor, configure the computing device to perform the method according to any one of claims 1 to 9.

12. A system for designing a process to be performed by a robotic cell, comprising: A library module, the library module comprising: a library of configurations, each configuration comprising a digital representation of a component of the robotic cell or a geometric transformation of the robotic cell; a library of skills, each skill represented by a respective programming block and each skill comprising a functional description of performing a skill objective using a robot of the robotic cell interacting with a physical environment; and a simulation module comprising a simulation engine configured to simulate a plurality of designs, wherein each design is represented by a combination of configurations and skills to achieve a specified process goal, and thereby obtain a set of feasible designs that satisfy one or more specified process constraints; and The recommendation module is configured to cluster designs that are substantially similar to each other by a clustering algorithm and output a recommended design from the group of feasible designs, wherein only one representative design of the clustered designs is output.

13. The system according to claim 12, wherein: The simulation module includes a generative model based on machine learning to generate a design for simulation based on one or more process constraints.

14. The system according to any one of claims 12 and 13, wherein: The simulation module is configured to generate a design for simulation by generating a transformation of a baseline design based on identifying at least one bottleneck skill in the baseline design, the baseline design comprising a current design of the robotic cell or another reference design, the bottleneck skill being a skill that introduces the greatest constraint when executed in the design.

15. The system according to claim 14, wherein: The recommendation module is configured to output a recommendation based on the programming block that performs the bottleneck skill.

Images