System and method for real-time control of a robot using a robot animation system
The robot animation system addresses integration and real-time editing challenges by synchronizing databases for instant motion editing and visualization, enhancing robot control efficiency and user interaction.
Patent Information
- Application Number
- JP2024527562
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-11-10
- Filing Date
- 2022-11-10
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2042-11-10
AI Technical Summary
Conventional control systems lack integration, real-time editing capabilities, and visualization, requiring multiple software packages and lengthy iteration times, hindering efficient robot motion control.
A robot animation system that synchronizes user and controller motion databases for real-time visualization and interactivity, allowing users to edit robot motions instantly and reflect changes on the robot interface and device.
Enables lifelike, continuous robot motion with improved workflow efficiency by allowing real-time synchronization and editing of robot commands, reducing iteration times and enhancing user interface interaction.
Smart Images

Figure 2024544927000001_ABST
Abstract
Description
[Technical field]
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority under 35 U.S.C. §119(e) to earlier U.S. Provisional Patent No. 63 / 278,793, filed November 12, 2021, the disclosure of which is incorporated herein by reference in its entirety.
[0002] Field The present disclosure relates to control systems, and more particularly to control systems that operate in real-time using database connectivity. [Background technology]
[0003] background Currently, various control systems exist with individualized sets of capabilities that are not integrated into one system and made to work together as a single system. For example, a traditional control system workflow may enable motion editing through a system integrated with a programmable logic controller (PLC) that requires exporting and importing motion editing data. These traditional control systems do not allow real-time editing and may require multiple software packages to provide the target device with the ability to edit the required motion parameters. In addition, traditional control systems do not provide the ability to visualize and interact with the target device in real-time. Even if the subsystems can work together, the connections between the subsystems require additional work, slowing the workflow and the time to iterate can take hours or longer. Summary of the Invention [Means for solving the problem]
[0004] The present disclosure addresses these challenges and issues, whereby a control system may include two databases with the ability to synchronize during programming to enable real-time visualization, interactivity, and kinematic modeling of the target device.
[0005] overview In accordance with one or more embodiments, a robot animation system for controlling a robot with real-time control is provided. The system includes a user computing device having a user interface, a user motion database communicatively coupled to the user computing device, a controller, a controller motion database communicatively coupled to the controller, and a target device communicatively coupled to the controller. The user computing device can be configured to connect the user motion database and the controller motion database to share corresponding sets of motion instructions in real-time in response to receiving a synchronization instruction from the user computing device. The target device can be configured to implement the corresponding sets of motion instructions on the target device in real-time.
[0006] According to another embodiment, a method for controlling a robot in real time is provided. The method can comprise receiving a synchronization instruction at a user motion database from a user computing device, the user computing device comprising a user interface. The method can also comprise connecting the user motion database and the controller motion database to share corresponding sets of motion instructions in real time. The method can further comprise implementing the corresponding sets of motion instructions shared as a result of the connecting on a target device in real time via the controller.
[0007] Further areas of applicability of the present disclosure will become apparent from the detailed description, claims, and drawings. The detailed description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the disclosure. [Brief description of the drawings]
[0008] The present disclosure will become more fully understood from the detailed description and the accompanying drawings.
[0009] [Figure 1] FIG. 1 is an example user interface of a robot animation system according to some embodiments.
[0010] [Diagram 2] FIG. 2 is a block diagram of an exemplary computing device according to some embodiments.
[0011] [Diagram 3] FIG. 3 is an example functional block diagram of a robot animation system according to some embodiments.
[0012] [Figure 4] FIG. 4 is an exemplary user interface for controlling the motion of a robot through a robot animation system according to some embodiments.
[0013] [Diagram 5] FIG. 5 is an exemplary flowchart of a method of operating a robotic animation system according to some embodiments.
[0014] In the drawings, reference numbers may be reused to identify similar and / or identical elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0015] Detailed Description A control system, such as a robot animation system, enables life-like motion to be created on a robot through a series of interconnected subsystems. More specifically, the robot animation system implements multiple databases, including a user motion database and a controller motion database, to enable user modifications to the robot instructions before synchronizing the databases and then providing life-like and continuous motion of the robot. Furthermore, the robot animation system enables an improved workflow that is not only iterative but also fast.
[0016] That is, the method of providing a user computing device with a user interface communicatively coupled to a user motion database allows a user or animator to add, edit, or otherwise update controller instructions to the intended motion of a robot and have those changes immediately reflected in the robot's user interface and the robot itself. A robot animation system may include multiple modules or subsystems with various parameters, such as, for example, the type of movement, timing, and speed of the robot's motion, that a user can modify on the user computing device to adjust or change any robot motion to a desired look, angle, form, etc.
[0017] While this disclosure is directed to a specific system for controlling the motion of a robot, the robot animation system is generally directed to gathering information from a user, allowing that information to be modified in real time, and then applying that information to the robot by implementing the motion on the robot. Thus, any target device other than a robot that uses a real-time data stream with configurable parameters that can be inserted can be used.
[0018] Furthermore, the robot animation system is designed to program the motions of the robot in real time. As mentioned above, the robot animation system comprises at least two databases that are kept synchronized during programming. These databases comprise joint space motion data for the animated figure (e.g., robot, etc.). The controller motion database communicatively coupled to the controller provides or stores instructions for operation, which instruct what motions to perform, in what sequence, and at what point in time, allowing the robot to operate smoothly without being affected by any potential delays on the user computing device. In various implementations, the user may select a particular motion for a set of motions, such as moving the robot's head or moving the robot's arm, or the user may select a corresponding item or part of the robot along with the timing of adding the motion to the sequence of motions. Furthermore, the user may sequence various motions, such as moving the arm or moving the head, along a timeline, and drag and drop motions before or after other motions.
[0019] For example, at the beginning of an operating or programming session, a user may connect to the controller motion database by synchronizing with the controller motion database and connecting to a number of modules or stored data for operating the real-time system. Synchronizing with the controller motion database opens a bidirectional communication path between the user system and the robot system that operates in real time. The user may make edits to either the robot motion instructions or directions on a user interface of the robot motion profile displayed on the user computing device of the robot. The user interface provides a number of user interface elements that are selectable by the user and designed to correspond to different types and angles of motion for different parts or components of the robot. The user interface may also include a user interface element that corresponds to the speed of the robot motion or a timeline of a sequence of the robot motion. For example, a user may select a user interface element that corresponds to a particular motion to adjust which part of the robot performs the motion or at what point in time a particular motion occurs in relation to other motions.
[0020] Thereafter, when a user desires to have a motion reflected or implemented by the robot, the user may select a user interface element on the robot animation application or program to instantly synchronize the user motion database with the controller motion database, the synchronization being performed using packed data or at least one packet data or command message across the distributed communication system or distributed network. These commands may comprise frame numbers corresponding to specific motions at specific times along a timeline of a sequence of motions. Once the two databases are in sync, various control modes become available for implementation to move the robot. The control modes may comprise a play mode, a go to specific point mode, etc., as described further below. The various control modes are user selectable as icons on the user interface of the robot animation application and target specific points in the motion profile of the command. The control modes may also target the robot and / or the program including implementing or implementing the motion at full speed on the robot on the user interface.
[0021] When using the play mode, the user interface may send a command to the controller that sends the actual motion command to the robot, starts playing or implementing the received motion command, and starts listening for the current frame number. When the controller receives the play command, the controller commands the robot to start executing the animation and sends the current frame number to the user interface that depicts the actual real-time motion of the robot to the user. By receiving the current frame number, the user interface can display the actual motion of the robot in sync with the controller that commands the robot to perform the motion. When using the go to point mode, the user interface commands the controller in real time which frame to implement. That is, the user operating the user computing device can command the robot in real time to perform a specific motion related to a specific frame through the user interface of the robot animation application.
[0022] A user interface model of the robot is displayed on the user interface of the user computing device to provide the user with a way to program the motion of the robot, for example by defining specific position or motion parameters for different frames, as well as to display the real-time motion of the robot to the user. The robot animation system may have a variety of different user interfaces with different features provided to the user for robot control and robot display on the robot animation application.
[0023] In various implementations, the robot animation application provides a three-dimensional visual representation of the robot (e.g., as shown in FIG. 1) and allows a rig to be built within the robot animation application, which provides a method of robot manipulation. Additionally or alternatively, the user interface may be simpler, with capabilities such as recording data points of position information that may include few or many different position parameters. The robot animation system is specifically targeted to providing a user with a way to view the robot motion and to direct the motions the robot should perform, with the user interface designed to be separate from other parts of the robot animation system.
[0024] In various implementations, the same model is created for use in the real-time controller. These two models allow the system to be programmed into the space used in the user interface, which is then replayed and converted into the space used by the robot's actuators to implement motions for the robot as commanded by the controller to move. The model implemented in the real-time controller converts data between the spaces at the same rate that the controller is operating.
[0025] The robot animation system may also include a set of real-time controller settings that affect the motion of the robot. These settings are variables used in equations to create the motion of the robotic figure on the user interface in various modes. Exemplary settings may include a gain multiplier in the motion proportional-integral (PI) loop, an initial direction of homing, and an expected count during homing. For example, more than 100 individual settings exist for each motor. These settings can be modified to individually change the behavior of each axis even while the robot is in motion (see, for example, FIG. 4). These settings allow for shaping each actuator response to a motion profile. The described settings can be edited not only through the application programming interface (API) of the controller, but also by the user interface. That is, for example, the controller may include multiple robot control options as well as an interface for manipulation by the operator. Thus, the described API may allow the settings of the robot to be modified during motion to fit various needs from the robot.
[0026] In various embodiments, the robot animation system provides an external visualization tool that is used to view and potentially control the robot. Some existing systems, such as Maya, Cinema4D, and Blender, naturally support this. Others (such as lighting systems) do not have this support and therefore another tool, such as Sim Mechanics, Unity, Unreal, must be used to provide the external visualization. However, the external visualization is integrated into all aspects of the robot animation system, allowing anyone on the network (which may be local or across a distributed communication system) to visualize the location of all the joints of the robot in real time. In general, an accurate representation of the doll that can be viewed from any angle is very useful, especially when access to the doll is limited. In various implementations, the robot animation system uses network communication along with packed data to send the position of each joint through the network to the user computing device and potentially other network devices as the position is changed.
[0027] The robot animation system is configured to allow interactivity between the robot and user input, as well as pre-programmed data. In various implementations, the robot animation system includes features that allow external data streams to be used with each motion. That is, external data can be used to affect the motion of the robot. For example, motion data is presented to the robot animation system as position data, where the position data is expressed as a 32-bit floating point number between 0 and 1. These data streams can be received from various sources, such as joysticks, artificial intelligence (AI), gaming systems, and others. In implementations with interactive systems, the sensor data can be used to create on-the-fly motion profiles, where this data can be blended or transitioned into and out of the motion profile data.
[0028] As mentioned above, the method of storing, synchronizing, and using the received and stored data implemented in the user and controller motion databases is unique compared to any existing system. For example, using an exact copy of data in two places (databases) that can be instantly synchronized and then used to keep the user interface robot and the robot moving simultaneously is new and unique to a robot animation system. As mentioned above, synchronization is initiated by the user at any time the user chooses (e.g., by selecting a button or user interface element labeled "synchronize"). For example, the user may implement synchronization when he or she has finished editing on the user interface of the robot animation application and wants to see the new edited motion of the robot.
[0029] Due to the speed and ease with which a user can implement synchronization, synchronization allows even small changes to be immediately visible. The ability to synchronize is part of a robot animation system that provides the ability to produce the best possible performance on a robot. Furthermore, the robot animation system can implement synchronization to provide visualization of the robot through the puppet and implemented on the robot, which allows interactivity to be easily integrated into the robot animation system and further allows motion parameters to be edited in real time by the user and the API. It will be apparent to those skilled in the art that the robot animation system of the present disclosure has advantages over other systems used to control robots. For example, the robot animation system allows a single system to control and display a robot in real time through synchronization between the user and the controller motion database.
[0030] Furthermore, the robot animation system supports a workflow in which a significant amount of the underlying science and technology can be varied while still achieving the same results. For example, the user motion database can be implemented using Autodesk Maya (which can be substituted with Blender or a proprietary program) as the primary data store, and the functionality of the robot animation system will still be the same.
[0031] The robot animation system defines a workflow that allows users to work with familiar tools that have features that allow them to quickly and efficiently perform the tasks required to adjust or update the robot's motion. As mentioned above, the capabilities of the robot animation system include the ability to instantly view edits on the puppet and robot in the user interface, the ability to move to a frame with a single click, and the ability to scrub the timeline live on the puppet. When the creative workflow is finished, the data in the robot animation system is already updated and the puppet is ready to play. This is because the user motion database and the controller motion database are updated during the synchronization step. The robot animation system also obtains all the parameters required to shape the robot's motion and puts them into a single easy-to-use tool. Furthermore, additional tools such as visualization, interactivity, and real-time kinematic modeling and models are excluded from other systems.
[0032] 1, an exemplary user interface 100 of a robot animation system according to some embodiments is shown. As depicted, the exemplary user interface 100 shows a puppet robot 101 in an exemplary modification application being operated on a user computing device. That is, the modification application is visible to a user, in which the user can see the real-time motion of the robot through the puppet robot 101, and by using various toolbars (e.g., toolbar 102, etc.) on the exemplary user interface 100, the user can modify and update any desired motions to command the robot.
[0033] Referring to Figure 2, a block diagram of an exemplary computing device 200 is shown in accordance with some embodiments. Figure 2 illustrates an example computing device 200 that can be used by the disclosed system or to perform the disclosed methods. A computing device 200, such as a user computing device or controller of Figure 3, can, for example, implement one or more of the functions described herein. However, it should be understood that other computing device configurations are possible.
[0034] Computing device 200 may include one or more processors 201, memory 202, one or more input / output devices 203, a transceiver 204, one or more communication ports 207, and a display 206, all operatively coupled to one or more data buses 208. Data bus 208 enables communication between various devices. Data bus 208 may include wired or wireless communication channels. Data bus 208 connects one or more devices.
[0035] Processor 201 may comprise one or more distinct processors, each having one or more cores. Each distinct processor may have the same or different architecture. Processor 201 may comprise one or more central processing units (CPUs), one or more graphics processing units (GPUs), application specific integrated circuits (ASICs), digital signal processors (DSPs), and the like.
[0036] The processor 201 may be configured to perform a function or operation by executing code, which is code stored on an instruction memory and that implements the function or operation. For example, the processor 201 may be configured to perform one or more of any of the functions, methods, or operations disclosed herein.
[0037] The memory 202 may comprise an instruction memory that may store instructions that may be accessed (e.g., read) and executed by the processor(s) 201. For example, the instruction memory may be a non-transitory computer-readable storage medium, such as a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a removable disk, a CD-ROM, any non-volatile memory, or other suitable memory. For example, the instruction memory may store instructions that, when executed by the one or more processors 201, cause the one or more processors 201 to perform one or more of the functions of the image reconstruction system.
[0038] The memory 202 may also comprise a working memory. The processor 201 may store data in the working memory and read data from the working memory. For example, the processor 201 may store a working set of instructions, e.g., instructions loaded from an instruction memory, in the working memory. The processor 201 may also use the working memory to store dynamic data that is created during operation of the computing device 200. The working memory may be a random access memory (RAM), e.g., a static random access memory (SRAM) or a dynamic random access memory (DRAM), or other suitable memory.
[0039] The input / output device(s) 203 may comprise any suitable device that allows for data input or output. For example, the input / output device(s) 203 may comprise one or more of a keyboard, a touchpad, a mouse, a stylus, a touch screen, physical buttons, a speaker, a microphone, or other suitable input or output device.
[0040] The communications port 207 may comprise, for example, a serial port, such as a universal asynchronous receiver-transmitter (UART) connection, a universal serial bus (USB) connection, or other suitable communications port or connection. In some examples, the communications port 207 allows for the programming of executable instructions in an instruction memory. In some examples, the communications port 207 allows for the transport (e.g., uploading or downloading) of data.
[0041] The display 206 can display the user interface 205. The user interface 205 can enable user interaction with the computing device 200. In some examples, a user can interact with the user interface 205 by engaging with the input / output devices 203. In some examples, the display 206 can be a touch screen, and the user interface 205 is displayed on the touch screen.
[0042] The transceiver 204 may enable communication with a network, such as a Wi-Fi network, an Ethernet network, a cellular network, or other suitable communication network. For example, the transceiver 204 is configured to enable communication with the cellular network when operating on the cellular network. The processor 201 is operable to receive data from the network or send data to the network via the transceiver 204.
[0043] Referring to FIG. 3, an exemplary functional block diagram of a robot animation system 300 according to some embodiments is shown. The robot animation system 300 includes a user computing device 304, such as a handheld computing device, laptop, tablet, desktop, etc., that can operate a robot animation application through which a user can modify and view the motion of a target device 306, such as the robot illustrated in FIG. 3. As mentioned before, the target device 306 includes a plurality of actuators located at a plurality of positions 308a-308f throughout the target device 306 to perform or execute movements. While only six actuators and positions are shown, it can be understood that a plurality of actuators can be located anywhere on the target device 306 that can be commanded to move and is not limited to the example shown in FIG. 3.
[0044] The user computing device 304 updates the user motion database 310 with any changes to the motion information for commanding the target device 306. As a result of the synchronization, two-way communication between the user motion database 310 and the controller motion database 312 (e.g., a robot motion database, etc.) is initiated to allow the motion parameter module 316, the motion data parameter module 320, the visualization module 324, the interactivity module 328, and the control command module 332 to also be synchronized between the user computing device 304 and the controller 336 (e.g., a robot controller, etc.). The user computing device 304 and the controller 336 are synchronized in response to a user selection to allow real-time visualization of the doll and the target device 306 on a user interface (e.g., user interface 205, etc.) of the user computing device 304, the visualization resulting from the controller 336 outputting control signals to the target device 306 as instructed in the user motion database 310 as a result of the synchronization.
[0045] For example, the control command module 332 may comprise specific information regarding which frames to follow with specific motion instructions from the motion parameter module 316 and / or the motion data parameter module 320, which dictate which motion to implement. In various implementations, the visualization module 324 enables real-time feedback to the user computing device 304 for the user to visualize the motion of the target device 306. Additionally, the interactivity module 328 enables real-time user input via the user computing device 304 (e.g., through the input / output device 203 and / or the display 206) to control the target device 306.
[0046] 4, an exemplary user interface 400 for controlling the motion of a robot through a robot animation system according to some embodiments is shown. For example, the exemplary user interface 400 depicts an example robot animation application, in which the motion parameters of the robot 401 can be viewed and set through the example robot animation application. A user can adjust various motions at various points on the robot and at various times. The user interface 400 can also provide indications 402 of variations between the commanded motion settings (e.g., position and velocity, etc.) of the robot 401 versus the actual motion of the robot 401. In some embodiments, the user interface 400 can also include fault indications 403. These fault indications 403 can comprise faults to or from the robot 401 and / or other components of the robot animation system (e.g., the user motion database 310 or the controller motion database 312, etc.). The user interface 400 can also comprise play interface elements 404 to allow a user to start and stop the robot 401.
[0047] 5, a flowchart of an exemplary method 500 of operating the robot animation system 300 according to some embodiments is shown. The method 500 begins at step 502. At step 504, a synchronization instruction is received at the user motion database 310 from the user computing device 304, which may further comprise the user interface 100. At step 506, the user motion database 310 and the controller motion database 312 are connected to share corresponding sets of motion instructions in real time. At step 508, the target device 306 implements the corresponding sets of motion instructions shared as a result of the connection in real time via the controller 336. The method 500 ends at step 510.
[0048] In some embodiments, the method 500 can comprise displaying the doll 101 on the user interface 100 of the user computing device 304 and performing movements corresponding to the corresponding set of implemented motion instructions in real time using the doll 101. In further embodiments, the method 500 can comprise receiving user input at the user computing device 304 and updating the corresponding set of motion instructions in real time in response to a synchronization instruction being sent from the user computing device 304. In some embodiments, the method 500 can comprise retrieving a set of motion instructions from a user motion database 310, the set of motion instructions having been previously uploaded and stored in the user motion database 310. The method 500 can then generate and display user-selectable user interface elements (e.g., elements found in the toolbar 102, etc.) on the user interface 100 of the user computing device 304 corresponding to each motion of the set of motion instructions. In some embodiments, the method 500 can further comprise implementing the corresponding set of motion instructions on the target device 306 in real time in response to receiving a play instruction from the user computing device 304.
[0049] The system includes a user computing device having a user interface, a user motion database communicatively coupled to the user computing device, a controller, a controller motion database communicatively coupled to the controller, and a target device communicatively coupled to the controller. The user computing device is configured to connect the user motion database and the controller motion database for sharing corresponding sets of motion instructions in real time in response to receiving a synchronization indication from the user computing device. The target device is configured to implement the corresponding sets of motion instructions on the target device in real time.
[0050] In some embodiments, a doll is displayed on a user interface of a user computing device, the doll being configured to perform movements in real time that correspond to a corresponding set of implemented motion instructions.
[0051] In some embodiments, the user computing device is configured to receive user input and update the corresponding set of motion instructions in real-time in response to a synchronization indication being sent from the user computing device.
[0052] In some embodiments, the user computing device is configured to retrieve the set of motion instructions from a user motion database, the set of motion instructions having been previously uploaded and stored in the user motion database, and the user computing device may generate and display a plurality of user-selectable user interface elements on a user interface of the user computing device corresponding to each motion in the set of motion instructions.
[0053] In some embodiments, the target device is configured to implement a corresponding set of motion instructions in real time on the target device in response to receiving play instructions from a user computing device.
[0054] In some embodiments, the corresponding set of motion instructions is implemented on the target device by the user computing device sending a set of frames to the target device in a predetermined order.
[0055] In some embodiments, each motion in the set of motion instructions comprises (i) instructions to perform the motion via the doll and (ii) instructions to perform the motion on the target device.
[0056] In some embodiments, the target device comprises multiple actuators for performing movements on the target device.
[0057] In some embodiments, the synchronization indication is sent in response to a selection of a user-selectable interface element displayed on a user interface of a user computing device.
[0058] In some embodiments, the controller receives a set of motion instructions from a controller motion database, and the controller selects a set of motion instructions to send to the target device that relate to instructions for performing movements on the target device.
[0059] In some embodiments, the method comprises receiving a synchronization indication at a user motion database from a user computing device, the user computing device comprising a user interface. The method also comprises connecting the user motion database and the controller motion database to share corresponding sets of motion instructions in real time. The method further comprises implementing the corresponding sets of motion instructions shared as a result of the connecting on the target device via the controller in real time.
[0060] In some embodiments, the method comprises displaying a doll on a user interface of a user computing device and performing in real time movements using the doll that correspond to a corresponding set of implemented motion instructions.
[0061] In some embodiments, the method comprises receiving user input at a user computing device and updating a corresponding set of motion instructions in real time in response to a synchronization indication being sent from the user computing device.
[0062] In some embodiments, the method comprises obtaining a set of motion instructions from a user motion database, the set of motion instructions having been previously uploaded and stored in the user motion database, and generating and displaying a plurality of user-selectable user interface elements on a user interface of a user computing device corresponding to each motion in the set of motion instructions.
[0063] In some embodiments, the method comprises implementing a corresponding set of motion instructions in real time on the target device in response to receiving play instructions from a user computing device.
[0064] In some embodiments of the method, the corresponding set of motion instructions is implemented on the target device by the user computing device sending a set of frames to the target device in a predetermined order.
[0065] In some embodiments of the method, each motion in the set of motion instructions comprises (i) instructions to perform the motion via the doll and (ii) instructions to perform the motion on the target device.
[0066] In some embodiments of the method, the target device comprises a plurality of actuators for performing movements on the target device.
[0067] In some embodiments of the method, the synchronization indication is sent in response to a selection of a user-selectable interface element displayed on a user interface of the user computing device.
[0068] In some embodiments of the method, the controller receives a set of motion instructions from a controller motion database, and the controller selects a set of motion instructions related to instructions for performing movements on the target device for sending to the target device.
[0069] Although the methods described above involve reference to flowcharts, it will be understood that many other ways of performing the acts involved in the methods may be used. For example, the order of some of the operations may be changed and some of the operations described may be optional.
[0070] In addition, the methods and systems described herein can be at least partially implemented in the form of computer-implemented processes and apparatuses for practicing those processes. The disclosed methods can also be at least partially implemented in the form of a tangible non-transitory machine-readable storage medium encoded with computer program code. For example, the steps of the method can be implemented in hardware, executable instructions (e.g., software) executed by a processor, or a combination of the two. The medium may comprise, for example, a RAM, a ROM, a CD-ROM, a DVD-ROM, a BD-ROM, a hard disk drive, a flash memory, or other non-transitory machine-readable storage medium. When the computer program code is loaded into a computer and executed by the computer, the computer becomes an apparatus for practicing the method. The method can also be at least partially implemented in the form of a computer into which the computer program code is loaded or executed, thereby making the computer a special-purpose computer for practicing the method. When implemented on a general-purpose processor, the computer program code segments configure the processor to create specific logic circuits. Alternatively, the method can be at least partially implemented in an application-specific integrated circuit for performing the method.
[0071] In this application with the definitions below, the term "module" or the term "controller" may be substituted for the term "circuitry." The term "module" may refer to, be a part of, or comprise processor hardware (shared, specialized, or a group) that executes code and storage hardware (shared, specialized, or a group) that stores code executed by the processor hardware.
[0072] The module may include one or more interface circuits. In some examples, the interface circuit may implement a wired or wireless interface to connect to a local area network (LAN) or a wireless personal area network (WPAN). Examples of LANs are the Institute of Electrical and Electronics Engineers (IEEE) standard 802.11-2016 (also known as the WIFI wireless networking standard) and the IEEE standard 802.3-2015 (also known as the ETHERNET wired networking standard). Examples of WPANs are the BLUETOOTH wireless networking standard from the Bluetooth Special Interest Group and IEEE standard 802.15.4.
[0073] Modules may communicate with other modules using interface circuits. Although modules may be depicted in this disclosure as logically communicating directly with other modules, in various implementations, modules may actually communicate through a communication system. A communication system may comprise physical and / or virtual networking equipment, such as hubs, switches, routers, and gateways. In some implementations, a communication system may connect to or route through a wide area network (WAN), such as the Internet. For example, a communication system may comprise multiple LANs that connect to each other through the Internet or point-to-point leased lines using technologies including Multiprotocol Label Switching (MPLS) and Virtual Private Networks (VPNs).
[0074] In various implementations, the functionality of a module may be distributed among multiple modules connected via a communication system. For example, multiple modules may implement the same functionality that is distributed by a load balancing system. In a further example, the functionality of a module may be split between a server module (also known as remote or cloud) and a client (or user) module.
[0075] The foregoing is provided for the purposes of illustrating, explaining, and describing embodiments of these disclosures. Modifications and adaptations to these embodiments will be apparent to those skilled in the art, and may be made without departing from the scope or spirit of these disclosures.
Claims
1. a user computing device having a user interface; a user motion database communicatively coupled to the user computing device; A controller; a controller motion database communicatively coupled to the controller; a target device communicatively coupled to the controller; A system comprising: The user computing device comprises: configured to connect the user motion database and the controller motion database to share corresponding sets of motion instructions in real time in response to receiving a synchronization indication from the user computing device; the target device is configured to implement the corresponding set of motion instructions on the target device in real time. system.
2. 2. The system of claim 1, wherein a doll is displayed on the user interface of the user computing device, the doll being configured to perform movements in real time corresponding to a corresponding set of implemented motion instructions.
3. 2. The system of claim 1, wherein the user computing device is configured to receive user input and update the corresponding set of motion instructions in real-time in response to the synchronization indication being sent from the user computing device.
4. The user computing device comprises: obtaining the set of motion instructions from the user motion database, the set of motion instructions having been previously uploaded and stored in the user motion database; and generating and displaying a plurality of user-selectable user interface elements on the user interface of the user computing device corresponding to each motion in the set of motion instructions; The system of claim 1 configured to:
5. The system of claim 1 , wherein the target device is configured to implement the corresponding set of motion instructions on the target device in real time in response to receiving play instructions from the user computing device.
6. The system of claim 5 , wherein the corresponding set of motion instructions is implemented on the target device by the user computing device sending a set of frames to the target device in a predetermined order.
7. The system of claim 1 , wherein each motion in the set of motion instructions comprises: (i) an instruction to perform a movement via the doll; and (ii) an instruction to perform a movement on the target device.
8. The system of claim 1 , wherein the target device comprises a plurality of actuators for performing movements on the target device.
9. The system of claim 1 , wherein the synchronization indication is transmitted in response to a selection of a user-selectable interface element displayed on the user interface of the user computing device.
10. The system of claim 1 , wherein the controller receives the set of motion instructions from the controller motion database, and the controller selects the set of motion instructions to send to the target device that relate to instructions for performing movements on the target device.
11. receiving, at a user motion database, a synchronization indication from a user computing device, the user computing device comprising a user interface; connecting said user motion database and a controller motion database to share corresponding sets of motion instructions in real time; implementing in real time on a target device via a controller the corresponding set of motion instructions shared as a result of said connecting; The method includes:
12. displaying a doll on the user interface of the user computing device; and performing, in real time, movements corresponding to the corresponding set of implemented motion instructions using the doll; The method of claim 11 further comprising:
13. 12. The method of claim 11, further comprising: receiving a user input at the user computing device; and updating the corresponding set of motion instructions in real time in response to the synchronization indication being sent from the user computing device.
14. obtaining the set of motion instructions from the user motion database, the set of motion instructions having been previously uploaded and stored in the user motion database; and generating and displaying a plurality of user-selectable user interface elements on the user interface of the user computing device corresponding to each motion in the set of motion instructions; The method of claim 11 further comprising:
15. The method of claim 11 , further comprising: implementing the corresponding set of motion instructions in real time on the target device in response to receiving play instructions from the user computing device.
16. The method of claim 15 , wherein the corresponding set of motion instructions is implemented on the target device by the user computing device sending a set of frames to the target device in a predetermined order.
17. The method of claim 11 , wherein each motion in the set of motion instructions comprises: (i) an instruction to perform a movement via the doll; and (ii) an instruction to perform a movement on the target device.
18. The method of claim 11 , wherein the target device comprises a plurality of actuators for performing movements on the target device.
19. The method of claim 11 , wherein the synchronization indication is sent in response to a selection of a user-selectable interface element displayed on the user interface of the user computing device.
20. 12. The method of claim 11, wherein the controller receives the set of motion instructions from the controller motion database, and the controller selects the set of motion instructions to send to the target device that relate to instructions for performing movements on the target device.
Citation Information
Patent Citations
Mechanical arm motion control method based on simulation data
CN107443379A
Operation method for activation of home robot device and home robot device supporting the same
US20180178372A1
Autonomous Robotic Aide
US20180370028A1
Entertaining mobile application for animating a single image of a human body and applying effects
US20200410735A1
