Robot development system and method
By combining virtual environment training and physical testing, the cerebellum controller was optimized, solving the complexity problem in the development of robot cerebellum control systems and enabling rapid and efficient robot development.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- DELTA ELECTRONICS INC(CN)
- Filing Date
- 2025-06-12
- Publication Date
- 2026-07-23
Smart Images
Figure CN2025100570_23072026_PF_FP_ABST
Abstract
Description
Robot Development Systems and Methods Technical Field
[0001] This invention relates to robot development systems and methods, and more particularly to the development of a cerebellum control system for robots. Background Technology
[0002] A robot's cerebellum control system is a control architecture that mimics the function of a biological cerebellum. It is primarily responsible for motion coordination, action smoothing, and error correction. It receives real-time feedback from sensors (such as joint angles, acceleration, and torque), compares this feedback with the expected action, and dynamically adjusts the control output based on the error to ensure more stable and flexible robot movements. Cerebellum control systems often incorporate machine learning algorithms (such as reinforcement learning (RL) or feedforward neural networks (FNN)) to enable them to adapt to different tasks and environmental changes.
[0003] However, the development of cerebellar control systems is quite complex because they need to simulate the high degree of integration and adaptability of the biological cerebellum in motion coordination, error correction, and motor learning. Generally, the development process for a cerebellar control system includes: establishing a learnable control architecture (e.g., feedforward and feedback loops), designing error-based adjustment strategies (e.g., reinforcement learning or adaptive control), and repeatedly training the system in simulated environments or on physical platforms to improve its ability to perceive and correct motion errors. Furthermore, development also involves technologies such as sensor data processing, motion output, and parameter optimization.
[0004] Overall, the development of cerebellar control systems faces multiple challenges, including: (1) mathematical modeling and control theory design; (2) establishing a learning architecture (such as reinforcement learning or neural networks) capable of handling multiple sensor inputs and nonlinear motion outputs; (3) ensuring the system has the ability to react instantly and adjust dynamically; and (4) balancing factors such as the accuracy of the physical model and the error of the physical entity. These factors together significantly increase the difficulty of developing cerebellar control systems.
[0005] Therefore, a robot development system and method are needed to solve the above problems. Summary of the Invention
[0006] The disclosed embodiments provide a robot development system, including a cerebellum controller, a virtual component module, a virtual construction module, a virtual testing module, a compilation module, and a physical construction module. The virtual component module is configured to store multiple virtual components and at least one physical model corresponding to each virtual component. The virtual construction module is configured to construct a virtual robot using the cerebellum controller and the physical model corresponding to a selected virtual component. The virtual testing module is configured to train the virtual robot in a virtual environment to optimize the cerebellum controller's control over the virtual components. The compilation module is configured to compile the trained cerebellum controller of the virtual robot into firmware. The physical construction module is configured to construct a physical robot based on the virtual components used by the trained virtual robot and to deploy the firmware to the physical robot.
[0007] The disclosed embodiments provide a robot development method executed in a computer system, the computer system including a cerebellum controller. The robot development method includes: storing a plurality of virtual components and at least one physical model corresponding to each virtual component; constructing a virtual robot using the cerebellum controller and the physical models corresponding to selected virtual components; training the virtual robot in a virtual environment to optimize the control of the cerebellum controller for the virtual components; compiling the trained cerebellum controller of the virtual robot into firmware; constructing a physical robot based on the virtual components used by the trained virtual robot; and deploying the firmware to the physical robot.
[0008] In one embodiment, the robot development system further includes a physical testing module configured to: perform physical testing on the physical robot in a physical environment; determine whether the physical robot passes the physical test; and, in response to the physical robot failing the physical test, drive the virtual testing module to retrain the virtual robot.
[0009] In one embodiment, the physical testing module is further configured to acquire physical sensing data collected by the physical robot during physical testing, for use by the virtual testing module to retrain the virtual robot.
[0010] In one embodiment, the firmware includes a robotic operating system, and the physical robot transmits physical sensing data through the robotic operating system using an application program interface (API).
[0011] In one embodiment, the compilation module is further configured to recompile the cerebellum controller of the retrained virtual robot to obtain an updated version of the firmware; and the entity building module is further configured to transfer the updated version of the firmware to the physical robot via an application interface.
[0012] In one embodiment, the virtual testing module is further configured to retrain the virtual robot based on scene information and entity sensing data from the physical environment.
[0013] In one embodiment, the entity sensing data includes acceleration values, pressure values, and joint angle values.
[0014] In one embodiment, the physical testing module is further configured to determine whether the physical robot passes the physical test based on a motion test index, the motion test index being related to the task performed by the physical robot in the physical test.
[0015] In one embodiment, the robot development system further includes an input module configured to: receive user-defined virtual components; convert the file format of the user-defined virtual components to conform to the virtual environment; and store the converted virtual components in a virtual component module.
[0016] In one embodiment, the virtual component includes sensors, motors, servos, and joint modules.
[0017] The robot development system and method disclosed herein can accelerate the development of robots with different configurations or performing different tasks. Furthermore, by pre-installing components and a cerebellar control system, and by connecting automated compilation and application interfaces, the number of operations required to develop various robots can be reduced, thereby shortening the robot development time. Attached Figure Description
[0018] Figure 1 is a system architecture diagram of a robot development system according to an embodiment of the present invention;
[0019] Figure 2 is a data flow diagram of a robot development method according to an embodiment of the present invention;
[0020] Figure 3 is a data flow diagram of a robot retraining method according to an embodiment of the present invention.
[0021] Figure Number Explanation: 10: Robot Development System; 11: Virtual Component Module; 12: Virtual Construction Module; 13: Virtual Testing Module; 14: Cerebellum Controller; 15: Compilation Module; 16: Entity Construction Module; 17: Entity Testing Module; 20: Robot Development Methodology; 201-205: Steps; 30: Robot Retraining Methodology; 301-305: Steps; 21: Virtual Component; 22: Virtual Robot; 23: Firmware; 24: Entity Robot; 31: Entity Sensing Data Detailed Implementation
[0022] The following description illustrates various embodiments of the present invention, but is not intended to limit the scope of the invention. The actual scope of the invention is defined by the scope of the claims.
[0023] In the embodiments listed below, the same or similar elements or components will be represented by the same reference numerals.
[0024] The serial numbers in this specification and in the scope of the claims, such as "first", "second", etc., are for convenience only and there is no sequential relationship between them.
[0025] The descriptions of embodiments of the apparatus or system in this specification also apply to embodiments of the method, and vice versa.
[0026] Figure 1 is a system architecture diagram of a robot development system 10 according to an embodiment of the present invention. As shown in Figure 1, the virtual machine platform construction system 10 includes a virtual component module 11, a virtual construction module 12, a virtual testing module 13, a cerebellum controller 14, a compilation module 15, and an entity construction module 16.
[0027] In one embodiment, the robot development system 10 may be a computer system implemented using a general-purpose processing unit or dedicated hardware circuitry, such as a personal computer (e.g., a desktop computer or a laptop computer), a server computer, or a mobile device such as a tablet computer or a smartphone, but this disclosure is not limited thereto.
[0028] In one embodiment, the robot development system 10 may include a processing unit. The processing unit may include any one or more general-purpose or special-purpose processors and combinations thereof for executing instructions, such as a central processing unit (CPU) and / or a graphics processing unit (GPU). The processing unit may also include volatile memory, such as dynamic random-access memory (DRAM) and / or static random-access memory (SRAM), for temporarily storing processed data, but this disclosure is not limiting.
[0029] In one embodiment, any one of the virtual component module 11, virtual construction module 12, virtual testing module 13, cerebellum controller 14, compilation module 15, and entity construction module 16 can be implemented by software or firmware, which can be loaded into the aforementioned volatile memory and executed by the aforementioned processing unit.
[0030] In another embodiment, any of the virtual component module 11, virtual build module 12, virtual test module 13, compilation module 15, and physical build module 16 may be implemented using an integrated circuit (IC), such as an application-specific integrated circuit (ASIC), a system on a chip (SoC), or a field-programmable gate array (FPGA), but this disclosure is not limited thereto.
[0031] In one embodiment, the cerebellum controller 14 can be implemented using an artificial intelligence (AI) model. The AI model can be a reinforcement learning (RL) algorithm, a deep neural network (DNN) algorithm, or a deep reinforcement learning (DRL) algorithm, but this disclosure is not limited thereto.
[0032] Reinforcement learning algorithms can include, for example, Q-learning, State-Action-Reward-State-Action (SARSA), or Evolutionary Strategies (ES). Deep neural network algorithms can include, for example, feedforward neural networks (FNN), convolutional neural networks (CNN), or recurrent neural networks (RNN). Deep reinforcement learning algorithms can include, for example, deep Q-network (DQN), proximal policy optimization (PPO), or soft actor-critic (SAC).
[0033] Figure 2 is a data flow diagram of a robot development method 20 according to an embodiment of the present invention, executed in a robot development system 10 (as shown in Figure 1). As shown in Figure 2, the robot development method 20 includes steps 201-205. Steps 201-205 are described below.
[0034] In step 201, the virtual component module 11 can store multiple virtual components 21 and their corresponding at least one physical model. Virtual components 21 may include rods, sensors, motors, servo motors, and joint modules, but this disclosure is not limited thereto. A physical model refers to a mathematical or physical equation used to describe the physical properties of a component, such as its mass, shape, inertia, rigidity, friction, and collision characteristics; it can realistically simulate the motion, interaction, and reaction of the component in a virtual environment.
[0035] In one implementation, the virtual component module 11 can be implemented using a relational database system (such as MySQL, PostgreSQL, SQLite, etc.). In another implementation, the virtual component module 11 can be implemented using a file system (such as CSV files, Excel files) or a NoSQL database (such as MongoDB), but this disclosure is not limited thereto.
[0036] Table 1 is an example of the content stored in the virtual component module 11. As shown in Table 1, each virtual component has a corresponding physical model.
[0037] Table 1
[0038] It is worth noting that by pre-setting or building some commonly used components and their corresponding physical models into the system, and providing them to users for direct selection, the steps required for users to find third-party physical models or build their own physical models can be reduced, thereby improving the efficiency of robot development.
[0039] However, the preset or built-in components may not include the components the user wants to use, or the user may want to use custom components. Therefore, in one embodiment, the robot development system 10 may further include an input module. The input module can receive user-defined virtual components and convert their file formats to conform to the format required for subsequent simulations. For example, the input module can receive computer-aided design (CAD) drawings of the components input by the user and convert the CAD drawings into a Unified Robot Description Format (URDF). Then, the input module can store the converted virtual components in the virtual component module 11.
[0040] Similarly, the input module can also receive user-defined physical models or third-party physical models, and convert the file formats of these physical models into the formats required for subsequent simulations.
[0041] In step 202, the virtual construction module 12 can use the cerebellum controller 14 and the physical model corresponding to the virtual component 21 selected by the user to construct a virtual robot 22. For example, joint modules, servo motors, reducers, sensors, grippers, etc., combined with the cerebellum controller 14 can construct a virtual robotic arm. As another example, joint modules, servo motors, inertial measurement units, foot pressure sensors, stereo cameras, etc., combined with the cerebellum controller 14 can construct a virtual humanoid robot.
[0042] In step 203, the virtual testing module 13 can train the virtual robot 22 in a virtual environment to optimize the control of the cerebellum controller 14 for the virtual components 21 of the virtual robot. The virtual environment refers to a digital environment established by simulation software that can imitate the physical rules and interactive behaviors of the real world. For example, interactive elements such as terrain, obstacles, and objects (such as balls and boxes) can be added, or parameters such as weather and light source can be adjusted for robot training, testing, or development.
[0043] The training of the virtual robot 22 may involve defining the task objectives, inputs and outputs, and setting the reward function. Task objectives may include, for example, "walking to a specific location," "picking up an item," or "avoiding obstacles." Inputs may be sensor data such as position, angle, image, and force. Outputs may be control commands such as motor speed and joint angles. The reward function guides the learning direction and may be, for example, "giving positive points for approaching the target and deducting points for hitting obstacles."
[0044] Next, the virtual robot 22 can repeatedly perform tasks (such as walking) in the virtual environment and adjust the control strategy for the virtual component 21 (such as adjusting neural network parameters, motor angles and speeds) based on the task performance results to maximize rewards. This task will be repeated until the control strategy gradually converges.
[0045] Table 2 provides an example of training the virtual robot 22 to walk forward. As shown in Table 2, by awarding positive scores for forward movement and stable gait, the virtual robot 22 can be guided to complete the task of "walking forward steadily." Therefore, even if the virtual robot 22 may receive lower rewards in the early stages of training due to random movements and falls, it can eventually achieve the task objective by continuously adjusting its control strategy (i.e., joint angles) and thus obtaining increasingly higher rewards.
[0046] Table 2
[0047] Table 3 provides an example of training the virtual robot 22 to grasp objects. As shown in Table 3, by awarding positive points for successfully grasping an object and negative points for the rest, the virtual robot 22 can be guided to complete the task of "grabbing objects".
[0048] Table 3
[0049] In step 204, when the virtual robot 22 has completed training (i.e. can complete the target task), the compilation module 15 can compile the cerebellum controller 14 into firmware 23.
[0050] In step 205, when the virtual robot 22 has completed training (i.e. can complete the target task), the entity building module 16 can build an entity robot 24 with reference to the virtual components 21 used by the virtual robot 22, and deploy the firmware 23 to the entity robot 24.
[0051] For example, a virtual robotic arm can include virtual joint modules, servo motors, reducers, sensors, grippers, and a cerebellum controller 14. Therefore, the physical construction module 16 can use physical joint modules, servo motors, reducers, sensors, grippers, etc., in conjunction with the aforementioned firmware to construct a physical robotic arm.
[0052] For example, a virtual humanoid robot may include virtual joint modules, servo motors, inertial measurement units, foot pressure sensors, stereo cameras, and a cerebellum controller 14. Therefore, the physical construction module 16 can use physical joint modules, servo motors, inertial measurement units, foot pressure sensors, and stereo cameras, along with the aforementioned firmware, to construct a physical humanoid robot.
[0053] It is worth noting that since the hardware architecture, drivers, or control interfaces used in the virtual environment and the physical platform may differ, users may need to cross-compile or rewrite the program, which are quite complex and time-consuming. Therefore, using the compilation module 15 can achieve the following benefits: (1) simplifying the migration process from the simulation environment to the physical platform, reducing manual modification and error setting, improving compilation and deployment efficiency, and accelerating the development cycle; (2) automatically adjusting the target hardware architecture and driver dependencies, improving software compatibility and stability; and (3) lowering the threshold for cross-platform development, allowing developers to focus more on control strategies and application implementation, and improving the overall productivity and reliability of robot development.
[0054] In one embodiment, firmware 23 may include a Robot Operating System (ROS), such as ROS2 and Isaac ROS. The robot operating system is not a traditional operating system, but rather a middleware layer running on platforms such as Linux. It provides many functions, including hardware abstraction, low-level device control, information transmission (e.g., exchange of data and control commands), software module encapsulation and reuse, and simulation, visualization, and debugging tools, making robot development more efficient and flexible.
[0055] It's worth noting that the robot operating system can run on various operating systems and hardware platforms, such as Linux and Windows, and is also compatible with different processor architectures (e.g., x86, ARM). This allows robot developers to develop and test robot programs on various hardware devices and easily port programs from one platform to another. This increases flexibility and applicability while reducing the cost and time of redevelopment.
[0056] Figure 3 is a data flow diagram of a robot retraining method 30 according to an embodiment of the present invention. In this embodiment, the robot development system may further include a physical testing module 17, which may be implemented by software, firmware or application-specific integrated circuits, but the present disclosure is not limited thereto.
[0057] As shown in Figure 3, the robot retraining method 30 includes steps 301-304. Steps 301-304 are described below.
[0058] In step 301, the physical testing module 17 can perform a physical test on the physical robot 24 in a physical environment and determine whether the physical robot 24 passes the physical test. If the physical robot 24 fails the physical test, the physical testing module 17 can notify the virtual testing module 13 to retrain the virtual robot 22.
[0059] A physical environment refers to a real-world space and conditions, including actual terrain, objects, obstacles, lighting, sound, weather, and the actual robots and sensors in operation. Testing or operating in a physical environment is subject to real physical limitations (such as friction, collisions, and material properties) and unpredictable factors (such as errors and interference). For example, for a robotic vacuum cleaner, the physical environment can include furniture, walls, carpets, and stairs to simulate a residential space; glass doors, reflective objects, and narrow passages to simulate an office environment; or moving obstacles (such as people or boxes) to simulate real-world interference.
[0060] In one embodiment, the physical testing module 17 can determine whether the physical robot passes the physical test based on a motion test index. This motion test index is not unique and constant, but depends on the task performed by the physical robot in the physical test. For example, the motion test index for a self-driving delivery robot can be "whether the delivery task is completed" or "whether the delivery task can be completed within a predetermined time". As another example, for a warehouse handling robot, it can be "whether the pickup and handling operation is completed" or "whether the pickup and handling operation is completed within a specified time".
[0061] In step 302, the physical testing module 17 can acquire and store the physical sensing data 31 collected during the physical testing from the physical robot 24. Then, the virtual testing module 13 can use the physical sensing data 31 to retrain the virtual robot 22.
[0062] The type of entity sensing data 31 will depend on the sensors equipped on the entity robot 24. In one embodiment, the entity robot 24 is equipped with an inertial measurement unit (IMU), a force-torque sensor (FSR), and an encoder, so the entity sensing data may include acceleration values, pressure values, and joint angle values, but this disclosure is not limited thereto.
[0063] In one embodiment, the sensor has limited computing power. In this case, the sensor can only transmit the entity sensing data 31 it collects back to the robot operating system of the entity robot 24. Then, the robot operating system of the entity robot 24 uses an application program interface (API) to transmit the entity sensing data 31 back to the entity testing module 17. The form of the application interface can be, for example, a publisher / subscriber, a client / server, or a parameter server, but this disclosure is not limited to these.
[0064] In another embodiment, the sensor has sufficient computing power. In this case, the sensor can be installed on a robotic operating system and use the application interface supported by the robotic operating system to transmit the entity sensing data 31 back to the entity testing module 17. In other embodiments, the sensor can transmit the entity sensing data 31 back to the entity testing module 17 via other network communication protocols, such as TCP / IP, UDP, MQTT, and WebSocket, but this disclosure is not limited thereto.
[0065] In one embodiment, in addition to entity sensing data 31, the virtual testing module 13 can also use scene information from the physical environment to retrain the virtual robot. The scene information from the physical environment may include spatial and terrain information, object and obstacle information, environmental perception conditions, communication and signal conditions, temporal and dynamic characteristics, and task-related configurations, but this disclosure is not limited thereto.
[0066] Spatial and terrain information can include, for example, terrain type (flat ground, slope, uneven road surface, stairs), floor material (concrete, wood, grass, carpet), spatial dimensions (room size, corridor width), or elevation differences and ground undulations. Object and obstacle information can include, for example, fixed objects (walls, tables, chairs, doors), movable objects (people, animals, other mobile robots), the shape, size, material of obstacles, the position and dynamic changes of objects (e.g., whether furniture will be rearranged), etc.
[0067] Environmental perception conditions can include, for example, light source brightness and direction (natural light / artificial light, flickering), lighting changes (daytime / nighttime, shadows), sound sources and noise interference, and climate factors (wind, rain, temperature, humidity). Communication and signal conditions can include, for example, Wi-Fi, Bluetooth signal strength and stability, GPS coverage and accuracy (outdoor / indoor), and electromagnetic interference (e.g., in industrial environments).
[0068] Temporal and dynamic characteristics can include, for example, pedestrian flow density and its temporal variation, environmental rhythm (e.g., day-night differences between offices and factories), and task time constraints and rhythm requirements (e.g., immediate response, long-term patrols). Task-related configurations can include, for example, the location of specific task items (e.g., moving objects, checkpoints), work area division (e.g., permitted areas, restricted areas), and task flow requirements (e.g., grab → move → release).
[0069] In step 303, after the virtual robot 22 has been retrained, the compilation module 15 can recompile the cerebellum controller 14 to obtain an updated version of firmware 23. In step 304, the entity building module 16 can redeploy the updated version of firmware 23 back to the physical robot through the aforementioned application interface.
[0070] It is worth noting that through the application interface, physical sensing data can be directly exchanged between the virtual and the real world, enabling automatic updates and integration. This avoids delays, formatting errors, or omissions caused by manual input by the user, and also facilitates the recording and tracking of data flow, improving data consistency and timeliness.
[0071] It is worth noting that retraining can leverage real-world feedback to enhance the accuracy and practicality of virtual robots, thereby improving the performance of trained control strategies in real-world applications. This allows virtual robots to learn more realistically and contributes to more stable and efficient deployment of physical robots in the future.
[0072] The robot development system and method disclosed herein can accelerate the development of robots with different configurations or performing different tasks. Furthermore, by pre-installing components and a cerebellar control system, and by connecting automated compilation and application interfaces, the number of operations required to develop various robots can be reduced, thereby shortening the robot development time.
[0073] The preceding paragraphs describe various embodiments. Clearly, the teachings herein can be implemented in many ways, and any particular architecture or functionality disclosed in the examples is merely representative. Based on the teachings herein, it should be understood in the art that the various embodiments disclosed herein can be implemented independently, or two or more embodiments can be implemented in combination.
[0074] Although this disclosure has been described above with reference to embodiments, it is not intended to limit this disclosure. Anyone skilled in the art may make some changes and modifications without departing from the spirit and scope of this disclosure. Therefore, the scope of protection of the invention shall be determined by the scope of the appended claims.
Claims
1. A robot development system, comprising: Cerebellar controller; A virtual component module, configured to store multiple virtual components and at least one physical model corresponding to each virtual component; A virtual construction module is configured to construct a virtual robot using the cerebellum controller and the multiple physical models corresponding to the selected multiple virtual components; A virtual testing module, configured to train the virtual robot in a virtual environment to optimize the cerebellum controller's control of the plurality of virtual components of the virtual robot; A compilation module configured to compile the cerebellar controller of the trained virtual robot into firmware; as well as An entity building module is configured to build an entity robot based on the plurality of virtual components used by the trained virtual robot, and to deploy the firmware to the entity robot.
2. The robot development system according to claim 1 further includes a physical testing module, the physical testing module being configured to: The physical robot is subjected to physical testing in a physical environment; Determine whether the physical robot passes the physical test; and In response to the physical robot failing the physical test, the virtual test module is instructed to retrain the virtual robot.
3. The robot development system according to claim 2, wherein the entity testing module is further configured to: The entity sensing data collected by the physical robot during the physical test is obtained for the virtual test module to retrain the virtual robot.
4. The robot development system according to claim 3, wherein the firmware includes a robot operating system; and The physical robot transmits the physical sensing data through the application interface of the robot operation system.
5. The robot development system of claim 4, wherein the compilation module is further configured to recompile the cerebellar controller of the retrained virtual robot to obtain an updated version of the firmware; and The entity building module is further configured to transmit the updated version of the firmware to the entity robot via the application interface.
6. The robot development system according to claim 3, wherein the virtual testing module is further configured to: The virtual robot is then trained based on the scene information of the physical environment and the physical sensing data.
7. The robot development system according to claim 2, wherein the entity testing module is further configured to: Based on the motion test indicators, determine whether the physical robot has passed the physical test; in, The motion test metrics are related to the tasks performed by the physical robot in the physical test.
8. The robot development system according to claim 1 further includes an input module, the input module being configured to: Receive the user-defined multiple virtual components; The file formats of the user-defined virtual components are converted to conform to the virtual environment; and The converted virtual components are stored in the virtual component module.
9. The robot development system according to claim 1, wherein the plurality of virtual components includes sensors, motors, servo motors, and joint modules.
10. A robot development method, executed on a computer system, said computer system including a cerebellum controller, the robot development method comprising: Store multiple virtual components and at least one physical model corresponding to each virtual component; A virtual robot is constructed using the cerebellum controller and the multiple physical models corresponding to the selected multiple virtual components; The virtual robot is trained in a virtual environment to optimize the cerebellum controller's control of the multiple virtual components of the virtual robot; Compile the cerebellar controller of the trained virtual robot into firmware; and A physical robot is constructed based on the plurality of virtual components used by the trained virtual robot, and the firmware is deployed to the physical robot.