Control device integrating cerebellum model and cross-modal attention
By using a three-layer collaborative control framework and a cerebellum model, combined with multimodal perception data to output real-time compensation torque, the problem of low sample efficiency and motion jitter of robots in unstructured environments in existing technologies is solved, and efficient and low-energy dynamic anti-interference capability is achieved.
Patent Information
- Application Number
- CN202511360374.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-23
- Publication Date
- 2025-10-28
- Estimated Expiration
- 2045-09-23
AI Technical Summary
Existing reinforcement learning algorithms suffer from low sample efficiency and slow convergence in unstructured environments, and fail to fully utilize multimodal sensing information. Traditional neural networks lack the ability to perform detailed time series modeling, resulting in robot motion jitter, high energy consumption, poor safety, and difficulty in real-time deployment on computing-limited platforms.
A three-layer collaborative control framework is adopted, including a high-level policy network, a mid-level cerebellar network, and a low-level actuator. The mid-level cerebellar network adopts a spiking neural network morphology and embeds plastic synapses. It is optimized through time difference distillation and adaptive experience replay pooling, and combined with multimodal perception data to output real-time compensation torque.
It significantly improves the robot's anti-interference ability in unstructured environments, reduces torque compensation delay, solves the strategy drift problem, improves dynamic response capability and safety, and reduces energy consumption.
Smart Images

Figure CN120839814A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent robot technology, and more specifically to a control device that integrates a cerebellar model and cross-modal attention. Background Art
[0002] Humanoid robots need to perform complex tasks such as grasping, transporting, obstacle avoidance, and interaction in unstructured environments. Existing reinforcement learning algorithms usually rely on large-scale simulation data and lengthy real-world interaction training, resulting in low sample efficiency, slow convergence, and the simple splicing of multimodal sensor information into deep networks, failing to fully utilize the complementary characteristics of different modalities in the spatiotemporal scale. At the same time, traditional neural networks lack the ability to finely model time series in motion control, leading to motion jitter, high energy consumption, poor safety, and difficulty in real-time deployment on AGV chassis and dual robotic arm systems with limited computing power.
[0003] Therefore, improving the anti-interference ability of intelligent robots in unstructured environments is a problem that needs to be solved. Summary of the Invention
[0004] The purpose of this invention is to propose a control device that integrates a cerebellar model and cross-modal attention, which can improve the anti-interference ability of intelligent robots in unstructured environments.
[0005] To achieve the above objectives, the present invention provides a control device that integrates a cerebellar model and cross-modal attention, comprising a three-layer collaborative control framework: High-level policy network: Receives global state data, outputs macroscopic action primitive instructions, and guides the overall direction of task completion; Mid-level cerebellar network: Real-time fusion of multimodal proprioceptive data, outputting compensating torque; Low-level actuator: The joint torque output is executed through feedforward PID control.
[0006] In an optional embodiment, the mid-level cerebellar network adopts a spiking neural network morphology and embeds plastic synapses.
[0007] In an optional embodiment, the mid-layer cerebellar network adopts a biomimetic cerebellar microcircuit structure, including: Granular layer: Receives body-sensing data and encodes it as sparse pulses; Purkinje layer: Receives prediction instructions from higher-level policy networks and calculates temporal difference error online, driving synaptic plasticity based on temporal difference error; Lower olive pit: Generate teaching signals based on actual torque error.
[0008] In an optional embodiment, the high-level policy network and the mid-level cerebellar network share the value function network through "temporal differential distillation"; The low-frequency temporal difference information of the high-level policy network was distilled and injected into the mid-level cerebellar network for training.
[0009] In the optional scheme, the training process adopts an adaptive experience replay pool, which dynamically adjusts the sample weights based on the instantaneous predicted torque error output by the mid-level cerebellar network.
[0010] In an optional configuration, the adaptive experience replay pool prioritizes learning key perturbation samples.
[0011] In an optional configuration, the high-level policy network operates at a frequency lower than 10 Hz, while the mid-level cerebellar network operates at a frequency greater than 200 Hz.
[0012] The deployment process for the optional solution includes: Construct the target task scenario in a simulation environment and perform pre-training; If the simulation pre-training reaches the expected performance threshold, the trained policy network model is transferred to the real robot hardware platform; otherwise, it returns to continue pre-training. Migrating to a real hardware platform, freezing the parameters of the high-level policy network, and receiving torque error from the low-level actuator and proprioceptive signals from multimodal sensors in real time; in the mid-level cerebellar network, the inferior olivary nucleus generates teaching signals based on the torque error, and the granular layer performs sparse encoding on the proprioceptive signals; using the temporal difference error to drive the synaptic plasticity adjustment of the Purkinje layer, learning to generate precise compensating torque output. The bottom-level actuator receives motion primitive instructions from the high-level policy network and compensation torque generated by the mid-level cerebellum network based on teaching signals. It calculates the final joint torque output through a feedforward PID control algorithm and drives the servo motor to perform fine movements. Real-time monitoring of task execution performance and torque error; if performance fails to meet preset standards or error persists, dynamically adjust the mid-level cerebellar network synaptic learning rate or return to continue strengthening online learning; if the standard is met, proceed to the next step; The high-level policy network, mid-level cerebellum network, and low-level actuators that have been trained and optimized are integrated and deployed.
[0013] In an optional embodiment, the pre-training includes: In the simulation environment, the initial action primitive instructions generated by the high-level policy network and the multimodal sensor data stream are combined to drive the simulated robot to perform tasks, while the initial learning of the mid-level cerebellar network is initiated. During the pre-training process, the weights of the sample data in the experience replay pool are dynamically adjusted based on the task execution results and the error signals fed back by the sensors, prioritizing the learning of key or difficult samples. By utilizing sample data and combining multimodal perception information and task semantic understanding, the parameters of the high-level policy network are iteratively optimized to generate more accurate action primitive instructions. The beneficial effects of the present invention are: This invention can correct high-level motion errors online, significantly improving dynamic anti-interference capabilities; reduce torque compensation delay, effectively addressing disturbances such as ground slippage and sudden load changes; and solve the policy drift problem in traditional end-to-end training. Attached Figure Description
[0014] The above and other objects, features and advantages of the present invention will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.
[0015] Figure 1 This is a framework diagram of a control device that integrates a cerebellar model and cross-modal attention in one embodiment of the present invention.
[0016] Figure 2 This is a flowchart illustrating the model training process of a control device that integrates a cerebellar model and cross-modal attention in one embodiment of the present invention. Detailed Implementation
[0017] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer from the following description and drawings. However, it should be noted that the concept of the technical solution of the present invention can be implemented in many different forms and is not limited to the specific embodiments described herein. The accompanying drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.
[0018] It should be understood that when an element or layer is referred to as "on," "adjacent to," "connected to," or "coupled to" other elements or layers, it may be directly on, adjacent to, connected to, or coupled to other elements or layers, or there may be intervening elements or layers. Conversely, when an element is referred to as "directly on," "directly adjacent to," "directly connected to," or "directly coupled to" other elements or layers, there are no intervening elements or layers. It should be understood that although the terms first, second, third, etc., may be used to describe various elements, components, areas, layers, and / or portions, these elements, components, areas, layers, and / or portions should not be limited by these terms. These terms are only used to distinguish one element, component, area, layer, or portion from another element, component, area, layer, or portion. Therefore, without departing from the teachings of this invention, the first element, component, area, layer, or portion discussed below may be referred to as the second element, component, area, layer, or portion.
[0019] Spatial relation terms such as “below,” “under,” “below,” “under,” “above,” “above,” etc., are used herein for convenience of description to describe the relationship between one element or feature shown in the figure and other elements or features. It should be understood that, in addition to the orientation shown in the figure, spatial relation terms are intended to also include different orientations of the device in use and operation. For example, if the device in the figure is flipped, then the element or feature described as “below” or “under” the other element or feature will be oriented “above” the other element or feature. Therefore, the exemplary terms “below” and “under” can include both upper and lower orientations. The device may be otherwise oriented (rotated 90 degrees or otherwise) and the spatial descriptive terms used herein will be interpreted accordingly.
[0020] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. When used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising” and / or “including,” when used in this specification, identify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups. When used herein, the term “and / or” includes any and all combinations of the associated listed items.
[0021] Example Reference Figure 1 This embodiment provides a control device that integrates a cerebellar model and cross-modal attention, including a three-layer collaborative control framework: High-level policy network: Receives global state data, outputs macroscopic action primitive instructions, and guides the overall direction of task completion; Mid-level cerebellar network: Real-time fusion of multimodal proprioceptive data, outputting compensating torque; Low-level actuator: The joint torque output is executed through feedforward PID control.
[0022] Specifically, this embodiment constructs a three-layer collaborative control framework aimed at achieving robust execution and millisecond-level response for complex robot tasks. The high-level policy network, operating at a frequency below 10 Hz, serves as the decision-making core. It receives RGB image information and global state data (such as LiDAR data, low-frequency versions of joint angles, postures, etc.) to help the high-level decision network understand the robot's overall state and output macroscopic action primitive commands, such as "move forward 0.5 meters" or "grab the box," guiding the overall direction of task completion. The mid-level cerebellum network (CereNet), operating at 200 Hz, focuses on high-speed dynamic control and disturbance suppression. It fuses high-frequency sensing data from joint angles, current, torque, and the chassis IMU in real time, outputting precise compensation torques to compensate for instability caused by environmental disturbances or modeling errors. The low-level actuators, operating at 1 kHz, are responsible for the fastest command tracking. They primarily employ a PID algorithm combined with feedforward control to ensure that the compensation torques of the high-level policy network and CereNet can be executed quickly and accurately by the joint and chassis drive systems, forming a complete closed-loop control chain.
[0023] In this embodiment, the mid-level cerebellar network adopts a spiking neural network morphology and embeds plastic synapses. The mid-level cerebellar network employs a biomimetic cerebellar microcircuit structure, including: a granular layer: receiving proprioceptive data and encoding it as sparse pulses; a Purkinje layer: receiving prediction instructions from the high-level policy network and calculating temporal difference errors online, driving synaptic plasticity based on these errors; and a lower olive nucleus: generating teaching signals based on actual torque errors. The high-level policy network and the mid-level cerebellar network share a value function network through "temporal difference distillation"; low-frequency temporal difference information from the high-level policy network is distilled and injected into the mid-level cerebellar network for training. The training process employs an adaptive experience replay pool, dynamically adjusting sample weights based on the instantaneous predicted torque error output by the mid-level cerebellar network. The adaptive experience replay pool prioritizes learning key perturbation samples.
[0024] The key to CereNet lies in its bio-inspired "cerebellar microcircuit" differentiable computational unit structure. The network consists of a granular layer that receives proprioceptive signals (composed of angle, current, torque, and chassis IMU data) and encodes them as sparse impulse representations (with activation rates kept below 5% for efficiency). The Purkinje layer receives prediction instructions from the higher-level policy network and calculates temporal difference errors online, dynamically driving synaptic plasticity to correct prediction biases. The lower olive nucleus generates critical instruction signals based on torque errors during actual execution. The entire network employs a spiking neural network architecture and embeds plastic synapses, maintaining ultra-low power consumption significantly below 0.8 watts while ensuring millisecond-level response times. To reconcile the differences between the control frequencies of the high and low layers and the target objectives, a multimodal reinforcement learning strategy is employed. The higher-level policy network uses the Soft Actor-Critic algorithm to optimize the weight coefficients, balancing the reward function comprised of task completion, energy efficiency, and posture stability. Specifically, the high-level policy network shares a value function network with CereNet, and effectively extracts and injects the low-frequency high-level temporal difference information into the high-frequency CereNet training through a "temporal difference distillation" mechanism, which effectively suppresses policy drift. The training process also introduces an adaptive experience replay pool, which dynamically adjusts the sample weights based on the instantaneous prediction error of CereNet output, significantly improving the utilization rate of effective samples.
[0025] This device follows a strict end-to-end process. The first training phase involves pre-training in the Isaac Sim simulation environment, where environmental factors such as friction coefficient, load mass, and lighting conditions are randomized to achieve co-optimization and parameter initialization of the high-level policy network and CereNet. The second phase moves to a real hardware platform for fine-tuning. Here, the high-level policy network parameters are frozen, and CereNet's plasticity synaptic parameters are fine-tuned online solely based on torque errors generated during actual execution. This process converges rapidly within half an hour and recovers over 90% of the simulation performance.
[0026] In terms of system deployment, the perception layer includes dual-channel RGB-D vision sensors, a high-frequency joint encoder, current and torque sensors, and a chassis IMU. Computational tasks are assigned to an NVIDIA Jetson AGX Orin (primarily responsible for high-level strategies) and an STM32H7 microcontroller (with a dedicated on-chip accelerator for real-time SNN computation of CereNet). The execution layer consists of four AGV hub motors and twelve robotic arm servo motors, all communicating synchronously at millisecond levels via an EtherCAT bus. By quantizing and compressing the CereNet model, its inference latency is significantly reduced, meeting the stringent performance requirements of embedded deployments.
[0027] Reference Figure 2 The deployment process is as follows: Step 1: System Initialization: Configure multimodal sensor interfaces: RGB camera, LiDAR, motor encoder, microphone; load high-level policy network, including Soft Actor-Critic algorithm framework, multimodal vision processing module, task semantic understanding module and low-level actuator control module, including feedforward PID controller, servo motor drive interface; initialize Cerenet cerebellar neural network model parameters. Step 2: Simulation Environment Startup: Build the target task scenario in the MuJoCo physics simulation environment and start the simulation loop; Step 3: Collaborative pre-training execution: In the simulation environment, the preliminary action primitive instructions generated by the high-level policy network and the multimodal sensor data stream are combined to drive the simulated robot to perform tasks, while the initial learning of Cerenet is initiated. Step 4: Dynamic adjustment of sample weights: During the pre-training process, the weights of sample data in the experience replay pool are dynamically adjusted based on the task performance (such as goal achievement and efficiency) and the error signals fed back by the sensors, prioritizing the learning of key or difficult samples. Step 5: Policy Network Training and Optimization: Based on the Soft Actor-Critic algorithm, using weighted sample data, combined with multimodal perception information (vision, LiDAR point cloud, ontology perception) and task semantic understanding, the parameters of the high-level policy network are iteratively optimized to generate more accurate action primitive instructions. Step 6: Fine-tuning in the real hardware environment: If the simulation pre-training reaches the expected performance threshold, then transfer the trained policy network model to the real robot hardware platform; otherwise, return to step 3 to continue pre-training. Step 7: Cerenet Cerebellar Neural Network Online Learning: During the real hardware fine-tuning stage or the later stage of simulation, torque error data from the underlying actuator and proprioceptive signals from multimodal sensors are received in real time; in CereNet, the lower olive kernel generates key teaching signals based on the torque error, and the granular layer performs sparse encoding on the proprioceptive signals; the synaptic plasticity adjustment of the Purkinje layer is driven by the time difference error to learn and generate accurate compensation torque output.
[0028] Step 8: Actuator Control and Motion Output: The bottom-level actuator receives motion primitive instructions from the high-level policy network and compensation torque signals generated by Cerenet based on the teaching signals. It calculates the final joint torque output through a feedforward PID control algorithm to drive the servo motor to perform fine movements. Step 9: Performance Evaluation and Synaptic Adjustment: Monitor task execution performance (such as grasping success rate, motion smoothness, energy consumption) and torque error in real time; if the performance does not meet the preset standard or the error persists, dynamically adjust the Cerenet synaptic learning rate or return to Step 7 to enhance online learning; if the standard is met, proceed to the next step. Step 10: Algorithm Deployment and Application: Integrate and deploy the trained and optimized high-level policy network, Cerenet cerebellar neural network, and low-level control module to enable the robot to complete the specified task in a complex environment based on multimodal perception information through reinforcement learning and online fine-tuning of the cerebellum.
[0029] The mid-cerebellar network employs a cerebellar-like sparse pulse coding mechanism (Granule layer activation rate <5%) to convert high-dimensional proprioceptive signals (joint angles, currents, torques) into low-power pulse streams. Combined with the temporal difference error-driven synaptic plasticity of the Purkinje layer, it online corrects high-level motion errors, significantly improving dynamic anti-interference capabilities. The mid-cerebellar network utilizes a synergistic design of spiking neural networks and plastic synapses, maintaining power consumption <0.8W under 200Hz high-frequency control. Sub-millisecond latency for torque compensation is achieved through an STM32H7 embedded accelerator, effectively addressing disturbances such as slippage and sudden load changes. High-level vision (30Hz)... The RGB image and the underlying ontology perception (1kHz joint / current signal) share a value network through "temporal differential distillation", injecting low-frequency task semantics (such as obstacle avoidance intention) into the mid-level cerebellum network to solve the policy drift problem in traditional end-to-end training; the adaptive experience replay pool based on CereNet prediction error improves the utilization rate of key perturbation samples by 3.7 times, accelerates policy convergence and reduces simulation pre-training time by more than 50%.
[0030] This embodiment demonstrates significantly faster convergence speed than traditional deep reinforcement learning algorithms with the same number of training interactions, improved motion trajectory smoothness, reduced energy consumption, and increased grasping and obstacle avoidance success rates. It also exhibits strong robustness under conditions such as partial camera occlusion, LiDAR noise, and motor parameter drift. Due to the rapid compensation capability of the mid-level cerebellar network for motion errors, system safety is improved and the probability of collisions is reduced. The algorithm can be deployed in real time on humanoid robot platforms with limited computing resources, effectively shortening the time and cost of migrating from simulation to reality.
[0031] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.
Claims
1. A control device integrating a cerebellar model and cross-modal attention, characterized in that, Includes a three-tiered collaborative control framework: High-level policy network: Receives global state data, outputs macroscopic action primitive instructions, and guides the overall direction of task completion; Mid-level cerebellar network: Real-time fusion of multimodal proprioceptive data, outputting compensating torque; Low-level actuator: The joint torque output is executed through feedforward PID control.
2. The control device integrating a cerebellar model and cross-modal attention as described in claim 1, characterized in that, The mid-level cerebellar network adopts a spiking neural network morphology and incorporates plastic synapses.
3. The control device integrating a cerebellar model and cross-modal attention as described in claim 2, characterized in that, The mid-layer cerebellar network adopts a biomimetic cerebellar microcircuit structure, including: Granular layer: Receives body-sensing data and encodes it as sparse pulses; Purkinje layer: Receives prediction instructions from the higher-level policy network and calculates the temporal difference error online, driving synaptic plasticity based on the temporal difference error; Lower olive pit: Generate teaching signals based on actual torque error.
4. The control device integrating a cerebellar model and cross-modal attention as described in claim 1, characterized in that, The high-level policy network and the mid-level cerebellar network share the value function network through "temporal differential distillation"; The low-frequency temporal difference information of the high-level policy network was distilled and injected into the mid-level cerebellar network for training.
5. The control device integrating a cerebellar model and cross-modal attention as described in claim 4, characterized in that, The training process employs an adaptive experience replay pool, dynamically adjusting sample weights based on the instantaneous predicted torque error output by the mid-level cerebellar network.
6. The control device integrating a cerebellar model and cross-modal attention as described in claim 5, characterized in that, The adaptive experience replay pool prioritizes learning key perturbation samples.
7. The control device integrating a cerebellar model and cross-modal attention as described in claim 1, characterized in that, The high-level policy network operates at a frequency below 10 Hz, while the mid-level cerebellar network operates at a frequency greater than 200 Hz.
8. The control device integrating a cerebellar model and cross-modal attention as described in claim 1, characterized in that, The deployment process includes: Construct the target task scenario in a simulation environment and perform pre-training; If the simulation pre-training reaches the expected performance threshold, the trained policy network model is transferred to the real robot hardware platform; otherwise, return to continue pre-training. Migrating to a real hardware platform, freezing the parameters of the high-level policy network, and receiving torque error from the low-level actuator and proprioceptive signals from multimodal sensors in real time; in the mid-level cerebellar network, the inferior olivary nucleus generates teaching signals based on the torque error, and the granular layer performs sparse encoding of the proprioceptive signals; using the temporal difference error to drive the synaptic plasticity adjustment of the Purkinje layer, learning to generate accurate compensating torque output. The bottom-level actuator receives motion primitive instructions from the high-level policy network and compensation torque generated by the mid-level cerebellum network based on teaching signals. It calculates the final joint torque output through a feedforward PID control algorithm and drives the servo motor to perform fine movements. Real-time monitoring of task execution performance and torque error; if performance fails to meet preset standards or error persists, dynamically adjust the mid-level cerebellar network synaptic learning rate or return to continue strengthening online learning; if the standard is met, proceed to the next step; The high-level policy network, mid-level cerebellum network, and low-level actuators that have been trained and optimized are integrated and deployed.
9. The control device integrating a cerebellar model and cross-modal attention as described in claim 8, characterized in that, The pre-training includes: In the simulation environment, the initial action primitive instructions generated by the high-level policy network and the multimodal sensor data stream are combined to drive the simulated robot to perform tasks, while the initial learning of the mid-level cerebellar network is initiated. During the pre-training process, the weights of the sample data in the experience replay pool are dynamically adjusted based on the task execution results and the error signals fed back by the sensors, prioritizing the learning of key or difficult samples. By utilizing sample data and combining multimodal perception information and task semantic understanding, the parameters of the high-level policy network are iteratively optimized to generate more accurate action primitive instructions.
Citation Information
Patent Citations
Distributed type control system of hydraulic quadruped robot and control method
CN103279113A
Universal system of intelligent robot with body, construction method and use method
CN117549310A
Thumb independent optimization five-finger manipulator control method based on hierarchical strategy control
CN119567270A
Mechanical arm power compliance interaction control method based on multi-brain-region joint model
CN119952711A
Multi-layer intelligent development framework of multi-modal LLM Agent for unmanned aerial vehicle system
CN119987724A
Cited By
Robot motion control method and system based on cerebellum reinforcement learning
CN121756368A
A robot motion control method and system based on cerebellum reinforcement learning
CN121756368B
Logistics robot control method and system based on online transfer learning
CN122172754A
