A control device fusing a cerebellar model and cross-modal attention

Abstract

The application relates to the technical field of intelligent robots, and provides a control device fusing a cerebellum model and cross-modal attention, which comprises a three-layer cooperative control framework: a high-layer strategy network: receiving global state data, outputting macroscopic action element instructions, guiding the overall direction of task completion; a middle-layer cerebellum network: fusing multi-modal ontology perception data in real time, outputting compensation torques; and a bottom-layer actuator: executing joint torque output through feedforward PID control. The application can correct high-layer action errors online, significantly improves dynamic anti-interference capability, reduces torque compensation delay, effectively deals with disturbances such as ground skidding and load mutation, and solves the strategy drift problem in traditional end-to-end training.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of intelligent robots, and in particular to a control device fusing a cerebellar model and cross-modal attention. BACKGROUND

[0002] A humanoid robot needs to complete complex tasks such as grabbing, carrying, obstacle avoidance and interaction in an unstructured environment. Existing reinforcement learning algorithms usually rely on large-scale simulation data and lengthy real interaction training, have low sample efficiency, slow convergence, and multi-modal sensing information is often simply spliced into a deep network, failing to fully utilize the complementary characteristics of different modalities in time and space scales. At the same time, traditional neural networks lack the ability to finely model time series in motion control, resulting in action jitter, high energy consumption, poor safety, and difficulty in real-time deployment on AGV chassis and dual-arm systems with limited computing power.

[0003] Therefore, how to improve the anti-interference capability of intelligent robots in an unstructured environment is a problem to be solved at present. SUMMARY

[0004] The purpose of the application is to provide a control device fusing a cerebellar model and cross-modal attention, which can improve the anti-interference capability of intelligent robots in an unstructured environment.

[0005] To achieve the above purpose, the application provides a control device fusing a cerebellar model and cross-modal attention, which comprises a three-layer cooperative control framework.

[0006] A high-level strategy network receives global state data and outputs macro action primitive instructions to guide the overall direction of task completion.

[0007] A middle-layer cerebellar network fuses multi-modal proprioceptive data in real time and outputs compensation torque.

[0008] A bottom-layer actuator executes joint torque output through feedforward PID control.

[0009] In an optional solution, the middle-layer cerebellar network adopts a spiking neural network form and embeds plastic synapses.

[0010] In an optional solution, the middle-layer cerebellar network adopts a bionic cerebellar microcircuit structure, which comprises:

[0011] A granular layer receives proprioceptive data and encodes them into sparse spikes.

[0012] A Purkinje layer receives prediction instructions from the high-level strategy network, calculates time difference errors online, and drives synaptic plasticity based on the time difference errors.

[0013] An inferior olive nucleus generates a teaching signal according to an actual torque error.

[0014] In an optional solution, the high-level policy network and the middle-level cerebellar network share the value function network through "time difference distillation";

[0015] The low-frequency time difference information of the high-level policy network is distilled into the middle-level cerebellar network for training.

[0016] In an optional solution, an adaptive experience replay pool is used in the training process, and the sample weight is dynamically adjusted based on the instantaneous predicted torque error output by the middle-level cerebellar network.

[0017] In an optional solution, the adaptive experience replay pool preferentially learns key perturbation samples.

[0018] In an optional solution, the running frequency of the high-level policy network is lower than 10 Hz, and the running frequency of the middle-level cerebellar network is greater than 200 Hz.

[0019] In an optional solution, the deployment process includes:

[0020] A target task scenario is constructed in a simulation environment for pre-training;

[0021] If the simulation pre-training reaches an expected performance threshold, the trained policy network model is migrated to a real robot hardware platform; otherwise, the pre-training is continued.

[0022] The migration to the real hardware platform freezes the parameters of the high-level policy network, and real-time torque errors from the bottom-level actuator and body perception signals from the multi-modal sensor are received. In the middle-level cerebellar network, the inferior olive nucleus generates a teaching signal based on the torque error, and the granular layer performs sparse coding on the body perception signal. The synaptic plasticity adjustment of the Purkinje layer is driven by the time difference error to learn and generate accurate compensation torque output.

[0023] The bottom-level actuator receives the action primitive instruction from the high-level policy network and the compensation torque generated by the middle-level cerebellar network based on the teaching signal, calculates the final joint torque output through a feedforward PID control algorithm, and drives the servo motor to perform fine actions.

[0024] The task execution effect and torque error are monitored in real time. If the performance does not meet the preset standard or the error persists, the synaptic learning rate of the middle-level cerebellar network is dynamically adjusted or the online learning is continued. If the standard is met, the next step is performed.

[0025] The high-level policy network, the middle-level cerebellar network, and the bottom-level actuator that have completed training and optimization are integrated and deployed.

[0026] In an optional solution, the pre-training includes:

[0027] In the simulation environment, the initial action primitive instruction generated by the high-level strategy network is combined with the multi-modal sensor data stream to drive the simulation robot to perform the task, and the initial learning of the middle-layer cerebellum network is started;

[0028] In the pre-training process, according to the task execution effect and the error signal fed back by the sensor, the weight of the sample data in the experience replay pool is dynamically adjusted, and the key or difficult sample is preferentially learned;

[0029] By using the sample data, the multi-modal perception information and the task semantic understanding are combined to iteratively optimize the parameters of the high-level strategy network, and more accurate action primitive instructions are generated.

[0030] The beneficial effects of the present application are as follows:

[0031] The present application can correct the high-level action error online, significantly improve the dynamic anti-interference ability, reduce the delay of torque compensation, effectively cope with ground slip, load mutation and other disturbances, and solve the strategy drift problem in traditional end-to-end training. BRIEF DESCRIPTION OF DRAWINGS

[0032] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description of illustrative embodiments thereof, which proceeds with reference to the accompanying drawings.

[0033] Figure 1 The framework diagram of the control device fusing the cerebellum model and the cross-modal attention in an embodiment of the present application.

[0034] Figure 2 The model training flowchart of the control device fusing the cerebellum model and the cross-modal attention in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The present application will be further described below in conjunction with the drawings and specific embodiments. According to the following description and drawings, the advantages and features of the present application will be more apparent, however, it should be noted that the technical solutions of the present application can be implemented in various forms, and are not limited to the specific embodiments described herein. The drawings are greatly simplified and use non-precise proportions, and are only used to facilitate and clarify the purpose of illustrating the embodiments of the present application.

[0036] It will be understood that when an element or layer is referred to as being "on" or "connected to" another element or layer, it can be directly on or connected to the other element or layer or intervening elements or layers can be present. In contrast, when an element is referred to as being "directly on" or "directly connected to" another element or layer, there are no intervening elements or layers present. It will be understood that, although the terms first, second, third, etc. can be used herein to describe various elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Thus, a first element, component, region, layer or section discussed below could be termed a second element, component, region, layer or section without departing from the teachings of the present application.

[0037] Spatially relative terms, such as "beneath", "below", "lower", "under", "above", "upper" and the like, can be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if a device in the figures is turned over, elements described as "below" or "beneath" other elements or features would then be oriented "above" or "over" the other elements or features. Thus, the exemplary term "below" can encompass both an orientation of above and below. The device can be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.

[0038] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising", when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0039] Embodiments

[0040] Reference Figure 1 The embodiments provide a control device fusing a cerebellar model and cross-modal attention, including a three-layer cooperative control framework:

[0041] High-level policy network: receives global state data, outputs macroscopic action primitive instructions, and guides the overall direction of task completion;

[0042] Middle-layer cerebellar network: real-time fusion of multi-modal proprioceptive data, output compensation torque;

[0043] Bottom-layer actuator: execute joint torque output through feedforward PID control.

[0044] Specifically, the embodiment constructs a three-layer collaborative control framework, aiming to realize the robust execution and millisecond-level response of complex tasks of the robot. The high-level policy network, with a running frequency lower than 10 Hz, serves as the decision core, receives RGB image information and global state data (such as laser radar data, low-frequency version of some joint angles, pose, etc.), helps the high-level decision network to understand the overall state of the robot, and outputs macroscopic action primitive instructions such as “advance 0.5 meters” or “hold the box”, guiding the overall direction of task completion. The middle-layer cerebellar network (CereNet), with a running frequency of 200 Hz, focuses on high-speed dynamic control and disturbance suppression, real-time fusion of high-frequency perception data of joint angles, currents, torques, and chassis IMU, and outputs fine compensation torque to compensate for unstable factors caused by environmental disturbance or modeling error. The bottom-layer actuator, with a running frequency of 1 kHz, is responsible for the fastest instruction tracking, mainly adopts the PID algorithm combined with feedforward control, ensures that the compensation torque of the high-level policy network and CereNet can be quickly and accurately executed by the joint and chassis driving system, and constitutes a complete closed-loop control chain.

[0045] In the embodiment, the middle-layer cerebellar network adopts a spiking neural network form and embeds plastic synapses. The middle-layer cerebellar network adopts a bionic cerebellar microcircuit structure, including: a granular layer that receives proprioceptive data and encodes it into sparse spikes; a Purkinje layer that receives high-level policy network prediction instructions and calculates online time difference error, and drives synaptic plasticity based on the time difference error; and a lower olive nucleus that generates teaching signals according to actual torque error. The high-level policy network and the middle-layer cerebellar network share the value function network through “time difference distillation”; the low-frequency time difference information of the high-level policy network is distilled into the middle-layer cerebellar network for training. The training process adopts an adaptive experience replay pool, dynamically adjusts the sample weight based on the instantaneous predicted torque error output by the middle-layer cerebellar network. The adaptive experience replay pool prioritizes learning key disturbance samples.

[0046] The key to CereNet lies in its bio-inspired "cerebellar microcircuit" differentiable computational unit structure. The network consists of a granular layer that receives proprioceptive signals (composed of angle, current, torque, and chassis IMU data) and encodes them as sparse impulse representations (with activation rates kept below 5% for efficiency). The Purkinje layer receives prediction instructions from the higher-level policy network and calculates temporal difference errors online, dynamically driving synaptic plasticity to correct prediction biases. The lower olive nucleus generates critical instruction signals based on torque errors during actual execution. The entire network employs a spiking neural network architecture and embeds plastic synapses, maintaining ultra-low power consumption significantly below 0.8 watts while ensuring millisecond-level response times. To reconcile the differences between the control frequencies of the high and low layers and the target objectives, a multimodal reinforcement learning strategy is employed. The higher-level policy network uses the Soft Actor-Critic algorithm to optimize the weight coefficients, balancing the reward function comprised of task completion, energy efficiency, and posture stability. Specifically, the high-level policy network shares a value function network with CereNet, and effectively extracts and injects the low-frequency high-level temporal difference information into the high-frequency CereNet training through a "temporal difference distillation" mechanism, which effectively suppresses policy drift. The training process also introduces an adaptive experience replay pool, which dynamically adjusts the sample weights based on the instantaneous prediction error of CereNet output, significantly improving the utilization rate of effective samples.

[0047] This device follows a strict end-to-end process. The first training phase involves pre-training in the Isaac Sim simulation environment, where environmental factors such as friction coefficient, load mass, and lighting conditions are randomized to achieve co-optimization and parameter initialization of the high-level policy network and CereNet. The second phase moves to a real hardware platform for fine-tuning. Here, the high-level policy network parameters are frozen, and CereNet's plasticity synaptic parameters are fine-tuned online solely based on torque errors generated during actual execution. This process converges rapidly within half an hour and recovers over 90% of the simulation performance.

[0048] In terms of system deployment, the perception layer includes dual-channel RGB-D vision sensors, a high-frequency joint encoder, current and torque sensors, and a chassis IMU. Computational tasks are assigned to an NVIDIA Jetson AGX Orin (primarily responsible for high-level strategies) and an STM32H7 microcontroller (with a dedicated on-chip accelerator for real-time SNN computation of CereNet). The execution layer consists of four AGV hub motors and twelve robotic arm servo motors, all communicating synchronously at millisecond levels via an EtherCAT bus. By quantizing and compressing the CereNet model, its inference latency is significantly reduced, meeting the stringent performance requirements of embedded deployments.

[0049] Reference Figure 2 The deployment process is as follows:

[0050] Step 1: System Initialization: Configure multimodal sensor interfaces: RGB camera, LiDAR, motor encoder, microphone; load high-level policy network, including Soft Actor-Critic algorithm framework, multimodal vision processing module, task semantic understanding module and low-level actuator control module, including feedforward PID controller, servo motor drive interface; initialize Cerenet cerebellar neural network model parameters.

[0051] Step 2: Simulation Environment Startup: Build the target task scenario in the MuJoCo physics simulation environment and start the simulation loop;

[0052] Step 3: Collaborative pre-training execution: In the simulation environment, the preliminary action primitive instructions generated by the high-level policy network and the multimodal sensor data stream are combined to drive the simulated robot to perform tasks, while the initial learning of Cerenet is initiated.

[0053] Step 4: Dynamic adjustment of sample weights: During the pre-training process, the weights of sample data in the experience replay pool are dynamically adjusted based on the task performance (such as goal achievement and efficiency) and the error signals fed back by the sensors, prioritizing the learning of key or difficult samples.

[0054] Step 5: Policy Network Training and Optimization: Based on the Soft Actor-Critic algorithm, using weighted sample data, combined with multimodal perception information (vision, LiDAR point cloud, ontology perception) and task semantic understanding, the parameters of the high-level policy network are iteratively optimized to generate more accurate action primitive instructions.

[0055] Step 6: Fine-tuning in the real hardware environment: If the simulation pre-training reaches the expected performance threshold, then transfer the trained policy network model to the real robot hardware platform; otherwise, return to step 3 to continue pre-training.

[0056] Step 7: Cerenet Cerebellar Neural Network Online Learning: During the real hardware fine-tuning stage or the later stage of simulation, torque error data from the underlying actuator and proprioceptive signals from multimodal sensors are received in real time; in CereNet, the lower olive kernel generates key teaching signals based on the torque error, and the granular layer performs sparse encoding on the proprioceptive signals; the temporal difference error is used to drive the synaptic plasticity adjustment of the Purkinje layer to learn and generate accurate compensation torque output.

[0057] Step 8: Actuator Control and Motion Output: The bottom-level actuator receives motion primitive instructions from the high-level policy network and compensation torque signals generated by Cerenet based on the teaching signals. It calculates the final joint torque output through a feedforward PID control algorithm to drive the servo motor to perform fine movements.

[0058] Step 9: Performance Evaluation and Synaptic Adjustment: Monitor task execution performance (such as grasping success rate, motion smoothness, energy consumption) and torque error in real time; if the performance does not meet the preset standard or the error persists, dynamically adjust the Cerenet synaptic learning rate or return to Step 7 to enhance online learning; if the standard is met, proceed to the next step.

[0059] Step 10: Algorithm Deployment and Application: Integrate and deploy the trained and optimized high-level policy network, Cerenet cerebellar neural network, and low-level control module to enable the robot to complete the specified task in a complex environment based on multimodal perception information through reinforcement learning and online fine-tuning of the cerebellum.

[0060] The mid-cerebellar network employs a cerebellar-like sparse pulse coding mechanism (Granule layer activation rate <5%) to convert high-dimensional proprioceptive signals (joint angles, currents, torques) into low-power pulse streams. Combined with the temporal difference error-driven synaptic plasticity of the Purkinje layer, it online corrects high-level motion errors, significantly improving dynamic anti-interference capabilities. The mid-cerebellar network utilizes a synergistic design of spiking neural networks and plastic synapses, maintaining power consumption <0.8W under 200Hz high-frequency control. Sub-millisecond latency for torque compensation is achieved through an STM32H7 embedded accelerator, effectively addressing disturbances such as slippage and sudden load changes. High-level vision (30Hz)... The RGB image and the underlying ontology perception (1kHz joint / current signal) share a value network through "temporal differential distillation", injecting low-frequency task semantics (such as obstacle avoidance intention) into the mid-level cerebellum network to solve the policy drift problem in traditional end-to-end training; the adaptive experience replay pool based on CereNet prediction error improves the utilization rate of key perturbation samples by 3.7 times, accelerates policy convergence and reduces simulation pre-training time by more than 50%.

[0061] This embodiment demonstrates significantly faster convergence speed than traditional deep reinforcement learning algorithms with the same number of training interactions, improved motion trajectory smoothness, reduced energy consumption, and increased grasping and obstacle avoidance success rates. It also exhibits strong robustness under conditions such as partial camera occlusion, LiDAR noise, and motor parameter drift. Due to the rapid compensation capability of the mid-level cerebellar network for motion errors, system safety is improved and the probability of collisions is reduced. The algorithm can be deployed in real time on humanoid robot platforms with limited computing resources, effectively shortening the time and cost of migrating from simulation to reality.

[0062] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A control device that fuses a cerebellar model and cross-modal attention, characterized by, The three-layer cooperative control framework comprises: a high-level strategy network: receiving global state data, outputting macro action primitive instructions, guiding the overall direction of task completion; a middle-level cerebellar network: real-time fusion of multi-modal proprioceptive data, outputting compensation torque; a bottom-level actuator: executing joint torque output through feedforward PID control; the high-level strategy network and the middle-level cerebellar network share the value function network through "time difference distillation"; low-frequency time difference information of the high-level strategy network is distilled into the middle-level cerebellar network for training.

2. The control device of the fusion cerebellar model and cross-modal attention according to claim 1, wherein, The middle-level cerebellar network adopts a spiking neural network form and embeds plastic synapses. 3.The control device of the fusion cerebellar model and cross-modal attention according to claim 2, wherein, The middle-level cerebellar network adopts a bionic cerebellar microcircuit structure, comprising: a granular layer: receiving proprioceptive data and encoding them into sparse spikes; a Purkinje layer: receiving high-level strategy network prediction instructions, and online calculating time difference errors, and driving synaptic plasticity based on time difference errors; a inferior olive: generating teaching signals according to actual torque errors.

4. The control device of fusion cerebellar model and cross-modal attention according to claim 1, wherein, The training process adopts an adaptive experience replay pool, and dynamically adjusts sample weights based on instantaneous prediction torque errors output by the middle-level cerebellar network.

5. The control device of fusion cerebellar model and cross-modal attention according to claim 4, wherein, The adaptive experience replay pool preferentially learns key disturbance samples.

6. The control device of fusion cerebellar model and cross-modal attention according to claim 1, wherein, The running frequency of the high-level strategy network is lower than 10 Hz, and the running frequency of the middle-level cerebellar network is greater than 200 Hz.

7. The control device of fusion cerebellar model and cross-modal attention according to claim 1, wherein, The deployment process comprises: building a target task scene in a simulation environment, and pre-training; if the simulation pre-training reaches an expected performance threshold, migrating the trained strategy network model to a real robot hardware platform; otherwise, returning to continue pre-training; migrating to a real hardware platform, freezing the high-level strategy network parameters, and real-time receiving torque errors from the bottom-level actuator and proprioceptive signals from multi-modal sensors; in the middle-level cerebellar network, the inferior olive generates teaching signals based on the torque errors, and the granular layer performs sparse coding on the proprioceptive signals; the synaptic plasticity adjustment of the Purkinje layer is driven by the time difference error to learn and generate accurate compensation torque output; the bottom-level actuator receives the action primitive instructions from the high-level strategy network and the compensation torque generated by the middle-level cerebellar network based on the teaching signals, calculates the final joint torque output through a feedforward PID control algorithm, and drives the servo motor to execute fine actions; real-time monitoring of task execution effect and torque error; if the performance does not reach the preset standard or the error persists, dynamically adjusting the synaptic learning rate of the middle-level cerebellar network or returning to continue online learning; if the standard is reached, the next step is performed; integrating and deploying the high-level strategy network, the middle-level cerebellar network and the bottom-level actuator after training and optimization.

8. The control apparatus of fusion cerebellar model and cross-modal attention according to claim 7, wherein, The pre-training comprises: in a simulation environment, combining the preliminary action primitive instructions generated by the high-level strategy network with the multi-modal sensor data stream to drive the simulation robot to execute tasks, and starting the preliminary learning of the middle-level cerebellar network; in the pre-training process, dynamically adjusting the weights of sample data in the experience replay pool according to the task execution effect and the error signal fed back by the sensor, and preferentially learning key or difficult samples; using sample data, combining multi-modal perception information and task semantic understanding, and iteratively optimizing the parameters of the high-level strategy network to generate more accurate action primitive instructions.

Images