Robot Operating System and Method Based on Hardware Isolation and Dynamic Task Engine

By using a robot operating system based on hardware isolation and a dynamic task engine, the contradiction between real-time performance and intelligence in robot operating systems is resolved, achieving efficient task routing and closed-loop collaborative control, thereby improving development efficiency and system performance.

CN121349953BActive Publication Date: 2026-11-10HARBIN INST OF TECH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202511316257.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-09-16
Publication Date
2026-11-10
Estimated Expiration
2045-09-16

AI Technical Summary

Technical Problem

Traditional robot operating systems struggle to simultaneously meet the demands of hard real-time computing and large-scale non-real-time computing, making it difficult to reconcile the conflict between real-time performance and intelligence. Existing technical solutions suffer from latency uncertainties, communication bottlenecks, and development complexity.

Method used

The robot operating system adopts hardware isolation and dynamic task engine, and achieves electrical isolation between real-time domain and non-real-time domain through static partitioning of heterogeneous multi-core SoC and Type-1 Hypervisor. Data transmission is carried out using inter-core communication bus, and task configuration and deployment are carried out through integrated development toolchain.

Benefits of technology

It achieves microsecond-level deterministic scheduling of real-time tasks, improves algorithm iteration speed, enhances development efficiency, and strengthens system integration, while balancing high performance and high security, and supports flexible task routing and closed-loop collaborative control.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121349953B_ABST
    Figure CN121349953B_ABST
Patent Text Reader

Abstract

This invention proposes a robot operating system and method based on hardware isolation and a dynamic task engine, belonging to the field of artificial intelligence technology. It addresses the long-standing issue of the binary opposition between real-time performance and intelligence in the robotics field at the architectural level. The system comprises: a heterogeneous multi-core SoC, a general-purpose operating system deployed in the non-real-time domain, a kernel-based real-time operating system deployed in the real-time domain, an inter-core communication bus based on shared physical memory, and an integrated development toolchain based on VSCode plugins. The method includes: building a robot operating system based on hardware isolation and a dynamic task engine; generating, routing, and deploying tasks; and while executing tasks in the real-time domain, reporting high-frequency status data to the non-real-time domain via the inter-core communication bus, forming a closed-loop collaborative control.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to a robot operating system and method based on hardware isolation and a dynamic task engine, belonging to the field of artificial intelligence technology. Background Technology

[0002] With the deep integration of artificial intelligence and robotics, modern robots are developing towards higher levels of autonomy and intelligence, expanding their application scenarios from structured industrial environments to unstructured, dynamically changing home and public spaces. This leap in complexity presents an unprecedented and almost contradictory technical challenge to their underlying operating system: On the one hand, the physical interactions of robots, such as the precise driving of high-frequency servo motors (which typically requires control frequencies of 1kHz or even higher), the maintenance of full-body dynamic balance based on multi-sensor fusion (IMU, torque sensor, encoder), and the planning and execution of complex motion trajectories, all fall under the category of hard real-time computing. These tasks have extremely stringent requirements for time determinism and must be completed within strict deadlines. Any microsecond-level, unpredictable jitter can accumulate into centimeter-level motion errors, or even cause the robot to become unstable and tip over, resulting in physical damage or safety accidents.

[0003] On the other hand, advanced intelligent behaviors of robots, such as real-time visual scene understanding using deep learning models (SLAM, object recognition), complex task planning and decision-making reasoning (e.g., understanding abstract instructions like "go to the kitchen and get me a bottle of water"), and fluent natural language human-computer interaction, all fall under the category of large-scale, high-computing-power non-real-time computing. These tasks require powerful CPU / GPU / NPU computing power. More importantly, they rely on a feature-rich and mature general-purpose operating system (GPOS), such as Linux, Android, or other operating systems, to allow developers to utilize the vast amount of existing open-source libraries (such as OpenCV, PyTorch, ROS2), middleware, and development tools.

[0004] Traditional technical solutions all have obvious technical flaws when attempting to reconcile this fundamental contradiction.

[0005] Option 1: Use real-time patching (such as Linux RT-PREEMPT) on a general operating system. While this option can significantly reduce average latency, under high system loads (such as heavy I / O and complex graphics rendering), there are still non-preemptible critical sections and excessively long interrupt shutdown times in the kernel. This means that the worst-case execution time (WCET) cannot be strictly guaranteed, scheduling jitter still exists, and it is difficult to meet the robot's requirements for extreme stability.

[0006] Option 2: Physical Dual-Master Control Solution. This solution uses a high-performance ARM processor running GPOS for intelligent computing, while an external microcontroller (MCU) running an RTOS for real-time control. Although this solution ensures the purity of the real-time domain, it introduces new bottlenecks: the ARM and MCU typically communicate via physical buses such as SPI, UART, or USB, which have limited bandwidth (usually in the Mbps range), high protocol overhead, and high end-to-end latency (up to hundreds of microseconds or even milliseconds). This makes high-frequency (KHz level) status feedback and command issuance impossible, severely limiting the speed and accuracy of the robot's dynamic response and preventing true "body-brain collaboration." Furthermore, the development, debugging, firmware upgrade, and deployment processes for the two systems are completely separate, leading to complex project management and low development efficiency.

[0007] In recent years, mature virtualization technologies in the server and desktop fields have provided new solutions to this problem. However, applying them to the resource-constrained and demanding field of embedded robotics still faces many challenges. For example, traditional Type-2 hypervisors, which run on GPOS, cannot provide real-time guarantees. While heavyweight Type-1 hypervisors (such as Xen and KVM) can provide isolation, they are also quite complex, introducing performance overhead and scheduling uncertainties, and requiring a huge amount of work for adaptation and customization on embedded SoCs. Summary of the Invention

[0008] This invention addresses the long-standing issue of the binary opposition between real-time performance and intelligence in the field of robotics architecture, and proposes a robot operating system and method based on hardware isolation and a dynamic task engine.

[0009] The technical solution adopted by this invention to solve the above problems is as follows: The robot operating system based on hardware isolation and dynamic task engine proposed in this invention includes: In a heterogeneous multi-core SoC, the CPU, memory, and peripherals are divided into real-time and non-real-time domains that are electrically isolated from each other and do not preempt each other during operation through a statically partitioned Type-1 Hypervisor. A general-purpose operating system deployed in a non-real-time domain is used to build a unified task management platform; A kernel-based real-time operating system deployed in the real-time domain for integrating a dynamic task engine; Based on the shared physical memory configuration, the inter-core communication bus adopts a single producer and single consumer lock-free queue as the core data structure of the inter-core communication bus to realize the communication connection between the non-real-time domain and the real-time domain. An integrated development toolchain based on VSCode plugins, used for graphical configuration, code generation, and deployment of tasks.

[0010] Robot operation methods based on hardware isolation and dynamic task engines include: Step 1: Build a robot operating system based on hardware isolation and a dynamic task engine; Step 2: Generate a task request containing algorithm logic and metadata through an integrated development toolchain, and send the task request to a unified task management platform built in a non-real-time domain; Step 3: After receiving a task request, the unified task management platform analyzes the task attributes based on a preset intelligent routing algorithm and determines the target execution domain corresponding to the task. Step 4: If the decision is a non-real-time task, it will be directly scheduled and executed by the general-purpose operating system in the non-real-time domain; Step 5: If the decision is a real-time task, the task request is sent to the real-time domain via the inter-core communication bus, and the dynamic task engine in the real-time domain loads and schedules the execution. Step 6: While executing tasks, the real-time domain reports high-frequency status data to the non-real-time domain through the inter-core communication bus, forming a closed-loop collaborative control.

[0011] Furthermore, step 2 generates a task request containing algorithm logic and metadata, including: Step 2.1: Configure task parameters through the graphical interface. Task parameters include, but are not limited to, motion cycle and joint angle limitations. Step 2.2: The toolchain automatically generates script code in the corresponding format based on the configuration template; Step 2.3: Encapsulate the script code and task parameters into a unified JSON object and transmit it to the unified task management platform via command-line tools.

[0012] Furthermore, step 3 determines the target execution domain corresponding to the task, including: Step 3.1: Analyze the real-time requirements of the tasks and prioritize routing periodic and high-frequency tasks to the real-time domain; Step 3.2: Analyze the computational complexity of the task and route tasks that require a large number of floating-point operations and one of the complex algorithm libraries to the non-real-time domain; Step 3.3: Analyze the resource dependencies of the task and route the task according to the access requirements of the peripherals; Step 3.4: Execute routing policies, including user-specified forced routes, intelligent routes automatically selected by the system, and dynamic migrations performed at runtime based on load.

[0013] Furthermore, in step 5, the task request is sent to the real-time domain via the inter-core communication bus, including: Step 5.1: Before sending, encode the task request using a zero-copy serialization protocol; Step 5.2: The non-real-time domain notifies the real-time domain of the arrival of a new task by triggering an inter-processor interrupt; Step 5.3: Before loading the script, the real-time domain's dynamic task engine performs a security check through the API sandbox mechanism, exposing only predefined secure function interfaces; Step 5.4: The dynamic task engine uses execution time limits, step-by-step garbage collection control, and a double-buffered hot-switching mechanism.

[0014] Furthermore, high-frequency status data is reported to the non-real-time domain via the inter-core communication bus to form a closed-loop control, including: Step 6.1: The control task in the real-time domain acquires IMU data, encoder readings, and force sensor data of the robot body at a fixed high frequency; Step 6.2: Write the collected multi-source data into the uplink channel area of ​​shared memory through the SPSC lockless circular queue; Step 6.3: Adopt a back pressure control strategy. When the uplink channel is full, use the latest data to overwrite the oldest data to ensure that the status information obtained in the non-real-time domain has the highest timeliness. Step 6.4: In the event of an emergency, the non-real-time domain can be forced to immediately process the reported emergency status data by triggering an IPI interrupt.

[0015] The beneficial effects of this invention are: 1. This invention fundamentally resolves the core contradiction between real-time performance and intelligence: through Jailhouse hardware isolation, it provides a clean environment for real-time control tasks, equivalent to that of a physical MCU, ensuring microsecond-level scheduling determinism. This solves the problem of increased jitter under high load in traditional RT-PREEMPT patching schemes and avoids the communication bottleneck of physical dual-master schemes.

[0016] 2. This invention achieves unprecedented algorithmic flexibility and iteration speed: the innovative real-time domain dynamic task engine and double-buffered hot update mechanism allow developers to safely and seamlessly update the underlying core control algorithm without stopping the robot's operation. This reduces the robot algorithm iteration cycle from several hours (traditional firmware compilation and flashing) to several seconds, greatly accelerating the research and development process.

[0017] 3. This invention significantly improves development efficiency and lowers the technical threshold: the integrated graphical development toolchain fully automates and graphically represents the complex processes of underlying system configuration, cross-core communication, and task deployment, allowing algorithm developers to focus entirely on the application logic itself without needing to become embedded system experts, thus greatly expanding the talent pool for robot development.

[0018] 4. This invention offers superior system integration and performance: all computing units reside on the same SoC and communicate via on-chip shared memory, resulting in latency and bandwidth far exceeding any inter-board physical bus, providing the optimal hardware foundation for achieving high-bandwidth, low-latency "body-brain collaboration." Simultaneously, the unified task management platform provides upper-layer applications with a simple and consistent API, shielding them from the heterogeneous complexity of the underlying layers.

[0019] 5. This invention balances high performance and high security: a strict API sandbox mechanism ensures that dynamically loaded scripts cannot damage the core of the real-time system, achieving a balance between flexibility and reliability. Hardware isolation also naturally provides fault isolation capabilities; software crashes outside the real-time domain will not affect the robot's basic motion safety. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the structure of a robot operating system based on hardware isolation and a dynamic task engine; Figure 2 This is a schematic diagram of the six-layer structure from hardware to development tools of a robot operating system based on hardware isolation and a dynamic task engine. Figure 3 This is a schematic diagram of the static partition configuration of the CPU core, memory and peripherals on the RK3568 processor according to the present invention. Figure 4 Detailed layout of the inter-core communication bus based on shared memory and schematic diagram of the working principle of SPSC lockless queue; Figure 5 This is a flowchart illustrating a robot operation method based on hardware isolation and a dynamic task engine. Figure 6 This is a flowchart of the intelligent routing algorithm based on a multi-dimensional decision model in the DeploymentManager platform of this invention.

[0021] Figure 7This is a timing diagram of the double-buffered hot update mechanism of the real-time domain dynamic task engine of this invention. Detailed Implementation

[0022] Specific implementation method one: as follows Figure 1 As shown, the structure of the robot operating system based on hardware isolation and dynamic task engine described in this embodiment includes: Heterogeneous multi-core SoC, general-purpose operating system deployed in non-real-time domain, kernel real-time operating system deployed in real-time domain, inter-core communication bus based on shared physical memory configuration, and integrated development toolchain based on VSCode plugin.

[0023] The CPU, memory, and peripherals of a heterogeneous multi-core SoC are divided into a real-time domain and a non-real-time domain at the hardware level through a statically partitioned Type-1 Hypervisor. These domains are electrically isolated from each other and do not preempt each other during runtime. In the non-real-time domain, a unified task management platform is built through a deployed general-purpose operating system. In the real-time domain, a dynamic task engine is integrated through a deployed kernel real-time operating system. The inter-core communication bus uses a single-producer, single-consumer lock-free queue as the core data structure for communication between the non-real-time domain and the real-time domain. An integrated development toolchain is used for graphical configuration, code generation, and deployment of tasks.

[0024] like Figure 4 As shown, the inter-core communication bus is designed based on a 4MB shared physical memory to achieve true zero-copy communication. This memory area is carefully divided into: 1) uplink data area (2MB); 2) downlink instruction area (1MB); 3) synchronization and control area (512KB); 4) reserved expansion area (512KB).

[0025] The synchronization and control area is the core of lock-free implementation. It contains a fine control structure, such as head and tail pointers (read and write indices), and uses atomic operation instructions (such as LDXR / STXR) and memory barrier instructions (such as DMB) provided by the ARMv8 architecture to ensure data consistency and operation order when multiple cores access these pointers. This enables thread-safe queue operations without the intervention of operating system locks.

[0026] To ensure data transmission efficiency and protocol flexibility, Google Flatbuffers is adopted as the serialization protocol. Its "zero-copy" feature allows the receiver to directly access any field in its memory via offset after data serialization, without any parsing or memory copying. This is crucial in high-frequency communication scenarios at 1kHz, as it minimizes the CPU overhead of each data exchange. Furthermore, Flatbuffers' schema evolution feature supports forward / backward compatibility of the protocol, ensuring that new and old versions of components can still communicate normally during system upgrades.

[0027] The unified task management platform, running as a high-privilege background system service of the operating system, is the "brain" of the entire system. Its intelligent routing algorithm is a multi-dimensional decision-making model based on weighted scoring, comprehensively considering: 1) task metadata; 2) static code analysis; 3) real-time system load; 4) resource and peripheral dependencies; and 5) power consumption and energy efficiency. In mobile scenarios, the platform evaluates the energy consumption model of tasks, tending to place low-power, periodic tasks on energy-efficient real-time cores and allowing non-real-time cores to enter a deeper sleep state when idle.

[0028] In addition to intelligent routing, DeploymentManager is also responsible for the full lifecycle management of tasks: it monitors the health status (heartbeat, resource usage) of all deployed tasks, and can handle non-real-time domain tasks that exit abnormally according to preset policies (such as restart, alarm); when it receives abnormal feedback such as "execution timeout" from the real-time domain, it can immediately notify the upper layer application or trigger the system to enter the security protection mode.

[0029] The core innovation of this invention is that the dynamic task engine uses Lua 5.4 as its base engine and performs a deep real-time (Real-TimeCapable) modification on it.

[0030] 1) Memory allocator redirection: Redirects all Lua memory requests (malloc / free) to a fixed-block memory pool provided by Uniproton with O(1) time complexity, completely eliminating the uncertainty latency and memory fragmentation problems caused by standard dynamic memory allocation.

[0031] 2) API Security Sandbox: Build a strict API whitelist. For example, allowed APIs include robot.set_joint_angle(id, pos, vel) (deterministic hardware operation), robot.read_imu_quat() (deterministic sensor reading), and math.sin(x) (pure computation); prohibited APIs include io.open() (file operation, variable latency), os.execute() (process creation, disrupts real-time performance), and socket.connect() (network operation, variable latency).

[0032] 3) Preemptive Time Slicing and GC Control: By injecting an instruction counting hook into the Lua interpreter's virtual machine (VM), the maximum execution time of any Lua code segment (e.g., 100μs) can be precisely controlled. If a timeout occurs, the script will be forcibly suspended or terminated, preventing malicious or erroneous code from crippling the entire real-time system. Simultaneously, Lua's automatic GC is changed to a controlled mode, where a low-priority RTOS task performs GC operations incrementally and in stages during periods of low system load, ensuring that each GC run is extremely short (e.g., <50μs) and does not block high-priority control tasks.

[0033] 4) Double-buffered hot update: The detailed steps of this mechanism are as follows: ① The DeploymentManager issues an update command containing the new script; ② The dynamic task engine allocates space in the standby Lua state machine (State B); ③ The new script is compiled, loaded, and statically verified in State B; ④ Once everything is ready, a "pending switch" flag is set; ⑤ At the moment before the next cycle begins after the current control cycle (State A) has finished executing, the main execution pointer is switched from State A to State B through an atomic pointer swap operation; ⑥ After the new cycle begins, the system seamlessly executes the new script logic; ⑦ When the system is idle, the background task safely destroys and reclaims the resources of State A.

[0034] In the process of establishing a robot operating system based on hardware isolation and a dynamic task engine, this implementation method runs the system on an industrial robot with Rockchip RK3568 as the core main control board. The overall system, from hardware to development tools, is divided into a six-layer structure, such as... Figure 2As shown, it includes: a development tool layer, an application management layer, an inter-core communication layer, an operating system layer, a virtualization layer, and a hardware platform layer. The development tool layer uses the VSCCode plugin for graphical configuration, one-click deployment, and real-time monitoring; the application management layer includes DeploymentManager for unified task management and intelligent routing; the inter-core communication layer includes a shared memory bus for implementing SPSC lock-free queues and Flatbuffers serialization; the operating system includes non-real-time and real-time domains; the virtualization layer includes Jaithouse Hyperxisor for hardware resource isolation; and the hardware platform layer includes the RK3568 ARM Cortex A55 processor.

[0035] This implementation method uses, for example: Figure 3 The Rockchip RK3568 and other multi-core ARMv8-A architecture processors shown natively support ARM Virtualization Extensions. This invention selects Jailhouse as the hypervisor, which, unlike heavyweight virtualization solutions such as KVM, is a statically partitioned, configuration-and-run hypervisor. Its technical advantages are: 1) Extremely lightweight and non-intrusive: Jailhouse has a very small codebase (several thousand lines) and introduces almost no runtime overhead or scheduling latency. It starts late in the system boot process, "coloring" the main operating system (Root Cell, i.e., the operating system) to confine it to the allocated resources, and then starts other partitions (Inmate Cell, i.e., Uniproton). After that, it no longer interferes with the internal scheduling of the Cell.

[0036] 2) Absolutely static partitioning: At system startup, Jailhouse precisely partitions all hardware resources at once based on a device tree-style configuration file, without performing any dynamic allocation during runtime. This fundamentally eliminates performance jitter caused by resource contention (such as memory bus, cache, interrupt controller), ensuring the "electrical isolation" and "temporal isolation" of real-time partitioning.

[0037] 3) Fault isolation: This strong physical-level isolation provides extremely high reliability. Any software failure in the non-real-time domain operating system, even a kernel panic, will never affect the normal operation of the real-time domain, ensuring that the robot can maintain the most basic attitude stability and safety protection actions under any circumstances.

[0038] 4) Configuration Details: Specifically, the CPU3 of the quad-core CPU, a specific address range of 1GB of physical memory, a dedicated SPI / I2C / GPIO controller, and the associated GIC interrupt signals are statically allocated to the real-time domain cell. All other resources belong to the root partition running the operating system. This allocation is physical and mandatory; once the real-time domain is started, it operates like an independent MCU with its own dedicated hardware.

[0039] Specific implementation method two: such as Figure 5 As shown, the steps of the robot operation method based on hardware isolation and dynamic task engine described in this embodiment include: S1: Build a robot operating system based on hardware isolation and a dynamic task engine; S2: Generate task requests containing algorithm logic and metadata through an integrated development toolchain, and send the task requests to a unified task management platform built in a non-real-time domain; The development team configures task parameters through a graphical interface. Task parameters include, but are not limited to, motion cycles and joint angle limitations. The toolchain automatically generates script code in the corresponding format based on the configuration template. The script code and task parameters are encapsulated into a unified JSON object and transmitted to a unified task management platform via a command-line tool.

[0040] S3: After receiving a task request, the unified task management platform analyzes the task attributes based on a preset intelligent routing algorithm and determines the target execution domain corresponding to the task. like Figure 6 As shown, the steps of the decision-making task are as follows: S301: Analyze the real-time requirements of the task and prioritize the routing of periodic and high-frequency tasks to the real-time domain; S302: Analyze the computational complexity of tasks and route tasks that require a large number of floating-point operations and one of the complex algorithm libraries to the non-real-time domain; S303: Analyze the resource dependencies of the task and route the task according to the access requirements of the peripheral device; S304: Execute routing policies, including user-specified forced routes, intelligent routes automatically selected by the system, and dynamic migrations performed at runtime based on load.

[0041] S4: If the decision is a non-real-time task, it will be directly scheduled and executed by the general-purpose operating system in the non-real-time domain; S5: If the decision is a real-time task, the task request is sent to the real-time domain through the inter-core communication bus, and the dynamic task engine in the real-time domain loads and schedules the execution. The specific execution steps for a task assignment include: S501: Before sending, the task request is encoded using a zero-copy serialization protocol; S502: The non-real-time domain notifies the real-time domain of the arrival of a new task by triggering an inter-processor interrupt; S503: The real-time domain dynamic task engine performs security checks through the API sandbox mechanism before loading scripts, exposing only predefined secure function interfaces; S504: The dynamic task engine utilizes execution time limits, step-by-step garbage collection control, and a double-buffered hot-switching mechanism.

[0042] The above process involves task generation, routing, and deployment. Using the VSCode plugin of this invention, the developer first deploys a basic "periodic walking" Lua script. After being analyzed by the DeploymentManager, this script, due to its 1ms hard real-time loop, is correctly routed and sent to the dynamic task engine in the real-time domain for execution. The robot then begins to stably tread in place or walk in a straight line. Simultaneously, in the non-real-time domain operating system, the developer deploys an "obstacle detection" application based on the lightweight YOLOv5-Nano model. This model has been quantized and optimized using the RKNN toolkit and can run efficiently on the RK3568 NPU. After being analyzed by the DeploymentManager, this application, due to its reliance on a camera, NPU, and AI computing library, is correctly routed to the non-real-time domain for execution.

[0043] S6: While executing tasks, the real-time domain reports high-frequency status data to the non-real-time domain through the inter-core communication bus, forming a closed-loop collaborative control.

[0044] The execution steps for high-frequency status data reporting include: S601: The control task in the real-time domain acquires IMU data, encoder readings and force sensor data of the robot body at a fixed high frequency. S602: The collected multi-source data is written to the uplink channel area of ​​shared memory through the SPSC lockless circular queue; S603: Employs a back pressure control strategy. When the uplink channel is full, such as... Figure 7 As shown, a double-buffered hot update mechanism is adopted, using the latest data to overwrite the oldest data, in order to ensure that the status information obtained from the non-real-time domain has the highest timeliness. S604: In the event of an emergency, the non-real-time domain can be forced to immediately process the reported emergency status data by triggering an IPI interrupt.

[0045] The goal of the VSCode plugin is to create a "digital twin" development experience. Its monitoring panel acquires massive amounts of robot status data in real time through the uplink channel of the inter-core communication bus and displays it in a graphical curve format, forming a virtual model that is synchronized with the physical robot's status in real time.

[0046] When developers drag sliders on the graphical interface to adjust the robot's walking stride, the plugin will: 1) generate new Lua script fragments in real time; 2) push them to the DeploymentManager via HDC; 3) after verification by the DeploymentManager, distribute them to the real-time domain; and 4) execute hot updates via the dynamic task engine. The entire process is completed within hundreds of milliseconds, allowing developers to immediately see the effects of parameter adjustments on the monitoring panel and the actual robot, achieving a truly "what you see is what you get" real-time robot debugging experience. Furthermore, this toolchain integrates version control functionality, allowing developers to associate different versions of control scripts and parameter configurations with Git repositories for one-click deployment, rollback, and comparison.

[0047] The workflow of closed-loop coordinated control includes: Perception (non-real-time domain): The "obstacle detection" application in the operating system continuously processes the video stream from the camera. When a ball rolls into the field of view and is recognized by the NPU, the application immediately calculates the ball's position, velocity, and predicted trajectory in the robot's own coordinate system.

[0048] Decision and Instruction Issuance (Non-Real-Time Domain -> Real-Time Domain): Instead of issuing a new task, the application generates a high-priority "obstacle avoidance parameter update" instruction. This instruction (e.g., a Flatbuffers structure containing fields such as {target_foot_placement_offset_x: 0.2, target_foot_placement_offset_y: 0.1}) is sent to the real-time domain via the downlink channel of the inter-core communication bus.

[0049] Execution and Control (Real-Time Domain): The dynamic task engine executing the "periodic walking" script exposes a callback function `on_param_update()` in its API that can be invoked by downlink commands. Upon receiving a command, this function is triggered, the script reads the new obstacle avoidance offset, and immediately applies it to the next step of the Central Pattern Generator (CPG) algorithm, dynamically adjusting the phase and amplitude of the gait. As a result, the robot's next footing point is shifted to the side and forward by a corresponding distance from its original position, thus cleverly avoiding the rolling ball.

[0050] Status feedback (real-time domain -> non-real-time domain): Throughout the process, the real-time domain continuously feeds back the raw data such as the IMU's attitude data (roll angle, pitch angle), the actual angles and torques of each joint, and the plantar pressure distribution at a frequency of 1KHz to the non-real-time domain through the uplink channel.

[0051] State estimation and upper-level adjustments (non-real-time domain): A "Kalman filter attitude estimation" application in the non-real-time domain receives these massive amounts of raw data in real time, fuses the IMU and kinematic model, and calculates a more accurate and smoother full-body robot attitude. This fused accurate attitude is used for display on the monitoring interface and as input for higher-level planning (such as path planning). For example, if the attitude estimation algorithm detects that the robot's ZMP (zero moment point) is close to the edge of the supporting polygon when performing obstacle avoidance, it can immediately issue a "lower center of gravity" or "reduce stride" command to the real-time domain for secondary dynamic adjustments to ensure absolute stability during the dynamic process.

[0052] As can be seen from this embodiment, the system architecture constructed by the present invention perfectly supports the operation of the three modules of perception, decision-making and control in their respective optimal environments, and tightly couples them into an organic whole through a high-performance inter-core communication bus, ultimately realizing complex, dynamic and intelligent robot behavior.

[0053] The design of this invention has good scalability and can be adjusted according to specific application scenarios. For example, the dynamic task engine can be replaced with the WebAssembly (WASM) runtime to support control logic written in more high-performance languages ​​such as C++ / Rust.

[0054] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the scope of the present invention, based on the technical essence of the present invention and within the spirit and principles of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A robot operation method based on hardware isolation and a dynamic task engine, characterized in that, include: Step 1: Build a robot operating system based on hardware isolation and a dynamic task engine; The robot operating system includes: The heterogeneous multi-core SoC, wherein the CPU, memory and peripherals of the heterogeneous multi-core SoC are divided into real-time domain and non-real-time domain at the hardware level through a static partitioned Type-1 Hypervisor, which are electrically isolated from each other and do not preempt each other during operation. A general-purpose operating system deployed in the non-real-time domain is used to build a unified task management platform; A kernel-based real-time operating system deployed in the real-time domain is used to integrate a dynamic task engine; Based on the shared physical memory configuration, the inter-core communication bus adopts a single producer and single consumer lock-free queue as the core data structure of the inter-core communication bus to realize the communication connection between the non-real-time domain and the real-time domain. An integrated development toolchain based on the VSCode plugin, used for graphical configuration, code generation, and deployment of tasks; Step 2: Generate a task request containing algorithm logic and metadata through an integrated development toolchain, and send the task request to a unified task management platform built in a non-real-time domain; Step 3: After receiving a task request, the unified task management platform analyzes the task attributes based on a preset intelligent routing algorithm and determines the target execution domain corresponding to the task. Step 4: If the decision is a non-real-time task, it will be directly scheduled and executed by the general-purpose operating system in the non-real-time domain; Step 5: If the decision is a real-time task, the task request is sent to the real-time domain via the inter-core communication bus, and the dynamic task engine in the real-time domain loads and schedules the execution. Step 5 involves sending the task request to the real-time domain via the inter-core communication bus, including: Step 5.1: Before sending, encode the task request using a zero-copy serialization protocol; Step 5.2: The non-real-time domain notifies the real-time domain of the arrival of a new task by triggering an inter-processor interrupt; Step 5.3: Before loading the script, the real-time domain's dynamic task engine performs a security check through the API sandbox mechanism, exposing only predefined secure function interfaces; Step 5.4: Dynamic task engine execution time limit, step-by-step garbage collection control, and double-buffered hot-switching mechanism; Step 6: While executing tasks, the real-time domain reports high-frequency status data to the non-real-time domain through the inter-core communication bus, forming a closed-loop collaborative control.

2. The robot operation method based on hardware isolation and dynamic task engine according to claim 1, characterized in that, Step 2 generates a task request containing algorithm logic and metadata, including: Step 2.1: Configure task parameters through the graphical interface. Task parameters include, but are not limited to, motion cycle and joint angle limitations. Step 2.2: The toolchain automatically generates script code in the corresponding format based on the configuration template; Step 2.3: Encapsulate the script code and task parameters into a unified JSON object and transmit it to the unified task management platform via command-line tools.

3. The robot operation method based on hardware isolation and dynamic task engine according to claim 1, characterized in that, Step 3 determines the target execution domain corresponding to the task, including: Step 3.1: Analyze the real-time requirements of the tasks and prioritize routing periodic and high-frequency tasks to the real-time domain; Step 3.2: Analyze the computational complexity of the task and route tasks that require a large number of floating-point operations and one of the complex algorithm libraries to the non-real-time domain; Step 3.3: Analyze the resource dependencies of the task and route the task according to the access requirements of the peripherals; Step 3.4: Execute routing policies, including user-specified forced routes, intelligent routes automatically selected by the system, and dynamic migrations performed at runtime based on load.

4. The robot operation method based on hardware isolation and dynamic task engine according to claim 1, characterized in that, Step 6 involves reporting high-frequency status data to the non-real-time domain via the inter-core communication bus, forming a closed-loop control, including: Step 6.1: The control task in the real-time domain acquires IMU data, encoder readings, and force sensor data of the robot body at a fixed high frequency; Step 6.2: Write the collected multi-source data into the uplink channel area of ​​shared memory through the SPSC lockless circular queue; Step 6.3: Adopt a back pressure control strategy. When the uplink channel is full, use the latest data to overwrite the oldest data to ensure that the status information obtained in the non-real-time domain has the highest timeliness. Step 6.4: In the event of an emergency, the non-real-time domain can be forced to immediately process the reported emergency status data by triggering an IPI interrupt.

Citation Information

Patent Citations

  • Industrial robot unified control architecture system and universal robot controller

    CN120620191A