Robot vision edge computing system and computing method thereof

Abstract

The invention discloses a robot vision edge computing system and a computing method thereof. The system comprises a sensing layer, a reconfigurable computing platform and a robot decision-making layer, the reconfigurable computing platform comprises: an input adaptation and buffer module, which is used for receiving a multi-source sensing data stream of a sensing layer and carrying out hardware-level timestamp synchronization and format unification; the dynamic scheduler is used for receiving a mode instruction from a robot decision-making layer and generating a hardware control signal according to the mode instruction so as to control a switching path of a multi-source sensing data stream and distribute computing resources in the platform; the algorithm container loading and executing unit is used for loading and operating the algorithm container; and the time slice scheduler is activated and controlled by the dynamic scheduler and is used for periodically switching the algorithm containers running in the algorithm container loading and executing unit at a fixed time slice on the hardware level. The invention further comprises a calculation method of the robot vision edge calculation system. According to the invention, deterministic scheduling and efficient multiplexing of computing resources can be realized.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a robot vision edge computing system and its computing method. Background Technology

[0002] Robot vision systems are a core component of intelligent devices such as mobile robots and automated guided vehicles, responsible for extracting, understanding, and outputting environmental information from images or video streams. As application scenarios become increasingly complex, the demands of robots on vision systems present a dynamic, heterogeneous, and diverse set of contradictions: on the one hand, during high-speed movement and obstacle avoidance navigation, a single vision algorithm (such as visual odometry or obstacle detection) is required to exclusively utilize computing resources to ensure deterministic, low-latency real-time performance (e.g., latency <100ms); on the other hand, during fixed-point monitoring and precision operations, multiple vision algorithms (such as face recognition, behavior analysis, and instrument readings) need to run simultaneously to maximize the utilization of computing resources and scene understanding capabilities.

[0003] Currently, there are two main types of computing solutions supporting robot vision, but neither can simultaneously achieve real-time performance, flexibility, efficiency, and collaboration. Category 1: Dedicated embedded vision systems. These systems adopt a fixed or semi-fixed hardware and algorithm integration scheme, directly deploying specific algorithms (such as a specific object recognition algorithm) on embedded processors (such as ARM+FPGA). Its advantage is that it has good real-time performance. However, its inherent disadvantages seriously restrict the intelligent development of robots: (1) Rigid functions and poor scalability: The algorithm functions are fixed once they leave the factory and cannot be dynamically added, deleted or upgraded according to new tasks; if new functions are to be added, hardware modules usually need to be replaced or added, resulting in high costs and bloated systems. (2) Low resource utilization: When performing a single task, most of the hardware's computing power is idle; when multiple tasks need to be parallelized, it cannot be achieved due to architectural limitations, or it can only be achieved by stacking multiple sets of hardware, resulting in a huge waste of physical resources and energy. (3) High threshold for algorithm deployment and optimization: The deployment of deep learning models (such as CNN-based detectors) heavily relies on manual optimization and lacks a unified automated toolchain. For example, it is difficult to easily quantize the FP32 model into the INT8 model to improve efficiency, which often forces developers to use models with high computational cost and low energy efficiency, thus limiting the performance ceiling of the system.

[0004] The second category: general computing platforms based on software containerization. With the popularization of containerization technology (such as Docker), some solutions attempt to deploy and manage multiple vision algorithms on the robot's main control computer (such as an industrial control computer) through software containers. This method solves the problems of algorithm environment isolation and dynamic deployment ("plug and play" algorithm) and improves development flexibility. However, its fundamental defect is that it cannot meet the robot's demanding requirements for determinism and real-time performance: (1) High task switching latency: Container scheduling and context switching at the software level are managed by general operating systems (such as Linux), which has huge overhead, usually in the range of tens to hundreds of milliseconds, resulting in slow algorithm switching response. (2) Uncertain resource scheduling: The scheduling strategy of general operating systems cannot guarantee the precise execution time of vision tasks, and multiple tasks are prone to mutual interference, making it impossible to guarantee real-time performance. (3) Difficult cross-task collaboration: Communication between containers is usually based on networks or files, which has high latency and makes it difficult to achieve low-latency perception data sharing across containers. For example, if one container detects a target at time t1, and another container needs to track it at time t2, due to the lack of an efficient sharing mechanism, the tracking container often needs to be re-detected, resulting in computational redundancy and breakage of temporal information, making it impossible to form continuous and coherent scene perception.

[0005] In short, this invention addresses the core problem of existing robot vision computing systems, which struggle to balance functionality, real-time performance, flexibility, and efficiency. Specifically, it solves the following five interrelated technical problems: In summary, existing technologies lack a hardware system architecture that can unify the aforementioned contradictions, particularly in areas such as dynamic working mode adaptation, plug-and-play algorithm deployment, real-time switching and collaboration of multiple tasks, computational efficiency optimization, and system reliability. Therefore, the robotics field urgently needs a visual computing system that can provide hard real-time, deterministic computational guarantees, support flexible and ecological algorithm deployment and efficient resource sharing, and achieve low-latency cross-task collaboration and efficient model execution, in order to solve at least one of the aforementioned technical problems. Summary of the Invention

[0006] The purpose of this invention is to overcome the above-mentioned shortcomings of the prior art and provide a robot vision edge computing system and computing method thereof, which achieves deterministic scheduling and efficient reuse of computing resources through dedicated hardware logic and reconfigurable architecture.

[0007] The technical solution of this invention is: One aspect of the present invention is a robot vision edge computing system, comprising a perception layer, a reconfigurable computing platform, and a robot decision layer connected in sequence; the reconfigurable computing platform includes: The input adaptation and buffering module is used to receive multi-source sensor data streams from the perception layer and perform hardware-level timestamp synchronization and format unification. A dynamic scheduler is used to receive mode instructions from the robot's decision-making layer and generate hardware control signals according to the mode instructions to control the switching path of multi-source sensor data streams and allocate computing resources within the platform. The algorithm container loading and execution unit is used to load and run algorithm containers; A time-slicing scheduler, whose activation is controlled by the dynamic scheduler, is used at the hardware level to periodically switch the algorithm containers loaded and running in the execution unit at fixed time slices. The dynamic scheduler is configured to: respond to a first mode instruction from the robot decision layer, control the multi-source sensor data stream of the input adaptation and buffer module to be directly connected to the specified algorithm container in the algorithm container loading and execution unit, and disable the time-slicing scheduler; respond to a second mode instruction from the robot decision layer, control the multi-source sensor data stream of the input adaptation and buffer module to be distributed among multiple algorithm containers in the algorithm container loading and execution unit, and activate the time-slicing scheduler to control the time-sharing execution of the multiple algorithm containers.

[0008] Furthermore, the time-slicing scheduler includes a programmable timer and a context fast save / restore circuit; the programmable timer is used to generate periodic hardware interrupt signals, and the context fast save / restore circuit is used to respond to the hardware interrupt signals by saving the execution context of the current algorithm container at the hardware level through a direct memory access channel and loading the execution context of the next algorithm container.

[0009] Furthermore, the algorithm container loading and execution unit includes the following sub-modules that work together: A container image repository is used to store multiple algorithm container image files; The unified quantization engine module is used to perform standardized quantization operations on deep learning models and output optimized model files to the container image repository. The container loading engine module is used to load the specified algorithm container image file from the container image repository according to the loading instruction, and deploy it to the isolated memory area divided by the hardware memory management unit; The running sandbox, created by the container loading engine module when loading the algorithm container image file, provides the algorithm container with a running environment that is forcibly isolated by the hardware memory management unit and the dynamic scheduler.

[0010] Furthermore, the reconfigurable computing platform also includes a cross-container shared context management unit, which is connected to the algorithm container loading and execution unit and is used to provide a memory region that can be shared and accessed by multiple algorithm containers, so that algorithm containers executing in different time slices can exchange intermediate perception data through the shared memory region.

[0011] Furthermore, the reconfigurable computing platform also includes an output fusion and synchronization module, which is used to collect the processing results from each of the running sandboxes and the fusion information in the shared cross-container shared context management unit, perform timestamp alignment and fusion on the output results of each algorithm container, and generate structured perception information to send to the robot decision layer.

[0012] Furthermore, under the second mode instruction, the time-slicing scheduler and the cross-container shared context management unit work together so that after the first algorithm container executed in the first time slice writes the result to the shared memory area, the second algorithm container executed in the second time slice can directly read the result, forming a virtual collaborative perception pipeline.

[0013] Furthermore, the reconfigurable computing platform also ensures system reliability through the collaboration of the following modules: The algorithm container loading and execution unit is configured to provide fault isolation through its hardware-isolated runtime sandbox; The container loading engine module is configured to perform a restart operation on the specified algorithm container in response to an external reset signal; The dynamic scheduler is configured to trigger the system to switch back from the second mode to the first mode in response to an external mode switching signal.

[0014] One of the calculation methods of the present invention based on the aforementioned robot vision edge computing system includes a mode switching method: Receive pattern instructions from the robot's decision-making layer; The mode instructions are parsed by a dynamic scheduler and hardware control signals are generated. If it is the first mode instruction, execute the first configuration: turn off the time-slicing scheduler, pass the input multi-source sensor data stream directly to the algorithm container specified in the algorithm container loading and execution unit, and centrally allocate computing resources to the algorithm container; If it is the second mode instruction, execute the second configuration: activate the time-slicing scheduler, cyclically distribute the input multi-source sensor data stream among multiple algorithm containers in the algorithm container loading and execution unit, and have the time-slicing scheduler control the multiple algorithm containers to take turns executing according to time slices.

[0015] Furthermore, in the second configuration, a collaborative working method is also included: The time-slicing scheduler generates hardware interrupts at fixed intervals to trigger hardware-level context switching in order to schedule multiple algorithm containers in a round-robin fashion. In any Nth time slice, the scheduled algorithm container writes its output perception data into the shared memory region managed by the cross-container shared context management unit, where N≥1; In the following N+1 time slice, another scheduled algorithm container directly reads the perceived data from the shared memory region for processing; Through the shared memory region and periodic time slice scheduling, a data relay and collaborative perception pipeline is realized among multiple algorithm containers.

[0016] Furthermore, it also includes the deployment and execution process of the algorithm container, with the following steps: S1: Use the unified quantization engine module to perform standardized quantization on the trained deep learning model to generate an optimized model file; package the optimized model file with the corresponding algorithm executable program and dependent libraries to form an algorithm container image file that conforms to the platform specifications; S2: Upload and store the algorithm container image file in the container image repository; S3: In response to the loading command, the specified algorithm container image file is read from the container image repository through the container loading engine module and deployed to the runtime sandbox, where an isolated and resource-controlled runtime environment is allocated. S4: Run the algorithm container in the running sandbox; when the system is working in the second mode, the execution process of the algorithm container is scheduled by the time-slicing scheduler in a time-sharing manner using hardware interrupts; the running algorithm container exchanges data with other algorithm containers through the shared memory area managed by the cross-container shared context management unit to achieve collaborative perception.

[0017] The beneficial effects of this invention are: (1) Achieved the unity of hard real-time guarantee and efficient resource utilization: Through hardware-level mode switching of the dynamic scheduler, exclusive and deterministic computing resources are provided for key tasks in mobile mode, ensuring the ultimate real-time performance (end-to-end latency <100ms); in stationary mode, through hardware-level time-sharing multiplexing of the time-slicing scheduler, a single set of hardware can execute dozens of algorithms in parallel, and the hardware resource utilization rate is increased from less than 30% of the traditional dedicated system to more than 80%, which is equivalent to completing the work of multiple sets of equipment with one set of equipment, and the hardware cost is estimated to be reduced by more than 50%; thus, the present invention can dynamically and quickly switch between two completely different working modes of "providing exclusive resource guarantee for a single task" and "providing efficient time-sharing multiplexing resources for multiple tasks" according to the different working states of the robot (mobile / stationary), overcoming the contradiction of the fixed structure of the traditional system; (2) It provides the ultimate flexibility of "plug and play" algorithms: containerized encapsulation and dynamic loading mechanism make the deployment and updating of new algorithms as easy as installing an application on a mobile phone, reducing the time from the traditional "hourly" level to the "minute" level. The hardware-isolated runtime sandbox allows more than 20 different dependent algorithms to coexist without conflict. A single algorithm failure can be isolated and automatically restarted without recompiling or deploying the entire system, improving the overall availability of the system by more than an order of magnitude.

[0018] (3) Microsecond-level task switching and efficient algorithm collaboration were achieved: the task switching latency was reduced from tens of milliseconds in the software solution to the microsecond level through the time-slicing scheduler, meeting the robot's need for real-time switching in rapid scene changes; and combined with the cross-container shared context management unit, different algorithms can efficiently relay sensing data, avoiding redundant calculations. Experiments show that in the fixed-point guarding scenario, the "detection-tracking" time-sharing collaborative scheme reduces the overall computation time by 40% compared to the independent full-process operation of each algorithm, while achieving stable and continuous multi-target tracking.

[0019] (4) Significantly improves the energy efficiency of deep learning inference: The built-in unified quantization engine module provides standardized INT8 model optimization capabilities; compared with the original FP32 model, deploying the INT8 quantized model can achieve an inference speed increase of 3-4 times and a memory consumption reduction of about 75%, provided that the accuracy loss is acceptable (<1%). This enables the system to support more complex models or run more tasks in parallel under the same hardware conditions, with an overall energy efficiency (performance / watt) improvement of more than 100%.

[0020] (5) A highly reliable and scalable robot vision platform has been built: This system realizes the elastic allocation of computing resources, strong isolation of tasks and ecological management of algorithms from the hardware architecture level, providing a solid, reliable and infinitely scalable underlying support platform for robots to cope with complex and ever-changing vision tasks in the future. Attached Figure Description

[0021] Figure 1 This is a system structure block diagram according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the workflow of the reconfigurable computing platform according to an embodiment of the present invention. Figure 3 This is a schematic diagram of the deployment and execution process of the algorithm container in an embodiment of the present invention. Detailed Implementation

[0022] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] like Figure 1 and Figure 2As shown: A robot vision edge computing system includes a perception layer 100, a reconfigurable computing platform 200 and a robot decision layer 300 connected in sequence.

[0024] Furthermore, the perception layer 100 includes various image and environmental sensors, such as one or more image sensors (e.g., cameras), lidar, millimeter-wave radar, etc., to collect raw environmental data, obtain multi-source sensor data streams (e.g., images and lidar point clouds), and send the collected multi-source sensor data streams to the reconfigurable computing platform 200.

[0025] Furthermore, the reconfigurable computing platform 200 is the core module of this embodiment. As a heterogeneous computing device designed with hardware and software co-design, its hardware foundation integrates a system-on-a-chip (SOC), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a neural network processor (NPU), a graphics processing unit (GPU), and dynamic random access memory (DDR). The platform internally includes the following key modules: The input adapter and buffer module 201, connected to the perception layer 100, receives multi-source sensor data streams sent by the perception layer 100, performs hardware-level timestamp synchronization and format unification processing on the multi-source sensor data streams, and finally writes the data into the buffer memory. Specifically, the input adapter and buffer module 201 includes a multi-input interface and a buffer memory (FIFO). Its main functions are: to receive sensor data streams of different rates and protocols (such as MIPI CSI-2, GigE Vision, CAN); to perform hardware-level timestamp synchronization and alignment on multi-source sensor data streams such as images and LiDAR point clouds; to unify the synchronized data format into an internal processing format (such as RGB image blocks, point cloud matrices), and write it into the high-speed buffer memory, providing a stable, aligned, and low-latency data source for subsequent processing.

[0026] The dynamic scheduler 202, a hardware logic unit, serves as the core control hub of this system. It receives mode instructions from the robot decision layer 300, including at least movement mode instructions and stationary mode instructions. Based on these instructions, it controls the switching of data paths and the allocation of computing resources. Specifically, the dynamic scheduler 202 is implemented using programmable logic circuits (such as state machines) in an FPGA or ASIC. The dynamic scheduler 202 receives mode instruction electrical signals from the robot decision layer 300 (such as the main controller). These instructions include at least two types: movement mode instructions (such as "high-speed navigation and obstacle avoidance") and stationary mode instructions (such as "stationary duty and inspection"). After parsing the instructions, the dynamic scheduler 202 generates hardware-level control signals, directly controlling the switching of data paths and the allocators corresponding to computing resources (such as NPU, GPU, and CPU cores), thereby reconfiguring the platform's operating mode. After parsing instructions, the dynamic scheduler 202 generates hardware-level control signals that can switch the output destination of the sensor data stream, allowing it to be directly connected to a single algorithm container or cyclically distributed to multiple algorithm containers; or it can allocate and reclaim computing resources such as NPU, GPU, and CPU cores, concentrating them on a single algorithm container or distributing them to multiple algorithm containers in a time-slice cycle, thereby achieving real-time switching of working modes at the hardware level. For example, in mobile mode, its output signal will turn off the time-slicing scheduler 203 and directly connect the sensor data stream to the computing unit occupied by the designated exclusive algorithm container.

[0027] The time-slicing scheduler 203 is a dedicated hardware module consisting of a programmable high-precision timer (such as a crystal oscillator-based timing circuit) and a context fast save / restore circuit, serving as a dedicated timing and switching circuit. This module only starts upon receiving a fixed-point mode activation signal from the dynamic scheduler. The programmable high-precision timer can be configured for fixed durations (e.g., 10ms, 20ms, 50ms). At the end of each time slice, the timer generates a hardware interrupt, which triggers the context fast save / restore circuit. Through a dedicated direct memory access (DMA) channel, the complete computational context of the currently running algorithm container, including processor register states, critical data in the on-chip cache, and model weights (such as slices of weights from the neural network model being used), is quickly saved to a dedicated high-speed static random access memory (SRAM). Simultaneously, the context computational unit of the next algorithm container to be run is loaded. The entire save-load process is completed entirely at the hardware level, with switching overhead controlled within microseconds (e.g., less than 5μs), significantly lower than the software task switching overhead based on the operating system, thus achieving hardware-level deterministic time-sharing multiplexing.

[0028] The algorithm container loading and execution unit 204 is responsible for loading and isolating the algorithm container, and contains the following sub-modules that work together: Container image repository 204a is a protected partition located on non-volatile memory (such as eMMC, SSD) of a reconfigurable computing platform, used to store multiple (≥20) pre-packaged algorithm container image files. Each image file is a self-contained read-only file system containing the algorithm executable, required dependency libraries (such as specific versions of OpenCV, TensorRT), configuration files, and optimized neural network model files.

[0029] The Unified Quantization Engine Module 204b, integrated into the platform management software, provides standardized INT8 quantization services for deep learning models. Specifically, it offers standardized INT8 quantization services to algorithm developers. After a developer submits a trained FP32 deep learning model, this module automatically performs calibration (determining the activation value distribution using a representative dataset) and quantization-aware training steps to generate an optimized INT8 model. The quantized model can improve inference speed by 3-4 times and reduce memory usage by approximately 75% with minimal accuracy loss (e.g., less than 1% decrease in mean average precision (mAP) in typical object detection tasks).

[0030] The container loading engine module 204c is used to receive loading instructions from the platform management software, read the specified algorithm container image file from the container image repository 204a, decompress it and deploy it to a dedicated memory area that is pre-allocated and physically isolated by the hardware memory management unit (MMU). This process is completed within 1-2 seconds, and this area constitutes the running sandbox 204d of the algorithm container.

[0031] Sandbox 204d is used to run each successfully loaded algorithm container. The boundaries of the sandbox are jointly defined by the Hardware Memory Management Unit (MMU) and the Dynamic Scheduler 202, ensuring that each algorithm container can only access its allocated memory address space, a specific number of CPU cores, and a preset quota of GPU / NPU computing power. In other words, during runtime, the memory address space accessible to each loaded algorithm container, the number of CPU cores it can use, and its GPU / NPU computing power quota are all strictly limited and isolated by the MMU and the Dynamic Scheduler 202. This strong hardware isolation mechanism ensures that a container crashing or experiencing a memory leak due to an algorithm defect will not affect the operation of other containers or even the core system services, achieving fault isolation.

[0032] The cross-container shared context management unit 205 manages a shared memory region that can be securely and in a controlled manner accessed by all algorithm containers. This region stores and exchanges common perception state data (such as timestamped detection results) across time slices. It works in conjunction with the time-slicing scheduler 203 to ensure the correctness of data read / write timing, enabling the time-sharing containers to work collaboratively like a parallel pipeline. Specifically, the cross-container shared context management unit 205 manages a shared memory region that can be securely, in a controlled manner, and with low latency accessed by all algorithm containers through memory mapping. This region stores common perception states across time slices, and its data structure can be designed as a timestamped circular buffer. For example, the "face detection container" running in time slice A writes the detected face bounding boxes, feature vectors, confidence scores, and timestamps to this region; the "person re-identification container" running in the next time slice B does not need to rerun the detection network and can directly read this information for feature matching, thus achieving a near-parallel processing pipeline in the time-sharing architecture. This unit works in conjunction with the time-slicing scheduler 203 to ensure the timing correctness of data writing and reading through hardware signals, thus avoiding data races.

[0033] The output fusion and synchronization module 206 collects processing results from each running sandbox 204d and fusion information from the shared cross-container shared context management unit 205. It performs timestamp alignment, fusion, and redundancy removal on the results output by different algorithms, ultimately encapsulating them into unified structured perception information, which is then stably sent to the robot decision layer 300. Specifically, the output fusion and synchronization module 206 is responsible for timestamp alignment, data fusion (such as target association and trajectory smoothing), and redundancy removal on results from different algorithms that may have different frequencies and delays. It then encapsulates these results into unified structured perception information (such as target list, location, attributes, and trajectory), and outputs it stably and with low latency to the robot decision layer 300 via high-speed interfaces (such as PCIE and Ethernet) for navigation, decision-making, and control.

[0034] Furthermore, the robot decision layer 300 is the robot's main controller or upper-level planning system, used to send pattern instructions to the reconfigurable computing platform 200 and receive its feedback structured perception information in order to make decisions such as navigation, obstacle avoidance, and operation.

[0035] In this embodiment, the system can dynamically and quickly switch between the following two working modes based on the robot's state: Movement Mode: When the robot is in scenarios with extremely high real-time requirements, such as high-speed movement or emergency obstacle avoidance, the robot decision layer 300 sends a "movement mode" command to the dynamic scheduler 202. The dynamic scheduler 202 then generates control signals to: 1) shut down the time-slicing scheduler 203 and stop its periodic interruptions; 2) directly connect the data output path of the input adaptation and buffer module 201 to the pre-specified running sandbox 204d containing the algorithm container with the highest real-time requirements (such as the "visual odometry container" or the "near-range obstacle detection container"); 3) exclusively allocate the platform's main computing resources (such as 90% of the NPU computing power and large-capacity cache) to this container. In this mode, the end-to-end latency from sensor data input to the container's output is strictly guaranteed to be within 100 milliseconds, and can even reach 20-50 milliseconds, meeting the real-time requirements of high-speed movement.

[0036] Fixed-point mode: When the robot is in a scenario requiring the execution of multiple algorithms, such as fixed-point duty or precision operations, the robot decision layer 300 sends a "fixed-point mode" command. The dynamic scheduler 202 then performs the following actions: 1) Activating and configuring the time-slicing scheduler 203, for example, setting the time slice length to 20ms; 2) Circularly distributing the sensor data stream to the input buffers of multiple algorithm containers arranged in a scheduling sequence; 3) Setting the computing resources to be allocated in a time-slice rotation. The time-slicing scheduler 203 starts working, switching the currently active algorithm container at the hardware level in 20ms cycles. For example, in the first 20ms, the face detection container exclusively uses computing resources to run; at the end of the 20ms, the algorithm container loading and execution unit 204 quickly saves its context and loads the context of the behavior analysis container to start the execution of the next 20ms. This cycle continues, allowing a single hardware unit to execute dozens of algorithms in parallel, increasing resource utilization from less than 30% in traditional dedicated systems to over 80%.

[0037] In this embodiment, the system can achieve "plug-and-play" and efficient execution of algorithm containers. For example... Figure 3 As shown, the ecological deployment and execution process of algorithm containers is as follows: S101: Developers use the Unified Quantization Engine Module 204b to perform INT8 quantization on their trained FP32 deep learning model. The quantization engine automatically completes the calibration, generates the optimized INT8 model file, and packages it together with the algorithm executable program and the required dependency libraries to form an algorithm container image file that conforms to the platform specifications.

[0038] S102: Upload the algorithm container image file to the platform's container image repository 204a. This process can be completed via network or local interface.

[0039] S103: When the algorithm needs to be executed, the platform management software issues a loading command to the container loading engine module 204c. The loading engine extracts the algorithm container image file from the container image repository 204a and allocates an isolated, resource-controlled runtime environment in the runtime sandbox 204d.

[0040] S104: The algorithm container executes within the running sandbox 204d. In fixed-point mode, its execution is strictly controlled by the time-slicing scheduler 203; the algorithm container achieves collaborative awareness by reading and writing shared memory managed by the cross-container shared context management unit 205 to exchange data with other algorithm containers with low latency; finally, the algorithm execution result is output to the output fusion and synchronization module 206.

[0041] In this embodiment, regarding the time-slicing scheduling and collaborative workflow in fixed-point mode, the following example of a typical fixed-point monitoring scenario illustrates how multiple containers can work collaboratively through hardware slicing and shared memory: Time slice 0 (0-20ms): The face detection container is scheduled to execute. It reads a frame of image from the input buffer, runs its INT8 quantized detection network, writes the location, bounding box, and feature vector of all detected faces, along with the current timestamp, to a designated area of ​​the cross-container shared context management unit 205, and then suspends.

[0042] Time slice 1 (20-40ms): The people tracking container is scheduled to execute. Instead of rerunning the detection network, it directly reads all face detection results from the data written to the shared region in the previous slice. Then, it runs a lightweight tracking algorithm (such as Kalman filtering or correlation filtering), correlates the current detection results with historical trajectories, updates each person's motion trajectory and state, writes the updated trajectory information back to the shared region, and then suspends.

[0043] Time slice 2 (40-60ms): The behavior recognition container is scheduled to execute. Based on the tracking trajectory of people in the shared area, it extracts the region of interest (ROI), runs the behavior recognition network, judges the people's behavior (such as waving, running), associates the behavior label with the corresponding person ID, and outputs the final result.

[0044] Time slice 3 (60-80ms): can schedule other containers, such as instrument reading containers.

[0045] Time slice 4 (80-100ms): The scheduling cycle returns to the face detection container to start processing the next frame image, achieving continuous perception.

[0046] Throughout the process, the programmable high-precision timer of the time-slicing scheduler 203 precisely controls the start and end of each container, and the context fast save / restore circuit completes switching in microseconds. Perception data between different algorithms flows efficiently through the cross-container shared context management unit 205, forming a virtual pipeline of "detection-tracking-recognition," avoiding redundant calculations, ensuring the continuity of target IDs and the consistency of perception, and improving overall computational efficiency by more than 40% compared to running multiple full-function algorithms independently.

[0047] In this embodiment, the system's reliability is ensured through the following mechanism: Fault isolation: Each algorithm container runs in a 204d sandbox, where resource access is strictly limited by hardware. A crash, memory leak, or abnormally high load in a single container will not affect other containers or the system's core modules.

[0048] Health monitoring and self-healing: The platform management software continuously monitors the operational status of each container (such as heartbeat signals and output validity). Once an anomaly is detected in a container, it can automatically send a command to the container loading engine module 204c to restart the abnormal container without affecting the operation of other containers.

[0049] Mode downgrade: If a critical container (such as a detection container) repeatedly fails in stationary mode, the platform management software can send a command to the dynamic scheduler 202 to trigger the system to downgrade from "stationary mode" to "mobile mode" to ensure that at least one core function (such as obstacle avoidance) is available and to ensure the basic safety of the robot.

[0050] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A robot vision edge computing system, characterized in that, It includes a perception layer, a reconfigurable computing platform, and a robot decision-making layer connected in sequence; The reconfigurable computing platform includes: The input adaptation and buffering module is used to receive multi-source sensor data streams from the perception layer and perform hardware-level timestamp synchronization and format unification. A dynamic scheduler is used to receive mode instructions from the robot's decision-making layer and generate hardware control signals according to the mode instructions to control the switching path of multi-source sensor data streams and allocate computing resources within the platform. The algorithm container loading and execution unit is used to load and run algorithm containers; A time-slicing scheduler, whose activation is controlled by the dynamic scheduler, is used at the hardware level to periodically switch the algorithm containers loaded and running in the execution unit at fixed time slices. The dynamic scheduler is configured to: respond to a first mode instruction from the robot decision layer, control the multi-source sensor data stream of the input adaptation and buffer module to be directly connected to the specified algorithm container in the algorithm container loading and execution unit, and disable the time-slicing scheduler; respond to a second mode instruction from the robot decision layer, control the multi-source sensor data stream of the input adaptation and buffer module to be distributed among multiple algorithm containers in the algorithm container loading and execution unit, and activate the time-slicing scheduler to control the time-sharing execution of the multiple algorithm containers.

2. The robot vision edge computing system according to claim 1, characterized in that, The time-slicing scheduler includes a programmable timer and a context fast save / restore circuit. The programmable timer is used to generate periodic hardware interrupt signals, and the context fast save / restore circuit is used to respond to the hardware interrupt signals by saving the execution context of the current algorithm container at the hardware level through a direct memory access channel and loading the execution context of the next algorithm container.

3. The robot vision edge computing system according to claim 1, characterized in that, The algorithm container loading and execution unit further includes the following sub-modules that work together: A container image repository is used to store multiple algorithm container image files; The unified quantization engine module is used to perform standardized quantization operations on deep learning models and output optimized model files to the container image repository. The container loading engine module is used to load the specified algorithm container image file from the container image repository according to the loading instruction, and deploy it to the isolated memory area divided by the hardware memory management unit; The running sandbox, created by the container loading engine module when loading the algorithm container image file, provides the algorithm container with a running environment that is forcibly isolated by the hardware memory management unit and the dynamic scheduler.

4. The robot vision edge computing system according to claim 3, characterized in that, The reconfigurable computing platform also includes a cross-container shared context management unit, which is connected to the algorithm container loading and execution unit and is used to provide a memory region that can be shared and accessed by multiple algorithm containers, so that algorithm containers executing in different time slices can exchange intermediate perception data through the shared memory region.

5. The robot vision edge computing system according to claim 4, characterized in that, The reconfigurable computing platform also includes an output fusion and synchronization module, which collects the processing results from each of the running sandboxes and the fusion information from the shared cross-container shared context management unit, performs timestamp alignment and fusion on the output results of each algorithm container, and generates structured perception information to be sent to the robot decision layer.

6. The robot vision edge computing system according to claim 4 or 5, characterized in that, Under the second mode instruction, the time-slicing scheduler works in collaboration with the cross-container shared context management unit, so that after the first algorithm container executed in the first time slice writes the result to the shared memory area, the second algorithm container executed in the second time slice can directly read the result, forming a virtual collaborative awareness pipeline.

7. The robot vision edge computing system according to claim 3, 4, or 5, characterized in that, The reconfigurable computing platform also ensures system reliability through the collaboration of the following modules: The algorithm container loading and execution unit is configured to provide fault isolation through its hardware-isolated runtime sandbox; The container loading engine module is configured to perform a restart operation on the specified algorithm container in response to an external reset signal; The dynamic scheduler is configured to trigger the system to switch back from the second mode to the first mode in response to an external mode switching signal.

8. A calculation method based on the robot vision edge computing system according to any one of claims 1 to 7, characterized in that, Including mode switching methods: Receive pattern instructions from the robot's decision-making layer; The mode instructions are parsed by a dynamic scheduler and hardware control signals are generated. If it is the first mode instruction, execute the first configuration: turn off the time-slicing scheduler, pass the input multi-source sensor data stream directly to the algorithm container specified in the algorithm container loading and execution unit, and centrally allocate computing resources to the algorithm container; If it is the second mode instruction, execute the second configuration: activate the time-slicing scheduler, cyclically distribute the input multi-source sensor data stream among multiple algorithm containers in the algorithm container loading and execution unit, and have the time-slicing scheduler control the multiple algorithm containers to take turns executing according to time slices.

9. The calculation method of the robot vision edge computing system according to claim 8, characterized in that, In the second configuration, a collaborative working method is also included: The time-slicing scheduler generates hardware interrupts at fixed intervals to trigger hardware-level context switching in order to schedule multiple algorithm containers in a round-robin fashion. In any Nth time slice, the scheduled algorithm container writes its output perception data into the shared memory region managed by the cross-container shared context management unit, where N≥1; In the following N+1 time slice, another scheduled algorithm container directly reads the perceived data from the shared memory region for processing; Through the shared memory region and periodic time slice scheduling, a data relay and collaborative perception pipeline is realized among multiple algorithm containers.

10. The calculation method of the robot vision edge computing system according to claim 8 or 9, characterized in that, It also includes the deployment and execution process of the algorithm container, with the following steps: S1: Use the unified quantization engine module to perform standardized quantization on the trained deep learning model to generate an optimized model file; package the optimized model file with the corresponding algorithm executable program and dependent libraries to form an algorithm container image file that conforms to the platform specifications; S2: Upload and store the algorithm container image file in the container image repository; S3: In response to the loading command, the specified algorithm container image file is read from the container image repository through the container loading engine module and deployed to the runtime sandbox, where an isolated and resource-controlled runtime environment is allocated. S4: Run the algorithm container in the running sandbox; When the system is operating in the second mode, the execution process of the algorithm container is scheduled in a time-sharing manner by the time-slicing scheduler using hardware interrupts; the running algorithm container exchanges data with other algorithm containers through the shared memory area managed by the cross-container shared context management unit to achieve collaborative perception.

Images