A robotic base architecture system and method

Abstract

The application discloses a kind of robot bottom layer architecture system and method, for the technical pain points of high serial delay, mismatch of computing resources, semantic understanding gap, system behavior unpredictable existing in existing robot bottom layer architecture, system is by sensor array, dynamic data flow engine, semantic understanding and prediction unit, decision and planning unit, resource-aware controller and the whole link closed loop architecture of actuator drive array composition, supporting collection distribution semantic decision control execution optimization control method.By dynamic data flow engine, path dynamic arrangement and resource hard isolation are realized, resource-aware controller realizes global resource elastic scheduling, semantic understanding and prediction unit constructs two-way closed loop feedback, whole link adapts high dynamic scene demand.The beneficial effect is in: high computing power utilization, accurate semantic identification, deterministic behavior output and full-scene autonomous adaptation, significantly improve emergency response efficiency and operation reliability.

Description

Technical Field

[0001] This invention belongs to the field of robotics technology, specifically relating to a robot underlying architecture system and method. Background Technology

[0002] With the development of robotics technology, robots have rapidly evolved from traditional pre-programmed execution to full-scenario autonomous intelligence. In this process, the underlying control architecture of robots faces a fundamental challenge: the contradiction between the powerful computing power required for autonomous intelligence and the real-time deterministic requirements in safety-critical scenarios. Traditional autonomous robots need to be equipped with various sensors such as vision, LiDAR, and IMU (Inertial Measurement Unit) to achieve full-dimensional perception of the environment. The size of a single frame of raw data can reach hundreds of megabytes. This high-throughput and strongly temporally correlated data characteristic places higher demands on the perception, decision-making, and motion control capabilities of modern autonomous robots. This leads to stringent requirements for low latency and high determinism, especially in critical tasks such as dynamic obstacle avoidance, human-robot interaction, and precision operations, where millisecond or even sub-millisecond latency requirements have become the norm.

[0003] The currently widely adopted robot architecture in the industry is a layered architecture based on a publish-subscribe model, such as the ROS / ROS2 architecture. This architecture achieves system decoupling through modular design, but its internal logic is still limited by a serial pipeline pattern from perception acquisition, data preprocessing, decision planning to motion control. While this structure offers some improvements in optimizing single-point performance, it cannot fundamentally solve the technical shortcomings of autonomous intelligent tasks in highly dynamic and unstructured scenarios. In particular, the serialized data flow becomes the main bottleneck for end-to-end system latency, especially in scenarios with multiple sensors operating concurrently, easily leading to data congestion and command delays, thus severely impacting the robot's responsiveness.

[0004] Furthermore, existing allocation strategies are typically pre-configured based on preset computing units, meaning that once computing resources are allocated, dynamic adjustments are difficult. In real-world dynamic scenarios, the computing power required for visual perception can reach over 80% during high-speed movement, while for precise force control or human-computer interaction scenarios, priority must be given to ensuring computing power for real-time force control algorithms. This static binding model leads to a mismatch of computing resources, potentially resulting in insufficient computing power support for critical tasks or idle resources, failing to maximize efficiency across all scenarios. This mechanism also makes the originally open allocation of computing power rigid, further exacerbating the imbalance in resource utilization and reducing the overall real-time performance and stability of the system.

[0005] Therefore, in order to address the aforementioned technical problems, it is necessary to provide a robot underlying architecture system and method.

[0006] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0007] The purpose of this invention is to provide a robot underlying architecture system and method that can solve the technical problems mentioned in the background art.

[0008] To achieve the above objectives, a specific embodiment of the present invention provides the following technical solution: A robot underlying architecture system includes a sensor array, a dynamic data stream engine, a semantic understanding and prediction unit, a decision-making and planning unit, a resource-aware controller, and an actuator drive array. The sensor array is used to collect raw perception data of the robot's own state and the surrounding environment. The dynamic data stream engine communicates bidirectionally with the sensor array to dynamically generate and adjust the data stream processing path with temporal constraints and resource isolation attributes based on real-time task requirements, semantic feedback, and scene prediction information, and to complete the priority distribution of perception data. The semantic understanding and prediction unit communicates bidirectionally with the dynamic data stream engine to receive data along the processing path, fuse it to generate the current scene semantic state vector, and predict the future short-time window scene evolution. The trend generates a predicted state vector and feeds back two types of vectors to the dynamic data flow engine decision and planning unit and the semantic understanding and prediction unit. The dynamic data flow engine communicates bidirectionally to combine the semantic and predicted state vectors, the target task, and the system resource status to complete multi-objective optimization decisions and generate decision instructions containing the expected behavior sequence. The resource-aware controller communicates bidirectionally with the decision and planning unit and the upstream core units to receive decision instructions, collect the system resource status in real time to generate a global resource status vector, dynamically generate control parameters, and feed back the resource status to the upstream units. The actuator drive array communicates with the resource-aware controller to receive control parameters and generate drive signals to control the actuator actions.

[0009] In one or more embodiments of the present invention, the dynamic data flow engine dynamically generates data flow processing paths, specifically by: analyzing the core perception requirements and computing power requirements of the current task scenario in real time, selecting matching data processing operators from a preset operator library, arranging the data processing operators according to processing logic and timing constraints, generating data flow processing paths with clear data dependencies, timing deadlines and priority attributes, and allocating computing, caching and bus bandwidth resources with hard isolation attributes to the paths.

[0010] In one or more embodiments of the present invention, the dynamic data stream engine adjusts the data stream processing path in real time, specifically as follows: when receiving the semantic state vector and the prediction state vector, for high-priority semantic targets and high-risk prediction scenarios, the corresponding data distribution priority is increased and the path computing resources are expanded; when receiving the global resource state vector and identifying resource overload, a degradation strategy is executed to reduce the sampling frequency of non-core data, switch lightweight operators, and suspend non-critical branches; when receiving task priority and computing power requirement information, the path is pre-arranged and resources are pre-allocated for the sequence of behaviors to be executed.

[0011] In one or more embodiments of the present invention, the semantic understanding and prediction unit generates a semantic state vector by: performing spatiotemporal synchronization alignment on the original perception data based on the hardware timestamps and spatial calibration parameters of each sensor to generate a multimodal synchronization data frame with unified timestamps and matched spatial coordinate systems; inputting the synchronization data frame into a multimodal fusion deep neural network model to complete cross-dimensional feature extraction, association and fusion to generate a high-dimensional fusion feature vector representing the core attributes of the scene; and mapping the high-dimensional fusion feature vector to a preset standardized semantic space to generate a semantic state vector containing the robot's own state, environmental object attributes, relative pose and dynamic parameters.

[0012] In one or more embodiments of the present invention, the semantic understanding and prediction unit generates a predicted state vector by: taking the semantic state vector synchronized at multiple consecutive time points as input and importing it into a preset temporal prediction model. The temporal prediction model extrapolates and predicts the scene semantic state at each time point within a preset short time window based on the temporal evolution law of historical semantic states and combined with the robot's own decision-making behavior trend, generating a predicted state vector that includes the dynamic change trend of the environment, potential risk areas, and changes in the state of the task target, and synchronously feeding it back to the dynamic data stream engine.

[0013] In one or more embodiments of the present invention, the decision-making and planning unit performs multi-objective optimization decision-making and generates decision instructions, specifically by: using the semantic state vector as the initial state constraint, the predicted state vector as the future scenario constraint, and the global resource state vector as the computing power constraint, creating a multi-objective optimization cost function with the objectives of satisfying preset task efficiency, system energy consumption, behavior safety, and state uncertainty thresholds, searching for the desired behavior sequence that satisfies the real-time and safety constraints in the robot's feasible behavior space, and generating corresponding decision instructions.

[0014] In one or more embodiments of the present invention, the resource-aware controller generates control parameters by: collecting the CPU load rate, GPU computing power utilization rate, memory utilization rate, bus bandwidth utilization rate, and actuator operating status of each computing unit in the entire system at a preset fixed period, generating a global resource state vector, parsing the decision instructions into the control objectives, accuracy requirements, and response time constraints of the actuators, importing them into a preset resource-aware control model, and dynamically adjusting the computational order, sampling frequency, and computational accuracy of the control algorithm under the premise of meeting the response time constraints, and solving to generate control parameters that meet the preset control performance and system stability indicators.

[0015] In one or more embodiments of the present invention, the semantic understanding and prediction unit dynamically adjusts the model size and inference accuracy of the multimodal fusion deep neural network model based on the received global resource state vector; when it detects that computing resources are tight, it automatically switches to a lightweight inference model to reduce computing power consumption and ensure the real-time performance of semantic output.

[0016] To achieve the above objectives, a specific embodiment of the present invention provides the following technical solution: A robot underlying architecture method includes the following steps: S1. Collect raw perception data of the robot's own state and the surrounding environment through a sensor array; S2. Through the dynamic data flow engine, based on the real-time requirements of the task, semantic feedback and scene prediction information, the data flow processing path with time constraints and resource isolation attributes is dynamically generated and adjusted to complete the priority distribution of perceived data. S3. The semantic understanding and prediction unit receives data along the processing path, generates a semantic state vector through multimodal fusion, predicts the future short-term window scene evolution trend to generate a prediction state vector, and feeds the two types of vectors back to the dynamic data stream engine. S4. By combining semantic and predicted state vectors, target tasks and system resource states through the decision and planning unit, multi-objective optimization decisions that meet preset constraints are executed to generate decision instructions containing expected behavior sequences. S5. Receive decision instructions through the resource-aware controller, collect the resource status of the entire system in real time to generate a global resource status vector, dynamically generate control parameters that meet preset constraints, and feed back the resource status to the corresponding unit. S6. Receive control parameters through the actuator drive array and generate drive signals to control the actuator action.

[0017] In one or more embodiments of the present invention, the dynamic data stream engine dynamically adjusts the data stream processing path and computing power allocation based on the real-time received semantic state, prediction state, global resource state and task priority information; the semantic understanding and prediction unit dynamically adjusts the model inference accuracy based on the global resource state; and the decision-making and planning unit dynamically adjusts the decision constraints and behavior sequence planning granularity based on the global resource state to complete the full-link closed-loop optimization.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: By using the parallel path orchestration and hard resource isolation design of the dynamic data flow engine, the problems of high end-to-end latency, data congestion and instruction lag caused by traditional serial pipeline architecture are solved, thereby improving response time. By using the global resource status acquisition of the resource-aware controller and the computing power elastic scheduling mechanism of the dynamic data flow engine, the resource mismatch problem caused by the static binding of computing resources and task requirements in the existing architecture is solved, avoiding the situation of insufficient computing power for core tasks and idle resources for non-core tasks, and improving the computing power utilization rate of the whole scenario. By using multimodal fusion semantic generation, temporal scene prediction, and bidirectional closed-loop feedback with the data stream engine, the understanding gap between raw data and semantic decision-making is solved, as well as the semantic misjudgment, missed detection, and wasted computing power caused by one-way open-loop transformation, thereby improving the accuracy of semantic recognition and the targeting of data processing. By allocating compute, cache, and bus bandwidth resources with hard isolation attributes to the data stream processing path, and combining this with explicit timing constraints, the problem of unpredictable system behavior caused by resource preemption during multi-task concurrency is solved. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is an overview diagram of a robot underlying architecture system according to an embodiment of the present invention; Figure 2 This is a flowchart of the path generation process of the dynamic data flow engine in one embodiment of the present invention; Figure 3 This is a flowchart of the path adjustment process of the dynamic data stream engine in one embodiment of the present invention; Figure 4 This is a flowchart of semantic state vector generation in one embodiment of the present invention; Figure 5 This is a closed-loop diagram of resource awareness and execution control in one embodiment of the present invention; Figure 6 This is a basic flowchart of a robot underlying architecture method in one embodiment of the present invention; Figure 7 This is a closed-loop optimization process for a robot underlying architecture method in one embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions in this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0022] like Figures 1-5 As shown, a robot underlying architecture system according to one embodiment of the present invention includes a sensor array, a dynamic data stream engine, a semantic understanding and prediction unit, a decision and planning unit, a resource-aware controller, and an actuator drive array. The sensor array includes LiDAR, a vision camera, an inertial measurement unit (IMU), joint encoders, force sensors, etc., used to collect raw perception data of the robot's own state and surrounding environment. Each data frame carries a unique hardware timestamp, sensor ID, and spatial calibration parameters. These spatial calibration parameters are obtained in advance through a calibration algorithm to ensure data accuracy. Simultaneously, it can receive sampling frequency and sampling precision adjustment commands from the dynamic data stream engine, providing high-quality, highly synchronous raw data for subsequent data processing.

[0023] The dynamic data stream engine communicates bidirectionally with the sensor array to dynamically generate and adjust data stream processing paths with time constraints and resource isolation attributes based on real-time task requirements, semantic feedback, and scene prediction information, thereby completing the priority distribution of perceived data.

[0024] Specifically, the dynamic data stream engine selects matching data processing operators from a pre-set operator library based on real-time task requirements, feedback from the semantic understanding and prediction unit, and scene prediction results. This enables functions such as filtering, feature extraction, timing synchronization, and data compression. The data processing operators are arranged according to processing logic and timing constraints to generate data stream processing paths with clear data dependencies, timing deadlines, and priority attributes. Each path is allocated computation, cache, and bus bandwidth resources with hard isolation attributes to avoid multi-path resource contention and fundamentally solve the serial latency problem. Simultaneously, the engine has a path fault tolerance mechanism; when a path fails, it can quickly switch to a backup path to ensure data stream continuity.

[0025] Specifically, the path generation and adjustment of the dynamic data flow engine includes: Generation Path: The core perception and computing power requirements of the current task scenario are analyzed in real time. Matching data processing operators, such as SIFT feature extraction for visual data and Kalman filtering for LiDAR data, are selected from a pre-defined operator library. The path is arranged according to the processing logic of "data preprocessing → feature extraction → data fusion" and timing constraints. Computational, cache, and bus bandwidth resources with hard isolation attributes are allocated to the path, and the timing deadline is set according to the task type. In other words, data processing operators are arranged according to processing logic and timing constraints to generate a data flow processing path with clear data dependencies, timing deadlines, and priority attributes, and computational, cache, and bus bandwidth resources with hard isolation attributes are allocated to the path.

[0026] Path adjustment: Based on feedback from the semantic understanding and prediction unit, increase the computing power quota and data distribution priority of paths corresponding to high-priority semantic targets and high-risk scenarios; based on resource overload information from the resource-aware controller, implement degradation strategies such as reducing the sampling frequency of non-core data, switching lightweight operators, and pausing non-critical branches; based on task priority and computing power requirements from the decision-making and planning unit, pre-arrange paths and pre-allocate resources for the sequence of behaviors to be executed to avoid delays and jitter caused by task switching.

[0027] The semantic understanding and prediction unit communicates bidirectionally with the dynamic data stream engine to receive data along the processing path, fuse it to generate the current scene semantic state vector, predict the future short-time window scene evolution trend to generate a predicted state vector, and feed the two types of vectors back to the dynamic data stream engine.

[0028] Specifically, the semantic understanding and prediction unit receives data distributed directionally by the dynamic data stream engine, generates a semantic state vector for the current scene through spatiotemporal synchronization and multimodal fusion, and simultaneously predicts the scene evolution trend in short time windows (50-100ms) based on a time-series prediction model, generating a predicted state vector. These two types of vectors are fed back to the dynamic data stream engine for dynamic adjustment of the data stream processing path; and sent to the decision-making and planning unit to provide accurate data support for decision-making. Meanwhile, the semantic understanding and prediction unit can dynamically adjust the size and inference accuracy of its multimodal fusion deep neural network model based on the global resource state vector fed back by the resource-aware controller, balancing recognition accuracy and computational power consumption—that is, switching to a lightweight model when resources are scarce and switching to a high-precision model when resources are abundant.

[0029] The semantic understanding and prediction unit dynamically adjusts the model size and inference accuracy of the multimodal fusion deep neural network model based on the received global resource state vector. When it detects that computing resources are scarce, it automatically switches to a lightweight inference model to reduce computing power consumption and ensure the real-time nature of semantic output.

[0030] The dynamic data stream engine adjusts the data stream processing path in real time, specifically as follows: Upon receiving the semantic state vector and the prediction state vector, for high-priority semantic targets and high-risk prediction scenarios, the corresponding data distribution priority is increased and the path computing resources are expanded; When the global resource state vector is received and resource overload is detected, a degradation strategy is executed, which includes reducing the sampling frequency of non-core data, switching lightweight operators, and pausing non-critical branches. Upon receiving task priority and computing power requirements, the system pre-arranges paths and pre-allocates resources for the sequence of actions to be executed.

[0031] The semantic state vector in the semantic understanding and prediction unit is generated through spatiotemporal synchronization alignment (using linear interpolation to unify timestamps and using rotation matrix and translation vector mapping to match spatial coordinate systems), multimodal fusion (using a deep neural network model with encoder and decoder structure to complete cross-sensor feature extraction and association fusion), and semantic space mapping (mapping high-dimensional fused feature vectors to a preset standardized semantic space). The vector dimension is 64-dimensional, including the robot's own state (16-dimensional), environmental object attributes (24-dimensional), relative pose (16-dimensional), and dynamic parameters (8-dimensional).

[0032] The predicted state vector in the semantic understanding and prediction unit is based on the semantic state vector synchronized for 10 consecutive time steps as input. It is then imported into a preset LSTM temporal prediction model. The model extrapolates and predicts the scene semantic state at each time step within the next 50~100ms based on the temporal evolution of historical semantic states and combined with the robot's own decision-making behavior trends. It generates a predicted state vector that includes the dynamic change trend of the environment, potential risk areas, and changes in the state of the task target, and synchronously feeds it back to the dynamic data stream engine.

[0033] In other words, the semantic understanding and prediction unit generates a semantic state vector, specifically: Based on the hardware timestamps and spatial calibration parameters of each sensor, the raw sensing data is spatiotemporally synchronized and aligned to generate multimodal synchronized data frames with unified timestamps and matching spatial coordinate systems. The synchronous data frame is input into the multimodal fusion deep neural network model to complete cross-dimensional feature extraction, association and fusion, and generate a high-dimensional fusion feature vector that represents the core attributes of the scene; The high-dimensional fused feature vector is mapped to a pre-defined standardized semantic space to generate a semantic state vector containing the robot's own state, environmental object attributes, relative pose, and dynamic parameters.

[0034] The semantic understanding and prediction unit generates a prediction state vector, specifically as follows: The semantic state vectors synchronized at multiple consecutive moments are used as input and imported into a preset temporal prediction model. Based on the temporal evolution of historical semantic states and combined with the robot's own decision-making behavior trends, the temporal prediction model extrapolates and predicts the scene semantic state at each moment within a preset short time window in the future, generating a predicted state vector that includes the dynamic change trend of the environment, potential risk areas, and changes in the state of the task target, and synchronously feeds it back to the dynamic data stream engine.

[0035] The decision-making and planning unit communicates bidirectionally with the semantic understanding and prediction unit and the dynamic data flow engine. This allows them to combine semantic and predicted state vectors, target tasks, and system resource states to complete multi-objective optimization decisions and generate decision instructions containing desired behavior sequences. Specifically, the decision-making and planning unit combines the semantic state vector and predicted state vector output by the semantic understanding and prediction unit, along with system-preset target tasks such as precision assembly, dynamic obstacle avoidance, and path navigation, and the global resource state vector fed back by the resource-aware controller, to complete multi-objective optimization decisions. This is achieved by creating a multi-objective optimization cost function to search for desired behavior sequences that satisfy real-time and safety constraints in the robot's feasible behavior space, generating corresponding decision instructions such as joint motion angles, end effector posture, and motion speed. Simultaneously, the current task priority and computational requirements are fed back to the dynamic data flow engine, enabling collaborative adaptation between tasks and resources.

[0036] The resource-aware controller communicates bidirectionally with the decision-making and planning unit and upstream core units to receive decision commands, collect the resource status of the entire system in real time to generate a global resource status vector, dynamically generate control parameters, and feed back the resource status to upstream units.

[0037] Under normal circumstances, the resource-aware controller collects real-time data on the entire system's resource status, including CPU load rate, GPU computing power utilization, memory utilization, bus bandwidth utilization, and sensor and actuator operating status, at a preset fixed period. It generates a global resource status vector and broadcasts this vector to all upstream core units, providing a basis for end-to-end resource adaptation. Simultaneously, it receives decision commands from the decision and planning unit, parses them into the actuator's control objectives, accuracy requirements, and response time constraints, imports them into a preset resource-aware control model, and dynamically adjusts the computational order, sampling frequency, and computational accuracy of the control algorithm while meeting the response time constraints. It then solves for and generates control parameters that meet preset control performance and system stability indicators, ensuring the real-time performance and determinism of the control commands.

[0038] The decision-making and planning unit performs multi-objective optimization decisions and generates decision instructions. Specifically, it uses semantic state vectors as initial state constraints, predicted state vectors as future scenario constraints, and global resource state vectors as computing power constraints. It creates a multi-objective optimization cost function with the objectives of satisfying preset task efficiency, system energy consumption, behavior safety, and state uncertainty thresholds. It searches for expected behavior sequences that meet real-time and safety constraints in the robot's feasible behavior space and generates corresponding decision instructions.

[0039] More specifically, a multi-objective optimization cost function is created with semantic state vectors as initial state constraints, predicted state vectors as future scenario constraints, and global resource state vectors as computing power constraints, aiming to meet preset task efficiency, system energy consumption, behavior safety, and state uncertainty thresholds. A genetic algorithm is used to search for expected behavior sequences that meet real-time and safety constraints in the robot's feasible behavior space, and corresponding decision instructions are generated. The decision instruction update cycle is consistent with the system control cycle.

[0040] The actuator drive array communicates with the resource-aware controller to receive control parameters and generate drive signals to control the actuator's actions. The actuator drive array receives control parameters from the resource-aware controller, generates precise drive signals to control the actuator's actions, and simultaneously collects real-time operating status data such as actuator position, speed, torque, and temperature through built-in sensors, feeding this data back to the resource-aware controller to form a complete control closed loop. This ensures that decision commands are accurately executed and that execution anomalies are promptly reported to guarantee system safety.

[0041] Among them, the resource-aware controller generates control parameters, specifically by collecting the CPU load rate, GPU computing power utilization rate, memory utilization rate, bus bandwidth utilization rate and actuator running status of each computing unit in the entire system at a preset fixed period to generate a global resource status vector. The decision instructions are parsed into the control objectives, accuracy requirements and response time constraints of the actuator, and imported into the preset resource-aware control model. Under the premise of meeting the response time constraints, the computation order, sampling frequency and computation accuracy of the control algorithm are dynamically adjusted to solve and generate control parameters that meet the preset control performance and system stability indicators.

[0042] Under normal circumstances, the CPU load rate, GPU computing power utilization rate, memory utilization rate, bus bandwidth utilization rate, and actuator operating status of each computing unit in the entire system are collected at fixed intervals. A global resource state vector is generated through normalization processing. The decision instructions issued by the decision and planning unit are parsed into the control objectives, accuracy requirements, and response time constraints of the actuators. The preset resource-aware control model of PID + fuzzy control fusion is imported into the model. Under the premise of meeting the response time constraints, the computation order, sampling frequency, and computation accuracy of the control algorithm are dynamically adjusted to solve and generate control parameters that meet the preset control performance and system stability indicators, so as to ensure that the actuators move accurately and stably.

[0043] In this embodiment, the workflow of the robot's underlying architecture system is as follows: the sensor array collects raw data of the robot itself and its environment and outputs it to the dynamic data flow engine; the dynamic data flow engine dynamically orchestrates data flow paths with resource isolation based on the task, semantic feedback, and resource status, and distributes data to the semantic understanding and prediction unit; the semantic understanding and prediction unit generates semantic and scene prediction vectors, feeds them back to the engine, and sends them to the decision and planning unit; the decision unit generates decision instructions by combining multiple types of constraints, the resource-aware controller collects global resource status, parses instructions, generates control parameters, and sends them to the actuator drive array; the actuator completes the action and reports the running status, and the linkage of each unit realizes full-link closed-loop optimization, ensuring low-latency and high-reliability operation of the system.

[0044] like Figures 5-6 As shown, a robot underlying architecture method according to an embodiment of the present invention includes the following steps: S1. Collect raw perception data of the robot's own state and the surrounding environment through a sensor array; Specifically, the sensor array collects raw perception data of the robot's own state and surrounding environment according to a preset cycle or the adjusted sampling instructions issued by the dynamic data stream engine. Each data frame carries a unique hardware timestamp, sensor ID, and spatial calibration parameters. After preliminary noise reduction processing, the data is output to the dynamic data stream engine. If a sensor is detected to have data loss, abnormal signal, or other faults, a fault alarm is triggered, and the system switches to a backup sensor to collect data.

[0045] S2. Through the dynamic data flow engine, based on the real-time requirements of the task, semantic feedback and scene prediction information, the data flow processing path with time constraints and resource isolation attributes is dynamically generated and adjusted to complete the priority distribution of perceived data. Specifically, after receiving the raw data output by the sensor array, the dynamic data stream engine dynamically generates and adjusts the data stream processing path with time constraints and resource isolation attributes based on the current target task requirements, the semantic and prediction state vectors fed back by the semantic understanding and prediction unit, and the global resource state vectors fed back by the resource-aware controller. The raw sensing data is then distributed to the corresponding processing operators of the semantic understanding and prediction unit according to the path priority.

[0046] S3. The semantic understanding and prediction unit receives data along the processing path, generates a semantic state vector through multimodal fusion, predicts the future short-term window scene evolution trend to generate a prediction state vector, and feeds the two types of vectors back to the dynamic data stream engine. Specifically, the semantic understanding and prediction unit receives targeted data along the processing path allocated by the dynamic data flow engine. First, the data is spatiotemporally synchronized and aligned (timestamps are unified to the system reference time, and spatial coordinates are matched to the robot's body reference coordinate system). Then, a multimodal fusion deep neural network model is used to complete cross-dimensional feature extraction, association, and fusion to generate the current scene semantic state vector. Subsequently, the semantic state vectors of 10 consecutive time steps are input into the LSTM time-series prediction model to predict the scene evolution trend within the next 50-100ms and generate a predicted state vector. Finally, the two types of vectors are fed back to the dynamic data flow engine (for path optimization) and simultaneously sent to the decision and planning unit (for decision support).

[0047] S4. By combining semantic and predicted state vectors, target tasks and system resource states through the decision and planning unit, multi-objective optimization decisions that meet preset constraints are executed to generate decision instructions containing expected behavior sequences. Specifically, the decision-making and planning unit receives the semantic state vector and predicted state vector output by the semantic understanding and prediction unit, the global resource state vector fed back by the resource-aware controller, and the system's preset target task instructions. Combining the three types of constraints, it creates a multi-objective optimization cost function, uses a genetic algorithm to search for the desired behavior sequence that satisfies the real-time and safety constraints in the robot's feasible behavior space, generates decision instructions containing the desired behavior sequence, and sends them to the resource-aware controller. At the same time, it feeds back the current task priority and computing power requirements to the dynamic data flow engine to achieve collaborative adaptation between tasks and resources.

[0048] The dynamic data stream engine dynamically adjusts the data stream processing path and computing power allocation based on the real-time received semantic state, prediction state, global resource state and task priority information. The semantic understanding and prediction unit dynamically adjusts the model's inference accuracy based on the global resource status; The decision-making and planning unit dynamically adjusts the decision constraints and behavior sequence planning granularity based on the global resource status to complete the full-link closed-loop optimization.

[0049] S5. Receive decision instructions through the resource-aware controller, collect the resource status of the entire system in real time to generate a global resource status vector, dynamically generate control parameters that meet preset constraints, and feed back the resource status to the corresponding unit. Specifically, the resource-aware controller receives decision commands from the decision and planning unit, collects the resource status of the entire system in real time at a fixed period of 1ms, generates a global resource status vector after normalization processing, and broadcasts the vector to the dynamic data flow engine, semantic understanding and prediction unit, and decision and planning unit. At the same time, it parses the decision commands into the control objectives, accuracy requirements and response time constraints of the actuators, imports them into the resource-aware control model, dynamically adjusts the control algorithm parameters, solves and generates control parameters that meet the preset control performance and system stability indicators, and sends them to the actuator drive array.

[0050] S6. Receive control parameters through the actuator drive array and generate drive signals to control the actuator action; Specifically, the actuator drive array receives control parameters from the resource-aware controller, generates precise drive signals, and controls the actuators of each joint of the robot and the end effector to act according to the decision instructions. At the same time, it collects the real-time operating status of the actuators through built-in sensors, and feeds them back to the resource-aware controller after A / D conversion, forming a control closed loop and completing one control cycle.

[0051] The beneficial effects of a robot underlying architecture system and method in this invention include at least the following: By using a dynamic data stream engine to achieve parallel targeted processing and hard resource isolation, the system's end-to-end latency is significantly reduced compared to the existing architecture. Core tasks can reliably meet sub-millisecond real-time requirements and are adapted to high-speed dynamic operation scenarios.

[0052] By linking the resource-aware controller with the dynamic data flow engine, the dynamic allocation and recycling of computing resources are realized, and the computing power utilization rate is greatly improved. This not only avoids insufficient computing power for core tasks, but also solves the problem of idle resources for non-core tasks, thereby maximizing the computing power utilization efficiency across all scenarios and reducing system energy consumption.

[0053] Through a two-way closed loop between the semantic understanding and prediction unit and the dynamic data flow engine, the semantic recognition results and scene prediction information are fed back to the data processing link in real time. This optimizes the data collection and processing strategies, improves the confidence of semantic recognition, reduces the probability of misjudgment and missed detection, effectively eliminates the understanding gap between raw data and semantic information, and improves system reliability.

[0054] Allocate compute, cache, and bus bandwidth resources with hard isolation attributes to each data stream processing path, and set clear timing constraints and deadlines to avoid resource contention when multiple tasks are concurrent.

[0055] By using a scenario prediction mechanism to pre-arrange data flow paths and pre-allocate resources, the traditional reactive processing architecture is upgraded to predictive pre-adaptation, which can respond to scenario changes and potential risks in advance. This improves the operational stability of robots in highly dynamic and unstructured scenarios, broadens the application scenarios of robots, and adapts them to various types of autonomous robots.

[0056] It will be apparent to those skilled in the art that this disclosure is not limited to the details of the exemplary embodiments described above, and that this disclosure can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of this disclosure is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within this disclosure. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0057] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A robot underlying architecture system, characterized in that, include: Sensor arrays are used to collect raw perception data of the robot's own state and the surrounding environment; The dynamic data stream engine communicates bidirectionally with the sensor array to dynamically generate and adjust data stream processing paths with time constraints and resource isolation attributes based on real-time task requirements, semantic feedback and scene prediction information, thereby completing the priority distribution of perceived data. The semantic understanding and prediction unit communicates bidirectionally with the dynamic data stream engine. It receives data along the processing path, fuses it to generate the current scene semantic state vector, predicts the future short-time window scene evolution trend to generate a prediction state vector, and feeds the two types of vectors back to the dynamic data stream engine. The decision-making and planning unit communicates bidirectionally with the semantic understanding and prediction unit and the dynamic data flow engine. It combines semantic and predicted state vectors, target tasks and system resource states to complete multi-objective optimization decisions and generate decision instructions containing expected behavior sequences. The resource-aware controller communicates bidirectionally with the decision-making and planning unit and upstream core units. It is used to receive decision commands, collect the resource status of the entire system in real time to generate a global resource status vector, dynamically generate control parameters, and feed back the resource status to upstream units. The actuator drive array communicates with the resource-aware controller to receive control parameters and generate drive signals to control the actuator actions.

2. The robot underlying architecture system according to claim 1, characterized in that, The dynamic data flow engine dynamically generates data flow processing paths, specifically by: analyzing the core perception and computing power requirements of the current task scenario in real time, selecting matching data processing operators from a preset operator library, arranging the data processing operators according to processing logic and timing constraints, generating data flow processing paths with clear data dependencies, timing deadlines, and priority attributes, and allocating computing, caching, and bus bandwidth resources with hard isolation attributes to the paths.

3. The robot underlying architecture system according to claim 2, characterized in that, The dynamic data stream engine adjusts the data stream processing path in real time, specifically as follows: Upon receiving the semantic state vector and the prediction state vector, for high-priority semantic targets and high-risk prediction scenarios, the corresponding data distribution priority is increased and the path computing resources are expanded; When the global resource state vector is received and resource overload is detected, a degradation strategy is executed, which includes reducing the sampling frequency of non-core data, switching lightweight operators, and pausing non-critical branches. Upon receiving task priority and computing power requirements, the system pre-arranges paths and pre-allocates resources for the sequence of actions to be executed.

4. The robot underlying architecture system according to claim 3, characterized in that, The semantic understanding and prediction unit generates a semantic state vector, specifically as follows: Based on the hardware timestamps and spatial calibration parameters of each sensor, the raw sensing data is spatiotemporally synchronized and aligned to generate multimodal synchronized data frames with unified timestamps and matching spatial coordinate systems. The synchronous data frame is input into the multimodal fusion deep neural network model to complete cross-dimensional feature extraction, association and fusion, and generate a high-dimensional fusion feature vector that represents the core attributes of the scene; The high-dimensional fused feature vector is mapped to a preset standardized semantic space to generate a semantic state vector containing the robot's own state, environmental object attributes, relative pose, and dynamic parameters.

5. A robot underlying architecture system according to claim 4, characterized in that, The semantic understanding and prediction unit generates a prediction state vector, specifically as follows: The semantic state vectors synchronized at multiple consecutive moments are used as input and imported into a preset temporal prediction model. Based on the temporal evolution of historical semantic states and combined with the robot's own decision-making behavior trends, the temporal prediction model extrapolates and predicts the scene semantic state at each moment within a preset short time window in the future, generating a predicted state vector that includes the dynamic change trend of the environment, potential risk areas, and changes in the state of the task target, and synchronously feeds it back to the dynamic data stream engine.

6. The robot underlying architecture system according to claim 1, characterized in that, The decision-making and planning unit performs multi-objective optimization decision-making and generates decision instructions. Specifically, it uses the semantic state vector as the initial state constraint, the predicted state vector as the future scenario constraint, and the global resource state vector as the computing power constraint. It creates a multi-objective optimization cost function with the objectives of satisfying preset task efficiency, system energy consumption, behavior safety, and state uncertainty thresholds. It searches for the expected behavior sequence that satisfies the real-time and safety constraints in the robot's feasible behavior space and generates the corresponding decision instructions.

7. The robot underlying architecture system according to claim 1, characterized in that, The resource-aware controller generates control parameters by: collecting the CPU load rate, GPU computing power utilization rate, memory utilization rate, bus bandwidth utilization rate and actuator running status of each computing unit in the entire system at a preset fixed period to generate a global resource status vector; The decision instructions are parsed into the control objectives, accuracy requirements, and response time constraints of the actuator, and then imported into a preset resource-aware control model. Under the premise of meeting the response time constraints, the computational order, sampling frequency, and computational accuracy of the control algorithm are dynamically adjusted to solve and generate control parameters that meet the preset control performance and system stability indicators.

8. The robot underlying architecture system according to claim 1, characterized in that, The semantic understanding and prediction unit dynamically adjusts the model size and inference accuracy of the multimodal fusion deep neural network model based on the received global resource state vector; when it detects that computing resources are tight, it automatically switches to a lightweight inference model to reduce computing power consumption and ensure the real-time nature of semantic output.

9. A robot underlying architecture method, characterized in that, Applied to the robot underlying architecture system according to any one of claims 1 to 8, the method includes the following steps: S1. Collect raw perception data of the robot's own state and the surrounding environment through a sensor array; S2. Through the dynamic data flow engine, based on the real-time requirements of the task, semantic feedback and scene prediction information, the data flow processing path with time constraints and resource isolation attributes is dynamically generated and adjusted to complete the priority distribution of perceived data. S3. The semantic understanding and prediction unit receives data along the processing path, generates a semantic state vector through multimodal fusion, predicts the future short-term window scene evolution trend to generate a prediction state vector, and feeds the two types of vectors back to the dynamic data stream engine. S4. By combining semantic and predicted state vectors, target tasks and system resource states through the decision and planning unit, multi-objective optimization decisions that meet preset constraints are executed to generate decision instructions containing expected behavior sequences. S5. Receive decision instructions through the resource-aware controller, collect the resource status of the entire system in real time to generate a global resource status vector, dynamically generate control parameters that meet preset constraints, and feed back the resource status to the corresponding unit. S6. Receive control parameters through the actuator drive array and generate drive signals to control the actuator action.

10. A robot underlying architecture method according to claim 9, characterized in that, The dynamic data stream engine dynamically adjusts the data stream processing path and computing power allocation based on the real-time received semantic state, prediction state, global resource state and task priority information. The semantic understanding and prediction unit dynamically adjusts the model's inference accuracy based on the global resource status; The decision-making and planning unit dynamically adjusts the decision constraints and behavior sequence planning granularity based on the global resource status to complete the full-link closed-loop optimization.

Images