Cloud-edge collaborative embodied intelligent humanoid robot service orchestration method

Abstract

This invention belongs to the field of robotics technology and discloses a cloud-edge-device collaborative method for orchestrating services of embodied intelligent humanoid robots. For tasks with high real-time and control precision requirements, such as dynamic obstacle avoidance and precise grasping, the edge device is given priority to execute these tasks, avoiding network latency that could break the perception and execution loop. For resource-intensive but moderately precise tasks, such as book classification and recognition, the edge device and cloud device work together to achieve global coordination, relying on a cloud-based digital twin model and a global environmental semantic map. During the scheduling process, model predictive control algorithms and cloud-based digital twin control model simulations are introduced to optimize multi-robot collaborative action strategies. In the adaptive adjustment phase, initial adjustment parameters are quickly generated through a meta-learning framework, and online fine-tuning optimizes the control parameters, enabling real-time responses to environmental changes such as object position shifts and new obstacles, as well as task deviations such as insufficient grasping force.

Description

Technical Field

[0001] This invention belongs to the field of robotics technology, specifically a cloud-edge-device collaborative method for orchestrating services of an embodied intelligent humanoid robot. Background Technology

[0002] Embossed intelligent humanoid robots serve as the core carrier connecting the physical and digital worlds, and service orchestration technology is the key support for coordinating this closed-loop process, optimizing resource allocation, and improving task execution efficiency. Currently, service orchestration technologies for embodied intelligent humanoid robots are mostly based on traditional edge-cloud dual architectures or simple edge-assisted modes, and the following technical problems still exist in practical applications: The cloud often handles all complex decision-making and orchestration tasks, while the edge devices only serve as data transmission relays or simple execution nodes. This can lead to a break in the perception and execution loop when robots are handling tasks with high real-time requirements due to network latency, resulting in inaccurate movements or even task failure. On the other hand, if there is an over-reliance on local orchestration at the edge devices, the limited computing power of the edge nodes will prevent the achievement of global optimization goals such as multi-robot collaboration and cross-scenario task planning. This creates a contradiction where prioritizing real-time performance leads to global inefficiency, while global optimization results in insufficient real-time performance.

[0003] Existing service orchestration methods often use preset rules or fixed models to split and allocate tasks, which makes it difficult to cope with dynamic changes in real-world environments. For example, in home service scenarios, when a robot is faced with a vague instruction such as "clean the table," traditional symbolic planning methods cannot flexibly parse multimodal environmental information and cannot dynamically adjust task steps based on real-time perceived environmental changes, resulting in rigid orchestration strategies and low task execution success rates.

[0004] Significant differences exist in computing power, storage, and communication resources among the cloud, edge, and terminal nodes of embodied intelligent robots. Existing technologies lack a unified resource abstraction and scheduling mechanism, resulting in the isolated distribution of heterogeneous resources. For example, multimodal data collected by edge sensors is uploaded to the cloud without targeted preprocessing, consuming a large amount of bandwidth resources; idle computing power at the edge cannot be fully utilized to process local complex tasks, while the cloud suffers from task queuing delays due to overload, resulting in low overall resource utilization. Summary of the Invention

[0005] The purpose of this invention is to provide a cloud-edge-device collaborative embodied intelligent humanoid robot service orchestration method to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a cloud-edge-device collaborative embodied intelligent humanoid robot service orchestration method, comprising: Preferably, a lightweight multimodal perception module is deployed on the edge side in the layered perception stage to collect raw data such as environmental images, physical properties of objects, sound signals and the robot's own joint states in real time. Data noise reduction and format standardization preprocessing are completed through an embedded chip on the edge side. At the same time, key feature data are filtered based on preset control thresholds to generate a perception and control correspondence table. Data denoising employs a median filtering algorithm to handle environmental image noise and a moving average filtering algorithm to handle time-series data such as force control and joint states. Format standardization converts all types of raw data into floating-point data formats, with image data normalized to the [0, 1] interval, force control data standardized to Newtons (N), joint angle data standardized to radians, and joint angular velocity data standardized to radians per second (rad / s). Preset control thresholds are set based on robot hardware parameters and scene safety requirements, such as the safety distance threshold for dynamic obstacle avoidance and the force control threshold for grasping actions. The filtering logic retains feature data exceeding the threshold fluctuation range as key control features.

[0007] A multi-source data fusion and control constraint verification unit is deployed at the edge to receive key feature data uploaded from the edge side. A lightweight deep learning model is used to complete cross-modal data fusion, generate a unified scene perception tensor, and synchronously call the control constraint rule library to verify whether the fused data meets the local control requirements. For data that does not meet the requirements, a second collection is triggered at the edge side. The control constraint rule base stores constraints categorized by data type, including three types: perceptual data accuracy rules, data integrity rules, and hardware adaptation rules. Perceptual data accuracy rules specify the allowable error range for various feature data types, such as requirements related to position data error and force control data error. Data integrity rules stipulate the required fields and the percentage of valid data in key fields. Hardware adaptation rules relate to the processing capabilities of the robot's local hardware, such as data transmission rate and computational response speed. The verification logic involves matching the three types of rules one by one. If the data meets all rules, it is determined to meet the local control requirements. If any rule is not met, a secondary data collection is triggered on the edge side, during which specific data dimensions requiring optimization are simultaneously sent.

[0008] A global perception enhancement and control accuracy prediction unit is deployed in the cloud. It receives scene perception tensors uploaded by each edge node, combines historical control data with digital twin control models, constructs a global environmental semantic map that includes control accuracy requirements, simulates the control effect under different perception data accuracies through digital twin models, predicts the impact of dynamic scene changes on control performance, and generates global perception accuracy optimization commands to be sent to the edge.

[0009] Preferably, the requirement modeling stage achieves the binding of requirement modeling and control strategy through preset performance control indicators. Based on the scene perception tensor and global environment semantic map generated in the layered perception stage, the edge terminal receives multimodal service requirements such as user voice and gestures, calls a lightweight large language model to parse the core task objectives of the requirements, and simultaneously extracts implicit control constraints and control priorities. The lightweight large language model is based on a simplified Transformer architecture, retaining only the core semantic parsing module and the control constraint extraction module. The input is structured data of user multimodal needs (speech-to-text, gesture recognition vectors), and the output is core task target keywords, control constraint keywords (e.g., collision avoidance, accuracy), and priority labels. A physical constraint sandbox module is added at the edge to perform physical rule verification on the ontology control parameters mapped by the output results of the lightweight large language model, solving the problems of the large language model's misunderstanding of physical concepts and the generation of "illusions". The model pre-training uses a lightweight human-computer interaction corpus, focusing on scenarios such as home services and simple collaborative work. During the fine-tuning stage, the association mapping between control constraint keywords and robot control parameters is strengthened to ensure that the parsing results can be directly connected to the subsequent control modeling process.

[0010] The physical constraint sandbox features a dual-library architecture: a hardware physical constraint library and an environmental physical rule library. The hardware physical constraint library defines the robot's physical limits, including constraints strongly related to hardware performance such as joint range of motion, actuator output power limits, force control threshold boundaries, and extreme motion speeds. The environmental physical rule library stores physical rules adapted to different scenarios, including spatial size limitations, object physical attribute adaptation rules, and environmental interaction safety rules. The verification logic is as follows: first, the core task objectives, control constraint keywords, and priority labels output by the lightweight large language model are mapped to initial on-body control parameters; then, the initial parameters are matched one by one with the physical constraints in the two libraries to determine whether the parameters are within the physically feasible range. The verification results are processed in two categories: if the initial parameters meet all physical rules, the parsing results are directly uploaded to the cloud; if the initial parameters exceed the physical constraint threshold, the parameter correction mechanism is triggered. The physical constraint sandbox generates quantitative correction suggestions based on the compliance thresholds of the dual databases, which are fed back to the lightweight large language model to re-optimize the output results. At the same time, the physical concept understanding deviation type, parameter deviation value and correction scheme of the large language model are recorded in the local log library to provide data support for sample supplementation and feature enhancement for subsequent model fine-tuning.

[0011] The preliminary analysis results are uploaded to the cloud. The cloud uses a large-scale embodied intelligent language model to perform deep semantic understanding and control target modeling, generating a structured control task model that includes task objectives, control indicators, and constraint thresholds. Based on preset performance control indicators, the control performance indicators of each sub-task are quantified, and a multi-objective optimization function is constructed. The multi-objective optimization function focuses on control accuracy, real-time performance, and resource consumption as its core optimization objectives. These objectives are linked by weight coefficients, which can be dynamically adjusted based on scenario types such as home services and industrial collaboration. The function consists of objective terms, constraint terms, and penalty terms. Objective terms define the ideal value range for each optimization objective, constraint terms limit the hardware-permissible boundaries of control parameters, and penalty terms set correction coefficients for parameters exceeding the constraint range. The edge-end adaptation and correction logic combines environmental constraints such as spatial size and obstacle distribution in the local scene perception tensor with robot hardware parameters such as joint range of motion and actuator power limits to adjust the values ​​of the objective terms and the boundaries of the constraint terms in the optimization function, ensuring that the control scheme output by the function can be implemented and executed in the local scene.

[0012] By combining the local scene perception tensor with the robot's own control capabilities at the edge, the structured control task model issued from the cloud is adapted and modified to form the final demand and control correspondence model.

[0013] Preferably, the task decomposition stage is based on the requirement-control correspondence model formed in the requirement modeling stage, and the edge end adopts a three-dimensional evaluation model of complexity, real-time performance and control accuracy (a specific implementation of multi-dimensional evaluation) to decompose the core task into sub-tasks with control attributes. Each dimension of the three-dimensional evaluation model is divided into 5 rating levels. The complexity dimension is divided according to the combined amount of computation and data, and the higher the level, the greater the required computation and data processing pressure. The real-time dimension is divided according to the maximum allowable delay of the task, and the higher the level, the stricter the delay requirement. The control accuracy dimension is divided according to the allowable error range, and the higher the level, the more stringent the error requirement. The granularity of the task breakdown is as follows: core tasks with a comprehensive score of ≥12 points (out of 15) are broken down into fine-grained subtasks, each containing only a single control action; tasks with a score of 8-11 points are broken down into medium-grained subtasks, each containing 2-3 related control actions; and tasks with a score of ≤7 points are broken down into coarse-grained subtasks, each containing more than 3 related control actions. This ensures that the broken-down subtasks are both easy to execute and meet resource allocation requirements.

[0014] For each subtask with control attributes, the real-time requirements and resource consumption are evaluated by combining the edge perception data obtained in the hierarchical perception stage with the edge resource status. A control dependency analysis algorithm is introduced to identify the control dependencies between subtasks and generate a subtask control execution sequence. By combining edge resource status and control performance indicators, a dynamic priority sorting algorithm is adopted to mark subtasks with high real-time requirements and strict control precision requirements as edge-priority control execution, and to mark subtasks with high resource consumption but moderate control precision requirements as edge and cloud collaborative control execution.

[0015] Preferably, in the scheduling optimization stage, the cloud receives sub-task information, resource status data and control performance requirements generated in the task splitting stage uploaded by each edge node, constructs a dual-objective scheduling model of control and resources, introduces a model predictive control algorithm, and predicts the control effect under different resource allocation schemes through a rolling optimization strategy to ensure that the resource scheduling scheme meets the control performance requirements. For multi-robot collaborative tasks, a distributed coordination control model is constructed using the alternating direction multiplier method to coordinate and allocate control permissions and resources for each robot. Combined with the global environmental semantic map from the hierarchical perception stage, the control strategy is simulated and practiced through a digital twin control model to optimize the collaborative execution path and action coordination strategy. Based on the optimization results, scheduling instructions containing resource allocation schemes and control parameters are generated and sent to the corresponding edge nodes. Simultaneously, the scheduling schemes and control model parameters are backed up to the cloud database to establish a scheduling and control association file.

[0016] The cloud database backup content includes three categories: basic information, core data, and related mapping tables. The basic information includes task ID, robot ID, edge node ID, and execution timestamp; the core data includes complete scheduling schemes, control model parameters, and simulation results. The scheduling schemes include resource allocation ratios and priority configurations, and the control model parameters include algorithm coefficients and threshold settings. The association mapping table records the correspondence between scheduling schemes and control effects; the update mechanism is a combination of real-time incremental backup and periodic full update: after the scheduling instruction is issued, the newly added data is backed up incrementally immediately, and the full data is updated for consistency verification and deduplication every morning. The backup data retains nearly 90 days of historical records and supports quick retrieval and retrieval by task type, robot ID, and time range.

[0017] Preferably, during the deployment and implementation phase, the edge device receives the sub-task allocation scheme generated during the scheduling optimization phase from the cloud. To avoid the impact of minor lags during container startup, destruction, and resource preemption on the real-time performance of high-frequency control tasks, the module containerization architecture is designed in layers. The execution module is not included in the Docker containerization packaging scope. Only the perception module and control module are packaged and instantiated into independent perception-control integrated service containers using Docker containerization technology. The execution module is directly embedded in the edge-side real-time control kernel. For non-high-frequency control sub-tasks that require flexible scheduling at the edge device, unikernel lightweight virtualization technology can also be used to encapsulate the modules, balancing scheduling flexibility and control real-time performance. Computing power and memory resources are dynamically allocated to the perception-control integrated service container according to the sub-task control requirements, and resource preemption permissions are set for containers of high-priority control tasks. The execution module embedded in the edge-side real-time control kernel adopts a kernel-level instruction direct-drive architecture, eliminating the intermediate scheduling layer and preloading the execution instruction set of high-frequency control actions (such as basic execution instructions for precise grasping and dynamic obstacle avoidance). It also optimizes instruction response latency through real-time kernel patches to ensure that the execution module can directly receive control instructions issued by the edge perception-control integrated service container without container scheduling lag. The lightweight virtualization solution is built on a simplified version of the runC runtime, removing the redundant system call layer of traditional containers, reducing virtualization overhead to less than 5%, and only adapting to non-high-frequency control subtasks on the edge that require dynamic start and stop. It forms a complementary architecture with the edge-side kernel-embedded execution module.

[0018] The resource dynamic allocation rules are executed according to the principle of matching demand and reserving reserve. The computing power allocation is based on the number of computing instructions required by the perception-control integrated service container, and the memory allocation is based on the maximum number of bytes of data to be processed. At the same time, 20% of computing power and memory are reserved for each edge node as an emergency reserve. The high-priority container preemption mechanism is as follows: when the perception-control integrated service container of a high-priority task starts or the computing power / memory demand increases, the reserved emergency reserve will be used first. If the reserve is insufficient, non-critical resources will be reclaimed proportionally in the order of low-priority task containers → medium-priority task containers. The reclamation ratio shall not exceed 30% of the resources already allocated to low / medium priority containers. Before reclamation, the corresponding container will be notified to temporarily store data to avoid interruption of task execution.

[0019] For the edge and cloud collaborative execution of sub-tasks marked in the task decomposition stage, a low-latency communication link based on 5G is established at the edge. A deterministic real-time communication system is built by integrating Application Data Distribution Service (DDS) and Time-Sensitive Network (TSN). An optimized dual-protocol architecture with on-demand adaptation of MQTT control protocol and DDS protocol is adopted. The accuracy of data interaction is ensured by adding control command verification fields, while achieving low-latency data interaction. When the latency exceeds the control threshold, it automatically switches to the local control mode at the edge. The edge receives and executes sub-tasks and control commands issued by the edge perception-control integrated service container. It drives the actuator to complete the action through the embedded real-time control system, and simultaneously collects control status data during the execution process and transmits it back to the edge in real time, forming a control closed loop of deployment, execution and feedback.

[0020] Preferably, the adaptive adjustment phase is based on the control status data transmitted back through the deployment, execution, and feedback control closed loop formed during the deployment and implementation phase. The control status monitoring and anomaly diagnosis unit deployed at the edge collects sub-task execution status data, environmental perception data updated during the hierarchical perception phase, and control parameter operation data in real time. The control performance evaluation index is used to determine whether there is a control deviation or dynamic environmental change. The control performance evaluation indicators include three core indicators: tracking error, settling time, and steady-state error. Tracking error is the difference between the actual executed parameters and the target control parameters; settling time is the time it takes for the system to reach a steady state from responding to a command; steady-state error is the deviation between the actual executed parameters and the target parameters after the system stabilizes. The judgment logic is as follows: when the tracking error exceeds the preset error range, or the settling time exceeds the maximum allowable duration, or the steady-state error persists for more than the set duration, it is determined that there is a control deviation; when the changes in key information such as the position of objects and the distribution of obstacles in the environmental perception data exceed the preset fluctuation range, it is determined that there is a dynamic change in the environment.

[0021] A meta-learning algorithm framework is introduced, based on an offline-trained multi-scenario control parameter initial library. To avoid the misleading influence of abnormal monitoring data caused by noise interference and hardware wear on parameter adjustment, a time-series anomaly detection module is added at the edge before the monitoring data is input into the meta-learning framework. This module performs full-dimensional time-series anomaly detection and data purification on control deviation monitoring data and environmental dynamic change perception data. Only data that passes the detection and meets the requirements can be used for subsequent parameter generation. When uploading high-level analysis data from the edge and related data from the cloud, time-series anomaly detection and processing records must be attached. The cloud performs a secondary time-series anomaly verification simultaneously to ensure the accuracy of parameter adjustment from the data source. Initial adjustment parameters are generated for real-time monitored control deviations or environmental dynamic changes. Combined with preset performance control indicators, the control parameters are optimized through online fine-tuning to achieve adaptation to dynamic environmental changes. The time-series anomaly detection module is custom-designed to address anomalies caused by noise and hardware wear. It employs a three-tiered detection logic: sliding window time-series sequence construction, multi-feature fusion anomaly judgment, and hierarchical anomaly processing. First, a time-series data sequence is constructed using a 50ms sliding window step size and a 300ms window length. This sequence contains control deviations (such as gripping force deviations and positional offsets) and environmental perception data (such as obstacle positions and object coordinates). The module extracts the core features of trend, fluctuation, and abrupt change in the sequence to meet the detection requirements of slow-changing anomalies from hardware wear and sudden anomalies from noise interference. Second, it integrates the isolated forest algorithm and the ARIMA time-series prediction residual analysis algorithm for dual-algorithm judgment. Outlier mutations in the sequence are identified by using isolated forests, and slow drift anomalies are determined by the residual threshold between the predicted values ​​and the actual monitored values ​​of the ARIMA model. Data that is deemed normal by both algorithms is considered compliant data. Data that is deemed abnormal by a single algorithm enters the review process. The final determination is made in combination with the robot's hardware operating status (such as joint wear and sensor aging coefficient). Finally, a hierarchical outlier processing is implemented. Minor outliers are corrected by linear interpolation. Moderate outliers are smoothed by historical normal time series data of the same scene. Severe outliers (such as continuous data exceeding the threshold due to hardware wear) are directly removed and marked as outliers, and hardware status warnings are triggered at the same time.

[0022] A compliance verification mechanism is set up for the data upload process: data used for high-level analysis within the edge must pass local compliance verification by the time-series anomaly detection module to generate a complete data archive containing "original data value - detection result - processed value - anomaly marker"; fault information, adjustment strategies, and supporting monitoring data that need to be uploaded to the cloud must include the time-series anomaly detection processing archive as a mandatory attachment. After receiving the data, the cloud performs a second sampling check on the core data using a lightweight time-series anomaly verification algorithm. If the verification passes, the data is included in the distributed shared database; if the verification fails, the data is fed back to the edge for re-collection and detection processing to ensure that there are no outlier interferences in the entire data chain.

[0023] The meta-learning framework employs a model-independent meta-learning algorithm, and the offline training steps are as follows: 1) Construct an initial library of control parameters for multiple scenarios, including 20 typical scenarios such as object position offset ±5mm and newly added obstacle diameter ≤10cm. Each scenario contains 1000 sets of control parameter samples, including PID parameters, path planning coefficients, etc. 2) During the pre-training phase, the scene type is used as the task, and the stochastic gradient descent method is adopted with a learning rate of 0.01 and 1000 iterations to learn the universal initialization parameters across scenes; 3) In the online fine-tuning stage, the input is real-time monitoring data, control deviation: such as grasping force deviation ΔF=2N; environmental changes: such as object position offset ΔX=3mm; fine-tuning is carried out in 3-5 steps using the gradient descent method with a learning rate of 0.001, and the output is the optimal adjustment parameters adapted to the current scenario, such as PID parameters Kp=5.2, Ki=0.3, Kd=0.1; The supporting fault-tolerant control strategy library is built using the C4.5 decision tree algorithm. The training samples are 10,000+ fault-redundancy scheme mapping data, such as visual sensor failure - enabling haptic-assisted positioning. The decision tree depth is set to 8 to ensure that the redundancy scheme is quickly called when a fault is triggered.

[0024] The online fine-tuning process consists of four steps: deviation identification (outputting deviation dimensions and types), parameter screening (outputting key adjustment parameters), gradient adjustment (outputting corrected parameters), and effect verification (outputting a conclusion on whether the target is met). Deviation identification clarifies the specific dimensions of the control deviation and the core types of environmental changes. Parameter screening selects the key parameters with the highest correlation to the deviation from the initial adjustment parameters. Gradient adjustment gradually corrects the key parameters according to a preset step size, which can be dynamically adjusted according to the magnitude of the deviation. Effect verification determines whether the preset performance control indicators are met by collecting the adjusted control state data. If the effect is met after a single fine-tuning, the adjustment stops. If it is not met, the above process is repeated, up to a maximum of 5 times. If it is still not met, a fault-tolerant control strategy is triggered.

[0025] When a serious control failure is detected, the edge-end fault-tolerant control strategy library is automatically triggered, redundant control schemes are invoked, and the fault information, adjustment strategy, and corresponding timing anomaly detection and processing files are simultaneously uploaded to the cloud.

[0026] Preferably, the optimization feedback stage is based on the fault information, adjustment strategy and full-link data of each stage uploaded in the adaptive adjustment stage. The control status data of the execution process on the end side, the scheduling optimization data and adaptive adjustment data on the edge end, and the global control effect data on the cloud are synchronized to the distributed shared database respectively to construct a control and resource data archive that includes the entire link of perception, modeling, splitting, scheduling, deployment and adjustment. The distributed shared database adopts a two-layer storage architecture design of structured data tables + distributed object storage compatible with the S3 protocol. The phased data table structure includes a perception data sub-table, a requirement modeling sub-table, a task splitting sub-table, a scheduling optimization sub-table, a deployment execution sub-table, and an adaptive adjustment sub-table. Each sub-table uses timestamp + robot ID + task ID as a composite primary key to store full-link structured control and resource data. The newly added distributed object storage layer compatible with the S3 protocol serves as a dedicated storage carrier for unstructured data, specifically storing raw sensor data collected from the edge side (including multi-view video time-series data and multi-modal perception raw stream data). It also establishes a dedicated synchronization mechanism between the edge side and the cloud for raw sensor data, realizing unified identification, associated storage, and collaborative synchronization of structured data and unstructured raw data. The cloud-deployed background automated scripts support online extraction and formatting transcoding of raw multi-view video time-series data encapsulated in the object storage without downloading large raw files, generating H.264 encoded MP4 standard video streams. This provides standardized and highly compatible video data support for cloud-visualized data auditing, control deviation playback, and high-precision ground truth annotation.

[0027] The edge-to-cloud raw sensor data synchronization mechanism works in synergy with the existing structured data incremental synchronization mechanism, employing a strategy of "fragmentation by composite primary key + incremental push + breakpoint resumption": The edge fragments and encapsulates the raw sensor data by timestamp + robot ID + task ID, generating a composite primary key identifier and MD5 checksum consistent with the structured data table for each fragment, and pushes it to the cloud's distributed object storage layer through a lightweight upload interface adapted to the S3 protocol. The structured data is synchronized to the data table layer according to the original rules. If a network interruption occurs during the synchronization process, the edge locally caches the fragmented data that has not been synchronized. After the connection is restored, the push continues based on the breakpoint position. The cloud automatically completes the association mapping between structured data and unstructured raw data through the composite primary key, ensuring the temporal consistency and uniqueness of the two types of data. The cloud-based automated backend script is a resident, lightweight processing script equipped with an online streaming data processing engine. It does not require downloading the original multi-view video timing data to the local machine. Instead, it directly reads the segmented data of the original video stream through the object segmentation access interface of the S3 protocol. After frame extraction, timing calibration, and encoding conversion, it generates an H.264 encoded MP4 standard video stream online. The transcoded video stream is still bound to the original composite primary key identifier and stored back in the video stream's dedicated storage area of ​​the distributed object storage. At the same time, metadata such as the video stream access address, transcoding parameters, and timing range are written into the corresponding structured sub-table, realizing the precise association between the video stream and the end-to-end control data.

[0028] Based on the transcoded H.264 / MP4 standard video stream, a cloud-based visualization data processing platform is built to achieve three core functions: First, data auditing, which visualizes and audits the perception, modeling, scheduling, and execution stages of the entire robot task execution process through synchronous linkage display of video stream and structured control data, quickly locating abnormal nodes in data acquisition or transmission; Second, control deviation playback, which retrieves the video stream and real-time control parameter data of the corresponding task based on the joint primary key, achieving millisecond-level synchronous playback of audio and video images and control deviation data, accurately reconstructing the scene where the control deviation occurred and the parameter change process, and analyzing the cause of the deviation; Third, high-precision ground truth annotation, which relies on the high compatibility and high definition of the standardized video stream, and directly performs manual or semi-automatic ground truth annotation on physical quantities such as the position, posture, and motion trajectory of objects in the video image in the visualization platform. After the annotation results are bound to the joint primary key, they are stored back to the distributed object storage and structured data table, providing high-precision annotation samples for the incremental learning of the global control model and the parameter library iteration of the meta-learning framework.

[0029] Data synchronization employs a real-time incremental synchronization mechanism. Structured control data from the edge is uploaded in batches every 50ms, edge data is aggregated and uploaded every 100ms, and cloud data is integrated and updated every 200ms. Raw unstructured sensor data from the edge is transmitted in fragments according to the S3 protocol, with the synchronization frequency consistent with the corresponding structured data to ensure timing matching. The synchronization process uses a data verification mechanism, verifying data integrity through MD5 checksums. If verification fails, data retransmission is triggered. The structured data table and the distributed object storage layer achieve mutual linkage of verification results through a composite primary key. If verification fails in any data layer, retransmission and verification of the corresponding associated data are triggered.

[0030] Based on full-link data, the global control model is dynamically updated using an incremental learning algorithm to optimize control parameter thresholds and performance evaluation indicators. Combined with the resource usage of each edge node, the global resource scheduling strategy is adjusted to achieve a balance between the dual objectives of optimal control performance and maximum resource utilization. The incremental learning algorithm employs a three-step strategy: data filtering, incremental parameter updates, and forgetting suppression. 1) Data screening: Screen samples with excessive control errors or low resource utilization from the full-link data, construct an incremental dataset, and add ≤10% of the total samples in a single iteration to avoid data redundancy; 2) Incremental parameter update: For the global control model (digital twin model, MPC model), the bottom 80% of the parameters are frozen and the top 20% of the parameters are fine-tuned. The optimizer is AdamW, the learning rate is 0.005, and the iteration is 20 rounds. 3) Forgetting suppression: The elastic weight consolidation algorithm is used to apply penalty terms to key parameters of the digital twin model, such as the physical attribute mapping coefficient, to ensure that the updated model maintains stable control accuracy over historical scenes. The updated output includes: control parameter thresholds (e.g., force control threshold adjusted to 4.8-8.2N), performance evaluation metrics (e.g., adjustment time ≤180ms), and resource scheduling strategies (e.g., 30% of edge computing power is tilted towards high-priority tasks). These are synchronized to each stage through a distributed shared database to achieve iterative optimization.

[0031] The cloud platform, relying on an integrated embodied intelligence training and inference platform, coordinates overall optimization. This platform deeply integrates three core functional modules: a dynamic computing power allocation module, an S3 protocol-compatible distributed object storage resource pool, and a unified edge node management and instruction delivery module. After optimizing and iterating the control model and scheduling strategy based on the full-link data archive, the platform distributes the optimized control model parameters and resource scheduling strategies to each edge node through its built-in standardized instruction delivery channel. This feedback is then fed back to various stages, including global perception accuracy optimization in the layered perception stage, task model correction in the requirement modeling stage, execution sequence optimization in the task splitting stage, and resource allocation scheme optimization in the scheduling optimization stage. The edge nodes, as edge-side nodes of the embodied intelligence training and inference platform, have completed platform management registration. After receiving optimization instructions from the platform, they further fine-tune the control parameters and deployment scheme based on local real-time data. The fine-tuned node status, parameter correction results, and execution feedback data are transmitted back to the cloud platform through the platform management channel. This forms a two-way iterative closed loop of cloud-based global optimization, edge-side local adaptation, and edge-side execution feedback, all supported by the platform, computing power and storage resource pools, and edge node management.

[0032] Among them, the platform's dynamic computing power allocation module constructs an integrated cloud-edge-device computing power resource pool, integrating cloud GPU / CPU cluster computing power and edge distributed computing power resources, and performs elastic scheduling of computing power based on task type: for computing power intensive tasks such as global control model training, incremental learning iteration, and video stream transcoding, cloud cluster computing power is prioritized and computing power isolation is guaranteed; for lightweight computing power needs such as local model fine-tuning and real-time data processing at the edge, the platform issues computing power allocation thresholds and schedules idle local computing power on edge nodes to complete the task, while reserving emergency computing power resources for high-priority tasks such as dynamic obstacle avoidance and precise capture, ensuring that computing power allocation is accurately matched with the real-time and complexity requirements of control tasks. The S3 protocol-compatible distributed object storage resource pool serves as the core data foundation of the platform. It handles all types of data uploaded from the edge, including raw sensor data, transcoded H.264 / MP4 standard video streams, end-to-end structured control data, model training annotation samples, and optimized model parameters. This enables unified storage, indexing, and management of both unstructured and structured data, providing highly available and scalable storage support for the training, inference, and cloud-based visualization auditing of embodied intelligent models. Simultaneously, it forms a data hierarchy with the edge local cache, with frequently accessed control parameters and scheduling strategies being pushed down to the edge cache, while low-frequency large file data is retained in the cloud resource pool, balancing data access efficiency and storage costs. The unified edge node management and command delivery module establishes a cloud platform for the full lifecycle management of edge nodes. It integrates functions such as edge node registration and network access, real-time status monitoring, resource load statistics, standardized command delivery, and feedback data transmission. It assigns a unique device identifier to each edge node, collects and visualizes data such as node computing power usage, memory usage, communication status, and task execution progress in real time, and delivers optimization commands in a platform-standardized data format, along with a unique node identifier and command priority. It pushes commands accurately through a low-latency communication link and supports real-time feedback and retransmission mechanisms for command delivery status, ensuring that optimization commands are delivered to the target edge nodes without omission or error.

[0033] The embodied intelligent training and inference platform simultaneously establishes a resource linkage scheduling mechanism for computing power, storage, and edge nodes: when edge nodes send back local fine-tuning data and execution feedback, the platform's dynamic computing power allocation module automatically adjusts the computing power resource ratio according to the computing power requirements of model re-optimization; the S3 object storage resource pool synchronously completes the archiving and indexing of feedback data, supplementing data samples for the next round of model training and strategy optimization; the edge node management module updates node status tags based on feedback data, providing a basis for node resource status for subsequent resource scheduling strategy issuance, realizing the coordinated linkage of computing power, storage, and edge nodes, and ensuring the efficient and stable operation of the two-way iterative closed loop.

[0034] The beneficial effects of this invention are as follows: 1. For tasks requiring high real-time performance and control precision, such as dynamic obstacle avoidance and precise grasping, this invention prioritizes execution on the edge side to avoid network latency causing a break in the perception and execution closed loop. For resource-intensive tasks with moderate precision requirements, such as book classification and recognition, the edge and cloud sides collaborate, relying on a cloud-based digital twin control model and a global environmental semantic map for overall coordination. During scheduling, model predictive control algorithms and cloud-based digital twin control model simulations are introduced to optimize multi-robot collaborative action strategies. This ensures low-latency response for edge tasks and achieves reasonable allocation of global resources, reducing inaccurate actions and task failures, and making the robot's task execution more stable in complex scenarios.

[0035] 2. In the requirement modeling stage of this invention, multimodal analysis and local adaptation are used to accurately transform fuzzy user requirements into quantifiable control targets. In the adaptive adjustment stage, initial adjustment parameters are quickly generated through a meta-learning framework, and combined with online fine-tuning to optimize control parameters, enabling real-time response to environmental changes such as object position shifts and the addition of obstacles, as well as task deviations such as insufficient grasping force. In the optimization feedback stage, incremental learning is used to dynamically update the control model and scheduling strategy, forming a two-way iterative closed loop. This allows the robot to flexibly adjust its execution strategy without relying on fixed rules, improving its adaptability in dynamic scenarios such as home services and collaborative work, and increasing the success rate of task execution.

[0036] 3. This invention establishes a unified resource scheduling and management system. At the edge, containerization technology is used to package integrated service modules, dynamically allocating resources according to task requirements and setting resource preemption permissions for high-priority tasks. Through 5G low-latency communication links and switching mechanisms, the collaborative efficiency between the edge and cloud is balanced. End-to-end data is synchronized to a distributed database to construct resource and control profiles, and incremental learning is used to optimize resource allocation strategies. This achieves reduced bandwidth usage through edge-side data preprocessing, full utilization of idle edge computing power, and balanced cloud resource load, improving issues such as redundant data transmission, wasted computing power, and cloud overload. While ensuring control performance, it also enhances cross-node resource utilization efficiency. Attached Figure Description

[0037] Figure 1 This is an overall flowchart of the method of the present invention; Figure 2 This is a detailed flowchart of the layered sensing stage of the present invention; Figure 3 This is a flowchart illustrating the linkage between task splitting and scheduling optimization in this invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] like Figures 1 to 3 As shown, this embodiment of the invention provides a cloud-edge-device collaborative embodied intelligent humanoid robot service orchestration method, including: The layered perception stage includes a lightweight multimodal perception module deployed on the edge, comprising visual, tactile, force, and auditory sensors. This module collects raw data in real time, including environmental images, physical properties of objects, sound signals, and the robot's own joint states. The physical properties of objects include hardness and weight; the robot's own joint states include angle and angular velocity. The module performs data noise reduction and format standardization preprocessing through an embedded chip on the edge. Simultaneously, it filters key control feature data based on preset control thresholds such as the safe distance threshold for dynamic obstacle avoidance and the force control threshold for grasping. These key feature data include the speed or position change rate of dynamic obstacles and the control intent vector of human action commands, generating a perception-control correspondence table. A multi-source data fusion and control constraint verification unit is deployed at the edge to receive key feature data uploaded from the end side. A lightweight deep learning model is used to complete cross-modal data fusion, generate a unified scene perception tensor, and synchronously call the control constraint rule library to verify whether the fused data meets local control requirements, such as whether the data accuracy meets the force control adjustment requirements. For data that does not meet the requirements, the end side is triggered to collect data again to ensure the control availability of the perception data. The lightweight deep learning model adopts a 3-layer convolutional layer + 1-layer cross-modal attention mechanism architecture. The input is multimodal key features uploaded from the edge, including visual features: a 224×224×3 RGB image tensor; tactile features: a 1×128 force control vector; and auditory features: a 1×256 sound spectrum vector. After local features are extracted by the convolutional layers, modal weights are assigned through the attention mechanism, with a visual modality weight of 0.4, a tactile modality weight of 0.3, and an auditory modality weight of 0.3. The final output is a 1×512-dimensional unified scene perception tensor. The model training uses labeled datasets from various scenarios such as home services and industrial collaboration. The optimizer is Adam, with an initial learning rate of 1e-4. Every 10 rounds, the learning rate decays to 0.8 of the previous round. After 50 rounds of iteration, the loss function converges to a preset threshold to ensure that the control availability after cross-modal data fusion meets local control requirements.

[0040] A global perception enhancement and control accuracy prediction unit is deployed in the cloud. It receives scene perception tensors uploaded by each edge node, combines historical control data with digital twin control models, constructs a global environmental semantic map that includes control accuracy requirements, simulates the control effect under different perception data accuracies through digital twin models, predicts the impact of dynamic scene changes on control performance, such as the grasping error that may be caused by object position shift, and generates global perception accuracy optimization instructions to be sent to the edge.

[0041] The global environmental semantic map comprises three layers: a basic environment layer, a control precision annotation layer, and a dynamic change layer. The basic environment layer stores static environmental information, such as room layout, fixed furniture positions, and equipment installation coordinates. The control precision annotation layer annotates control requirements by region, such as positioning precision annotation for fragile item placement areas and obstacle avoidance response speed annotation for movement channels. The dynamic change layer updates dynamic target information in real time, such as personnel positions, moving obstacle trajectories, and changes in object placement. The three layers are linked using a unified coordinate system and data update time is marked with timestamps, supporting quick retrieval of required information by region, control precision requirements, and dynamic status.

[0042] The digital twin control model consists of three layers: a physical entity mapping layer, a control rule engine layer, and a simulation and deduction layer. The physical entity mapping layer constructs a virtual twin based on robot hardware parameters (joint degrees of freedom, actuator power, etc.) and environmental physical properties (object mass, spatial dimensions, etc.). The control rule engine layer integrates robot kinematics, dynamics control logic, and scene adaptation rules; the simulation and inference layer simulates the robot's action effects and environmental interaction feedback under different combinations of control parameters based on the input scene perception tensor and historical control data. First, the real scene state is synchronized through the physical entity mapping layer. Then, the corresponding control strategy is matched by the control rule engine layer. Finally, the control effect prediction result is output through the simulation inference layer, providing a basis for the control accuracy labeling of the global environment semantic map.

[0043] In the requirement modeling phase, the requirement modeling and control strategies are deeply integrated through preset performance control indicators. The preset performance control indicators include three main categories: control accuracy, real-time performance, and resource consumption. Control accuracy indicators include positioning accuracy, force control accuracy, and attitude control accuracy. Real-time indicators cover command response time and task execution latency. Resource consumption indicators include CPU utilization, memory utilization, and communication bandwidth usage. The binding logic between various indicators and control strategies is as follows: based on the subtask priority, high-priority tasks prioritize meeting control accuracy and real-time performance indicators, while low-priority tasks balance control performance and resource consumption indicators. The binding relationship is stored in the edge control strategy dictionary in the form of key-value pairs.

[0044] Based on the scene perception tensor and global environment semantic map generated in the hierarchical perception stage, the edge device receives multimodal service requests from users, such as "tidy up the study and deliver books". It calls a lightweight large language model to parse the core task objectives of the request and extracts implicit control constraints at the same time. For example, "avoid collisions with fragile items" corresponds to a force control constraint with an impact force of ≤5N, and "precise delivery" corresponds to a position control constraint with a positioning accuracy of ≤5mm. It also determines the control priority, such as the real-time control priority of the delivery task being higher than that of sorting and tidying. The preliminary analysis results are uploaded to the cloud. The cloud uses a large-scale embodied intelligent language model to perform deep semantic understanding and control target modeling, generating a structured control task model that includes task objectives, control indicators, and constraint thresholds. Based on preset performance control indicators, the control performance indicators of each sub-task are quantified, such as trajectory tracking error not exceeding 3% and adjustment time not exceeding 200 milliseconds. A multi-objective optimization function is constructed with the objectives of maximizing control accuracy and minimizing energy consumption. The large-scale embodied intelligent language model is based on the Transformer architecture, containing 12 layers of encoders and 8 layers of decoders. The input is the multimodal requirement parsing result uploaded from the edge. The speech-to-text conversion is ≤512 tokens, the gesture recognition is a 1×64-dimensional motion vector, and a robot control domain dictionary is loaded, including a lexicon of control attributes such as force control threshold, positioning accuracy, and trajectory error. The model pre-training uses 100,000+ human-computer interaction scenario samples, including home services, collaborative handling, etc. The fine-tuning stage introduces 50,000+ robot control capability labeled data, associates task types with hardware parameter constraints, and uses the cross-entropy loss function to train until the loss value converges. The output structured control task model fields are clearly defined as: task ID, subtask name, control index, constraint threshold, and priority weight, ensuring that the quantified target can be directly mapped to the robot control logic.

[0045] At the edge, the local scene perception tensor is combined with the robot's own control capabilities, such as joint adjustment range and motor power, to adapt and correct the structured control task model sent from the cloud. For example, the smoothness control parameters of path tracking are adjusted according to the actual layout of the study room to ensure that the control target can be achieved within the robot's hardware capabilities, thus forming the final demand and control correspondence model.

[0046] The task decomposition stage is based on the requirement and control correspondence model formed in the requirement modeling stage. The edge end uses a three-dimensional evaluation model of complexity, real-time performance and control accuracy to decompose the core task into sub-tasks with control attributes. For example, "organizing books" is decomposed into "identifying the location of books, grabbing books (force control range of 5 to 8N), classifying and placing them (attitude control accuracy not exceeding 2 degrees), and cleaning the desktop (trajectory tracking error not exceeding 5%)". The complexity dimension is evaluated from two sub-dimensions: computational load (number of required operation instructions) and data volume (number of bytes of data processed), each with a weight of 50%. The real-time dimension is evaluated from two sub-dimensions: maximum allowable delay of the task and instruction response requirements, with weights of 60% and 40%, respectively. The control accuracy dimension is evaluated from three sub-dimensions: allowable range of positioning error, allowable range of force control error, and allowable range of attitude error, with weights of 40%, 30%, and 30%, respectively. The three-dimensional evaluation model calculates the comprehensive score of the sub-tasks through weighted summation, and the score result serves as the core basis for marking the granularity of sub-task decomposition and execution method.

[0047] For each subtask with control attributes, the edge perception data obtained in the hierarchical perception stage and the edge resource status are combined. The edge resource status includes computing power and power consumption. Real-time requirements and resource consumption are evaluated. A control dependency analysis algorithm is introduced to identify the control dependencies between subtasks. For example, the force control for grasping books needs to be based on the positioning control result to generate a subtask control execution sequence. The algorithm establishes a dependency matrix by parsing the input and output parameters of subtasks. Using subtasks as rows and columns of the matrix, if the output parameter of subtask A (such as positioning coordinates) is the input parameter of subtask B (such as the position reference for grasping actions), the corresponding position in the matrix is ​​marked as a strong dependency. If subtasks A and B share the same environmental perception data (such as obstacle distribution information) but have no direct input-output relationship, they are marked as weak dependencies. The identification logic prioritizes strong dependencies and sorts them in the order of input before output. Weakly dependent subtasks are sorted in ascending order of resource consumption, ultimately generating an ordered sequence of subtask control execution.

[0048] By combining edge resource status and control performance indicators, a dynamic priority sorting algorithm is adopted to mark subtasks with high real-time requirements and strict control accuracy requirements, such as dynamic obstacle avoidance and precise grasping, as edge-priority control execution, while subtasks with high resource consumption but moderate control accuracy requirements, such as auxiliary positioning for book classification and recognition, are marked as edge and cloud collaborative control execution.

[0049] The algorithm sets a priority scoring formula: priority score = real-time weight × real-time score + control precision weight × precision score + resource consumption weight × (1 - resource consumption score). The real-time weight, control precision weight, and resource consumption weight are preset according to the scenario type. For example, in the home service scenario, the real-time weight is 0.4, the control precision weight is 0.4, and the resource consumption weight is 0.2. Real-time performance is graded according to the maximum allowable latency of the task, with a higher score for shorter latency. Control accuracy is graded according to the allowable error range, with a higher score for smaller error. Resource consumption is graded according to the required computing power / bandwidth ratio, with a higher ratio for higher score. Subtasks with scores higher than a preset threshold are marked as edge-priority execution, while those with scores lower than the threshold but higher than the set standard are marked as edge and cloud collaborative execution.

[0050] In the scheduling optimization phase, the cloud receives sub-task information, resource status data, and control performance requirements generated during the task splitting phase from each edge node, constructs a dual-objective scheduling model for control and resources, introduces a model predictive control algorithm, and predicts the control effect under different resource allocation schemes through a rolling optimization strategy, such as the impact of edge computing power allocation on trajectory tracking accuracy (control objective) and resource utilization (resource objective), to ensure that the resource scheduling scheme meets the control performance requirements. The prediction time domain of the model predictive control algorithm is set to 10 steps, each step is 20ms, and the control time domain is set to 5 steps. The state variables are defined as [CPU utilization (%), memory usage (GB), positioning control error (mm), force control error (N)]; The objective function is: min(0.7 × weighted sum of control errors + 0.3 × weighted sum of resource consumption), where the weighted sum of control errors = 0.6 × positioning error + 0.4 × force control error, and the weighted sum of resource consumption = 0.5 × CPU utilization + 0.5 × memory usage; The constraints include edge CPU utilization ≤80%, memory usage ≤16GB, and communication latency ≤50ms. The resource allocation scheme is updated every 200ms through rolling optimization to ensure that the prediction results match the actual control effect.

[0051] For multi-robot collaborative tasks, such as jointly moving large objects, a distributed coordination control model is constructed using the alternating direction multiplier method. This model coordinates and allocates control permissions and resources for each robot. Combined with the global environmental semantic map from the hierarchical perception stage, the control strategy is simulated and practiced using a digital twin control model. This optimizes the collaborative execution path and action coordination strategy, such as the synchronous control parameters for the joint movements of multiple robots, to avoid control failures caused by uncoordinated actions. The distributed coordination control model is based on the alternating direction multiplier method, which decomposes the multi-robot collaborative goal into a local sub-problem and a global coordination problem: the input of the local sub-problem is the resource status (CPU / memory / battery) of a single robot and local environmental perception data, and the output is the action command of a single robot; The global coordination module takes the local output of each robot as input, calculates the synchronization error between robot actions to generate a correction coefficient in the range of [-0.1, 0.1], and feeds it back to each robot to adjust its action parameters. The model training uses scenario samples such as multi-robot collaborative handling and collaborative cleaning. The optimization objective is to minimize the synchronization error of collaborative actions to ensure that the actions of multiple robots are coordinated and consistent.

[0052] Based on the above optimization results, a scheduling instruction containing resource allocation schemes and control parameters is generated and sent to the corresponding edge nodes. Simultaneously, the scheduling scheme and control model parameters are backed up to the cloud database to establish a scheduling and control association file.

[0053] In the deployment and implementation phase, the edge device receives the sub-task allocation scheme generated by the scheduling and optimization phase from the cloud. Using Docker containerization technology, the perception module, control module and execution module corresponding to the sub-task are packaged and instantiated into an independent perception, control and execution integrated service container. The computing power and memory resources are dynamically allocated according to the control requirements of the sub-task, and resource preemption permissions are set for containers of high-priority control tasks. At the edge, a low-latency communication link based on 5G is established, and a deterministic real-time communication system is constructed by integrating Application Data Distribution Service (DDS) and Time-Sensitive Network (TSN). A dual-protocol architecture with optimized MQTT control protocol and DDS protocol adapted on demand is adopted. The accuracy of data interaction is ensured by adding a control command verification field. The high reliability and low jitter transmission of control commands are achieved by relying on the time synchronization and deterministic transmission capabilities of TSN and the low-latency publish-subscribe mechanism of DDS. The simple timeout switching logic is abandoned. A security baseline control module is added on the edge side to build a dual-dimensional judgment mechanism of communication status and edge-side security baseline. The latency risk is avoided from both the transmission link and execution end levels. When the switching trigger condition is met, the system automatically switches to the edge local control mode.

[0054] The TSN network achieves microsecond-level time synchronization between the edge and end-user sides through time-aware shaping and traffic scheduling mechanisms, allocating dedicated transmission time slots for high-frequency control commands to ensure deterministic command transmission. The DDS protocol is configured with low-latency QoS policies for robot control scenarios, employing a publish-subscribe model to achieve precise point-to-point distribution of control commands, eliminating redundant forwarding links, and adapting to stringent real-time control requirements such as dynamic obstacle avoidance and precise grasping. The MQTT protocol is adapted to conventional status feedback and non-high-frequency command interaction scenarios, complementing the DDS protocol. The edge communication scheduling module automatically switches protocols based on command type and real-time requirements. The end-user security baseline control module predefines a core security baseline indicator system, covering indicators strongly related to real-time control, such as end-user real-time kernel computing power utilization, actuator response latency, control command parsing efficiency, and hardware health status. It sets security thresholds and warning thresholds for each indicator and collects indicator data at a microsecond frequency through kernel-level monitoring threads, achieving full-time monitoring of end-user real-time control capabilities.

[0055] The dual-dimensional switching determination mechanism is as follows: The communication quality monitoring unit incorporates transmission delay, packet loss rate, TSN time synchronization accuracy, and DDS command distribution delay into the communication status determination indicators, and the end-side security baseline control module synchronously uploads baseline indicator monitoring data; when any communication status indicator exceeds the preset threshold three times consecutively, and the end-side security baseline indicator is in a warning / over-limit state, the local control mode switching is immediately triggered; if only the communication indicator slightly exceeds the standard but the end-side security baseline is in a normal state, the link redundancy compensation mechanism (such as 5G link frequency switching, DDS time slot reallocation) is activated instead of direct switching, realizing the delay risk avoidance strategy of "link compensation first, local control as a backup"; during the switching process, the continuity of control commands is ensured through command caching and breakpoint resumption mechanism to avoid interruption of action execution.

[0056] The core parameters monitored by the communication quality monitoring unit include TSN time synchronization accuracy, DDS command distribution delay, transmission delay, packet loss rate, and data transmission rate. Transmission delay is calculated using the timestamp difference method, where a timestamp is added when the sending end sends a command, and the difference is calculated when the receiving end receives it. Packet loss rate is calculated by checking and statistically analyzing consecutive command sequence numbers, showing the proportion of missing sequence numbers to the total number of sent sequence numbers. Data transmission rate is calculated per second as the number of valid transmitted data bytes. TSN time synchronization accuracy is determined based on the clock deviation between the edge end and the end side. DDS command distribution delay is the time from when a command is published at the edge end to when it is subscribed and received at the end side. The monitoring frequency is once every 10ms. Combined with the microsecond-level monitoring data from the end-side security baseline control module, a dual-dimensional switching judgment mechanism is used to trigger link compensation or local control mode switching.

[0057] The edge device receives and executes sub-tasks and control commands from the edge device, and drives the actuators (robotic arms, walking mechanisms) to complete the actions through the embedded real-time control system. At the same time, it collects control status data during the execution process, such as actual force control values ​​and positioning errors, and transmits them back to the edge device in real time, forming a control closed loop of deployment, execution and feedback.

[0058] The collected control status data includes three categories: execution parameters, effect feedback data, and resource consumption data. Execution parameters include joint angles, movement speed, force control values, and command execution time. Effect feedback data includes positioning error, action completion rate, and environmental interaction results, such as whether an object was successfully grasped. Resource consumption data includes CPU utilization, memory usage, and communication bandwidth usage. The data is stored in a JSON structure, with fields including timestamp, robot ID, task ID, data type, data value, and unit, ensuring that the edge device and the cloud can quickly parse and correlate the data.

[0059] The adaptive adjustment phase is based on the control status data transmitted back through the deployment, execution, and feedback control closed loop formed during the deployment and implementation phase. The control status monitoring and anomaly diagnosis unit deployed at the edge collects sub-task execution status data, environmental perception data updated during the hierarchical perception phase, and control parameter operation data in real time. By tracking error, adjustment time, and other control performance evaluation indicators, it judges whether there are control deviations such as insufficient grasping force or path deviation, or environmental dynamic changes such as new obstacles or object position shifts. A meta-learning algorithm framework is introduced, based on an offline-trained multi-scenario control parameter initial library. For control deviations monitored in real time, initial adjustment parameters are quickly generated. Combined with preset performance control indicators, control parameters are optimized through online fine-tuning, such as adjusting the PID parameters of the robotic arm joints and the speed planning parameters of the obstacle avoidance path, so as to achieve rapid adaptation to dynamic environmental changes. When serious control faults such as sensor failure or actuator jamming are detected, the edge-end fault-tolerant control strategy library is automatically triggered, and redundant control schemes are called. For example, when the vision sensor fails, tactile-assisted positioning is enabled. At the same time, the fault information and adjustment strategy are uploaded to the cloud to provide a basis for global control optimization.

[0060] The fault-tolerant control strategy library is categorized and stored according to fault type, including four main categories: sensor faults, actuator faults, communication faults, and control logic faults. The sensor fault category stores solutions such as enabling redundant sensors for single sensor failure and using historical data interpolation for multiple sensor anomalies. The actuator fault category stores solutions such as adjusting the motion path to avoid the faulty joint for single joint jamming and switching to a simplified motion mode for multiple joint failures. The communication fault category stores solutions such as independent control at the edge for cloud communication interruption and enabling local emergency command sets for communication anomalies between the edge and end-side devices. Each solution includes fault determination conditions, redundant component / algorithm calling logic, and control parameter adjustment range, which can be quickly retrieved by matching fault type keywords.

[0061] The optimization feedback phase is based on the fault information, adjustment strategy and full-link data of each stage uploaded in the adaptive adjustment phase. The control status data of the execution process on the end side, the scheduling optimization data and adaptive adjustment data on the edge end, and the global control effect data on the cloud are synchronized to the distributed shared database to build a control and resource data archive that includes the entire link of perception, modeling, splitting, scheduling, deployment and adjustment. Based on full-link data, the global control model (digital twin control model and MPC optimization model) is dynamically updated using incremental learning algorithms to optimize control parameter thresholds and performance evaluation indicators. Combined with the resource usage of each edge node, the global resource scheduling strategy is adjusted to achieve a balance between the dual objectives of optimal control performance and maximum resource utilization. The dual-objective balance adjustment logic adopts a dynamic priority switching and resource elastic allocation mechanism. When the control performance fails to meet the preset target, priority is given to ensuring control performance, allocating more computing power and bandwidth resources to the corresponding task, and appropriately reducing the resource allocation ratio of non-critical tasks. When the control performance meets the target but the resource utilization rate is lower than the preset standard, the resource allocation ratio is adjusted, and redundant resources are scheduled to other pending tasks or reserved as emergency resources. During the adjustment process, the historical optimal configuration in the full-link data archive is used as a reference benchmark, and the resource allocation weight is dynamically adjusted in combination with the current resource load status of each edge node to ensure the collaborative optimization of the two objectives. The resource load status includes CPU utilization, memory utilization, and bandwidth utilization.

[0062] The cloud distributes the optimized control model parameters and resource scheduling strategies to each edge node, which then feeds back to various stages, including global perception accuracy optimization in the layered perception stage, task model correction in the requirement modeling stage, execution sequence optimization in the task splitting stage, and resource allocation scheme optimization in the scheduling optimization stage. The edge device further fine-tunes the control parameters and deployment scheme based on local real-time data, forming a two-way iterative closed loop of cloud-based global optimization, edge-based local adaptation, and edge-side execution feedback, continuously improving the overall system control performance and service orchestration efficiency.

[0063] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0064] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A cloud-edge collaborative embodied intelligent humanoid robot service orchestration method, characterized in that, include: Layered perception stage: Through the collaborative work of a layered architecture of edge devices, cloud devices, and end devices, the matching of perceived data with control objectives is achieved; Multimodal raw data is collected at the edge, and key features are selected after preprocessing. Cross-modal data is fused at the edge and control availability is verified. The cloud combines historical data and digital twin control model to build a global environmental semantic map and generate perception optimization instructions to be sent to the edge. Demand modeling phase: Transform user multimodal service requirements into a quantitative control target system, and form a corresponding model of requirements and control through edge parsing, cloud deep modeling and local adaptation correction; Task decomposition phase: Based on the requirement and control correspondence model, core tasks are decomposed through multi-dimensional evaluation and dependency analysis, generating sub-task execution sequences and marking execution methods; Scheduling optimization phase: The cloud receives sub-task information, resource status data and control performance requirements uploaded by each edge node, verifies them through simulation and combines them with predictive control algorithms to optimize the resource allocation scheme; optimize the action coordination strategy for multi-robot collaborative scenarios, generate scheduling instructions and send them to the corresponding edge nodes and back up model parameters, so as to achieve synchronous optimization of global resource scheduling and control performance; Deployment and implementation phase: Containerization technology is used to deploy integrated service modules, dynamically configure resources and establish low-latency communication links, execute sub-tasks and collect control status data to form a closed loop; Adaptive adjustment phase: Based on closed-loop data, control deviation and dynamic environmental changes are monitored in real time. Initial adjustment parameters are generated and optimized online through preset performance control indicators and meta-learning. For severe control faults, the fault-tolerant control strategy library is called to call redundant control schemes, and fault information and adjustment strategies are uploaded to the cloud to achieve dynamic scenario adaptation. Optimization feedback phase: Integrate end-to-end data to build control and resource profiles, optimize control models and scheduling strategies through incremental learning and distribute them to the aforementioned phases to form an iterative closed loop.

2. The cloud-edge-end collaborative embodied intelligent humanoid robot service orchestration method according to claim 1, characterized in that, In the layered perception stage, a lightweight multimodal perception module is deployed on the edge side to collect raw data of the environment and the robot's own state. After preprocessing, key features are selected and a perception and control correspondence table is generated. At the edge, a multi-source data fusion and control constraint verification unit is deployed to fuse key feature data to generate a scene perception tensor and verify that it meets local control requirements. If it does not meet the requirements, a second acquisition is triggered on the edge side. A global perception enhancement and control accuracy prediction unit is deployed in the cloud. It receives scene perception tensors from each edge node, combines historical data with digital twin control models to construct a global environmental semantic map with control accuracy requirements, predicts the impact of dynamic scene changes on control performance, and issues perception accuracy optimization instructions.

3. The cloud-edge-end collaborative embodied intelligent humanoid robot service orchestration method according to claim 2, characterized in that, The requirement modeling phase establishes the association between requirement modeling and control strategies by pre-setting performance control indicators; the edge receives user multimodal service requirements, parses core task objectives and extracts implicit control constraints and control priorities, and uploads the preliminary results to the cloud; The cloud uses a large-scale embodied intelligent language model to perform deep semantic understanding and control target modeling, generate structured control task models, quantify sub-task control performance indicators, and construct multi-objective optimization functions. By combining local scene perception tensors with robot control capabilities at the edge, the structured control task model is adapted and modified to form a model that corresponds to the requirements and control.

4. The cloud-edge-end collaborative embodied intelligent humanoid robot service orchestration method according to claim 3, characterized in that, The task decomposition stage is based on the demand and control correspondence model. At the edge, a three-dimensional evaluation model of complexity, real-time performance, and control accuracy is used to decompose the core task into sub-tasks with control attributes. Assess the real-time requirements and resource consumption of each subtask, identify control dependencies between subtasks and generate execution sequences; combine edge resource status and control performance indicators, and mark the execution mode of subtasks through dynamic priority sorting: subtasks with high real-time and control precision requirements are marked as edge-priority execution, while subtasks with high resource consumption but moderate control precision requirements are marked as edge and cloud collaborative execution.

5. The cloud-edge-end collaborative embodied intelligent humanoid robot service orchestration method according to claim 4, characterized in that, In the scheduling optimization phase, the cloud receives sub-task information, resource status data, and control performance requirements uploaded by each edge node, constructs a dual-objective scheduling model for control and resources, and introduces a model predictive control algorithm to predict the resource allocation effect through a rolling optimization strategy. For multi-robot collaborative tasks, a distributed coordination control model is constructed to coordinate control permissions and resource allocation, and the collaborative path and action strategy are optimized through simulation and exercise of a cloud-based digital twin model combined with a global environmental semantic map. The system generates scheduling instructions containing resource allocation schemes and control parameters, sends them to the corresponding edge nodes, and backs up the scheduling schemes and model parameters to the cloud database.

6. The cloud-edge-end collaborative embodied intelligent humanoid robot service orchestration method according to claim 5, characterized in that, During the deployment and implementation phase, the edge terminal uses containerization technology to package the perception, control, and execution modules corresponding to the sub-tasks into an integrated service container, dynamically allocates resources, and sets resource preemption permissions for high-priority task containers. For the collaborative execution of sub-tasks between the edge terminal and the cloud, a low-latency communication link is established and a communication quality monitoring unit is deployed. When the communication latency exceeds a preset threshold, the system automatically switches to local control at the edge terminal. The edge terminal receives sub-tasks and control commands issued by the edge terminal, drives the actuator to complete actions, and collects control status data to send back to the edge terminal, forming a control closed loop.

7. The cloud-edge-end synergy embodied intelligent humanoid robot service orchestration method according to claim 6, characterized in that, In the adaptive adjustment phase, the edge terminal collects subtask execution status data, environmental perception data updated in the hierarchical perception phase, and control parameter operation data in real time through the control status monitoring and anomaly diagnosis unit. Based on the control performance evaluation index, it judges whether there is control deviation or dynamic environmental change. A meta-learning framework is introduced to generate initial adjustment parameters based on the offline trained multi-scenario control parameter initial library, which are then fine-tuned and optimized online to adapt to the dynamic environment. When a serious control failure is detected, the edge-end fault-tolerant control strategy library is triggered to call the redundancy scheme, and the fault information and adjustment strategy are simultaneously uploaded to the cloud.

8. The cloud-edge-device collaborative embodied intelligent humanoid robot service orchestration method according to claim 7, characterized in that, During the optimization feedback phase, the control status data at the mid-end, the scheduling optimization data and adaptive adjustment data at the edge, and the global control effect data at the cloud are synchronized to the distributed shared database to construct a full-link control and resource data archive. The global control model is dynamically updated using an incremental learning algorithm to optimize control parameter thresholds and performance evaluation indicators, and to adjust the global resource scheduling strategy to balance control performance and resource utilization. The cloud distributes the optimized control model parameters and scheduling strategies to each edge node, feeds back to the aforementioned stages, and fine-tunes them in conjunction with local real-time data, forming a two-way iterative closed loop of global optimization in the cloud, local adaptation at the edge, and execution feedback at the edge.

Images