Cloud-edge collaborative hardware and software integrated intelligent brain dock system

The cloud-edge collaborative hardware and software integrated intelligent brain dock system integrates edge computing, multimodal perception, model deployment and edge inference, and communication adaptation capabilities, which solves the problem of insufficient integration of robot edge intelligent carriers, realizes the collaboration between the cloud and the field, and improves the real-time performance and reliability of the robot in the field.

CN123068097APending Publication Date: 2026-11-10INST OF AUTOMATION CHINESE ACAD OF SCI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611272393.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-20
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

The lack of integrated edge intelligence carriers on the robot's field side results in a large system size, excessive power consumption, complex wiring, and significantly increased deployment and maintenance costs. Furthermore, the lack of an effective cloud-edge collaboration mechanism between cloud training management and real-time field execution makes it difficult to balance the continuous iteration capabilities of cloud models with the real-time control requirements of the robot on-site.

Method used

This paper presents a cloud-edge collaborative hardware and software integrated intelligent brain dock system, which integrates an edge computing module, a multimodal perception unit, a model deployment and edge inference unit, a communication adaptation unit, and a cloud-edge collaboration unit. It enables local inference and real-time control of the model on the edge computing module, reducing the dependence on real-time inference and network links in the cloud.

Benefits of technology

Reduce the wiring complexity and deployment and maintenance costs caused by splicing external equipment, while taking into account the iterative capabilities of pre-trained models in the cloud and the real-time control requirements of robots on site, thereby improving the real-time performance and reliability of robots in on-site operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123068097A_ABST
    Figure CN123068097A_ABST
Patent Text Reader

Abstract

This disclosure provides a cloud-edge collaborative hardware-software integrated intelligent brain dock system, comprising: an intelligent brain dock body, including an edge computing module; and multiple software functional units integrated into the intelligent brain dock body, including: a cloud-edge collaboration unit configured to receive description information of a pre-trained model sent from the cloud; a model deployment and edge inference unit configured to determine an edge inference engine compatible with the edge computing module based on the pre-trained model; a multimodal perception unit configured to collect environmental data of the robot's operating environment; the model deployment and edge inference unit is further configured to perform inference processing on the environmental data based on the edge inference engine to obtain control data; a communication adaptation unit configured to send control data to the robot and receive control result information fed back by the robot; and the cloud-edge collaboration unit is further configured to feed back model execution result information to the cloud, the model execution result information including information obtained based on the control result information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure generally relates to the field of artificial intelligence and robotics system integration technology, and more specifically, to a cloud-edge collaborative hardware-software integrated intelligent brain dock system. Background Technology

[0002] In recent years, cloud-based artificial intelligence models, represented by large-scale parametric deep neural networks, have made significant progress in robotic intelligence tasks such as object detection, semantic segmentation, grasping pose estimation, environmental perception, and task planning. However, the following prominent problems still exist in stably and efficiently deploying cloud-based artificial intelligence capabilities to actual robotic platforms.

[0003] (1) There is a lack of integrated edge intelligence carriers on the robot's field side. Existing solutions often use external industrial control computers, independent sensors and separate communication modules to achieve intelligent transformation, resulting in a large system size, excessive power consumption, complex wiring, and a significant increase in deployment and maintenance costs.

[0004] (2) There is a lack of effective cloud-edge collaboration mechanism between cloud-based training management and real-time on-site execution. The cloud excels at model training, version management, and task orchestration, while the robot on-site requires low-latency, high-reliability local perception, inference, and control execution. Existing solutions tend to focus on pure cloud-based inference or pure local deployment, making it difficult to balance the continuous iteration capabilities of cloud-based models with the robot's real-time control requirements on-site. Summary of the Invention

[0005] This disclosure provides a cloud-edge collaborative hardware and software integrated intelligent brain dock system to solve at least one of the above-mentioned problems.

[0006] According to a first aspect of the present disclosure, a cloud-edge collaborative hardware-software integrated intelligent brain dock system is provided, comprising: an intelligent brain dock body, including an edge computing module for providing computing resources; and multiple software functional units integrated into the intelligent brain dock body, the multiple software functional units including: a cloud-edge collaboration unit, a model deployment and edge inference unit, a multimodal perception unit, and a communication adaptation unit; wherein, the cloud-edge collaboration unit is configured to receive description information of a pre-trained model sent from the cloud, the pre-trained model being trained by the cloud; the model deployment and edge inference unit is configured to determine an edge inference engine adapted to the edge computing module based on the pre-trained model; the multimodal perception unit is configured to collect environmental data of the robot's operating environment; the model deployment and edge inference unit is further configured to perform inference processing on the environmental data based on the edge inference engine to obtain control data; the communication adaptation unit is configured to send the control data to the robot and receive control result information fed back by the robot; the cloud-edge collaboration unit is further configured to feed back model execution result information to the cloud, the model execution result information including information obtained based on the control result information.

[0007] Optionally, the cloud is used to perform model training, model management, and task orchestration. The cloud-edge collaboration unit is further configured to: receive description information and task instructions of the pre-trained model from the cloud and send them to the model deployment and edge inference unit; wherein the description information of the pre-trained model includes model files or model update information, and the task instructions are obtained by executing the task orchestration; receive inference results obtained by the model deployment and edge inference unit and control result information received by the communication adaptation unit, and determine the model execution result information; and feed back the model execution result information to the cloud.

[0008] Optionally, the model deployment and edge inference unit is further configured to execute a standardized model deployment and inference process, which includes: converting the model file of the pre-trained model into an open neural network exchange format to obtain an intermediate model decoupled from the training framework; compiling the intermediate model into the edge inference engine, wherein during the compilation process, operator fusion, layer optimization, or redundant node elimination are performed on the model computation graph, and the numerical accuracy is determined based on the hardware characteristics of the edge computing module; loading the edge inference engine, performing inference processing on the environmental data, and converting the inference result into the control data.

[0009] Optionally, the intelligent brain dock body further includes: a robot communication interface, which includes at least one of the following interfaces: Ethernet interface, CAN bus interface, USB interface, serial communication interface, and real-time data exchange interface; the communication adaptation unit includes: a physical interface layer configured to drive the robot communication interface; a protocol driver layer configured to support at least one of the following protocols: TCP / IP protocol, CAN protocol, RTDE protocol, and robot manufacturer's proprietary communication protocol; and a protocol unified abstraction layer configured to encapsulate the control interfaces and status feedback interfaces of heterogeneous robots into a unified robot control call interface, wherein the heterogeneous robots are robots that use different motion mechanisms, different controllers, different communication interfaces, or different communication protocols.

[0010] Optionally, the heterogeneous robot includes at least one of the following: a mobile robot, a collaborative robotic arm, an industrial robotic arm, a mobile operation composite robot, and a warehouse sorting robot.

[0011] Optionally, the pre-trained model is a model obtained through initial training or parameter configuration using data collected by the source robot as basic sample data. The plurality of software functional units further include a cross-platform rapid adaptation unit, configured to: use data collected by the target robot as target sample data; based on the target sample data, and according to hardware and operational scenario differences between the source robot and the target robot, perform model fine-tuning and parameter adaptation processing on the pre-trained model to obtain an adapted model; wherein the model deployment and edge inference unit is further configured to determine an edge inference engine compatible with the edge computing module based on the adapted model.

[0012] Optionally, the intelligent brain dock body further includes: a multimodal perception module, which includes at least one of the following: an RGB-D depth camera, a three-dimensional LiDAR, an inertial measurement unit, and a voice interaction module. The RGB-D depth camera is used to acquire color images and depth images of near-field targets, the three-dimensional LiDAR is used to acquire three-dimensional point cloud data of the working environment, the inertial measurement unit is used to acquire robot posture data and motion data, and the voice interaction module is used to collect voice commands or output voice information.

[0013] Optionally, the multimodal sensing unit is further configured to perform fusion processing on the data acquired by the multimodal sensing module.

[0014] Optionally, the edge computing module includes an embedded AI computing module that supports GPU parallel computing, neural network acceleration, or tensor computing.

[0015] Optionally, the intelligent brain dock body further includes: a power module for supplying power to the edge computing module; and a mounting component for fixing the intelligent brain dock body to a physical device on the local side of the robot.

[0016] The technical solutions provided by the embodiments of this disclosure offer at least the following beneficial effects: According to the cloud-edge collaborative hardware-software integrated intelligent brain dock system of this disclosure, the intelligent brain dock itself serves as the edge intelligence hub on the robot's field side. By integrating edge computing, multimodal perception, model deployment and edge inference, communication adaptation, and cloud-edge collaboration capabilities within the same hardware-software integrated carrier, it can reduce the wiring complexity and deployment and maintenance costs caused by splicing external devices. It also allows the cloud to focus on the training and management of the artificial intelligence model, while enabling the robot's field side to conveniently complete model deployment, real-time inference, and control execution. This is beneficial for balancing the cloud's pre-trained model iteration capabilities with the robot's real-time control requirements on-site. Simultaneously, the artificial intelligence model performs local inference on the edge computing module of the intelligent brain dock itself, and robot control data is sent to the on-site robot platform via the communication adaptation unit. This reduces the dependence on cloud-based real-time inference and network link stability, improving the real-time performance and reliability of the robot's on-site operations. Therefore, the embodiments of this disclosure solve the problems of insufficient integration of the robot's edge intelligence carrier and the difficulty in effectively coordinating cloud-based training management with on-site real-time execution in related technologies.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0019] Figure 1 This is a schematic diagram of the overall cloud-edge collaborative architecture of a hardware-software integrated intelligent brain dock system according to exemplary embodiments of the present disclosure.

[0020] Figure 2 This is a schematic diagram of a standardized model deployment and inference process according to exemplary embodiments of the present disclosure.

[0021] Figure 3 This is a flowchart illustrating a cross-platform rapid adaptation unit according to an exemplary embodiment of the present disclosure.

[0022] Figure 4 This is a schematic diagram of cloud and edge model library deployment according to exemplary embodiments of the present disclosure. Detailed Implementation

[0023] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0024] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0025] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. Another example is "performing at least one of step one and step two", which means the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0026] The cloud-edge collaborative hardware-software integrated smart docking system according to exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0027] An exemplary embodiment of this disclosure provides a cloud-edge collaborative hardware-software integrated intelligent brain dock system. This system is a rapidly adaptable hardware-software integrated system for intelligent robot perception and decision-making tasks. Depending on the robot's actual needs, its specific form can be a hardware-software system with sensors or a chip system integrating software. The intelligent brain dock system integrates multimodal perception and decision-making algorithms, possessing the ability to perform tasks collaboratively via the cloud, edge, and device, and can be widely deployed on various heterogeneous new robot platforms. The intelligent brain dock system may include a hardware intelligent brain dock body and multiple software functional units integrated into the intelligent brain dock body. The intelligent brain dock body includes an edge computing module for providing computing resources, such as, but not limited to, providing computing resources required for artificial intelligence model inference, sensor data processing, and control command generation.

[0028] Figure 1 This is a schematic diagram of the overall cloud-edge collaborative architecture of a hardware-software integrated intelligent brain dock system according to exemplary embodiments of the present disclosure. The overall architecture adopts a three-layer collaborative architecture: cloud—brain dock—robot. Figure 1This demonstrates the relationships between the cloud platform, the intelligent brain dock ontology, and the heterogeneous robot platform in terms of tasks, models, inference results, control commands, and state feedback. The cloud platform is used for model training, model management (including but not limited to model version maintenance), and task orchestration. It also deploys a large-scale perception-interaction-decision-scheduling model. For example, it can utilize high-performance GPU clusters to train perception models (including but not limited to object detection models, semantic segmentation models, and grasping pose estimation models). The trained model weight files are stored in PyTorch format (.pth), and task commands and model update packages are sent to the intelligent brain dock ontology via a network interface. The intelligent brain dock ontology is the core of the entire system, undertaking three main responsibilities: edge inference, perception processing, and robot control. It is used to complete model conversion and deployment, edge inference, robot control data generation, and state feedback processing on the robot's field side. It can interact and communicate with the large-scale model deployed on the cloud platform through a cloud-edge collaborative architecture, collect environmental data through a multimodal perception unit, perform AI inference through a local inference engine, and send control commands to the downstream robot platform through a multi-protocol communication adaptation unit. The robot end can include various heterogeneous robot platforms, which are used to receive control data sent by the intelligent brain dock body, execute corresponding tasks, and also feed back the execution results and status information (i.e., the control result information described below) to the intelligent brain dock body.

[0029] Reference Figure 1 The intelligent brain dock system comprises several software functional units, including: a cloud-edge collaboration unit, a model deployment and edge inference unit, a multimodal perception unit, and a communication adaptation unit. Among these units, the cloud-edge collaboration unit is configured to receive description information of a pre-trained model from the cloud, the pre-trained model being trained in the cloud; the model deployment and edge inference unit is configured to determine an edge inference engine compatible with the edge computing module based on the pre-trained model; the multimodal perception unit is configured to collect environmental data from the robot's operating environment; the model deployment and edge inference unit is also configured to perform inference processing on the environmental data based on the edge inference engine to obtain control data (e.g., including but not limited to control commands and task execution parameters); the communication adaptation unit is configured to send control data to the robot and receive control result information fed back by the robot; and the cloud-edge collaboration unit is also configured to feed back model execution result information to the cloud, including information obtained based on the control result information.

[0030] The cloud-edge collaborative hardware-software integrated intelligent brain dock system according to the exemplary embodiments of this disclosure uses the intelligent brain dock itself as the edge intelligence hub on the robot's field side. By integrating edge computing, multimodal perception, model deployment and edge inference, communication adaptation, and cloud-edge collaboration capabilities within the same hardware-software integrated carrier, it can reduce the wiring complexity and deployment and maintenance costs caused by splicing external devices. It also allows the cloud to focus on the training and management of artificial intelligence models, while enabling the robot's field side to easily complete model deployment, real-time inference, and control execution. This is beneficial for balancing the cloud's pre-trained model iteration capabilities with the robot's real-time control requirements on-site. Simultaneously, the artificial intelligence model performs local inference on the edge computing module of the intelligent brain dock itself, and robot control data is sent to the on-site robot platform via the communication adaptation unit. This reduces the dependence on cloud-based real-time inference and network link stability, improving the real-time performance and reliability of the robot's on-site operations. Therefore, the embodiments of this disclosure solve the problems of insufficient integration of robot edge intelligence carriers and the difficulty in effectively coordinating cloud-based training management and on-site real-time execution in related technologies.

[0031] In simple terms, the cloud-edge collaboration unit can be used to connect the cloud platform and the intelligent brain dock body; the model deployment and edge inference unit can be used to convert artificial intelligence models trained or managed in the cloud into inference models adapted to edge computing modules, and perform local inference based on environmental data collected by the multimodal perception unit; the multimodal perception unit can be used to collect on-site environmental data; and the communication adaptation unit can be used to transmit control data and receive status feedback between the intelligent brain dock body and the robot platform.

[0032] The following section provides a further description of a cloud-edge collaborative hardware-software integrated smart docking system according to exemplary embodiments of the present disclosure.

[0033] Optionally, the cloud-edge collaboration unit is further configured to: receive description information and task instructions of the pre-trained model from the cloud and send them to the model deployment and edge inference unit, wherein the description information of the pre-trained model includes model files or model update information, and the task instructions are obtained through task orchestration processing; receive the inference results obtained by the model deployment and edge inference unit and the control result information received by the communication adaptation unit, and determine the model execution result information; and feed back the model execution result information to the cloud. In other words, referring to... Figure 1The cloud-edge collaborative unit enables the interaction of task instructions, model description information, inference results, and system status information between the cloud platform and the intelligent brain dock itself. Task instructions are cloud-programmed instructions used to guide robot operations. The model description information includes the model file, which is the entire model file trained in the cloud. Model update information reflects changes in the model file after the pre-trained model is updated in the cloud, reducing data transmission volume. System status information is derived from control result information, including but not limited to task execution status and robot feedback status. Task execution status indicates the completion of task instructions issued from the cloud, while robot feedback status indicates the robot's joint angles, end-effector pose, and other states after task execution. System status information may also include system operating status, reflecting the intelligent brain dock system's operation. The inference results and system status information together constitute the model execution result information, reflecting the edge's operational status and providing a basis for cloud management and model optimization, forming a control closed loop. This cloud-edge collaborative mechanism balances the continuous iteration capability of the cloud-based pre-trained model with the low-latency control requirements of the robot on-site.

[0034] As an example, the intelligent brain dock system of the exemplary embodiments of this disclosure can operate according to the following process: First, the cloud platform completes the training, management, or task orchestration of the perception model and sends task instructions and model description information to the intelligent brain dock itself; second, the intelligent brain dock itself converts the pre-trained model in the cloud into an inference engine adapted to the edge computing module through the model deployment and edge inference unit; then, the multimodal perception unit collects image, depth, point cloud, posture, or voice interaction data in the robot's working environment, and the edge inference engine completes local inference; then, the inference results are converted into robot control instructions or task execution parameters and sent to the target robot platform through the communication adaptation unit; finally, the intelligent brain dock itself receives robot status feedback and uploads the task execution status, inference results, or system operation status to the cloud platform, thereby forming a closed-loop collaborative process of cloud training management, edge-side real-time inference, and robot on-site execution.

[0035] In related technologies, the model deployment process lacks unified standards, resulting in high adaptation costs. Model files trained in the cloud typically rely on specific deep learning frameworks (such as PyTorch and TensorFlow), while edge computing requires specific hardware architectures (such as GPUs and NPUs) for inference acceleration. Without a standardized conversion process from trained models to edge inference engines, each model update or edge hardware replacement necessitates repeated adaptation development, severely impacting model iteration efficiency and engineering deployment efficiency.

[0036] Optionally, the model deployment and edge inference unit is also configured to execute a standardized model deployment and inference process. This process includes: converting the model file of the pre-trained model into an open neural network exchange format to obtain an intermediate model decoupled from the training framework; compiling the intermediate model into an edge inference engine, wherein during compilation, operator fusion, layer optimization, or redundant node elimination are performed on the model computation graph, and numerical precision (e.g., including but not limited to FP32, FP16, or INT8 precision modes) is determined based on the hardware characteristics of the edge computing module; loading the edge inference engine, performing inference processing on environmental data, and converting the inference results into control data. By adopting the open neural network exchange format and the edge inference engine compilation process, cloud-trained or managed AI pre-trained models can be migrated to the robot's on-site edge computing module in a standardized manner. Edge inference efficiency is improved through operator fusion, layer optimization, and numerical precision selection, resulting in an inference model / edge inference engine adapted to the edge computing module. Local inference is then performed based on collected environmental data (e.g., including but not limited to image, depth, point cloud, pose, or voice interaction data).

[0037] As an example, Figure 2 This is a schematic diagram of a standardized model deployment and inference process according to exemplary embodiments of the present disclosure, illustrating the conversion process from a cloud-trained model to an intermediate model in an open neural network exchange format, an edge inference engine, and local inference output, enabling the migration of cloud-trained models to an edge inference engine. (Refer to...) Figure 2 This process may include, for example, the following specific steps.

[0038] (1) Phase 1: AI inference runtime environment configuration.

[0039] Deploy the AI ​​inference runtime environment on the edge computing module in the following order: Install the NVIDIA JetPack software development kit (version 5.0 or higher) to obtain the underlying GPU driver, CUDA (Compute Unified Device Architecture) runtime library, and hardware BSP (Board Support Package) support; Install the PyTorch deep learning framework compiled for the aarch64 (ARM 64-bit) architecture to enable the edge computing module to load and infer PyTorch models; Install the cuDNN (CUDA Deep Neural Network) deep neural network acceleration library to provide GPU acceleration for basic operators such as convolution and pooling; Install the TensorRT (Tensor Runtime) inference optimization library (version 8.0 or higher) to provide the core toolchain for subsequent model compilation and engine generation.

[0040] (2) Phase 2: PyTorch format model is converted to ONNX format.

[0041] After cloud training is complete, the built-in `torch.onnx.export()` interface of PyTorch is called to export the .pth format model weight file to .onnx format (ONNX, Open Neural Network Exchange). ONNX format uses a framework-independent computation graph representation, describing the sequence of operators in a directed acyclic graph (DAG), and storing the computation graph structure and weight parameters in a unified data structure. The core value of this conversion step is that it removes the binding constraint between the cloud-based PyTorch training framework and edge inference hardware, allowing the same trained model to be directly deployed on different hardware architectures that support ONNX (including x86, aarch64, DSP, NPU, etc.) without retraining for each type of hardware.

[0042] (3) Phase 3: ONNX format is converted to TensorRT inference engine.

[0043] Invoke the TensorRT compiler (the `trtexec` tool or the TensorRT Python API), load the .onnx format model file, and perform the following optimized compilation for the Ampere architecture GPU of the edge computing module.

[0044] Computation graph optimization and operator fusion: Merge adjacent convolutional layers with batch normalization layers, activation function layers, etc., to reduce the number of GPU kernel startups and the memory read / write overhead of intermediate results.

[0045] Accuracy calibration and quantization: Select FP32 (full precision), FP16 (half precision), or INT8 (8-bit integer quantization) inference precision based on deployment accuracy requirements. FP16 precision can improve inference throughput by approximately 1.5 to 2 times with minimal accuracy loss; INT8 quantization requires a calibration dataset for statistical distribution of activation values.

[0046] Automatic kernel tuning: TensorRT automatically selects the shortest execution time from a variety of candidate CUDA kernel implementations for the GPU's CUDA cores, Tensor Cores and other execution units on the current hardware.

[0047] Dynamic tensor memory management: Minimizes peak GPU memory usage during inference by using memory pools and asynchronous memory reuse mechanisms.

[0048] The above compilation process generates an .engine format inference engine file that is deeply optimized for current hardware. This file contains the complete network structure, optimized weights, and GPU kernel selection information, and can be directly loaded for inference without further optimization.

[0049] In one implementation, the model deployment and edge inference unit can perform the model transformation and engine generation process according to the following pseudocode: Input: cloud_model_file, hardware_profile, precision_mode Output: edge_inference_engine intermediate_model = export_to_intermediate_format(cloud_model_file) optimized_graph = optimize_graph(intermediate_model, fusion = true, remove_redundant_nodes = true) edge_inference_engine = compile_for_edge_hardware(optimized_graph, hardware_profile, precision_mode) verify_io(edge_inference_engine, robot_control_schema) return edge_inference_engine The pseudocode above is used to illustrate the logical flow of model deployment and edge inference unit. Here, cloud_model_file represents the model file trained in the cloud, hardware_profile represents the hardware architecture information of the edge computing module, precision_mode represents the inference precision mode, and robot_control_schema represents the data structure constraints required for the robot control interface.

[0050] (4) Stage 4: Edge closed loop reasoning.

[0051] After loading the .engine inference engine, the intelligent brain dock enters a real-time inference closed loop, including operations such as the following.

[0052] Data acquisition: The RGB-D (Red-Green-Blue-Depth) depth camera continuously outputs color and depth images at a specified frame rate (usually 30fps), and the 3D LiDAR outputs real-time point cloud frames.

[0053] Preprocessing: Perform preprocessing operations such as scaling and normalization on the image to convert the data into the format and precision required by the input tensor of the inference engine.

[0054] GPU inference: The preprocessed input tensor is copied to the GPU memory, triggering the TensorRT inference engine to perform forward inference and obtain the perceptual result tensor (target bounding box, confidence score, class label or pixel segmentation mask).

[0055] Post-processing: Perform post-processing operations such as non-maximum suppression (NMS) on the inference output to filter redundant detection results.

[0056] Pose calculation: For the grasping pose estimation task, the target detection results are transformed from the image coordinate system to the camera coordinate system using the depth image. Then, based on the pre-calibrated spatial transformation relationship between the camera coordinate system and the robot base coordinate system, the three-dimensional position and grasping posture of the target object relative to the robot base are obtained.

[0057] Command generation and issuance: The perception results are encapsulated into structured control commands, which are then issued by the multi-protocol communication adapter unit to select and match the communication protocol of the current robot platform.

[0058] Status feedback and closed-loop update: Receive the robot's execution status (joint angles, end-effector pose, positioning signal, etc.), update the system status based on feedback, and trigger the next round of perception and reasoning.

[0059] Optionally, the intelligent brain dock also includes: a robot communication interface, which includes at least one of the following interfaces: Ethernet interface, CAN bus interface, USB interface, serial communication interface, and real-time data exchange interface; a communication adaptation unit including: a physical interface layer configured to drive the robot communication interface; a protocol driver layer configured to support at least one of the following protocols: TCP / IP protocol, CAN protocol, RTDE protocol, and robot manufacturer's proprietary communication protocol; and a protocol unification abstraction layer configured to encapsulate the control interfaces and status feedback interfaces of heterogeneous robots into a unified robot control call interface, wherein heterogeneous robots are robots that use different motion mechanisms, different controllers, different communication interfaces, or different communication protocols. Therefore, the upper-level model inference program and task control program do not need to directly adapt to the underlying communication details of each robot platform, thus shielding the differences in communication protocols between different robot platforms. This allows the same intelligent brain dock to access heterogeneous robot platforms that use different motion mechanisms, different controllers, different communication interfaces, or different communication protocols, improving the system's heterogeneous access capability and solving the problem of fragmented communication protocols among industrial robot platforms.

[0060] Specifically, the Ethernet interface (Gigabit / 100Mbps adaptive) can be used for TCP / IP protocol communication and LiDAR data transmission; the CAN bus interface can be used to communicate with robotic arms using the CAN protocol (such as, but not limited to, the Songling Piper robotic arm); the USB interface can be used to connect an RGB-D camera; and HDMI and other expansion interfaces can be used for debugging and peripheral connection.

[0061] As an example, the protocol driver layer may include the following communication sub-units.

[0062] Ethernet TCP / IP subunit: Based on Socket communication, it realizes reliable transmission of control commands and is suitable for robot platforms that communicate via network (such as, but not limited to, Songling MP1000 mobile robot, Songling P40 mobile robot, RS11 warehouse sorting robot, etc.).

[0063] CAN bus sub-unit: Based on the SocketCAN framework, it realizes the transmission and reception of CAN frames and is suitable for robotic arms that use CAN bus communication (such as, but not limited to, the Songling Piper 6-DOF collaborative arm).

[0064] RTDE Real-Time Data Exchange Subunit: Based on the Universal Robots Real-Time Data Exchange (RTDE) protocol, it can realize real-time status synchronization between the controller and the robotic arm at the 1kHz level, and is suitable for UR series robotic arms.

[0065] The unified protocol abstraction layer shields the upper-layer inference engine from the differences in underlying communication protocols, providing a unified robot control interface (for sending control data such as end-effector pose targets, joint angle targets, and gripper control commands) and a status query interface (for reading status information such as joint status, end-effector pose, and position feedback, which can be used as the content of control result information). When adding a new heterogeneous robot platform, only the corresponding communication subunit needs to be added to the protocol driver layer; the upper-layer inference engine does not need to be modified, enabling the access, control, and feedback of multiple heterogeneous robot platforms.

[0066] In one implementation, the protocol unified abstraction layer can select the corresponding protocol driver and issue control commands according to the following pseudocode: Input: robot_id, control_command Output: execution_state protocol_type = query_protocol(robot_id) driver = load_protocol_driver(protocol_type) encoded_command = driver.encode(control_command) driver.send(encoded_command) raw_state = driver.receive_state() execution_state = driver.decode_state(raw_state) return execution_state The pseudocode above illustrates the unified encapsulation logic of the communication adaptation unit for different underlying communication protocols. The upper-layer model deployment and edge inference units only need to generate control_commands in a unified format, without directly handling differences in TCP / IP, CAN, RTDE, or robot manufacturer-specific protocols.

[0067] Optionally, heterogeneous robots include at least one of the following: mobile robots, collaborative robotic arms, industrial robotic arms, mobile manipulation composite robots, and warehouse sorting robots. By integrating different types of robots, operational needs in different environments can be met, thus expanding the scope of application of this disclosure.

[0068] In related technologies, rapid adaptation between heterogeneous robot platforms is challenging. Different robot platforms exhibit significant differences in sensor configuration, operating scenarios, motion mechanisms, controller types, interface formats, and communication protocols. Common protocols include TCP / IP, CAN, RTDE, and proprietary protocols from various manufacturers. If the perception model, control parameters, and communication interfaces all need to be redeveloped for each platform type, it will greatly increase the cost of cross-platform migration and severely restrict the system's scalability and maintainability.

[0069] Optionally, the pre-trained model uses data collected by the source robot as basic sample data, obtained through initial training or parameter configuration. Multiple software functional units include a cross-platform rapid adaptation unit, configured to: use data collected by the target robot as target sample data; based on the target sample data, and according to hardware and operational scenario differences between the source and target robots, perform model fine-tuning and parameter adaptation on the pre-trained model to obtain an adapted model; the model deployment and edge inference unit is further configured to determine an edge inference engine compatible with the edge computing module based on the adapted model. Through this approach, model fine-tuning and parameter adaptation can be performed using source platform data and a small number of adapted samples from the target platform, thereby reducing the cost of migrating and deploying AI models, control parameters, and communication interfaces between different robot platforms. This enables the intelligent brain dock ontology to conveniently perform perception, reasoning, and control tasks on the target robot platform.

[0070] Figure 3 This is a flowchart illustrating a cross-platform rapid adaptation unit according to an exemplary embodiment of the present disclosure, showing the process of source robot platform data, a small number of adaptation samples for the target robot platform, model fine-tuning, and parameter adaptation.

[0071] Reference Figure 3 The cross-platform rapid adaptation unit is used to reduce the cost of migrating and deploying the intelligent brain dock system between different robot platforms. Different robot platforms may differ in sensor models, camera mounting poses, workspace ranges, target object distributions, motion mechanisms, controller interfaces, and communication protocols. Therefore, when the same perception model or control parameters are directly migrated to the target platform, problems such as decreased detection accuracy, coordinate mapping deviations, or control interface mismatches may occur.

[0072] In one implementation, the cross-platform rapid adaptation unit includes a source platform data management subunit, a target platform sample collection subunit, a model fine-tuning subunit, a parameter adaptation subunit, and a deployment and verification subunit.

[0073] The source platform data management subunit is used to store environmental data such as images, depth, and point clouds collected by the source robot platform, as well as robot status and task execution data, and to complete the initial training or parameter configuration of the perception model based on the above data. The target platform sample acquisition subunit is used to collect a small number of adaptation samples on the target robot platform. These adaptation samples include data collected by the target platform's sensors, target object annotation information, robot execution status, or task execution results.

[0074] The model fine-tuning subunit fine-tunes or corrects parameters of the perception model based on the sensor differences and operational scenario differences between the source and target robot platforms. The parameter adaptation subunit adapts control parameters, coordinate transformation parameters, and communication parameters according to the target robot platform's motion mechanism, workspace range, coordinate system configuration, and control interface type. The deployment and verification subunit deploys the adapted model and parameters to the intelligent brain dock body via the model deployment and edge inference unit, and verifies the matching relationship between the inference output format, robot control interface, and status feedback interface.

[0075] Through the aforementioned cross-platform rapid adaptation unit, model fine-tuning and parameter adaptation can be performed using source robot platform data and a small number of adaptation samples from the target robot platform. This enables the intelligent brain dock system to complete the migration and deployment of models, parameters, and interfaces for different robot platforms while maintaining the main hardware and software architecture unchanged.

[0076] In one implementation, the cross-platform rapid adaptation unit can complete the migration and adaptation from the source platform to the target platform according to the following pseudocode: Input: source_dataset, target_samples, source_model, target_robot_profile Output: adapted_engine, adapted_control_params base_model = train_or_load_model(source_dataset, source_model) adapted_model = fine_tune(base_model, target_samples) adapted_control_params = adapt_robot_parameters(target_robot_profile) intermediate_model = export_to_intermediate_format(adapted_model) adapted_engine = compile_for_edge_hardware(intermediate_model, target_robot_profile.hardware) validate(adapted_engine, adapted_control_params, target_robot_profile) return adapted_engine, adapted_control_params The pseudocode above is used to illustrate the processing flow of the cross-platform rapid adaptation unit. Here, source_dataset represents the source robot platform data, target_samples represents a small number of adaptation samples for the target robot platform, and target_robot_profile represents information such as the sensor configuration, hardware architecture, control interface, and communication protocol of the target robot platform.

[0077] As an example, the total cross-platform adaptation time, communication packet loss rate, and task execution success rate can be used to evaluate the system deployment effect. The total cross-platform adaptation time can be expressed as... ,in The sample collection time for the target platform. For model fine-tuning time, Time generation for edge inference engine Configure the time for the communication interface. This refers to the robot integration and verification time. The communication packet loss rate can be expressed as... ,in The number of lost data packets. The total number of data packets sent. The task execution success rate can be expressed as... ,in The number of times the task was successfully completed. This represents the total number of task executions. These metrics can be used to evaluate the cross-platform deployment efficiency, communication reliability, and on-site task execution capabilities of the intelligent brain dock system.

[0078] Optionally, the intelligent brain dock also includes a multimodal perception module, which integrates multiple types of sensors for robot on-site perception and interaction. The multimodal perception module may include at least one of the following: an RGB-D depth camera, a 3D LiDAR, an inertial measurement unit, and a voice interaction module.

[0079] RGB-D depth cameras (such as, but not limited to, the Intel RealSense D435i) are used to acquire color and depth images of near-field targets, and can simultaneously output color and depth image streams, supporting near-field target detection, semantic segmentation, and depth-based grasping pose estimation tasks. The camera can be connected to an edge computing module via a USB interface.

[0080] 3D LiDAR (including but not limited to Livox Mid-360) is used to acquire 3D point cloud data of the working environment, which can be used for mapping the robot's working environment, obstacle perception, and spatial structure perception. LiDAR can be connected to edge computing modules via an Ethernet interface.

[0081] The inertial measurement unit is used to acquire the attitude and motion data (motion data such as angular velocity, acceleration, etc.) of the intelligent brain dock body or robot, which can help complete the robot motion state estimation and multi-sensor data alignment.

[0082] The voice interaction module is used to collect voice commands or output voice information, and can support human-computer interaction tasks.

[0083] It should be understood that the specific locations of these different sensors can be reasonably selected according to the needs of data collection, and there is no need to centrally set up all the sensors together.

[0084] The aforementioned multimodal perception modules work collaboratively to cover a wide range of task requirements, from near-field fine manipulation to wide-area environmental perception and human-computer interaction. Specifically, the RGB-D depth camera provides color and depth information, the 3D LiDAR provides large-scale point cloud information, the inertial measurement unit provides attitude and motion state information, and the voice interaction module provides human-computer interaction information. Therefore, by collecting the aforementioned multimodal environmental data, the intelligent brain dock can simultaneously support tasks such as target detection, semantic segmentation, obstacle perception, localization and mapping, grasping pose estimation, and human-computer interaction.

[0085] Optionally, the multimodal sensing unit is also configured to perform fusion processing on the data acquired by the multimodal sensing module. By configuring the multimodal sensing unit to perform fusion processing on the collected environmental data, simple environmental data processing can be achieved at the edge, reducing dependence on the cloud and helping to improve the real-time performance of the robot's on-site operations.

[0086] As an example, Figure 4 This is a schematic diagram of the cloud and edge model library deployment according to an exemplary embodiment of the present disclosure, in which the above-described data fusion processing can be performed by the edge perception mini-model.

[0087] In related technologies, there is a computing power gap between pre-trained models in the cloud and robot controllers. Pre-trained models in the cloud typically rely on high-performance GPU clusters for training and inference, while the controllers on mobile robots, collaborative robotic arms, industrial robotic arms, or warehouse sorting robots are mostly used for motion control and equipment management, making it difficult to directly support the real-time inference tasks of complex deep learning models. This makes it difficult for cloud intelligence to be implemented on the edge.

[0088] Optionally, the edge computing module includes an embedded AI computing module that supports GPU parallel computing, neural network acceleration, or tensor computing. The edge computing module serves as the core computing power carrier for the intelligent brain dock. By employing an embedded AI computing module with GPU parallel computing, neural network acceleration, or tensor computing capabilities, the robot platform can run complex deep learning models directly without relying on its own controller, thereby reducing the computing power requirements of the robot's main controller.

[0089] As an example, the embedded AI computing module can be equipped with a multi-core central processing unit and integrate I / O resources such as Ethernet interfaces, CAN bus interfaces, USB interfaces, and universal serial interfaces, supporting direct connection between multimodal sensors and heterogeneous robot platforms. In one specific embodiment, the edge computing module can adopt the NVIDIA Jetson Orin NX SUPER embedded module. This module is equipped with an NVIDIA Ampere architecture graphics processor, containing CUDA computing cores and third-generation Tensor Core acceleration units, providing up to 157 TOPS (sparse precision) of AI computing power, suitable for model deployment and edge inference acceleration based on JetPack, CUDA, cuDNN, and TensorRT toolchains. It should be understood that the above models are only specific embodiments and do not constitute a limitation on the hardware model of the edge computing module.

[0090] Optionally, the intelligent brain dock also includes: a power module for supplying power to the edge computing module; and a mounting component for fixing the intelligent brain dock to a physical device on the robot's local side. By configuring the power module, an independent power supply can be provided to the intelligent brain dock, ensuring reliable operation. It should be understood that in embodiments where the intelligent brain dock also includes other power-consuming components such as a multimodal sensing module, the power module can also supply power to these power-consuming components. By configuring the mounting component, the fixed installation of the intelligent brain dock can be easily achieved, improving the ease of on-site deployment. As an example, the physical device on the robot's local side includes, but is not limited to, the robot platform itself, a robot workstation, and an edge control cabinet.

[0091] The following describes a cloud-edge collaborative hardware-software integrated smart docking system according to a specific embodiment of the present disclosure.

[0092] This specific embodiment uses the deployment and verification of the intelligent brain dock system on various heterogeneous robot platforms as an example to illustrate a feasible implementation process of the intelligent brain dock system of this disclosure. This specific embodiment is used to exemplify the composition, deployment, and verification methods of the intelligent brain dock system of this disclosure, and does not limit this disclosure to using only the same robot model, sensor model, computing module model, model structure, communication protocol, or experimental parameters.

[0093] 1. System hardware and platform construction.

[0094] The intelligent brain dock in this specific embodiment includes an embedded AI computing module, a multimodal perception module, a robot communication interface, a power supply module, and mounting components. The embedded AI computing module employs an embedded AI computing platform with GPU parallel computing capabilities to run model deployment programs, edge inference services, sensor data processing programs, and robot control programs. The multimodal perception module includes an RGB-D depth camera and a 3D LiDAR, used to acquire color images and depth images of near-field targets, as well as 3D point cloud data of the working environment, respectively. The robot communication interface includes an Ethernet interface, a CAN bus interface, and a real-time data exchange interface, used to connect to robot platforms employing different controllers and communication protocols.

[0095] In this specific embodiment, the heterogeneous robot platforms accessed by the intelligent brain dock system are shown in Table 1.

[0096] Table 1. Heterogeneous robot platforms connected to the intelligent brain dock system

[0097] 2. Model Deployment and Edge Inference Setup.

[0098] In this specific embodiment, the cloud platform completes the training of the target detection model and the management of model files. After receiving the model files from the cloud, the intelligent brain dock first converts the model files under the training framework into an intermediate model in the Open Neural Network Exchange Format, and then compiles the intermediate model into an edge inference engine adapted to the hardware architecture of the edge computing module. During the compilation process, operator fusion, layer optimization, and redundant node elimination are performed on the model computation graph, and the appropriate precision mode is selected according to the hardware characteristics of the edge computing module.

[0099] The edge inference service loads the compiled inference engine and performs local inference on the images and depth data acquired by the RGB-D depth camera to obtain target detection results, target category, confidence level, and target spatial location information. For tasks that require the robot to perform grasping or manipulation, the system converts the target spatial location information into robot control commands or task execution parameters according to the pre-calibrated spatial transformation relationship between the camera coordinate system and the robot base coordinate system.

[0100] 3. Rapid cross-platform adaptation process.

[0101] This specific embodiment uses data collected by the Songling Piper collaborative arm and its accompanying camera as the source platform's basic data, and the UR5 robotic arm and its accompanying RGB-D camera as the target platform for cross-platform adaptation. The source platform data is used to complete the initial training of the object detection model; the target platform collects a small number of adaptation samples to correct for differences caused by different camera imaging parameters, installation positions, workspace ranges, and task scenarios. The object detection model can adopt the YOLO (You Only Look Once) series of models or other object detection models suitable for edge inference deployment; the model structure itself is not a necessary limitation of this disclosure.

[0102] After collecting a small number of samples from the target platform, the system fine-tunes the perception model and adapts it to the target platform's coordinate transformation parameters, robot control interface parameters, and communication parameters. The adapted model is then deployed to the intelligent brain dock body after being converted and compiled by the model deployment and edge inference unit, and used to perform perception, inference, and control tasks on the target robot platform.

[0103] 4. Evaluation indicators.

[0104] This specific embodiment uses model detection performance, physical operation success rate, cross-platform adaptation time, system continuous operation stability, and communication reliability as evaluation indicators. Specifically, model detection performance is evaluated using precision (P), recall (R), and mAP50; physical operation success rate is calculated based on the number of successful physical object grasping tasks performed by the robot and the total number of tests; cross-platform adaptation time is used to evaluate the total time from target platform sample collection, model fine-tuning, model conversion, inference engine generation to robot integration; system continuous operation stability is used to evaluate system crashes or interruptions during continuous operation; and communication reliability is evaluated using the data packet loss rate statistically analyzed during operation.

[0105] 5. Experimental results.

[0106] The results of cross-platform adaptation and perception operation verification are shown in Table 2.

[0107] Table 2. Cross-platform adaptation and perception operation verification results

[0108] As shown in Table 2, in this specific embodiment, the intelligent brain dock system can quickly adapt the model to the target robot platform and deploy it at the edge. The precision P of the target detection task reaches 97.6%, the recall R reaches 93.4%, and the mAP50 reaches 98.4%, indicating that the model can still maintain good target recognition performance after migration. In 30 UR5 physical object grasping tests, 29 were successful, with a physical object grasping success rate of 96.67%, indicating that the perception results after deployment by the intelligent brain dock can effectively support the robot's physical object operation tasks.

[0109] The results of system stability and communication reliability verification are shown in Table 3.

[0110] Table 3. Verification results of system stability and communication reliability

[0111] As shown in Table 3, in this specific embodiment, the intelligent brain dock system was able to operate continuously for 6 hours on all seven heterogeneous robot platforms without any system crashes or communication interruptions. The data packet loss rate of each communication link was less than 1%, indicating that the communication adaptation unit can maintain good communication reliability under different communication protocols and robot platform conditions.

[0112] In summary, this specific embodiment demonstrates that the intelligent brain dock system of the exemplary embodiments of this disclosure can realize the deployment of pre-trained models from the cloud to the edge, edge inference, robot control command issuance, status feedback reception, and rapid cross-platform adaptation on heterogeneous robot platforms, and can support intelligent operation of robots on the field side.

[0113] In summary, the exemplary embodiments of this disclosure provide an intelligent brain dock system that deeply integrates edge computing, multimodal perception, model deployment and edge inference, communication adaptation, cross-platform rapid adaptation, and cloud-edge collaboration. This system enables cloud-trained artificial intelligence models to be efficiently migrated to the edge of the robot for execution. Through the cloud-edge collaborative architecture, it achieves collaborative interaction between large cloud models and small edge models. At the same time, it flexibly connects to various heterogeneous robot platforms through a unified interface, realizing unified access, edge inference, and rapid deployment of heterogeneous robot platforms. It enables standardized migration and deployment of pre-trained models from the cloud to the robot's field side, improving the heterogeneous robot access capability, edge inference efficiency, real-time performance of field operations, and cross-platform deployment efficiency.

[0114] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0115] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A cloud-edge collaborative hardware-software integrated intelligent brain dock system, characterized in that, include: The intelligent brain dock body includes an edge computing module for providing computing resources; and Multiple software functional units are integrated into the intelligent brain dock body, including: cloud-edge collaboration unit, model deployment and edge inference unit, multimodal perception unit, and communication adaptation unit; The cloud-edge collaboration unit is configured to receive description information of a pre-trained model sent from the cloud, and the pre-trained model is trained by the cloud. The model deployment and edge inference unit is configured to determine an edge inference engine that is compatible with the edge computing module based on the pre-trained model. The multimodal sensing unit is configured to collect environmental data of the robot's operating environment; The model deployment and edge inference unit is also configured to perform inference processing on the environmental data based on the edge inference engine to obtain control data; The communication adapter unit is configured to send the control data to the robot and receive control result information fed back by the robot. The cloud-edge collaboration unit is also configured to feed back model execution result information to the cloud, the model execution result information including information obtained based on the control result information.

2. The cloud-edge collaborative hardware-software integrated intelligent brain dock system as described in claim 1, characterized in that, The cloud is used to perform model training, model management, and task orchestration. The cloud-edge collaborative unit is further configured as follows: The system receives the description information and task instructions of the pre-trained model sent from the cloud and sends them to the model deployment and edge inference unit. The description information of the pre-trained model includes model files or model update information, and the task instructions are obtained by executing the task orchestration process. Receive the inference results obtained by the model deployment and edge inference unit and the control result information received by the communication adaptation unit, and determine the model execution result information; The model execution result information is fed back to the cloud.

3. The cloud-edge collaborative hardware-software integrated intelligent brain dock system as described in claim 1, characterized in that, The model deployment and edge inference unit is also configured to execute a standardized model deployment and inference process, which includes: The model file of the pre-trained model is converted into an open neural network exchange format to obtain an intermediate model decoupled from the training framework. The intermediate model is compiled into the edge inference engine, wherein during the compilation process, operator fusion, layer optimization or redundant node elimination are performed on the model computation graph, and the numerical accuracy is determined based on the hardware characteristics of the edge computing module. The edge inference engine is loaded to perform inference processing on the environmental data, and the inference results are converted into the control data.

4. The cloud-edge collaborative hardware-software integrated intelligent brain dock system as described in claim 1, characterized in that, The intelligent brain dock body also includes: a robot communication interface, which includes at least one of the following interfaces: Ethernet interface, CAN bus interface, USB interface, serial communication interface, and real-time data exchange interface; The communication adapter unit includes: The physical interface layer is configured to drive the robot's communication interface; The protocol driver layer is configured to support at least one of the following protocols: TCP / IP protocol, CAN protocol, RTDE protocol, and robot manufacturer's proprietary communication protocol; The protocol unification abstraction layer is configured to encapsulate the control interface and status feedback interface of heterogeneous robots into a unified robot control call interface, wherein the heterogeneous robots are robots that use different motion mechanisms, different controllers, different communication interfaces or different communication protocols.

5. The cloud-edge collaborative hardware-software integrated intelligent brain dock system as described in claim 4, characterized in that, The heterogeneous robot includes at least one of the following: mobile robot, collaborative robotic arm, industrial robotic arm, mobile operation composite robot, and warehouse sorting robot.

6. The cloud-edge collaborative hardware-software integrated intelligent brain dock system as described in claim 1, characterized in that, The pre-trained model is a model obtained through initial training or parameter configuration using data collected by the source robot as basic sample data. The multiple software functional units further include a cross-platform rapid adaptation unit, configured as follows: Use the data collected by the target robot as the target sample data; Based on the target sample data, and according to the hardware differences and operational scenario differences between the source robot and the target robot, the pre-trained model is fine-tuned and parameter adapted to obtain an adapted model. The model deployment and edge inference unit is further configured to determine an edge inference engine that is compatible with the edge computing module based on the adaptation model.

7. The cloud-edge collaborative hardware-software integrated intelligent brain dock system as described in claim 1, characterized in that, The intelligent brain dock also includes: A multimodal perception module, comprising at least one of the following: an RGB-D depth camera, a 3D LiDAR, an inertial measurement unit, and a voice interaction module, wherein the RGB-D depth camera is used to acquire color and depth images of near-field targets, the 3D LiDAR is used to acquire 3D point cloud data of the working environment, the inertial measurement unit is used to acquire robot posture and motion data, and the voice interaction module is used to collect voice commands or output voice information.

8. The cloud-edge collaborative hardware-software integrated intelligent brain dock system as described in claim 7, characterized in that, The multimodal sensing unit is also configured to perform fusion processing on the data acquired by the multimodal sensing module.

9. The cloud-edge collaborative hardware-software integrated intelligent brain dock system as described in claim 1, characterized in that, The edge computing module includes an embedded AI computing module, which supports GPU parallel computing, neural network acceleration, or tensor computing.

10. The cloud-edge collaborative hardware-software integrated intelligent brain dock system as described in any one of claims 1 to 9, characterized in that, The intelligent brain dock also includes: A power supply module is used to supply power to the edge computing module; Installation components are used to fix the intelligent brain dock body to the physical device on the local side of the robot.