Privacy protection-oriented robot large model cloud edge-end collaborative reasoning and federated learning system

Abstract

The invention belongs to the field of intelligent edge systems and privacy enhancement computing, and particularly relates to a privacy protection-oriented robot large model cloud edge end collaborative reasoning and federated learning system, which comprises a cloud server layer used for deploying a large-scale pre-training model and executing complex reasoning and global federated learning coordination; the edge calculation layer is used for deploying an intermediate layer model and executing local data aggregation, privacy protection processing and intermediate feature calculation; the terminal equipment layer is used for deploying a lightweight model and executing data acquisition, primary processing and lightweight reasoning; the federated learning framework is used for optimizing the model; the privacy protection module is used for integrating data localization, differential privacy, homomorphic encryption, secure multi-party computing and a block chain verification mechanism; the adaptive allocation module is used for dynamically adjusting computing resources. According to the method, the problems of privacy leakage risk, computing resource limitation, network delay, insufficient data isolation and the like of the traditional AI service in a robot scene are solved, and efficient privacy protection and data security isolation are realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of intelligent edge systems and privacy-enhanced computing, specifically a robot large model cloud-edge-end collaborative reasoning and federated learning system for privacy protection. BACKGROUND

[0002] With the rapid development of large-scale pre-training models such as GPT series, LLaMA and other multi-modal models, artificial intelligence technology is gradually being applied to the field of robots. However, the current technology has the following shortcomings:

[0003] 1. Privacy leakage risk: The data generated in the user environment has high privacy, including personal living habits, family environment information, personal conversation content, etc. Traditional cloud AI architecture needs to upload these data to the central server for processing, which has a serious risk of privacy leakage. According to the report of International Data Protection Association (IDPA), more than 78% of cloud AI services have different degrees of data leakage risk.

[0004] 2. Computing resource limitation: Large-scale AI models (parameter quantity exceeding 100 billion) have high computing resource requirements, and typical inference processes require 16-32GB of video memory and a large amount of computing power, making it difficult to fully deploy on resource-limited terminal devices. Robots are usually equipped with low-power processors and limited memory, which cannot support the complete running of large models.

[0005] 3. Network delay problem: The pure cloud inference scheme has a network delay problem, and the typical cloud request-response delay is hundreds of milliseconds, which can reach several seconds in network congestion, which cannot meet the requirements of robots for real-time interaction (<100ms response time).

[0006] 4. Insufficient data isolation: Existing solutions lack effective data isolation mechanisms and are difficult to guarantee user data security. Most cloud services use multi-tenant architecture, and there is a risk of cross-contamination of different user data during processing.

[0007] 5. Model optimization dilemma: The data in a single environment is limited and unevenly distributed, making it difficult to continuously optimize the performance of AI models for specific scenarios. Traditional centralized learning methods require centralized storage of data, which contradicts the demand for privacy protection.

[0008] Federated learning, as a distributed machine learning method, allows multiple participants to jointly train models without sharing raw data, which has the advantage of protecting privacy. However, existing federated learning solutions face the following technical challenges when large models are applied to the robot scenario:

[0009] 1. Model splitting problem: How to scientifically split the large model into components suitable for the cloud-edge-end three-layer architecture, while ensuring inference performance and privacy protection effect;

[0010] 2. Reasoning coordination complexity: how to achieve efficient coordination reasoning between parts of the model while protecting privacy;

[0011] 3. Privacy protection and performance balance: how to provide strong privacy protection while not significantly reducing model performance and user experience;

[0012] 4. Resource allocation optimization: how to dynamically adjust the computing allocation strategy according to the task characteristics and resource conditions.

[0013] The problems of related patent technologies are as follows:

[0014] 1. Application No. CN110363183B discloses a "service robot visual picture privacy protection method based on generative adversarial network", which uses an improved Cycle-GAN algorithm to convert pictures containing privacy content into pictures without privacy content, solving the problem of privacy leakage from the source. However, this solution only processes two-dimensional image data and does not consider the privacy protection needs of multi-modal data. Its privacy protection mechanism is limited to the field of visual data and cannot be extended to other modalities such as speech and text. Although this method uses a training data growth mechanism to improve generalization, it does not consider scenarios combined with large models and lacks optimization design for resource-constrained environments.

[0015] 2. Application No. CN111421554B discloses an "edge computing-based mechanical arm intelligent control system, method and device", which uses a three-layer architecture of central cloud, edge cloud and edge node to achieve low-latency, high-security intelligent control of industrial mechanical arms. However, this solution mainly focuses on control instruction processing and does not consider the deployment of AI large models and privacy protection. The system uses simple data encryption transmission, but does not implement local data processing and multi-level privacy protection, which poses a risk of data leakage. In addition, although its edge computing architecture reduces network dependence, it does not specifically optimize for the high computing demand of large model inference.

[0016] 3. Application No. CN113435604B discloses a "federated learning optimization method and device", which introduces a delayed global gradient mechanism and a local random average control algorithm to effectively overcome model bias and improve communication efficiency of federated learning. However, this solution is for general machine learning models and does not consider the specific computing complexity and parameter size of large models. It also does not design a specific privacy protection mechanism for the robot scenario. The method lacks model splitting and coordination reasoning capabilities, making it unsuitable for resource-constrained terminal devices. At the same time, although its optimization algorithm improves convergence speed, it has limited adaptability in a heterogeneous device environment.

[0017] At present, there is no large model cloud edge end collaborative system specially for robot scene, which takes into account privacy protection, real-time and intelligent level. The existing technology cannot effectively solve the problem of balancing privacy protection and computing efficiency when the robot uses AI large model, and an innovative technical solution is urgently needed. SUMMARY

[0018] The purpose of the present application is to provide a privacy protection oriented robot large model cloud edge end collaborative reasoning and federated learning system. Through the innovative three-layer architecture design, model splitting strategy, federated learning framework, multi-level privacy protection mechanism and adaptive resource allocation technology, the problem of balancing privacy protection and computing efficiency when the robot uses AI large model is effectively solved, and the problems in the prior art are solved, realizing efficient privacy protection and data isolation while maintaining the intelligent service capability of the large model.

[0019] The technical solution adopted by the present application to achieve the above purpose is: a privacy protection oriented robot large model cloud edge end collaborative reasoning and federated learning system, comprising: a three-layer architecture of a cloud server, an edge computing node and a terminal device connected through a secure communication protocol; and a model splitting strategy is used to split the large model into layers and split it into the cloud server, the edge computing node and the terminal device;

[0020] The cloud server layer is used to deploy the deep network of large-scale pre-training model and execute complex reasoning and global federated learning coordination;

[0021] The edge computing layer is used to deploy the intermediate layer model and execute local data aggregation, privacy protection processing and intermediate feature calculation;

[0022] The terminal device layer is used to deploy the lightweight model and execute data acquisition, preliminary processing and lightweight reasoning;

[0023] The federated learning framework is used to locally train the model, encrypt aggregation, global optimization and individual adaptation to realize model optimization;

[0024] The privacy protection module is used to integrate data localization, differential privacy, homomorphic encryption, secure multi-party computation and blockchain verification mechanism;

[0025] The adaptive allocation module is used to dynamically adjust the computing resources based on task complexity, network status and device load.

[0026] The cloud server layer comprises:

[0027] The global model storage module adopts a distributed storage architecture and is used to store the complete large-scale pre-training model parameters, supports parameter sharding and parallel access, and the storage capacity is expanded to PB level;

[0028] The federal learning coordinator is implemented based on a Parameter Server architecture, in which parameter server nodes are used to store and manage global model parameters, and worker nodes are used to perform computing tasks on local data; wherein the cloud is used as a parameter server node, and the edge device is used as a worker node, and efficient parameter updating and model distribution are achieved through an asynchronous communication mechanism;

[0029] The model update manager manages model iterations using a version control system, and is used for incremental updating and rollback, and detailed performance indicators and change logs are recorded each time;

[0030] The secure computing engine is implemented based on an Intel SGX trusted execution environment, and is used to perform parameter aggregation in an encrypted state, providing hardware-level security protection against privileged user access.

[0031] The API gateway adopts a microservice architecture, provides RESTful and gRPC interfaces, supports TLS1.3 encryption and JWT authentication, and the API call throttling threshold is configurable.

[0032] The edge computing layer is deployed in the user local area network, and includes:

[0033] The intermediate layer model storage module is used to store and execute intermediate layer computing of large models, and uses quantization compression technology to reduce model size, and the typical configuration is 4-8GB model parameters.

[0034] The local data processing engine is implemented based on Apache Flink for stream data processing, and is used for window operation and complex event processing, with a data processing delay of <50ms.

[0035] The privacy protection preprocessor is used to implement data desensitization, identifier removal and differential privacy addition, and custom privacy rules and sensitive data identification.

[0036] The local cache manager adopts a hierarchical caching mechanism, and is used for model parameter and intermediate result caching, with a total capacity of 16-64GB, and the cache update strategy is based on access frequency and time decay.

[0037] The secure communication module is used to implement encrypted communication with the cloud and the terminal side.

[0038] The terminal device layer is the robot body, and includes:

[0039] The multi-modal perception module includes an RGB camera, a stereo depth camera, a microphone array, a touch sensor, and a laser radar, and is used to collect user instructions and environmental data.

[0040] A local storage module is configured to store user data and privacy information by using an encrypted file system, and uses an AES-256 encryption algorithm, a key of which is protected by a hardware security module, and supports secure erasing and data hierarchical storage.

[0041] An interactive interface is configured to provide a multi-modal interactive mode of voice, vision and touch.

[0042] An execution control unit is implemented based on ROS2, adopts a hierarchical control architecture, is configured to perform task planning, path planning and bottom-layer control, has a control cycle of 1-10 ms, and performs real-time obstacle avoidance and adaptive motion control.

[0043] The implementation of the federated learning framework comprises:

[0044] (1) Local training:

[0045] A terminal device collects user interaction data including voice instructions, operation sequences and environmental states;

[0046] An edge node performs local model training by using a mini-batch gradient descent, dynamically adjusts a batch size to be 16-128, and sets a learning rate to be 0.0001-0.001;

[0047] A training process applies differential privacy, adds Gaussian noise σ, sets σ to be 1.2-3.5, and dynamically allocates a privacy budget ε to be 0.1-1.0;

[0048] (2) Encrypted aggregation:

[0049] In an edge computing layer, an edge node uses an improved CKKS homomorphic encryption scheme, wherein a polynomial modulus N is 8192, and a quantization parameter Δ is 2 35 Encrypted gradients;

[0050] A cloud end performs secure aggregation based on a threshold Paillier homomorphic encryption, and sets a key length to be 2048 bits;

[0051] (3) Global optimization:

[0052] A cloud server layer adopts an improved FedAvg algorithm, introduces an adaptive aggregation weight mechanism, combines momentum optimization and gradient clipping, and sets an update frequency to be 1-7 days / once;

[0053] (4) Personalized adaptation:

[0054] An edge node performs local fine-tuning by using a meta-learning framework, and supports 5-shot learning, wherein an inner-loop learning rate is 0.01, and an outer-loop learning rate is 0.001;

[0055] In combination with a knowledge distillation technology, wherein a teacher model is a global model, a student model is a personalized model, a temperature parameter T is 2.0, and a distillation weight α is 0.5.

[0056] Adopt elastic weight merging to prevent catastrophic forgetting, penalty coefficient λ = 5000.

[0057] The robot large model cooperative reasoning and federated learning method comprises the following steps:

[0058] S1: A model splitting strategy is adopted to split the large-scale pre-training model into end-side model, edge model and cloud model according to functions, and deploy them to cloud servers, edge computing nodes and terminal devices;

[0059] S2: Cooperative reasoning process: collect data and extract basic features on the end side; encrypt intermediate features on the end side and transmit them to the edge layer; the edge layer processes the encrypted features, and the local generates the results of simple tasks, and the edge layer forwards the encrypted features to the cloud for complex tasks; the cloud performs deep reasoning, and the results are returned to the edge layer; the edge layer synthesizes the results and transmits them back to the end side for execution;

[0060] S3: The federated learning framework sequentially performs local training, encrypted aggregation, global optimization and personalized adaptation of the model to realize model optimization;

[0061] S4: The privacy protection module processes data according to sensitivity, and high-sensitive data does not leave the terminal; the transmission data is homomorphic encrypted throughout the process, and differential privacy noise is added in the training process;

[0062] S5: The adaptive allocation module monitors the network, load and power in real time, dynamically migrates tasks to the optimal level, and realizes dynamic allocation of resources.

[0063] In step S1, the model splitting strategy is specifically:

[0064] The end-side model deployed on the terminal device layer is used for preliminary processing of user instructions and environmental data; it contains an input processing layer and the first N layers of neural network, and the model parameter amount is ≤ 150 MB and the calculation complexity is ≤ 5 GFLOPs;

[0065] The edge model deployed on the edge computing layer is used for intermediate feature processing and preliminary decision generation; it contains intermediate M layers of neural network, wherein M ≥ 3, the model parameter amount is 2-5 GB and the calculation complexity is 50-200 GFLOPs;

[0066] The cloud model deployed on the cloud server layer contains the remaining deep network and the output layer, and the model parameter amount is ≥ 20 GB and the calculation complexity is ≥ 500 GFLOPs;

[0067] Among them, the three-layer model exchanges data through a secure communication protocol and adopts an encrypted transmission mechanism to ensure the security of intermediate features and gradient information. That is, lightweight homomorphic encryption is used for feature transmission from the end-side model to the edge, and fully homomorphic encryption is used for feature transmission from the edge model to the cloud model to ensure the privacy and security of intermediate features during transmission and processing.

[0068] In step S3, personalized adaptation is specifically as follows:

[0069] Model layering: The first 8 layers of Transformers are shared base layers for global optimization; the last 4 layers of Transformers are personalized top layers for local updates.

[0070] Meta-learning for fast adaptation: the inner loop fine-tunes local data, while the outer loop optimizes global initialization parameters;

[0071] Knowledge distillation: The global model guides the personalized model, and the loss function is: hard label loss + soft label loss;

[0072] Continuous learning: By constraining the important parameter λ to 5000 through EWC and combining it with an experience replay buffer pool of 500-1000 samples, continuous learning is performed.

[0073] The step S4 is specifically as follows:

[0074] a. Data localization:

[0075] The data classification engine automatically identifies sensitivity, setting high sensitivity to 0.8-1.0, medium sensitivity to 0.4-0.8, and low sensitivity to 0-0.4. Highly sensitive data, including biometrics and personal photos, is processed only on the client side, while medium-sensitive data, including lifestyle data, is processed only at the edge layer. Low-sensitivity data is desensitized and uploaded to the cloud.

[0076] b. Differential Privacy:

[0077] Gaussian noise is added to the gradient; the privacy budget ε is allocated according to the task sensitivity: ε = 0.1-0.3 for highly sensitive tasks, ε = 0.3-0.6 for medium sensitive tasks, and ε = 0.6-1.0 for low sensitive tasks; an RDP accounting mechanism is implemented to track the accumulated privacy loss;

[0078] c. Homomorphic encryption:

[0079] Lightweight homomorphic encryption is used from the device to the edge, with encryption overhead less than 50ms and a data expansion factor less than 3; fully homomorphic encryption is used from the edge to the cloud.

[0080] d. Secure multi-party computation:

[0081] Based on the (2,3)-Shamir secret sharing scheme, the cloud, edge, and third-party verifiers jointly perform security aggregation;

[0082] e. Blockchain verification:

[0083] The private blockchain adopts a PoS consensus mechanism, and the block generation time is < 5 seconds, and the transaction throughput is > 1000 TPS; the smart contract executes the access control strategy and the audit process.

[0084] In the step S5, the adaptive computing resource allocation module is implemented, including:

[0085] A task classifier is used to classify tasks according to computing complexity, privacy sensitivity and response time requirements based on a lightweight decision tree, and determine an optimal level;

[0086] The task classifier determines the optimal level, including:

[0087] End-side processing: tasks with high privacy, low complexity and real-time response;

[0088] Edge processing: tasks with medium privacy, medium complexity and near real-time response;

[0089] Cloud processing: tasks with low privacy, high complexity and non-real-time response;

[0090] Hybrid processing: the task splitting and allocation scheme is determined according to specific conditions;

[0091] A network perception module is used to sample network delay and bandwidth every 10 seconds, smooth the data by EWMA, and predict network state for 5-10 minutes by an ARIMA model;

[0092] A load balancer is used to dynamically migrate tasks, and when the end-side load is > 80%, the tasks are migrated to the edge layer; when the edge layer load is > 85%, the non-privacy tasks are migrated to the cloud;

[0093] A cache optimizer is used to pre-cache high-frequency task model parameters by using an LFU-Aging algorithm;

[0094] An energy consumption optimizer is used to establish a comprehensive energy consumption model, activate the power saving mode when the power is < 30%, and realize dynamic frequency adjustment and component hibernation.

[0095] The present application has the following advantages and benefits:

[0096] 1. Significant privacy protection effect: Through data localization processing and multi-level privacy protection mechanism, the risk of privacy leakage is reduced by more than 85%, meeting the requirements of strict data protection regulations such as GDPR. This conclusion is based on the defense effect evaluation of common privacy attack methods (including member inference attack, model inversion attack and attribute inference attack). After adopting differential privacy mechanism (ε = 0.1-1.0), the attack success rate is reduced from 78% to 11.7%, while the model accuracy only decreases by 1.8%;

[0097] 2. Greatly improved computing efficiency: Through three-layer architecture and model splitting, the end-to-end delay is reduced by 62% and the energy consumption is reduced by 45% while maintaining the same inference accuracy (accuracy decrease <2%);

[0098] 3. Strong network adaptability: The adaptive resource allocation mechanism enables the system to adapt to different network conditions, and still maintains basic functions in weak network environment (bandwidth <1Mbps), and retains more than 70% of core capabilities in offline state;

[0099] 4. Continuous optimization of model performance: The federated learning framework realizes the collaborative use of multi-user data, and the model performance improves with the use time, and the task completion rate increases by 25% after using for 3 months, and the user satisfaction increases by 35%;

[0100] 5. Enhanced personalized service: Personalized adaptation technology improves the ability of the robot to meet the needs of specific users, and the personalized task accuracy is 40% higher than that of the general model;

[0101] 6. Comprehensive system security: Multi-level security mechanism provides data lifecycle protection, resisting more than 95% of common privacy attack methods, including member inference attack, model inversion attack, etc. BRIEF DESCRIPTION OF DRAWINGS

[0102] Figure 1 is a schematic diagram of the system architecture of the present application;

[0103] Figure 2 is a schematic diagram of the model splitting of the present application;

[0104] Figure 3 is a flowchart of the federated learning of the present application;

[0105] Figure 4 is a schematic diagram of the privacy protection mechanism of the present application;

[0106] Figure 5 is a flowchart of the adaptive computing resource allocation of the present application. DETAILED DESCRIPTION

[0107] The present application will be further described in detail below in conjunction with the drawings and examples.

[0108] As Figure 1 shown, it is a system architecture diagram of the application, showing the three-layer architecture of cloud server, edge computing node and robot terminal and their connection relationship;

[0109] The technical solution of the application includes a three-layer architecture of cloud server, edge computing node and terminal device, which realizes data privacy protection and model performance optimization through model splitting and collaborative reasoning, federated learning and differential privacy, secure multi-party computation and other technologies.

[0110] 1. System architecture level

[0111] The system architecture of the application includes the following three levels:

[0112] 1.1 Cloud layer: a high computing layer for deploying large-scale pre-trained models, responsible for storing global model parameters, performing complex computing tasks, and coordinating global federated learning processes; the cloud layer uses a distributed computing cluster equipped with high-performance GPU / NPU accelerators, supporting large-scale matrix operations and deep neural network calculations.

[0113] 1.2 Edge layer: deployed on edge computing devices in the local area network, such as gateways or dedicated edge servers, responsible for intermediate layer computing, local data aggregation and privacy protection processing; the edge layer uses processors with medium computing power and special AI acceleration chips, supporting medium-scale neural network calculations and encryption operations.

[0114] 1.3 End-side layer: the robot itself, equipped with basic perception modules and small models, responsible for data acquisition, preliminary processing and part of the lightweight inference tasks; the end-side layer uses low-power processors and special neural network processing units (NPU), supporting basic feature extraction and simple inference tasks.

[0115] The three layers are connected through a secure communication protocol, using encrypted transmission and two-way identity authentication mechanism to ensure data transmission security.

[0116] 2. Model splitting and collaborative reasoning mechanism

[0117] The application uses large model hierarchical splitting technology to divide large-scale pre-trained models into three parts according to function and computing complexity:

[0118] 2.1 End-side model: a lightweight model deployed on the robot, mainly including input processing layer, basic feature extraction layer and part of shallow network, used for preliminary processing of user instructions and environmental data; the end-side model parameter is controlled within 100MB, the computing complexity is not more than 5GFLOPs, suitable for running on low-power processors.

[0119] 2.2 Edge Model: Deployed in the middle layer of edge computing nodes, responsible for intermediate feature processing and preliminary decision generation; the parameter quantity of the edge model is controlled in the range of 2-5GB, the computational complexity is between 50-200GFLOPs, and it is suitable for running on medium-performance computing devices.

[0120] 2.3 Cloud Model: Deployed in the deep network part of the cloud server, responsible for complex inference and decision optimization; the cloud model contains the main parameters of the large model, the parameter quantity can reach dozens to hundreds of GB, the computational complexity is 1000+GFLOPs, and it needs to be run on a high-performance computing cluster.

[0121] The three-layer model exchanges data through a specially designed secure communication protocol, using encryption transmission mechanism to ensure the security of intermediate features and gradient information. Specifically, the feature transmission from the end side to the edge uses lightweight homomorphic encryption (LHE), and the feature transmission from the edge to the cloud uses full homomorphic encryption (FHE), ensuring the privacy and security of intermediate features during transmission and processing.

[0122] 3. Model optimization framework based on federated learning

[0123] The invention designs a federated learning framework for robot scenarios:

[0124] 3.1 Local training: Robots update local models on the end side and edge layer according to daily interaction data; local training uses the mini-batch gradient descent (mini-batch SGD) algorithm, and the batch size is dynamically adjusted according to the device memory (16-128), and the learning rate uses the cosine annealing strategy (initial value 0.001).

[0125] 3.2 Encrypted aggregation: Use a secure aggregation protocol to aggregate model updates from multiple users without exposing raw gradients; encrypted aggregation is based on the threshold Paillier homomorphic encryption scheme, with a key length of 2048 bits, supporting addition and scalar multiplication operations in an encrypted state, and introducing random masks during aggregation to prevent information leakage.

[0126] 3.3 Global optimization: The cloud server updates the global model based on the aggregation result; the global optimization uses an improved version of the FedAvg algorithm, introducing an adaptive aggregation weight mechanism to dynamically adjust the aggregation weight according to the data quality and quantity of the client, while applying momentum optimization and learning rate scheduling strategies.

[0127] 3.4 Personalization: The global model is adapted to the needs of each specific user through transfer learning methods; the personalization uses the meta-learning framework (Model-Agnostic Meta-Learning, MAML), which achieves rapid adaptation with a small amount of local data (5-20 samples), and combines knowledge distillation technology (temperature parameter T = 2.0) to transfer global knowledge to the personalized model, retaining general capabilities while improving performance in specific scenarios.

[0128] 3.5 Continuous learning mechanism: The system implements a continuous learning algorithm with catastrophic forgetting protection, using the Elastic Weight Consolidation (EWC) technology, with an important parameter penalty coefficient λ set to 5000, ensuring that the model retains the ability to learn new knowledge while retaining the ability to learn new knowledge.

[0129] 3.6 Asynchronous federated update: To adapt to the offline use scenario of robots, the system supports an asynchronous federated update mechanism that allows devices to submit accumulated model updates when they reconnect to the network, using a time decay factor (γ = 0.9) to adjust the weight of old updates, ensuring the continuity of model optimization.

[0130] 4. Multi-level privacy protection mechanism

[0131] The present application realizes a multi-level privacy protection mechanism:

[0132] 4.1 Data localization: The original data is strictly retained in the user's local environment (terminal and edge), and is not transmitted to the cloud; the system implements a data classification engine that automatically identifies high-sensitive data (such as biometric features, personal photos, private conversations, etc.) and limits processing on the end side, medium-sensitive data (such as lifestyle habits) is limited to edge layer processing, and only low-sensitive data is allowed to be used for cloud training after desensitization.

[0133] 4.2 Differential privacy: Differential privacy technology is applied in the model training and intermediate feature extraction process, adding carefully designed noise; the system uses the Gaussian mechanism to add calibrated noise, with a dynamic allocation of privacy budget ε (range 0.1-1.0), and the noise standard deviation σ is calculated according to the sensitivity and privacy budget (typical value 1.2-3.5), and the Rényi differential privacy (RDP) accounting mechanism is implemented to accurately track the cumulative privacy loss.

[0134] 4.3 Homomorphic encryption: Homomorphic encryption is applied to intermediate features and model parameters that need to be transmitted, allowing computation in an encrypted state; the system uses an improved CKKS (Cheon-Kim-Kim-Song) scheme to implement efficient semi-homomorphic encryption, with a polynomial modulus degree N = 8192 and a quantization parameter Δ = 2^35, supporting approximate real number operations, and improving the efficiency of encrypted computation through batch processing technology (batchsize = 4096).

[0135] 4.4 Secure Multi-Party Computation: Use secure multi-party computation protocol in federated learning aggregation phase to ensure the security of parameter aggregation process; the system implements a three-party secure computation protocol based on secret sharing, adopts a (2,3)-Shamir secret sharing scheme, supports cloud, edge and third party verifier to jointly perform secure aggregation, and any party cannot obtain the original gradient information.

[0136] 4.5 Blockchain Verification: Introduce lightweight blockchain technology to record model updates and access logs, realize traceable and auditable data usage; the system uses a private blockchain with proof of stake (PoS) consensus mechanism, the block generation time is controlled within 5 seconds, the transaction throughput is >1000TPS, and the access control strategy and audit process are automatically executed through smart contract.

[0137] 5. Adaptive Computing Resource Allocation

[0138] The present application proposes an adaptive computing resource allocation strategy based on task complexity, network status and device load:

[0139] 5.1 Task Classifier: Classify user requests to determine the most suitable processing level; the system uses a lightweight decision tree classifier (depth ≤ 5) to quickly classify based on task type, computing complexity (low / medium / high), privacy sensitivity (low / medium / high) and response time requirement (real-time / near real-time / non-real-time), decision time < 5ms.

[0140] 5.2 Network Perception Module: Real-time monitoring of cloud-edge-end network status, dynamic adjustment of data transmission strategy; the system samples network delay and bandwidth every 10 seconds, uses exponential weighted moving average (EWMA, α = 0.3) to smooth the measurement results, and based on historical data, builds a time series prediction model (ARIMA) to predict network quality changes 5-10 minutes in advance.

[0141] 5.3 Load Balancer: Automatically adjust model splitting ratio based on computing resource usage of each layer; the system monitors CPU / GPU utilization, memory usage and energy consumption, and when the end-side device load exceeds 80%, more computing tasks are automatically migrated to the edge layer, and when the edge layer load exceeds 85%, non-privacy sensitive tasks are migrated to the cloud.

[0142] 5.4 Cache Optimizer: Analyze user usage patterns and pre-cache high-frequency task-related model parameters to edge or end-side; the system uses LFU-Aging cache replacement algorithm combined with time series analysis to predict user behavior patterns, preloads model parameters that may be needed 5-15 minutes in advance, cache hit rate > 85%, significantly reducing response delay.

[0143] 5.5 Energy Optimizer: Evaluate the energy consumption overhead of different processing schemes and select the most energy-efficient scheme under the premise of meeting performance requirements; the system establishes an accurate energy consumption prediction model, considering the comprehensive energy consumption of computing load, network transmission and storage access, and designs a special power-saving mode for battery-powered robots, which can extend the endurance time by 30-50%.

[0144] Embodiment 1: Basic system architecture and components

[0145] As shown in Figure 1 , the system architecture includes three main parts: cloud server cluster, edge computing device and robot.

[0146] The cloud server cluster consists of the following modules:

[0147] (1) Global model storage module: adopts distributed storage architecture, uses HDFS to store complete large-scale pre-training model parameters, supports parameter sharding and parallel access, and the storage capacity can be expanded to PB level;

[0148] (2) Federated learning coordinator: based on Parameter Server architecture, which is a distributed machine learning system architecture composed of parameter server nodes and worker nodes. Parameter server nodes are responsible for storing and managing global model parameters, and worker nodes are responsible for performing computing tasks on local data. In this system, the cloud acts as a parameter server node and the edge device acts as a worker node, achieving efficient parameter updating and model distribution through an asynchronous communication mechanism. This architecture includes three sub-modules: task scheduler, aggregation server and model distributor, supporting both asynchronous and synchronous federated learning modes, with a maximum of 10,000 concurrent clients;

[0149] (3) Model update manager: uses version control system to manage model iterations, supports incremental updates and rollback functions, and records detailed performance indicators and change logs for each update;

[0150] (4) Secure computing engine: based on Intel SGX trusted execution environment, supports parameter aggregation in encrypted state, provides hardware-level security protection, and prevents privileged user access;

[0151] (5) API gateway: adopts microservices architecture, provides RESTful and gRPC interfaces, supports TLS1.3 encryption and JWT authentication, and API call throttling threshold is configurable (default 100 QPS / client).

[0152] The cloud server adopts the Kubernetes container orchestration platform to realize the multi-tenant architecture, ensures the logical isolation of different user data through namespace and network policy, and realizes resource isolation through resource quota and pod security policy. The server is configured as a high-performance GPU cluster and a high-speed network connection.

[0153] The edge computing device is deployed in the user's local area network and mainly includes:

[0154] (1) Intermediate layer model: stores and executes the intermediate layer calculation of the large model, uses quantization compression technology (INT8) to reduce the model size, and the typical configuration is 4-8GB model parameters;

[0155] (2) Local data processing engine: realizes stream data processing based on Apache Flink, supports window operation and complex event processing (CEP), and the data processing delay is <50ms;

[0156] (3) Privacy protection preprocessor: realizes data desensitization, identifier removal and differential privacy addition, supports custom privacy rules and sensitive data identification (based on regular expressions and NER model);

[0157] (4) Local cache manager: adopts hierarchical cache strategy (L1: RAM, L2: SSD), supports model parameter and intermediate result caching, total capacity 16-64GB, cache update strategy based on access frequency and time decay;

[0158] (5) Secure communication module: realizes encrypted communication with the cloud and the end side, adopts TLS1.3 protocol, supports certificate rotation and key negotiation, communication encryption strength AES-256-GCM.

[0159] The edge device hardware is configured as a medium-performance computing platform, and the typical configuration includes multi-core CPU and dedicated AI accelerator (such as Google Edge TPU or NVIDIA Jetson). The device can be a dedicated edge server, or an upgraded smart router or home smart hub.

[0160] The robot as an end-side device includes:

[0161] (1) Multi-modal perception module: including high-definition RGB camera (1080p@30fps), stereo depth camera, microphone array (6-8 microphones, beamforming), touch sensor, laser radar (optional), etc., collecting user instructions and environmental data;

[0162] (2) Lightweight inference engine: implemented based on TensorFlow Lite or ONNX Runtime, supports INT8 quantization model, optimized for ARM / RISC-V architecture, inference delay <20ms;

[0163] (3) Local storage module: uses encrypted file system to store user data and privacy information, uses AES-256 encryption algorithm, key is protected by hardware security module (HSM), supports secure erasure and data hierarchical storage, total capacity 64-128GB;

[0164] (4) Interactive interface: provides multi-modal interaction methods such as voice, vision, and touch, voice recognition supports offline mode (vocabulary <5000) and online mode (infinite vocabulary), display screen uses 7-10 inch touch screen, resolution 1920x1080;

[0165] (5) Execution control unit: based on ROS2 (Robot Operating System 2), uses a layered control architecture, including task planning, path planning, and bottom layer control, control period 1-10ms, supports real-time obstacle avoidance and adaptive motion control.

[0166] The robot hardware platform uses a low-power high-efficiency processor, and the typical configuration includes a quad-core ARM Cortex-A76 processor (2.0-2.5GHz), 4GB LPDDR4 RAM, and a dedicated NPU acceleration unit.

[0167] Example 2: Model splitting and collaborative inference

[0168] As shown in Figure 2 , it is shown in detail how the large model is divided into end-side model, edge model and cloud model according to the calculation complexity and function, including the specific structure, parameter size and calculation complexity of each layer model, the present application divides the large model into three parts according to the calculation complexity and function.

[0169] Model splitting strategy:

[0170] The model splitting strategy of the present application is applicable to various large model architectures, including but not limited to Transformer, BERT, CNN, RNN, GNN, etc. The following takes the Transformer architecture as an example for detailed description, and other architectures can be split according to similar principles:

[0171] (1) End-side model: including input embedding layer, position encoding layer and first 1-2 layers of Transformer encoder; in specific implementation, for 12-layer Transformer model, the first layer is deployed on the end side, with a parameter amount of about 100 MB and a calculation complexity of about 3 GFLOPs; for 24-layer model, the first 2 layers are deployed on the end side, with a parameter amount of about 150 MB and a calculation complexity of about 5 GFLOPs;

[0172] (2) Edge model: including middle 3-8 layers of Transformer encoder / decoder; in specific implementation, for 12-layer model, the second 6 layers are deployed on the edge side, with a parameter amount of about 3 GB and a calculation complexity of about 100 GFLOPs; for 24-layer model, the third 10 layers are deployed on the edge side, with a parameter amount of about 5 GB and a calculation complexity of about 180 GFLOPs;

[0173] (3) Cloud model: including deep 9-12+ layers of Transformer encoder / decoder and output layer; in specific implementation, for 12-layer model, the seventh 12 layers and output layer are deployed on the cloud side, with a parameter amount of about 20 GB and a calculation complexity of about 500 GFLOPs; for 24-layer model, the eleventh 24 layers and output layer are deployed on the cloud side, with a parameter amount of about 70 GB and a calculation complexity of about 1500 GFLOPs.

[0174] The selection of model splitting points is based on the following principles:

[0175] (1) Minimizing the transmission amount of intermediate feature data: by analyzing the activation value size of each layer of the Transformer, the layer with smaller activation value dimension is selected as the splitting point, and the typical splitting point position is after the self-attention module and before the feedforward network, at which time the feature dimension is the smallest (hidden dimension x sequence length);

[0176] (2) Balancing the calculation load of each layer: according to the calculation complexity (FLOPs) and parameter amount of each layer, the calculation task is reasonably distributed to ensure that the end-side load does not exceed the device capability (usually <5 GFLOPs) and the edge layer load is moderate (50-200 GFLOPs);

[0177] (3) Ensuring that privacy-sensitive information is processed locally: ensuring that the input layer and preliminary feature extraction are completed on the end side, preventing raw data from leaving the local environment, especially for multi-modal input such as voice and image, preliminary dimension reduction and feature extraction are completed on the end side.

[0178] Collaborative inference process:

[0179] S1 input processing: user instructions (such as voice, image) are preliminarily processed on the terminal side to extract basic features; for example, for voice input, voice activity detection (VAD), feature extraction (MFCC or Mel spectrum graph) and preliminary encoding are completed on the terminal side; for image input, preprocessing, normalization and preliminary feature extraction are completed on the terminal side;

[0180] (1) Encryption of intermediate features: homomorphic encryption of intermediate features on the terminal side, using a lightweight homomorphic encryption scheme (LHE), encryption overhead < 50ms, data expansion coefficient < 3 after encryption, then sent to the edge layer;

[0181] (2) Edge inference: the edge layer processes the encrypted features, supports partial homomorphic operations (addition and limited multiplication), and generates new intermediate features; for simple tasks that do not require cloud processing (such as basic instruction recognition), the edge layer directly generates the results;

[0182] (3) Cloud deep inference: for complex tasks (such as open domain dialogue, complex scene understanding), the edge layer sends the processed features to the cloud for deep inference; the cloud uses full homomorphic encryption (FHE) to protect data and supports complete matrix operations in an encrypted state;

[0183] (4) Result synthesis: the inference results are returned from the cloud to the edge layer and combined with the local processing results; the system uses a confidence weighted fusion algorithm to assign weights (0-1) according to the confidence of each layer's results to generate the final decision;

[0184] (5) Execution control: the final instruction is returned to the terminal side to control the robot to perform the corresponding operation; safety checks are performed before execution to ensure that the instructions meet the predefined safety rules and operation range.

[0185] In particular, for privacy-sensitive tasks (such as personal photo analysis, health data processing), the system automatically adjusts to perform only on the terminal side and the edge layer, without involving the cloud, ensuring that the original data does not leave the local environment. The system implements a task sensitivity automatic evaluation mechanism, which divides the task sensitivity into high (0.8-1.0), medium (0.4-0.8) and low (0-0.4) levels based on content type, user settings and context information. Tasks with a sensitivity greater than 0.8 are strictly limited to local processing.

[0186] Embodiment 3: Model optimization based on federated learning

[0187] As shown in Figure 3 , the complete process from local training, encrypted aggregation to global optimization and personalized adaptation is shown, including data flow, parameter update and privacy protection measures, and the present application designs a specific federated learning framework for the robot scene. The federated learning process in this embodiment is as follows:

[0188] (1) Task Definition: Cloud defines global optimization objectives and training tasks; System supports multiple task types, including supervised learning (classification, regression), self-supervised learning (masked language modeling, contrastive learning), and reinforcement learning (policy optimization); Each task definition includes objective function, evaluation metrics, and privacy budget;

[0189] (2) Model Distribution: Cloud distributes the current global model to each participating user's edge device; Model distribution uses incremental update mechanism, only transmitting changed parameters, typical update size is 5-15% of the full model, P2P network is used to accelerate distribution, supports resume;

[0190] (3) Local Training:

[0191] Robots collect user interaction data, including voice commands, operation sequences, environment status, etc.

[0192] Edge devices use local data to train models, using small batch gradient descent algorithm, batch size 16-128, learning rate 0.0001-0.001, training rounds 1-5 rounds;

[0193] Apply differential privacy technology to protect the training process, add calibrated noise to the gradient, noise level dynamically adjusted according to privacy budget ε (0.1-1.0);

[0194] (4) Secure Aggregation:

[0195] Edge devices calculate model updates (gradients or parameter differences);

[0196] Apply homomorphic encryption technology to encrypt gradients, use improved CKKS scheme, support approximate real number operations;

[0197] Cloud server performs secure aggregation, uses SecAgg protocol, supports offline fault tolerance (tolerates up to 30% client offline);

[0198] (5) Global Update: Cloud updates global model according to aggregation results; Update uses momentum SGD algorithm, momentum coefficient 0.9, weight decay 5e-4, and applies gradient clipping (threshold 10.0) to prevent abnormal updates; Global update frequency is adjusted according to task type and data change speed, typical value is once every 1-7 days;

[0199] (6) Personalized Adaptation: Updated global model is distributed to each user again, each edge device performs local fine-tuning to adapt to specific user needs; Fine-tuning uses low learning rate (1e-5 to 1e-4) and early stopping strategy (patience = 3) to prevent overfitting; For new users, use cold start strategy, select the most similar user group's model as the initial model based on user basic attributes.

[0200] Personalized learning mechanism: To solve the problem of user scenario diversity, the embodiment of the application proposes a personalized federated learning mechanism:

[0201] a. Hierarchical model structure: The model is divided into a shared base layer and a personalized top layer; the shared layer contains general knowledge and feature extraction capability, which is optimized in the global range; the personalized layer is customized for a specific user and only updated locally; for a 12-layer Transformer model, the first 8 layers are usually set as the shared layer and the last 4 layers are set as the personalized layer;

[0202] b. Meta-learning framework: Use model-independent meta-learning algorithm (MAML) to improve the model's rapid adaptation ability; the meta-learning process includes inner loop (local adaptation) and outer loop (global optimization), with inner loop learning rate 0.01 and outer loop learning rate 0.001, supporting 5-shot learning (only 5 samples are needed to achieve effective adaptation);

[0203] c. Knowledge distillation: Use knowledge distillation technology to transfer global knowledge to personalized models; in the distillation process, the global model is used as the teacher model and the personalized model is used as the student model, the temperature parameter T is in the range of 1.5-3.0, preferably 2.0, and the distillation weight α = 0.5, considering both hard label loss and soft label loss;

[0204] d. Continuous learning strategy: Design a continuous learning algorithm with a forgetting prevention mechanism to balance new knowledge learning and old knowledge retention; use elastic weight consolidation (EWC) technology to add regularization constraints to important parameters, with importance coefficient λ = 5000, and the importance matrix is estimated by Fisher information matrix; At the same time, the experience replay mechanism is realized, and a sample buffer pool with a size of 500-1000 is maintained, and old samples are mixed in the new task training (mixing ratio 0.3).

[0205] Embodiment 4: Multi-level privacy protection mechanism

[0206] As shown in Figure 4 , the implementation and collaborative workflow of multi-level privacy protection technology are shown, including data classification, differential privacy, homomorphic encryption, secure multi-party computation and blockchain verification, and the application realizes a multi-level privacy protection mechanism.

[0207] Data classification and processing strategy:

[0208] The system classifies data into four categories according to the degree of privacy sensitivity:

[0209] (1) Highly sensitive data: such as biometric features (face, fingerprint, iris), personal photos, health data, sensitive conversations (related to health, finance, politics, etc.), family safety information, etc., with a sensitivity score of 0.8-1.0, strictly local processing, not allowed to leave the device;

[0210] (2) Moderate sensitive data: such as habits, daily activity records, location trajectory, social interaction, etc. The sensitivity score is 0.4-0.8. It can be processed in the edge layer, but not uploaded to the cloud. The aggregated features after processing can be used for local model optimization after applying differential privacy;

[0211] (3) Low sensitive data: such as environmental information, non-personalized instructions, device status, etc. The sensitivity score is 0.1-0.4. It can be used for cloud training after encryption, but strong privacy protection measures are required;

[0212] (4) Non-sensitive data: such as public scene interaction, general instructions, public environmental information, etc. The sensitivity score is 0-0.1. It can be used for global model optimization, but basic privacy protection is still required.

[0213] The system realizes an automatic data classification engine, which identifies data categories based on content analysis, context understanding and predefined rules, with a classification accuracy of >95%. The classification process is executed on the edge side to ensure that the classification decision itself does not leak privacy.

[0214] a. Technical support measures include:

[0215] Differential privacy implementation:

[0216] Add calibration noise in the gradient calculation process, use Gaussian mechanism N(0,σ 2 ), and calculate the noise standard deviation σ according to the sensitivity S and privacy budget ε: Typical parameter values are S = 1.0, δ = 10 -5 , ε = 0.1-1.0;

[0217] Adaptive adjustment of privacy budget ε, dynamic allocation according to task sensitivity, high sensitive task ε = 0.1-0.3, medium sensitive task ε = 0.3-0.6, low sensitive task ε = 0.6-1.0;

[0218] Implement RDP (Rényi Differential Privacy) accounting mechanism, accurately track privacy loss, support RDP calculation with multiple α values (1+10 -k , k ∈ {0, 1, 2, 3, 4}), and convert RDP to (ε, δ)-DP guarantee through conversion theorem, to ensure that the cumulative privacy loss does not exceed the preset threshold;

[0219] Implement adaptive clipping mechanism, dynamically determine gradient clipping threshold C, initial value set to 1.0, automatically adjust according to gradient distribution statistics (median and interquartile range) every round, range 0.1-10.0;

[0220] Specific differential privacy mechanisms are designed for different data modalities, such as word-level perturbation for text data, feature-level perturbation for image data, and time series perturbation for sensor data.

[0221] b. Homomorphic encryption framework:

[0222] Semi-homomorphic encryption is implemented using the improved CKKS scheme, supporting addition and limited multiplication operations;

[0223] Polynomial modulus degree N = 8192, security level 128 bits, quantization parameter Δ = 2 35 , supporting 15-bit significant figure precision;

[0224] Optimize the key management mechanism, adopt a hierarchical key structure, including Master Key, Session Key and Operation Key, with key rotation periods of 1 year, 1 week and 1 day respectively;

[0225] Batch processing algorithm designed for federated learning, packing multiple scalars into a single polynomial, batch size 4096, improving encryption calculation efficiency by 10-20 times;

[0226] Implement homomorphic operation acceleration technology, including pre-computation, parallel processing and hardware acceleration (AVX2 / AVX512 instruction set), which can improve the speed of typical homomorphic matrix operations by 5-8 times.

[0227] c. Secure multi-party computation protocol:

[0228] Implement three-party secure computation based on secret sharing, using (2,3)-Shamir secret sharing scheme;

[0229] Cloud, edge and third-party verifier jointly perform secure aggregation, and any party cannot obtain the original gradient information;

[0230] The protocol includes initialization phase, sharing phase, calculation phase and reconstruction phase, supporting basic operations such as addition, multiplication and comparison;

[0231] Implement communication optimization techniques, including batch processing communication, compressed transmission and pre-computation, reducing communication overhead by 60-75%;

[0232] Support for offline fault tolerance, when at most one party is offline, the calculation can still be completed, and Beaver triple technology is used to implement multiplication operations.

[0233] c. Blockchain verification mechanism:

[0234] Lightweight blockchain record model access and update operations, using Proof of Stake (PoS) consensus mechanism;

[0235] Block structure includes block header (timestamp, previous block hash, Merkle root) and transaction body (operation type, participant, data digest, signature);

[0236] Smart contract automatically executes access control policy and verifies operation legality based on predefined rules;

[0237] Distributed ledger enables operation auditability and traceability, supporting efficient query and verification;

[0238] Optimized light client implementation for IoT devices, storage requirement <100MB, computing requirement <5% CPU, suitable for running on resource-constrained devices.

[0239] d. Data lifecycle management:

[0240] Implement data automatic expiration mechanism, set different retention periods according to data type, high-sensitive data 7-30 days, medium-sensitive data 30-90 days, low-sensitive data 90-365 days;

[0241] Secure data deletion technology, using multiple overwriting and physical erasing combination, meeting the U.S. Department of Defense DoD5220.22-M standard;

[0242] Data usage tracking system, records time, subject, purpose and scope of each data access, supports user query and audit;

[0243] Data minimization principle implementation, automatically identifies and collects only necessary data, regularly performs redundant data cleaning.

[0244] Embodiment 5: Adaptive computing resource allocation

[0245] As shown in Figure 5 , it shows how the system dynamically adjusts computing resource allocation according to task complexity, network status and device load, including decision-making process and optimization strategy, the present application proposes an adaptive computing resource allocation mechanism.

[0246] Task classification and distribution, the system classifies tasks according to the following factors:

[0247] (1) Computing complexity: according to the FLOPs and memory requirements required for model inference, tasks are classified into three levels: high (>100GFLOPs), medium (10-100GFLOPs), and low (<10GFLOPs);

[0248] (2) Privacy sensitivity: according to the privacy level of the processed data and the nature of the task, the task is divided into three levels: high (0.8-1.0), medium (0.4-0.8), and low (0-0.4);

[0249] (3) Response time requirement: According to user interaction characteristics and task urgency, tasks are classified into three levels: real-time (<100ms), near real-time (100-500ms), and non-real-time (>500ms).

[0250] Based on the above classification, the system automatically determines the optimal processing location for the task:

[0251] End-side processing: tasks with high privacy, low complexity, and real-time response, such as basic voice command recognition, simple gesture recognition, and expression recognition.

[0252] Edge processing: tasks with medium privacy, medium complexity, and near real-time response, such as complex instruction understanding, environment perception, and simple dialogue generation.

[0253] Cloud processing: tasks with low privacy, high complexity, and non-real-time response, such as large-scale knowledge retrieval, complex scene understanding, and long text generation.

[0254] Mixed processing: depending on specific circumstances, tasks are split and allocated, such as intent recognition completed on the edge and complex response generation completed on the cloud.

[0255] The system implements a fast task classifier based on a decision tree with a depth of ≤5, features including task type, input data type, computation demand estimation, privacy label, and response time requirement, with a classification accuracy of >92% and a decision time of <5ms.

[0256] Dynamic resource scheduling

[0257] The system implements dynamic resource scheduling based on the following factors:

[0258] 1) Network status perception:

[0259] Real-time monitoring of cloud-edge-end network latency and bandwidth, with a sampling period of 10 seconds.

[0260] Using an exponential weighted moving average (EWMA) to smooth the measurement results, α = 0.3.

[0261] Based on historical data, an ARIMA(2,1,2) time series prediction model is constructed to predict network quality changes 5-10 minutes in advance.

[0262] Network status is classified into three levels: good (latency <50ms, bandwidth >10Mbps), general (latency 50-200ms, bandwidth 1-10Mbps), and poor (latency >200ms, bandwidth <1Mbps).

[0263] When the network status changes, automatically adjust the task allocation strategy, such as migrating more tasks to local processing when the network status is poor.

[0264] 2) Load Balancing:

[0265] Monitor each layer's computing resource usage, including CPU / GPU utilization, memory usage, storage I / O, and network bandwidth;

[0266] Set resource usage thresholds, 80% for edge, 85% for edge layer, and 90% for cloud;

[0267] Trigger load adjustment when resource usage exceeds thresholds, migrate tasks to lower load layers;

[0268] Implement task queue priority management, calculate priority scores based on task urgency (1-10) and importance (1-10), and schedule high-priority tasks first;

[0269] Support task preemption mechanism, allow high-priority tasks to preempt low-priority tasks' resources, and automatically migrate or delay execution of preempted tasks.

[0270] 3) Energy Optimization:

[0271] Establish an accurate energy consumption prediction model, considering the comprehensive energy consumption of computing load (CPU / GPU power consumption), network transmission (wireless communication power consumption), and storage access (memory / flash power consumption);

[0272] Assign energy consumption scores (1-100) to different processing schemes, and select the scheme with the lowest energy consumption score under the premise of meeting performance requirements;

[0273] Design special power saving modes for battery-powered robots, including low-power processor core selection, dynamic frequency adjustment (DVFS), and selective component hibernation;

[0274] Implement adaptive strategies based on battery power, automatically activate power saving mode when power <30%, and only retain core functions when power <15%.

[0275] 4) Predictive Caching:

[0276] Analyze user usage patterns and time regularities, and build user behavior prediction models;

[0277] Identify high-frequency tasks and usage scenarios, such as morning weather queries and evening entertainment controls;

[0278] Pre-cache high-frequency task-related model parameters to the edge or end side, with cache warm-up time usually within 5-15 minutes before expected use;

[0279] Implement intelligent preloading based on contextual clues (such as user location, time, and schedule) to predict possible next operations;

[0280] The cache replacement policy adopts the LFU-Aging algorithm, considering access frequency and time decay factor (γ = 0.95), to prevent long-term unused high-frequency items from occupying the cache.

[0281] 5) Fault recovery mechanism:

[0282] Implement multi-level fault detection, including heartbeat detection (period 5 seconds), health state monitoring and performance anomaly detection;

[0283] Design fault recovery strategies, including automatic restart, service degradation and backup switching;

[0284] For cloud unavailability, automatically switch to edge-end collaborative mode, retaining 70% core functions;

[0285] For edge layer failure, the end side can directly communicate with the cloud or run completely locally (retaining 40% basic functions);

[0286] Implement state synchronization and recovery mechanism to ensure service continuity after fault recovery.

[0287] Embodiment 6: Implementation of specific application scenarios

[0288] Smart home service scenario, in the smart home service scenario, the system realizes as follows:

[0289] (1) End-side model focuses on:

[0290] Basic voice wake-up and command recognition (supporting 50-100 offline commands);

[0291] Home member identity recognition (based on voiceprint or simple visual features);

[0292] Basic gesture and motion recognition;

[0293] Environmental state perception (temperature, humidity, illumination, etc.).

[0294] (2) Edge model is responsible for:

[0295] Complex instruction understanding and intent recognition, home scene understanding and state tracking, multi-turn dialogue management, and home activity pattern learning and prediction.

[0296] (3) Cloud model handles:

[0297] Open domain question answering and knowledge retrieval, complex scene planning and decision making, cross-home data analysis and pattern mining (under the premise of privacy protection), global model optimization and update.

[0298] In this scenario, the system places particular emphasis on protecting family privacy data, such as highly sensitive information like family member activities, conversation content, and living habits, which are strictly limited to local processing. Only de-identified aggregated features are used for model optimization.

[0299] In the medical companion service scenario, the system achieves the following:

[0300] a. The end-side model focuses on:

[0301] Patient basic state monitoring (heart rate, respiration, activity, etc.); emergency identification (fall detection, abnormal behavior identification); basic medical instruction understanding; local processing of privacy data (such as medication records, symptom descriptions).

[0302] b. The edge model is responsible for:

[0303] Health status assessment and trend analysis, medical advice generation (based on pre-authorized knowledge base), treatment compliance supervision, and doctor-patient communication assistance.

[0304] c. The cloud model handles:

[0305] Anonymous medical data analysis; complex medical knowledge retrieval; cross-patient pattern recognition (under strict privacy protection); professional medical model updates.

[0306] In this scenario, the system implements a data processing flow that complies with medical privacy regulations such as HIPAA, using stricter differential privacy parameters (ε = 0.1) and stronger encryption protection (key length 4096 bits) to ensure the security of sensitive medical data.

[0307] Example 7: Smart Home Voice Assistant Application Case

[0308] The following is a specific smart home voice assistant application case that demonstrates the complete workflow of the system:

[0309] User scenario: User says to the home service robot "Help me adjust the bedroom temperature to 24 degrees and play my favorite light music"

[0310] S1: Data collection and preliminary processing (end-side):

[0311] The robot collects user voice through a microphone array;

[0312] The end-side model performs voice activity detection and preliminary feature extraction;

[0313] The system automatically classifies the voice command as "moderately sensitive data" (sensitivity score 0.6);

[0314] S2: Voice understanding and intent recognition (edge layer):

[0315] Voice features are transmitted to the edge layer after lightweight homomorphic encryption;

[0316] The edge model performs speech recognition and intent understanding, identifying two intents: "adjust temperature" and "play music";

[0317] The edge layer classifies the "adjust temperature" task as a local task and the "play music" task as a task that requires cloud support;

[0318] S3: Complex reasoning and knowledge retrieval (cloud):

[0319] "I like light music" in the "play music" intent requires access to user historical preferences;

[0320] The edge layer sends the desensitized query request to the cloud;

[0321] The cloud model performs personalized music recommendation calculation;

[0322] S4: Result synthesis and execution (edge layer and end-side):

[0323] The edge layer receives the music recommendation results returned by the cloud;

[0324] The edge layer generates temperature adjustment instructions;

[0325] The two task results are combined and returned to the end-side;

[0326] The robot performs temperature adjustment and music playback operations;

[0327] S5: Model optimization (federated learning):

[0328] After the interaction is completed, the system records user feedback (such as whether the recommended music is satisfactory);

[0329] Within the preset federated learning time window, the edge layer updates the model using local data;

[0330] The updated model parameters are processed by differential privacy and homomorphic encryption and participate in federated learning;

[0331] The cloud aggregates multiple user model updates to generate an improved global model;

[0332] Throughout the process, the original voice data and user preference information are always kept local, and the cloud only receives encrypted features and desensitized queries, effectively protecting user privacy.

[0333] The application realizes efficient privacy protection and data security isolation, while maintaining the intelligent service capability of a large model, and significantly improves the intelligence level of the robot under the premise of protecting user privacy. The system has wide application prospects in intelligent home, medical care, public service and education assistance and other scenes, and provides important support for the safe and compliant development of robot technology.

[0334] Through the implementation of the application, the robot can provide high-quality intelligent services under the premise of strictly protecting user privacy, effectively balancing the relationship between privacy protection, computing efficiency and service quality, and providing a new technical path for the healthy development of artificial intelligence technology in the field of robots.

[0335] Those skilled in the art can understand that the above description is only a preferred embodiment of the application, and the features described in each embodiment of the disclosure and / or the claims can be combined or combined, even if such combination or combination is not explicitly described in the disclosure. It is not intended to limit the application, although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art can modify the technical solutions described in the foregoing embodiments, or make equivalent replacement to part of the technical features, any modification, equivalent replacement, improvement, etc. within the spirit and principles of the application shall be included in the protection scope of the application.

[0336] Although the preferred embodiments of the application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they know the basic creative concept. Therefore, the appended claims are intended to include the preferred embodiments and all changes and modifications falling within the scope of the application. Obviously, those skilled in the art can make various modifications and changes to the application without departing from the spirit and scope of the application. Thus, if these modifications and changes of the application fall within the scope of the claims of the application and their equivalent technologies, the application also intends to include these modifications and changes.

Claims

1. A privacy-preserving large-scale robot model cloud-edge collaborative reasoning and federated learning system, characterized by: include: A three-tier architecture of cloud servers, edge computing nodes, and terminal devices connected via secure communication protocols; A model splitting strategy is used to split the large model into layers, which are then split into cloud servers, edge computing nodes, and terminal devices. The cloud server layer is used to deploy deep networks of large-scale pre-trained models, perform complex reasoning and coordinate global federated learning; The edge computing layer is used to deploy the middle-layer model, perform local data aggregation, privacy protection processing, and intermediate feature calculation; The terminal device layer is used to deploy lightweight models and perform data collection, preliminary processing, and lightweight inference; A federated learning framework for local training, encrypted aggregation, global optimization, and personalized adaptation of models to achieve model optimization; Privacy protection module: used to integrate data localization, differential privacy, homomorphic encryption, secure multi-party computing and blockchain verification mechanism; Adaptive allocation module, used to dynamically adjust computing resources based on task complexity, network status and device load.

2. The privacy-preserving robot large-scale model cloud-edge collaborative reasoning and federated learning system according to claim 1 is characterized in that: The cloud server layer includes: The global model storage module uses a distributed storage architecture to store complete large-scale pre-trained model parameters, supports parameter sharding and parallel access, and has storage capacity expanded to the petabyte level. The federated learning coordinator is implemented based on the Parameter Server architecture. Parameter server nodes are used to store and manage global model parameters, and worker nodes are used to perform computing tasks on local data. The cloud serves as the parameter server node, and edge devices serve as worker nodes. Efficient parameter updates and model distribution are achieved through asynchronous communication mechanisms. Model update manager, which uses a version control system to manage model iterations for incremental updates and rollbacks, and records detailed performance metrics and change logs for each update; A secure computing engine, based on the Intel SGX trusted execution environment, performs parameter aggregation in an encrypted state, provides hardware-level security, and prevents privileged user access. The API gateway adopts a microservice architecture, provides both RESTful and gRPC interfaces, supports TLS1.3 encryption and JWT authentication, and the API call current limit threshold is configurable.

3. The privacy-preserving robot large-scale model cloud-edge collaborative reasoning and federated learning system according to claim 1 is characterized in that: The edge computing layer is deployed in the user's local area network and includes: The intermediate model storage module is used to store and execute intermediate layer calculations of large models and uses quantization compression technology to reduce the model size. The typical configuration is 4-8GB of model parameters. A local data processing engine based on Apache Flink for stream data processing, used for window operations and complex event processing, with data processing latency less than 50ms. Privacy-preserving preprocessor for data desensitization, identifier removal, differential privacy addition, custom privacy rules, and sensitive data identification. Local Cache Manager: This uses a hierarchical caching mechanism for caching model parameters and intermediate results, with a total capacity of 16-64GB. The cache update strategy is based on access frequency and time decay. Secure communication module, used to implement encrypted communication with the cloud and the end side.

4. The privacy-preserving robot large-scale model cloud-edge collaborative reasoning and federated learning system according to claim 1 is characterized in that: The terminal device layer is the robot body, including: Multimodal perception module, including: RGB camera, stereo depth camera, microphone array, touch sensor, lidar, used to collect user commands and environmental data; The local storage module is used to store user data and privacy information using an encrypted file system, using the AES-256 encryption algorithm. The key is protected by a hardware security module and supports secure erasure and hierarchical data storage. Interactive interface, used to provide multimodal interaction methods such as voice, vision, and touch; The execution control unit is implemented based on ROS2 and adopts a hierarchical control architecture for task planning, path planning and low-level control. The control cycle is 1-10ms, and it provides real-time obstacle avoidance and adaptive motion control.

5. The privacy-preserving robot large-scale model cloud-edge collaborative reasoning and federated learning system according to claim 1 is characterized in that: The implementation of the federated learning framework includes: (1) Local training: The terminal device collects user interaction data including voice commands, operation sequences, and environmental status; Edge nodes perform local model training using mini-batch gradient descent, with batch size dynamically adjusted to 16-128 and a learning rate of 0.0001-0.001; Differential privacy is applied during training, adding Gaussian noise σ, σ = 1.2-3.5, and the privacy budget ε is dynamically allocated between 0.1 and 1.0; (2) Encrypted aggregation: In the edge computing layer, edge nodes use the improved CKKS homomorphic encryption scheme, where the polynomial modulus N = 8192 and the quantization parameter Δ = 2 35 , encrypted gradient; The cloud performs secure aggregation based on threshold Paillier homomorphic encryption with a key length of 2048 bits; (3) Global optimization: The cloud server layer uses an improved FedAvg algorithm, introduces an adaptive aggregation weight mechanism, and combines momentum optimization and gradient clipping; the update frequency is 1-7 days / time; (4) Personalized adaptation: Edge nodes use a meta-learning framework for local fine-tuning and support 5-shot learning, where the inner loop learning rate is 0.01 and the outer loop learning rate is 0.001; Combined with knowledge distillation technology, the teacher model is a global model, the student model is a personalized model, the temperature parameter T = 2.0, and the distillation weight α = 0.5; Elastic weight merging is used to prevent catastrophic forgetting, with a penalty coefficient of λ = 5000.

6. A robot large model collaborative reasoning and federated learning method based on the system according to any one of claims 1 to 5, characterized in that: The following steps are involved: S1: Use a model splitting strategy to split large-scale pre-trained models into device-side models, edge models, and cloud models based on their functions; and deploy them to cloud servers, edge computing nodes, and terminal devices. S2: Collaborative inference process: The device collects data and extracts basic features; the device encrypts intermediate features and transmits them to the edge layer; the edge layer processes the encrypted features, generating results locally for simple tasks and forwarding the encrypted features to the cloud for complex tasks; the cloud performs deep inference, and the results are returned to the edge layer; the edge layer synthesizes the results and transmits them back to the device for execution; S3: The federated learning framework sequentially performs local training, encrypted aggregation, global optimization, and personalized adaptation on the model to achieve model optimization. S4: The privacy protection module processes data according to sensitivity, ensuring that highly sensitive data does not leave the terminal. Data transmission is homomorphically encrypted throughout, and differential privacy noise is added during training. S5: The adaptive allocation module monitors the network, load, and power in real time, dynamically migrates tasks to the optimal level, and realizes dynamic allocation of resources.

7. The robot large model collaborative reasoning and federated learning method of the system according to claim 6 is characterized in that: In step S1, the model splitting strategy is specifically as follows: The device-side model deployed at the terminal device layer is used to initially process user commands and environmental data. It includes the input processing layer and the first N neural network layers. The model parameter size is ≤150MB and the computational complexity is ≤5GFLOPs. The edge model deployed at the edge computing layer is used for intermediate feature processing and initial decision generation. It contains an intermediate M-layer neural network, where M ≥ 3, a model parameter size of 2-5 GB, and a computational complexity of 50-200 GFLOPs. The cloud model deployed on the cloud server layer includes the remaining deep network and output layer, with model parameter volume ≥ 20GB and computational complexity ≥ 500GFLOPs; Among them, the three-layer model exchanges data through a secure communication protocol and adopts an encrypted transmission mechanism to ensure the security of intermediate features and gradient information. That is, lightweight homomorphic encryption is used for feature transmission from the end-side model to the edge, and fully homomorphic encryption is used for feature transmission from the edge model to the cloud model to ensure the privacy and security of intermediate features during transmission and processing.

8. The robot large model collaborative reasoning and federated learning method of the system according to claim 6 is characterized in that: In step S3, personalized adaptation is specifically as follows: Model layering: The first 8 layers of Transformers are shared base layers for global optimization; the last 4 layers of Transformers are personalized top layers for local updates. Meta-learning for fast adaptation: the inner loop fine-tunes local data, while the outer loop optimizes global initialization parameters; Knowledge distillation: The global model guides the personalized model, and the loss function is: hard label loss + soft label loss; Continuous learning: By constraining the important parameter λ to 5000 through EWC and combining it with an experience replay buffer pool of 500-1000 samples, continuous learning is performed.

9. The robot large model collaborative reasoning and federated learning method of the system according to claim 6 is characterized in that: The step S4 is specifically as follows: a. Data localization: The data classification engine automatically identifies sensitivity, setting high sensitivity to 0.8-1.0, medium sensitivity to 0.4-0.8, and low sensitivity to 0-0.

4. Highly sensitive data, including biometrics and personal photos, is processed only on the client side, while medium-sensitive data, including lifestyle data, is processed only at the edge layer. Low-sensitivity data is desensitized and uploaded to the cloud; b. Differential Privacy: Gaussian noise is added to the gradient; the privacy budget ε is allocated according to the task sensitivity: ε = 0.1-0.3 for highly sensitive tasks, ε = 0.3-0.6 for medium sensitive tasks, and ε = 0.6-1.0 for low sensitive tasks; an RDP accounting mechanism is implemented to track the accumulated privacy loss; c. Homomorphic encryption: Lightweight homomorphic encryption is used from the device to the edge, with encryption overhead less than 50ms and a data expansion factor less than 3; fully homomorphic encryption is used from the edge to the cloud. d. Secure multi-party computation: Based on the (2,3)-Shamir secret sharing scheme, the cloud, edge, and third-party verifiers jointly perform security aggregation; e. Blockchain Verification: The private blockchain adopts the PoS consensus mechanism, with a block generation time of <5 seconds and a transaction throughput of >1000TPS; the smart contract executes access control policies and audit processes.

10. The robot large model collaborative reasoning and federated learning method of the system according to claim 6 is characterized in that: The step S5 is implemented by an adaptive computing resource allocation module, including: A task classifier, which is used to classify tasks based on computational complexity, privacy sensitivity, and response time requirements based on a lightweight decision tree, and to determine the optimal tier; Among them, the task classifier determines the optimal level, including: On-device processing: tasks requiring high privacy, low complexity, and real-time response; Edge processing: tasks with medium privacy, medium complexity, and near real-time response; Cloud processing: low privacy + high complexity + non-real-time response tasks; Hybrid processing: determine the task splitting and allocation scheme based on specific circumstances; The network perception module samples network latency and bandwidth every 10 seconds, smooths the data using EWMA, and uses the ARIMA model to predict 5-10 minute network status. The load balancer is used to dynamically migrate tasks. When the end-side load exceeds 80%, it is migrated to the edge layer. When the edge layer load exceeds 85%, non-privacy tasks are migrated to the cloud. Cache optimizer, used to pre-cache high-frequency task model parameters using the LFU-Aging algorithm; Energy consumption optimizer is used to establish a comprehensive energy consumption model. When the power is less than 30%, the power saving mode is activated to achieve dynamic frequency adjustment and component hibernation.

Images