Method of input and output end U-shaped segmentation cooperative training architecture

Abstract

A method for an input and output end U-shaped segmentation cooperative training architecture belongs to the field of network models, and comprises the following steps: step 1, in order to realize integration and dynamic segmentation during ONNX operation, performing off-line preprocessing before model deployment, and generating a sub-model library for dynamic scheduling; step 2, performing multi-dimensional real-time acquisition and monitoring on performance data of the end-side equipment; 3, a user submits data at the end side, namely training samples or reasoning input, a model execution engine is started, and when an execution process reaches a preset potential segmentation point, decision callback is triggered; step 4, sub-model loading, trusted authentication and cooperative computing execution: according to a dynamic decision result, the system enters a safe and efficient cooperative computing stage, and meanwhile, a full-link multi-layer privacy protection system is started; and 5, in order to continuously optimize the accuracy of the dynamic segmentation decision, the system establishes a closed-loop feedback mechanism. According to the method, the optimal balance among privacy, efficiency and model effectiveness is realized.

Description

Technical Field

[0001] This invention belongs to the field of network models and relates to a method for a U-shaped segmentation collaborative training architecture at the input and output ends. Background Technology

[0002] With the deep penetration of artificial intelligence technology into fields such as the Internet of Things, mobile Internet, and industrial Internet, deep learning models have become the core engine driving industrial intelligence. However, the exponential growth in model size (such as large language models and multimodal models) has placed unprecedented demands on computing, storage, and communication resources. Against this backdrop, the traditional centralized cloud computing model faces severe challenges, such as data privacy and compliance issues. Sensitive information such as medical images, financial transactions, and personal voice and behavioral data, if directly uploaded to the cloud, faces significant privacy risks and may violate increasingly stringent data protection regulations. In addition, it faces network bandwidth bottlenecks: the sheer volume of raw data (such as images, videos, and time-series signals) generated by massive terminal devices means that continuous, large-scale uploading to the cloud will incur high communication costs and cause network congestion, making it difficult to meet the needs of real-time applications. Deploying all models on terminals is limited by their computing power: although the computing power of terminal devices (such as mobile phones, sensors, and drones) is improving, their computing, storage, and power resources remain limited, making it impossible to independently undertake the complete training task of large-scale deep learning models.

[0003] To address these challenges, academia and industry have proposed various distributed machine learning paradigms, but all of them have inherent flaws and cannot achieve an ideal balance between privacy, efficiency, computing power, and model performance. A classic solution, such as centralized cloud computing, uploads all raw data to a cloud data center for unified model training. As mentioned earlier, this method suffers from serious privacy leaks and security risks. Federated learning: Terminal devices train models locally using their own data, only uploading model updates (such as gradients) to the cloud for aggregation to generate a global model. Its disadvantages include heavy computational and storage burdens, the need for terminals to run the complete model, high resource requirements, and unsuitability for complex models or resource-constrained terminals. Simultaneously, it faces communication efficiency issues: the amount of model parameter updates may still be large, resulting in significant communication overhead. Regarding information privacy, shared gradient information can still be leaked through inversion attacks or membership inference attacks, thus compromising the privacy of the original data.

[0004] Traditional segmentation learning involves dividing a deep learning model at a specific "segmentation point." The terminal handles the computation of the first few layers, sending intermediate activation values ​​to the cloud to complete the computation of the remaining layers, including the final loss calculation. This method faces the problem of output and label exposure. Since the loss calculation is performed in the cloud, the model's output and the label data used for supervised learning typically need to be uploaded to the cloud, creating a new privacy leak point. Label data itself is often highly sensitive information. This method is actually not a fully closed-loop terminal approach; it fails to achieve the privacy standard of "both input and output on the terminal," and cannot meet the needs of scenarios where the output results also need to be kept confidential.

[0005] U-shaped segmentation learning is an evolution of traditional segmentation learning architecture. Its core idea is that the model's input and output layers remain on the terminal device, while only the intermediate layers are computed in the cloud. This architecture theoretically offers the possibility of achieving a completely privacy-preserving closed loop, but current research and practice are still in their early stages and face a series of profound engineering and security challenges. First, there is the bottleneck of communication efficiency: the intermediate activation values ​​and split-point gradients transmitted bidirectionally between the terminal and the cloud may still have a very large data dimension, becoming a major bottleneck restricting training speed and increasing communication costs. There is an urgent need for efficient compression, quantization, and error compensation algorithms to significantly reduce the communication load while ensuring model convergence.

[0006] Second, there is a need for enhanced privacy protection: Although the original data and tags do not leave the terminal, the intermediate activation values ​​may still contain information from the original input, posing a risk of being recovered by reconstruction attacks or model inversion attacks. The existing U-shaped architecture lacks a systematic and provable privacy protection mechanism (such as deep integration with differential privacy, homomorphic encryption, or trusted execution environments).

[0007] Thirdly, system reliability and fault tolerance: The cloud-edge environment is dynamic and unreliable. When cloud-side training instances are interrupted due to resource scheduling or failures, how to ensure seamless checkpoints, state transitions, and rapid recovery of U-shaped training tasks is an engineering problem that has not yet been well solved.

[0008] Fourth, there is a lack of security authentication and trust: How terminal devices verify the trustworthiness of the cloud computing environment and ensure that its intermediate layer computing has not been tampered with is the foundation for establishing trust in end-to-end cloud collaboration. Existing solutions lack integrated, lightweight remote authentication and secure execution environment verification mechanisms.

[0009] Fifth, the balance between personalization and model performance: In scenarios where multiple terminal devices are trained collaboratively, how to design an effective personalized output head synchronization scheme so that each terminal can utilize the powerful representation capabilities of the cloud while obtaining a personalized model that adapts to its own data distribution is the key to improving the practicality of the model. Summary of the Invention

[0010] To overcome the shortcomings of existing U-shaped segmentation learning in terms of communication efficiency, privacy and security, and dynamic adaptation, this invention provides a method for input-output U-shaped segmentation collaborative training architecture, which realizes dynamic and elastic segmentation of the three-segment computing tasks of "end-cloud-end", adjusting the segmentation point in real time according to the device load, rather than fixed segmentation; it constructs a full-link configurable privacy protection system integrating differential privacy, homomorphic encryption, trusted execution environment and two-way verification, and adopts the ONNX architecture to realize standard segmentation and deployment of cross-platform models.

[0011] The technical solution adopted by this invention to solve its technical problem is: A method for an input / output U-shaped segmentation collaborative training architecture includes the following steps: Step 1, Model Preprocessing and Deployment Preparation: To achieve integration and dynamic splitting with the ONNX runtime, offline preprocessing is performed before model deployment to generate a sub-model library that can be dynamically scheduled; Step 2: Real-time multi-dimensional acquisition and monitoring of performance data of end-side devices: The performance monitoring agent collects multiple hardware resource status data in real time on the end-side devices at a preset frequency (e.g., per second); Step 3: Dynamic splitting decision based on device type and real-time load: The user submits data (training samples or inference input) on the device side, the model execution engine starts, and when the execution process reaches the preset potential splitting point, a decision callback is triggered; Step 4, Sub-model Loading, Trusted Authentication and Collaborative Computation Execution: Based on the dynamic decision results, the system enters a safe and efficient collaborative computing phase, while simultaneously enabling a multi-layered privacy protection system across the entire chain to ensure the security of data, computation and training throughout the entire process; Step 5, Closed-loop feedback, strategy optimization and system adaptation: To continuously optimize the accuracy of dynamic segmentation decisions, the system establishes a closed-loop feedback mechanism.

[0012] Step 1 is completed in the development environment or cloud server, without consuming client-side resources, and the process is as follows: Step 1.1, Model Analysis and Labeling: Load the original model to be deployed using the ONNX Python API and traverse its computational graph. Insert custom label nodes at predefined potential split points (e.g., after operator boundaries or operator groups). These nodes are special operators without actual computation, used only to identify graph split boundaries. Step 1.2, Graph Segmentation and Sub-model Generation: Based on the marked nodes, the original model is segmented into two independent sub-models using ONNX serialization and deserialization functions. The edge sub-model contains all operators from the input layer to the current marked node, and the cloud sub-model contains all operators from the current marked node to the output layer. This operation is repeated for all predefined segmentation points to generate a sub-model library containing multiple sets of edge and cloud sub-models. Step 1.3, Metadata Mapping Table Creation: Create a "Splitting Point ID-Sub-Model" mapping table to map the splitting point ID to the corresponding edge and cloud-side sub-model files. This mapping table is pre-deployed to the edge-side dynamic scheduler, which supports it in quickly loading the corresponding sub-model based on the decision results.

[0013] In step 2, the hardware resource status data includes computing resources, storage resources, power consumption and thermal status, and network status. The computing resources include the utilization rate of each CPU core, the utilization rate of the GPU (if present), and the utilization rate of AI accelerators such as NPU / TPU. The storage resources include memory utilization rate, available memory amount, and virtual memory usage. The power consumption and thermal status include the device's current remaining battery power, instantaneous power consumption, battery temperature, and CPU / GPU core temperature. The network status includes the current available network bandwidth, round-trip time (RTT), and connection stability. These raw data constitute the input vector for performance evaluation.

[0014] The process of step 3 is as follows: Step 3.1 Real-time evaluation: The dynamic segmentation decision-maker receives the real-time performance data collected in Step 1, calculates it according to the preset evaluation model (basic weighted model, priority-aware model or intelligent prediction model), and outputs a quantified comprehensive load score S (range 0-100). Step 3.2, Policy Mapping: The decision-maker combines the pre-registered device type identifier D_type (e.g., 1-high-performance terminal, 2-general mobile terminal, 3-resource-constrained terminal) with the calculated load score S and queries the predefined two-dimensional segmentation policy mapping table; Step 3.3, Generate Instructions: The mapping table outputs the specific model splitting point L_split (e.g., "split at the 15th percentile of the total layers"). This instruction clarifies where the end-side sub-model should end and where the cloud-side sub-model should start in this task, thus dynamically forming the "three-segment U-shaped" structure of this collaborative computing.

[0015] The process of step 4 is as follows: Step 4.1, Sub-model loading: The edge-side dynamic scheduler quickly pulls the corresponding edge-side sub-model and cloud-side sub-model from the local cache or from the server according to the split point instruction, and completes the loading and initialization. Step 4.2, Establishment of secure channel and two-way authentication: Establish a secure communication link between the client and the cloud service (or a Trusted Execution Environment (TEE) instance); Two-way hash verification: To defend against man-in-the-middle and replay attacks, the sender calculates and appends the hash value of critical data (such as model parameters and important intermediate results) before transmission; the receiver then recalculates the hash and compares it. The cloud also includes a commitment hash of its calculation process when returning the calculation result for end-user verification. Step 4.3, Multi-layered Privacy Protection Across the Entire Chain: To ensure security throughout the entire process from data input to result output, the system employs a multi-layered privacy protection mechanism, specifically including: a. First layer (data concealment and obfuscation): The terminal side performs homomorphic encryption on the intermediate activation value or adds differential privacy noise to prevent the leakage of the original information; b. Second layer (computation integrity verification): Based on the Trusted Execution Environment (TEE) remote verification mechanism, it ensures that the computation executed in the cloud has not been tampered with; c. Third layer (privacy hardening during training): The gradient is compressed, sparsified, or encrypted during backpropagation to prevent gradient leakage; d. Fourth Layer (Data Integrity and Freshness Verification): This layer uses hash chains and timestamps to prevent replay attacks and man-in-the-middle attacks. The protection mechanisms described above can be flexibly configured according to the security requirements and performance budget of the application scenario. Step 4.4, Forward computation (inference / training), the process is as follows: a. End-side computation: Input data is computed by the loaded end-side sub-model to generate intermediate activation values; b. Privacy processing: According to the security policy, the intermediate activation value is preprocessed on the end side, such as: (1) Adding differential privacy noise: injecting calibrated Gaussian or Laplace noise to provide a provable privacy budget; (2) Homomorphic encryption: using the cloud public key for encryption to generate ciphertext; c. Cloud Computing: The processed data is sent to the cloud. The cloud-based sub-model loads computations on the received data (for encrypted data, computation is performed with homomorphic encryption support), producing the final output or intermediate results required for subsequent gradient calculations; d. Result Return: The cloud returns the calculation result (which may be encrypted) to the client. The client decrypts the result (if necessary) or uses it directly. Step 4.5, Backpropagation (specific to the training task), the process is as follows: a. End-side loss calculation and gradient initiation: Label data is always stored on the end side. The end side calculates the loss and starts backpropagation until the split point. b. Cloud-based gradient computation and privacy hardening: The gradient of the cloud-side part is computed in the cloud, and privacy hardening techniques such as gradient pruning, noise addition (differential privacy) or Top-k sparsity can be applied. c. Secure Gradient Aggregation and Backhaul: Within a Trusted Execution Environment (TEE), gradients from multiple devices can be securely aggregated. The hardened gradients are then securely backhauled to the endpoint. d. Parameter update: The edge integrates the local gradient and the gradient transmitted back from the cloud, updates the edge model parameters, and sends the updated information of the cloud model to the cloud.

[0016] The process of step 5 is as follows: Step 5.1, Data Collection: After the completion of this collaborative task, the system collects key performance indicators, including: local computing time on the device side, data encryption / decryption time, network transmission time, cloud computing time, total task completion time, and changes in device energy consumption. Step 5.2, Feedback Analysis: The dynamic segmentation decision-maker receives this feedback data and compares and analyzes it with the load state predicted during the decision-making process; Step 5.3, Strategy Optimization: Using the analysis results, the system can automatically or manually adjust the entries in the segmentation strategy mapping table, or optimize the parameters (such as weights and thresholds) of the performance evaluation model, so that the system can better adapt to changes in equipment performance and workload patterns during long-term operation.

[0017] The beneficial effects of this invention are mainly reflected in: comprehensive and configurable privacy and security protection from data, computation to training. Users can flexibly enable different levels of protection mechanisms according to the security requirements and performance budget of specific application scenarios, thereby achieving the optimal balance between privacy, efficiency and model utility, and truly realizing the privacy closed loop of "input and output not leaving the end" promised by the U-shaped architecture. Attached Figure Description

[0018] Figure 1 It is a diagram of a U-shaped structure.

[0019] Figure 2 This is a flowchart of the method for a U-shaped segmentation collaborative training architecture at the input and output ends.

[0020] Figure 3 This is a flowchart of the privacy protection encryption process.

[0021] Figure 4 It is a diagram of a multi-layered privacy protection architecture.

[0022] Figure 5 It is a swimlane diagram of data cloaking and obfuscation layers.

[0023] Figure 6It is a swimlane diagram for calculating the integrity verification layer.

[0024] Figure 7 This is a lane diagram with enhanced privacy during the training process.

[0025] Figure 8 It is a swimlane diagram for data integrity and freshness verification layer. Detailed Implementation

[0026] The present invention will now be further described with reference to the accompanying drawings.

[0027] Reference Figures 1-8 A method for a U-shaped input / output segmentation collaborative training architecture, where the U-shaped structure is as follows: Figure 1 As shown, the model to be trained is divided into two parts by two split points. The front and back parts are run on the edge side, and the middle part is run on the cloud side.

[0028] The method includes the following steps: Step 1: Model Preprocessing and Deployment Preparation: To achieve integration and dynamic splitting with the ONNX runtime, offline preprocessing is performed before model deployment to generate a sub-model library that can be dynamically scheduled. This is completed in the development environment or on a cloud server, without consuming end-side resources. The process is as follows: Step 1.1, Model Analysis and Labeling: Load the original model to be deployed using the ONNX Python API and traverse its computational graph. Insert custom label nodes at predefined potential split points (e.g., after operator boundaries or operator groups). These nodes are special operators without actual computation, used only to identify graph split boundaries. Step 1.2, Graph Segmentation and Sub-model Generation: Based on the marked nodes, the original model is segmented into two independent sub-models using ONNX serialization and deserialization functions. The edge sub-model contains all operators from the input layer to the current marked node, and the cloud sub-model contains all operators from the current marked node to the output layer. This operation is repeated for all predefined segmentation points to generate a sub-model library containing multiple sets of edge and cloud sub-models. Step 1.3, Metadata Mapping Table Creation: Create a "Splitting Point ID-Sub-Model" mapping table to map the splitting point ID to the corresponding edge and cloud-side sub-model files. This mapping table is pre-deployed to the edge-side dynamic scheduler, which supports it to quickly load the corresponding sub-model based on the decision results. Step 2: Real-time multi-dimensional acquisition and monitoring of performance data of end-side devices: The performance monitoring agent collects multiple hardware resource status data in real time on the end-side devices at a preset frequency (e.g., per second); Hardware resource status data includes computing resources, storage resources, power consumption and thermal status, and network status. The computing resources include CPU core utilization, GPU utilization (if present), and the utilization rate of AI accelerators such as NPU / TPU. The storage resources include memory utilization, available memory, and virtual memory usage. The power consumption and thermal status include the device's current remaining battery power, instantaneous power consumption, battery temperature, and CPU / GPU core temperature. The network status includes the current available network bandwidth, round-trip time (RTT), and connection stability. This raw data constitutes the input vector for performance evaluation.

[0029] Step 3: Dynamic splitting decision based on device type and real-time load: The user submits data (training samples or inference input) on the device side. The model execution engine starts, and when the execution process reaches the preset potential splitting point, a decision callback is triggered; the process is as follows: Step 3.1 Real-time evaluation: The dynamic segmentation decision-maker receives the real-time performance data collected in Step 1, calculates it according to the preset evaluation model (basic weighted model, priority-aware model or intelligent prediction model), and outputs a quantified comprehensive load score S (range 0-100). Step 3.2, Policy Mapping: The decision-maker combines the pre-registered device type identifier D_type (e.g., 1-high-performance terminal, 2-general mobile terminal, 3-resource-constrained terminal) with the calculated load score S and queries the predefined two-dimensional segmentation policy mapping table; Step 3.3: Generate instructions: The model splitting strategy mapping table outputs the specific model splitting point L_split (e.g., "split at the 15th percentile of the total layers"). This instruction clarifies where the end-side sub-model should end and where the cloud-side sub-model should start in this task, thus dynamically forming the "three-segment U-shaped" structure of this collaborative computing.

[0030] Step 4, Sub-model Loading, Trusted Authentication and Collaborative Computation Execution: Based on the dynamic decision results, the system enters a secure and efficient collaborative computing phase; Step 5, Closed-loop feedback, strategy optimization and system adaptation: To continuously optimize the accuracy of dynamic segmentation decisions, the system establishes a closed-loop feedback mechanism.

[0031] This embodiment describes three core methods for implementing dynamic segmentation decisions, a segmentation point ID-sub-model index mapping table, a model segmentation strategy mapping table, and a multi-layered privacy protection system across the entire chain.

[0032] The basic weighted model in step 3.1 adopts a basic load assessment model based on dynamic weighted moving average, which serves as the basic embodiment of this invention. Its core principle lies in: smoothing historical performance data to eliminate instantaneous fluctuations, and adaptively adjusting the weights of each performance indicator according to the recent trend of system resource pressure, thereby calculating a comprehensive score that better reflects the true load condition of the equipment; the implementation steps are as follows: Step 101, Data Smoothing Process: Let the current time be... The collected raw CPU utilization rate is The original memory utilization rate was Using weighted moving averages for past Data within the period is smoothed, and more recent data is given higher weights. The weight sequence is as follows: ,in Indicates the most recent moment. Indicates the farthest moment, and ; Smoothed CPU load and memory load The calculation formula is as follows: ; ; in, and Each of these is a historical moment The original performance data; Step 102, Dynamic Weight Adjustment: Define the dynamic weights of the CPU. Dynamic weights of memory ,satisfy Initially, both can be set to be equal (e.g., both 0.5). The weights are dynamically adjusted based on the historical pressure trend; for example, the recent average memory load can be compared with a preset threshold. Relationship: like ,but ; in, This is a relatively small adjustment step. This mechanism ensures that when a resource remains scarce, its influence on the overall score will increase accordingly. Step 103, Comprehensive Load Score Calculation: Calculate the time using the smoothed load value and its dynamic weights. Overall load score : ; This score This will be used to query a predefined score-splitting point mapping table to ultimately determine at which layer the model should split.

[0033] The priority-aware model in step 3.1 adopts a priority-aware adaptive load assessment model, which is an enhancement of the basic model. Its core principle is to identify the type or priority of the inference task and dynamically switch the weight calculation strategy according to the different sensitivities of different priority tasks to system resources, thereby achieving a more refined resource assessment that matches the task objectives. The implementation steps are as follows: Step 201, Priority Definition and Policy Mapping: Predefine a set of task priorities. For each priority level, a specific weight strategy is pre-stored in the decision maker; (High priority, such as interactive tasks): Latency-sensitive, weighting strategy biased towards CPU, for example... ; (Medium priority): Balancing strategy, for example ; (Low priority, such as background tasks): may be more sensitive to memory usage, and the weighting strategy is biased towards memory. Step 203, Overall Load Score Calculation: Obtain the current smoothed performance data. and (The calculation method is the same as in 2.1). Then, the weights obtained from the mapping table are used for calculation: ; This score By integrating real-time device status with task-specific requirements, the segmentation decisions made accordingly can better meet the service quality requirements of different tasks.

[0034] The intelligent prediction model in step 3.1 adopts a machine learning-based intelligent load assessment model. This model learns complex load change patterns from historical performance data and predicts future load trends or directly recommends the optimal load splitting point, thereby avoiding performance bottlenecks caused by system response latency. The implementation steps are as follows: Step 301, Model Training (Offline Phase); Feature engineering involves collecting large-scale historical performance data sequences from edge devices, including but not limited to CPU, memory, network bandwidth, and disk I / O. Feature vectors are then constructed. ,in Representative moment Multidimensional performance indicators The size of the time window; Tag generation: Tags The objective can be either a regression objective or a classification objective. The regression objective refers to the overall load score over a future period; the classification objective refers to the optimal segmentation point that minimizes the time consumption or maximizes the efficiency of edge-cloud collaborative inference under the given performance conditions. ; Model selection and training: using the training dataset To train a prediction model Applicable models include gradient boosting decision trees (such as XGBoost), random forests, or long short-term memory networks (LSTM). Step 302, Online Prediction and Decision Making (Online Phase): When the model reaches a potential split point, the performance monitoring agent constructs a feature vector from the current and recent historical performance data. The feature vector is then input into the deployed machine learning model. middle; If it is a regression model, the model outputs a predicted future load score. : ; The decision-maker then uses this predicted score Query the mapping table to determine the split point.

[0035] If it is a classification model, the model directly outputs the recommended split points. : ; The decision-maker can either directly adopt this recommendation or integrate it with rule-based decisions before execution.

[0036] In step 3, based on the dynamic segmentation decision mechanism of the predefined mapping table, the comprehensive load score of the device is calculated. Subsequently, the dynamic segmentation decision-maker needs to transform this score into specific model segmentation points. This embodiment employs a two-layer mapping mechanism based on device type and load score. This mechanism is implemented through a predefined segmentation strategy mapping table, ensuring the efficiency and reliability of the decision-making process.

[0037] Mapping table design principle: Recognizing that terminal devices with different performance levels should have fundamentally different segmentation strategies. For example, a high-performance PC with a load score of 60 may still have far more remaining computing power than an embedded device with a load score of 60. Therefore, a simple global segmentation function is not optimal.

[0038] The segmentation strategy mapping table designed in this invention is a two-dimensional lookup table, with its two key input dimensions being: Device type identifier ( ): Pre-classify edge devices, for example, 1 represents high-performance terminals (such as desktop computers and workstations), 2 represents general mobile terminals (such as smartphones and tablets), and 3 represents resource-constrained terminals; Overall load score ( ): A score between 0 and 100 calculated by the performance monitoring agent; The output of the mapping table is the model split point ( These are typically identified by the model layer number or the name of a specific split point.

[0039] Table 1 is a specific segmentation point ID-sub-model index mapping table of the present invention;

[0040] Table 1 is used in conjunction with the "Segmentation Strategy Decision Mapping Table" (Table 2). The decision table determines the segmentation point identifier based on the device status, and this table locates the specific sub-model file based on the segmentation point identifier, realizing the complete process from decision to loading.

[0041] Table 2 is a specific example of a mapping table in this invention, namely the model segmentation strategy mapping table mentioned above;

[0042] Where m represents the total number of layers in the model.

[0043] The decision-maker's workflow is as follows: Input: Receive device type from the performance monitoring agent and overall load score ; Look up the table: and For the index, query the above splitting strategy mapping table to determine the unique splitting point. ; Output: the split points Send it to the model execution engine.

[0044] To further enhance adaptability, the mapping table supports dynamic updates. During the closed-loop feedback phase, the system records the actual edge inference latency, cloud response time, and total system time after each decision. Through long-term monitoring, if it is found that the decision in a certain score range consistently leads to performance failure, the system can automatically or the administrator can manually fine-tune the split point of that range.

[0045] The mapping table decision scheme in this embodiment has the following significant advantages: High decision efficiency: The time complexity of the table lookup operation is O(n log n). It meets the stringent real-time requirements of model inference. The rules are clear: the strategy is readily apparent, easy to debug, verify, and deploy. It offers high flexibility: by dynamically updating the mapping table, the system strategy can be optimized without modifying the core decision-maker code, adapting to new models or device types. It supports complex logic: by introducing a device type dimension, it implements a more refined and reasonable segmentation strategy than simple piecewise functions, fully considering the performance differences of heterogeneous hardware.

[0046] In step 4, a multi-layered end-to-end privacy protection system is constructed to address the privacy and security challenges mentioned in the background, such as intermediate data leakage and untrusted computation. This system is not a single technology, but rather provides configurable protection strategies based on different security needs and performance overheads, ensuring protection throughout the entire process from data input to result output. The specific architecture diagram is shown below. Figure 4 As shown.

[0047] The first layer, data concealment, obfuscation, and noise injection layer: This layer of protection aims to prevent intermediate activation values ​​from leaking original data information, and provides two technical paths that can be used in combination: Path A (Confusion Encryption): As described in the original text, homomorphic encryption is used to encrypt the intermediate activation value to achieve ciphertext computation.

[0048] Path B (Differential Privacy Noise): As a lighter-weight option or one combined with encryption, random noise (such as Laplace noise) that meets differential privacy requirements is injected after the activation values ​​are output by the end-side sub-model but before transmission. The noise level is calculated based on a preset privacy budget (ε). This method significantly reduces communication and computational overhead at the cost of quantifiable privacy loss and effectively defends against model inversion attacks. See the swimlane diagram below. Figure 5 As shown.

[0049] The second layer, the computational integrity verification layer (for malicious cloud-based actions): This layer ensures that the cloud can correctly execute the agreed-upon model calculations, preventing the model from being tampered with or the calculation process from being maliciously exploited.

[0050] Based on a Trusted Execution Environment (TEE), the intermediate layer of the model to be executed in the cloud is deployed within the TEE. Before collaborative inference begins, the edge devices initiate a remote proof challenge. The cloud-based TEE environment (such as Intel SGX Enclave) generates a hardware-signed "attestation evidence" proving that the code running internally is indeed the expected, tamper-proof model code. During the training phase of this architecture, drawing on the secure aggregation concept of federated learning, the main role of the TEE is not to execute the computation of the entire model, but rather to act as a lightweight, trusted coordinator responsible for aggregating gradients or model updates from multiple edge devices and distributing the new global model.

[0051] The process is as follows: From the edge to the cloud: Sending remote proof challenges; From cloud TEE to edge: Returns a verification report signed by the hardware certificate; End-side: Verify the signature of the verification report and the hash value of the code within it; Only after successful verification will the endpoint send the intermediate data (which may have undergone the first layer of processing) to the TEE instance for computation.

[0052] This establishes a trusted chain from end to cloud, effectively defending against internal attacks and malware originating from cloud infrastructure. A swimlane diagram is shown below. Figure 6 As shown.

[0053] The third layer is the training process privacy reinforcement layer (to prevent backpropagation gradient leakage): During the model training phase, the gradients transmitted back from the cloud to the edge may also contain original data information. This layer reinforces this process.

[0054] Gradient compression and selective backpropagation: Instead of backpropagating all gradients, only the gradients with the largest absolute value are backpropagated. Top-k Sparsification, or quantization of gradients, reduces both communication overhead and the amount of information that might be leaked from the gradient.

[0055] Gradient encryption: The returned gradients are encrypted to ensure their confidentiality during transmission. This encryption can be combined with homomorphic encryption schemes to form an end-to-end encrypted training pipeline.

[0056] Specific lane diagrams are as follows: Figure 7 As shown.

[0057] The fourth layer, the data integrity and freshness verification layer (two-way verification based on hash chains): To prevent data from being tampered with, replayed, or attacked in the middle during transmission, this layer introduces a lightweight hash verification mechanism in key communication links.

[0058] End-to-cloud verification: Before sending important data (such as encrypted intermediate activation values ​​or gradient updates), the endpoint calculates the hash value `H_data = Hash(Data)` and sends `(Data, H_data)` together. Upon receiving the data, the cloud recalculates the hash value `H_data` of the received data and compares it with `H_data`. If they do not match, the data is discarded and a retransmission is requested or an alert is issued.

[0059] Cloud-to-edge verification: Before returning the computation results, the cloud can calculate the commitment hashes of key states in the entire computation process, such as hashing the input data, the cloud-side model version number, and the computation parameters, and return them along with the results. The edge can then use this to verify the integrity and consistency of the cloud computation.

[0060] Replay protection: By introducing timestamps or incrementing sequence numbers into the hash calculation, replay attacks can be effectively defended against.

[0061] Specific lane diagrams are as follows: Figure 8 As shown.

[0062] The privacy protection system in this embodiment provides comprehensive and configurable privacy and security protection from data and computation to training through the organic combination of the three layers mentioned above. Users can flexibly enable different layers of protection mechanisms according to the security requirements and performance budget of specific application scenarios, thereby achieving the optimal balance between privacy, efficiency and model utility, and truly realizing the privacy closed loop of "input and output never leaving the end" promised by the U-shaped architecture.

[0063] The embodiments described in this specification are merely examples of implementations of the inventive concept and are for illustrative purposes only. The scope of protection of this invention should not be considered limited to the specific forms described in these embodiments; rather, it extends to equivalent technical means conceived by those skilled in the art based on the inventive concept.

Claims

1. A method for a U-shaped input / output segmentation collaborative training architecture, characterized in that, The method includes the following steps: Step 1, Model Preprocessing and Deployment Preparation: To achieve integration and dynamic splitting with the ONNX runtime, offline preprocessing is performed before model deployment to generate a sub-model library that can be dynamically scheduled; Step 2: Real-time multi-dimensional acquisition and monitoring of performance data from end-side devices: The performance monitoring agent collects multiple hardware resource status data from the end-side devices in real time at a preset frequency; Step 3: Dynamic segmentation decision based on device type and real-time load: The user submits data on the device side, i.e., training samples or inference input. The model execution engine starts and triggers a decision callback when the execution process reaches the preset potential segmentation point. Step 4, Sub-model Loading, Trusted Authentication and Collaborative Computation Execution: Based on the dynamic decision results, the system enters a safe and efficient collaborative computing phase, while simultaneously enabling a multi-layered privacy protection system across the entire chain to ensure the security of data, computation and training throughout the entire process; Step 5, Closed-loop feedback, strategy optimization and system adaptation: To continuously optimize the accuracy of dynamic segmentation decisions, the system establishes a closed-loop feedback mechanism.

2. The method for an input / output U-shaped segmentation collaborative training architecture as described in claim 1, characterized in that, Step 1 is completed in the development environment or cloud server, without consuming client-side resources, and the process is as follows: Step 1.1, Model Analysis and Labeling: Load the original model to be deployed using the ONNXPythonAPI, traverse its computation graph, and insert custom labeling nodes at predefined potential split points. These nodes are special operators without actual computation and are only used to identify graph split boundaries. Step 1.2, Graph Segmentation and Sub-model Generation: Based on the marked nodes, the original model is segmented into two independent sub-models using ONNX serialization and deserialization functions. The edge sub-model contains all operators from the input layer to the current marked node, and the cloud sub-model contains all operators from the current marked node to the output layer. This operation is repeated for all predefined segmentation points to generate a sub-model library containing multiple sets of edge and cloud sub-models. Step 1.3, Metadata Mapping Table Creation: Create a "Splitting Point ID-Sub-Model" mapping table to map the splitting point ID to the corresponding edge and cloud-side sub-model files. This mapping table is pre-deployed to the edge-side dynamic scheduler, which supports it in quickly loading the corresponding sub-model based on the decision results.

3. The method for an input / output U-shaped segmentation collaborative training architecture as described in claim 1 or 2, characterized in that, In step 2, the hardware resource status data includes computing resources, storage resources, power consumption and thermal status, and network status. The computing resources include the utilization rate of each CPU core, the utilization rate of the GPU, and the utilization rate of the AI ​​accelerator. The storage resources include the memory utilization rate, the amount of available memory, and the virtual memory usage. The power consumption and thermal status include the device's current remaining battery power, instantaneous power consumption, battery temperature, and CPU / GPU core temperature. The network status includes the current available network bandwidth, round-trip latency, and connection stability. These raw data constitute the input vector for performance evaluation.

4. The method for an input / output U-shaped segmentation collaborative training architecture as described in claim 1 or 2, characterized in that, The process of step 3 is as follows: Step 3.1 Real-time evaluation: The dynamic segmentation decision-maker receives the real-time performance data collected in step 1, performs calculations based on a preset evaluation model, which includes a basic weighted model, a priority-aware model, or an intelligent prediction model, and outputs a quantified comprehensive load score S. Step 3.2, Strategy Mapping: The decision-maker combines the pre-registered device type identifier D_type with the calculated comprehensive load score S and queries the predefined two-dimensional segmentation strategy mapping table; Step 3.3: Generate instructions: The mapping table outputs the model splitting point L_split. This instruction clarifies where the end-side sub-model should end and where the cloud-side sub-model should start in this task, thus dynamically forming the "three-segment U-shaped" structure of this collaborative computing.

5. A method for an input / output U-shaped segmentation collaborative training architecture as described in claim 1 or 2, characterized in that, The process of step 4 is as follows: Step 4.1, Sub-model loading: The edge-side dynamic scheduler quickly pulls the corresponding edge-side sub-model and cloud-side sub-model from the local cache or from the server according to the split point instruction, and completes the loading and initialization. Step 4.2, Establishment of secure channel and two-way authentication: Establish a secure communication link between the endpoint and cloud services or Trusted Execution Environment (TEE) instances; Two-way hash verification: To defend against man-in-the-middle attacks and replay attacks, before transmitting critical data, the sender calculates the hash value of the data and attaches it; after receiving it, the receiver recalculates the hash and compares it. When the cloud returns the calculation result, it also attaches the promised hash of its calculation process for the end-side verification. Step 4.3: Multi-layered privacy protection across the entire chain; Step 4.4: Forward computation; Step 4.5, backpropagation.

6. The method for an input / output U-shaped segmentation collaborative training architecture as described in claim 5, characterized in that, The process of step 4.3 is as follows: a. First layer, data concealment and obfuscation: The terminal side performs homomorphic encryption on the intermediate activation value or adds differential privacy noise to prevent the leakage of the original information; b. Second layer, computational integrity verification: a remote proof mechanism based on a trusted execution environment to ensure that computations executed in the cloud have not been tampered with; c. Third layer, privacy hardening during training: compressing, sparsifying or encrypting gradients during backpropagation to prevent gradient leakage; d. Fourth layer, data integrity and freshness verification: prevent replay attacks and man-in-the-middle attacks through hash chain and timestamp mechanism.

7. The method for an input / output U-shaped segmentation collaborative training architecture as described in claim 5, characterized in that, The process of step 4.4 is as follows: a. End-side computation: Input data is computed by the loaded end-side sub-model to generate intermediate activation values; b. Privacy processing: According to the security policy, the intermediate activation value is preprocessed on the end side. The preprocessing process is as follows: (1) Add differential privacy noise: inject calibrated Gaussian or Laplace noise to provide a provable privacy budget; (2) Homomorphic encryption: use the cloud public key to encrypt and generate ciphertext; c. Cloud computing: The processed data is sent to the cloud, where the cloud-side sub-model loads the data to perform calculations and generate the final output or intermediate results required for subsequent gradient calculations. d. Result return: The cloud returns the calculation results to the terminal, which can then decrypt or use them directly.

8. The method for an input / output U-shaped segmentation collaborative training architecture as described in claim 5, characterized in that, The process of step 4.5 is as follows: a. End-side loss calculation and gradient initiation: Label data is always stored on the end side. The end side calculates the loss and starts backpropagation until the split point. b. Cloud-based gradient computation and privacy hardening: The gradient of the cloud-side portion is computed in the cloud, and privacy hardening techniques such as gradient pruning, noise addition, or Top-k sparsity are applied. c. Gradient Security Aggregation and Backhaul: Within the Trusted Execution Environment (TEE), gradients from multiple devices are securely aggregated, and the hardened gradients are securely backhauled to the edge. d. Parameter update: The edge integrates the local gradient and the gradient transmitted back from the cloud, updates the edge model parameters, and sends the updated information of the cloud model to the cloud.

9. A method for an input / output U-shaped segmentation collaborative training architecture as described in claim 1 or 2, characterized in that, The process of step 5 is as follows: Step 5.1, Data Collection: After the completion of this collaborative task, the system collects key performance indicators, including local computing time on the device side, data encryption / decryption time, network transmission time, cloud computing time, total task completion time, and changes in device energy consumption. Step 5.2, Feedback Analysis: The dynamic segmentation decision-maker receives this feedback data and compares and analyzes it with the load state predicted during the decision-making process; Step 5.3, Strategy Optimization: Using the analysis results, automatically or manually adjust the entries in the splitting strategy mapping table, or optimize the parameters of the performance evaluation model, so that the system can better adapt to changes in equipment performance and workload patterns during long-term operation.

Images