A large model cross-domain adaptation cloud edge collaborative learning method facing complex production environment
Patent Information
- Application Number
- CN202611121769.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-27
- Publication Date
- 2026-10-16
AI Technical Summary
（2）数据分布挑战：云侧训练数据（源域）与边缘设备在实际应用场景中遇到的数据（目标域）往往存在分布差异（即“域偏移”）；此外，由于环境变化，单个边缘设备也面临数据分布偏移
Smart Images
Figure CN122927436A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of cloud-edge collaborative learning methods, and more particularly to a cloud-edge collaborative learning method for cross-domain adaptation of large models in complex production environments. Background Technology
[0002] With the deep integration of artificial intelligence and intelligent manufacturing in the industrial sector, the tobacco industry is actively promoting the digital and intelligent transformation of tobacco processing workshops. Tobacco processing is a crucial process that transforms raw tobacco leaves into tobacco shreds that meet specific standards. It involves numerous technological steps and complex equipment linkages, demanding extremely high levels of continuity, stability, and consistency in process parameters. Employing artificial intelligence technology based on computer vision algorithms for intelligent monitoring of each production stage in the tobacco processing workshop is an important method for improving monitoring accuracy, reducing costs, and driving intelligent transformation.
[0003] These systems typically rely on a central intelligent analysis model deployed on a cloud-side server. However, when deploying the aforementioned intelligent monitoring model directly to various edge devices (such as inspection robots, fixed cameras, and sensor gateways) in the workshop, two major challenges arise: (1) Model deployment challenge: Edge devices typically have limited computing power and storage space, making it difficult to directly support and run large cloud-side models. Therefore, the model must be compressed before deployment, with common techniques including knowledge distillation, network pruning, and parameter quantization. (2) Data distribution challenge: There are often distribution differences between cloud-side training data (source domain) and the data encountered by edge devices in actual application scenarios (target domain) (i.e., "domain offset"). In addition, due to environmental changes, individual edge devices also face data distribution offsets. This offset can cause models that perform well on the cloud to experience a significant performance drop after deployment at the edge.
[0004] To address both challenges simultaneously, existing work has explored Cloud-Edge Collaborative Learning (CECL) frameworks. In CECL, large models are typically deployed on cloud servers, while lightweight models are derived from the cloud model and deployed on edge devices. Edge devices upload selected samples or parameter changes to the cloud server to update the cloud model. The updated knowledge is then propagated back from the cloud side to the edge model via parameter updates, resulting in better edge device performance. Some CECL frameworks focus on federated learning, assigning the model and sub-task models to edge devices, while the cloud side updates the global model through parameter fusion. However, these frameworks ignore the domain gaps between edge data. To address these domain gaps, recent CECL frameworks are adaptive during testing, uploading target domain data or domain vectors selected by the edge model to the cloud model. By updating the affine parameters in the batch normalization layer of the cloud model, they learn the feature information of the target domain.
[0005] The existing CECL framework has limitations when dealing with complex and ever-changing scenarios such as yarn processing workshops. It either requires uploading images or video streams that may contain sensitive production data from edge devices to the cloud, posing a risk of data privacy leakage, or it can only perform limited parameter fine-tuning, resulting in limited performance improvement and failing to effectively solve the problem of catastrophic forgetting of existing knowledge when adapting to new grades or new processes. Summary of the Invention
[0006] To address the aforementioned issues, this invention proposes a large-scale cross-domain adaptation cloud-edge collaborative learning method for complex production environments. This method aims to safely and efficiently migrate visual analysis models to specific scenarios within a tobacco processing workshop and support continuous adaptive optimization. By alternately performing passive domain adaptation at the edge and domain knowledge integration at the cloud, this method achieves model compression to adapt to edge computing power while effectively mitigating the catastrophic forgetting problem in multi-domain continuous learning. This enhances the domain generalization capabilities of both cloud and edge models, enabling accurate and robust monitoring of material flow anomalies and ensuring production quality and efficiency.
[0007] A cloud-edge collaborative learning method for cross-domain adaptation of large models in complex production environments includes the following steps:
[0008] S1: Initialization and deployment steps of the silk-making material flow monitoring model based on pruning networks;
[0009] S2: The passive domain adaptation step of the edge detection model based on the neighborhood fraction mask;
[0010] S3: Domain knowledge fusion steps based on gradient masking for large cloud-side models.
[0011] Furthermore, in step S1, the initialization and deployment steps of the filament material flow monitoring model based on the pruning network specifically involve training a pruning network as a cloud-side model using source domain data on a cloud-side server. Each trainable weight parameter in the pruning network is associated with a learnable importance score. Based on the computing resource constraints of the target edge device, structured pruning is performed on the cloud-side model to derive a compact edge model and deploy it to the edge device.
[0012] Furthermore, the initialization and deployment steps of the filament material flow monitoring model based on the pruning network also include: calculating a joint mask based on the maintained domain-specific score library, and sending the joint mask to the corresponding edge device; the joint mask is used to identify the positions of parameters that are crucial to the observed domain in the passive domain adaptation step on the edge side.
[0013] Furthermore, the joint mask is a binary mask obtained by taking the union of all domain-specific masks seen in the domain; wherein each domain-specific mask is obtained by selecting the top layer with the highest importance score from each layer of its corresponding domain importance score vector. The parameter index is generated by %.
[0014] Furthermore, in step S2, the passive domain adaptation step of the edge monitoring model based on the neighborhood score mask is specifically performed on the edge device using locally collected unlabeled target domain data to perform passive domain adaptation on the edge model. During the adaptation process, the gradient mask issued by the cloud server is applied to selectively freeze the gradient of the model parameters, only updating parameters that are not important to the seen neighborhood, and accumulating the processed gradient.
[0015] Furthermore, in the passive domain adaptation step of the edge device, the operation of applying the gradient mask is formally represented as follows:
[0016] (1)
[0017] in, The original gradient obtained from backpropagation. Let ⊙ represent the joint mask, and let ⊙ denote the Hadamard product. This is the gradient after masking.
[0018] Furthermore, in step S3, the domain knowledge fusion of the cloud-side large model based on gradient masking specifically involves receiving accumulated gradients from edge devices on the cloud-side server; filling the accumulated gradients to match the dimensions of the cloud-side model parameters, and then using them to update the weight parameters of the cloud-side model; and approximating the importance score based on the weight gradients to integrate new domain knowledge into the cloud-side model.
[0019] Furthermore, in the cloud-side large model knowledge fusion step, the filled gradient is utilized. Update cloud-side model weights The process is represented as:
[0020] (2)
[0021] in, The learning rate is used to approximately update importance scores based on the weight gradient. The process is represented as:
[0022] (3)
[0023] This approximation ignores terms involving high powers of the learning rate.
[0024] Furthermore, the cloud-side large model knowledge fusion step supports both synchronous and asynchronous modes: In asynchronous mode, the cloud-side server sequentially receives and processes gradients from a single edge device; In synchronous mode, the cloud-side server simultaneously receives gradients from multiple edge devices, and after handling gradient conflicts using gradient fusion technology, updates the cloud-side model all at once.
[0025] Furthermore, after the cloud-side large model knowledge fusion step is completed, the updated importance score vector is stored in the domain-specific score library; and the cloud-side model initialization and deployment steps are re-executed using the updated cloud-side model to generate a performance-enhanced edge model for deployment. Attached Figure Description
[0026] Figure 1 This is a flowchart of the overall solution.
[0027] Figure 2 This is a schematic diagram of the overall task flow of the present invention.
[0028] Figure 3 This describes the collaborative training and deployment process of cloud-edge models based on gradient masks. Detailed Implementation
[0029] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings. The following embodiments are merely exemplary and can only be used to explain and illustrate the technical solution of the present invention, and should not be construed as limiting the technical solution of the present invention.
[0030] This invention proposes a privacy-preserving cloud-edge collaborative learning (P²CECL) framework based on prunable networks. A prunable model trained on source domain data, known as the cloud-side model, is deployed on a cloud server. Structured pruning is performed on this model to obtain a compact model, known as the edge model, which is then deployed to multiple edge devices. The edge devices perform Source-Free Domain Adaptation (SFDA) on the edge model using real-world data from their respective environments and record the accumulated gradients. These gradients are then uploaded to the cloud server to update the cloud model. This process transfers the knowledge learned by the edge model in the new domain to the cloud model, enhancing the cloud model's domain generalization ability through domain knowledge integration. Subsequently, the updated cloud model can derive edge models with improved domain generalization capabilities, which can be deployed on existing or new edge devices, enabling the edge devices to achieve better generalization performance.
[0031] To mitigate the catastrophic forgetting problem during passive domain adaptation, this invention proposes a gradient masking mechanism that selectively freezes parameters important to the learned domain. Specifically, this invention utilizes importance scores learned for each weight in a pruning network to identify and protect parameters crucial to the learned domain. When adapting to a new domain, only unimportant parameters are updated, thereby maximizing the retention of old knowledge while integrating new knowledge.
[0032] This application provides a large-scale cross-domain adaptation cloud-edge collaborative learning system for complex production environments, which includes a method comprising: a cloud-side server for executing the cloud-side model initialization and deployment steps and the cloud-side knowledge fusion steps; and one or more edge devices, communicatively connected to the cloud-side server, for executing the edge-side passive domain adaptation steps.
[0033] This system is applied to intelligent monitoring scenarios in tobacco processing workshops. Edge devices are deployed on-site in the processing workshop to perform visual analysis of material flow, equipment status, or process parameters. The unlabeled target domain data consists of images or video data collected on-site in the workshop.
[0034] like Figures 1 to 3 As shown, this application provides a large-scale model cross-domain adaptation cloud-edge collaborative learning method for complex production environments, including the following steps to implement the proposed large-scale model cross-domain adaptation cloud-edge collaborative learning method for complex production environments:
[0035] S1: Initialization and deployment of a filament material flow monitoring model based on a pruning network;
[0036] S2: The edge detection model is based on passive domain adaptation using neighborhood fraction masks;
[0037] S3: Domain knowledge fusion based on gradient mask for large cloud-side models.
[0038] In this application, the initialization and deployment scheme of the silk-making material flow monitoring model based on pruning networks aims to generate a compact monitoring model suitable for edge devices for the cloud-side silk-making material flow monitoring model.
[0039] Its core lies in constructing a pruning network as the source model (i.e., the cloud model) for the cloud side. In this network, each trainable weight parameter Each is associated with a learnable importance score. Thus, each neuron connection is determined by strength. and importance Common parameterization.
[0040] The training process for a prunable network is as follows: On a cloud-based server, source domain data is used... cloud model Training is performed, and all network weights are jointly optimized during this process. and their corresponding importance scores This allows for the simultaneous learning of the entire network and its multiple nested subnetworks.
[0041] During cloud-based model training, a progressive subnetwork activation strategy is employed, with subnetworks of varying parameter sizes used for training. With cloud-side model By sharing parameters, subnetworks are sequentially incorporated into the optimization process to learn scores and weights more effectively.
[0042] The deployment process is as follows: After the cloud model is trained, structured pruning is performed on the cloud model according to the actual computing resource constraints of the target edge device to derive a compact edge model.
[0043] The specific operation of structured pruning is as follows: based on the importance scores obtained during training... In each network layer, the highest-scoring neuron is selected, and the lower-scoring connections are removed, thereby generating a sub-network with a predetermined parameter scale (e.g., retaining 100%, 75%, or 50% of the parameters of the original model) as a fringe model.
[0044] When deploying the edge model to edge devices, discard all importance scores attached to the edge model. Only retain the weight parameters after pruning. To adapt to the limited computing resources of edge devices.
[0045] Its advantage lies in the fact that the edge model derived through a pruning method using shared parameters has an isomorphic weight parameter space with the cloud-side source model. This lays the foundation for the cloud side to use gradients uploaded from the edge to update the cloud-side model and achieve knowledge integration, while avoiding the uploading of original workshop images / videos, thus meeting industrial data security requirements.
[0046] While pruning to generate the edge model, the cloud server calculates a joint mask based on the maintained domain-specific score library. The mask is then sent to the corresponding edge devices to selectively freeze gradient updates during the subsequent passive domain adaptation process of the edge model, in order to mitigate catastrophic forgetting.
[0047] In this application, the edge detection model is based on the passive domain adaptation method of neighborhood fraction mask:
[0048] This method is executed on a compact model deployed on an edge device, which adapts the model using unlabeled real-world data (target domain data) collected locally on the device, and protects knowledge of the seen domain through a gradient masking mechanism in the process.
[0049] The premise is that when the cloud server deploys the compact model to the edge device, it will simultaneously issue a joint mask. This mask is calculated and generated by the cloud side based on its maintained domain-specific score library, and is used to identify the locations of parameters that are critical to the seen domain.
[0050] The adaptive process employs a passive domain adaptation algorithm. The edge device uses locally collected target domain data. By minimizing a specific SFDA loss function (such as SHOT) [4] or TPDS [5] The model parameters are updated using the loss defined in the code.
[0051] Its core innovation lies in the introduction of gradient masking. During the backpropagation process of each iteration, the edge model weights are calculated. gradient Then, use the joint mask issued by the cloud side. The gradient is masked to generate the masked gradient. This operation can be formally represented as:
[0052] (1)
[0053] Where ⊙ represents the Hadamarda.
[0054] Among them, joint mask It is a binary mask, and its calculation method is as follows:
[0055] (2)
[0056] That is, the domain-specific mask for all seen domains (from the 0th source domain to the (n-1)th target domain). The union of the domains. Each domain-specific mask. It is through its corresponding domain importance score vector For each layer, select the top layer with the highest importance score. The parameter index (e.g., 10%) is generated.
[0057] The effect of gradient masking is to combine the masks. The gradients corresponding to positions marked "1" (i.e., important to the existing domain) are set to zero, thus freezing these key parameters during parameter updates; only the gradients corresponding to positions with a mask of "0" are updated, allowing the model to adapt to new domains. This effectively mitigates the catastrophic forgetting of previously learned domain knowledge when adapting to new domains.
[0058] To reduce communication overhead between edge devices and cloud servers, edge devices do not immediately upload gradients after each iteration. Instead, a gradient accumulation strategy is used to accumulate the masked gradients calculated over multiple iteration cycles (e.g., set to 15 iterations), and then upload the accumulated gradients. Uploaded to the cloud server. Indicates the first Uploaded once.
[0059] Throughout the edge-side adaptation process, the edge model only uses and updates its weight parameters. The scores attached to each weight in the cloud-based model to assess domain importance have been discarded during model deployment to accommodate the limited computing and storage resources of edge devices.
[0060] In this application, a domain knowledge fusion method based on gradient masking is proposed for large cloud-side models:
[0061] This method is executed on a cloud-side server. Its core objective is to use masked cumulative gradients uploaded from edge devices to update the cloud-side model, thereby integrating new domain knowledge and improving the generalization ability of the global model.
[0062] Its input is: the cloud-side server receives data from the first... The set of masked gradients accumulated over multiple iterations for each edge device. and the weights of the current cloud-side model. And importance score Its output is: updated cloud-side model weights. And importance score .
[0063] The first step is weight updating. Since the edge model is obtained by structurally pruning the cloud-side model, its parameter dimension is smaller than that of the original cloud-side model. Therefore, before updating the cloud-side weights using the uploaded gradients, a padding operation needs to be performed to match the dimension. This process is formally represented as:
[0064] (3)
[0065] in, It's the learning rate. This indicates a fill operation. This operation utilizes the fraction used during the last pruning. As a guide, the location of the gradient of the edge model is mapped back to the corresponding parameter location of the cloud side model.
[0066] Its key innovation lies in proposing an approximate score update mechanism. Because edge devices discard importance scores to conserve resources when deploying models, the uploaded gradients only contain weight gradient information. To update the scores in the cloud-side model, this invention approximates the calculation using the relationship between weight gradients and score gradients, based on the definition of a pruning network. This relationship is:
[0067] (4)
[0068] in, Let denote the loss function, and ⊙ denote the Hadamard product.
[0069] The specific operation for approximate score update is as follows: The uploaded, padded mask gradient is... and current cloud-side weights Substitute into formula (4) to estimate the fraction The gradient. The update process is formally represented as:
[0070] (5)
[0071] This approximation ignores the learning rate. Higher powers, due to The approximation is usually very small, and is safe and effective.
[0072] After completing the iterative update of the weights and scores of the cloud-side model, the final updated importance score vector will be... The scores are stored in a domain-specific score library. This score library records the importance assessment of each parameter to each domain after the model has adapted to that domain, providing a basis for generating new joint masks in S2 to protect the learned knowledge.
[0073] It supports both synchronous and asynchronous collaborative modes. In asynchronous mode, the cloud side sequentially receives and processes gradients from a single edge device. In synchronous mode, the cloud side simultaneously receives gradients from multiple (…). The cumulative mask gradient set of edge devices .
[0074] When performing knowledge integration in synchronous mode, the cloud side needs to process gradients from different edge devices, which may conflict due to different distributions of target domain data. To address this, this method employs gradient fusion techniques (such as PCGrad). [6] This is used to alleviate gradient conflicts, and then the weights and scores of the cloud-side model are updated all at once using the fused gradients.
[0075] Its overall advantage lies in the following: through masked gradient updates and approximate score updates, the cloud-side model can absorb and integrate knowledge from multiple new domains without touching any original edge data. Simultaneously, since the gradients uploaded from the edge have already undergone joint masking locally, parameters crucial to the already seen domains are frozen, effectively mitigating the catastrophic forgetting of old knowledge when integrating new knowledge, thereby continuously improving the domain generalization ability of the cloud-side model. After knowledge fusion is complete, the updated cloud-side model can be pruned again to generate a more powerful edge model, which can then be deployed to existing or new devices, forming a positive cycle of performance improvement.
[0076] To validate the effectiveness of our proposed strategy, experiments were conducted on the widely used domain adaptation benchmark dataset Office-Home. Office-Home contains four visually distinct domains: Art (Ar), Clipart (Cl), Product (Pr), and Real-World (Rw). It comprises 65 categories and 15,588 images. In the experiments, one domain (e.g., Rw) was left as an unseen domain, and sequential adaptation was performed on the other three domains to test the model's generalization ability to novel environments.
[0077] The experiment used three core performance metrics to measure the model's effectiveness: average accuracy, ): Calculates the average classification accuracy of the model across all known domains, used to measure the model's overall generalization performance. The formula is: ,in Is the model in the domain Accuracy on the scale. Average Forgetting Scale. This quantifies the degree to which a model forgets previously learned knowledge as it continuously adapts to new domains. A smaller value is better; ideally, it should be 0 or negative (indicating no performance degradation or even performance improvement). The formula is: , .in Is the model in the domain This represents the best historical accuracy achieved. Accuracy on Unseen Domain ): In the Office-Home dataset setting, the reported model accuracy on domains (such as Rw) that were not involved in training and adaptation directly assesses the model's ability to generalize to new and unknown environments.
[0078] This patented method was compared with the following advanced methods: Continuous passive domain adaptation methods: G-SFDA[2] and PaCAD[3]. These methods are designed for sequential adaptation to multiple target domains, but usually require knowing or estimating the domain ID during inference. Baseline continuous adaptation methods: In order to make a fair comparison, the advanced single-shot passive domain adaptation methods (SHOT[4] and TPDS[5]) were directly applied to the sequence scenario of cloud-edge collaboration, denoted as SeqDA+SHOT and SeqDA+TPDS. The experimental results are shown in Table 1:
[0079] Table 1. Performance of different methods on the Office-Home dataset (Rw is selected as the unknown domain).
[0080] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.
Claims
1. A cloud-edge collaborative learning method for cross-domain adaptation of large models in complex production environments, characterized in that, Includes the following steps: S1: Initialization and deployment steps of the silk-making material flow monitoring model based on pruning networks; S2: The passive domain adaptation step of the edge detection model based on the neighborhood fraction mask; S3: Domain knowledge fusion steps based on gradient masking for large cloud-side models.
2. The large-scale cross-domain adaptation cloud-edge collaborative learning method for complex production environments as described in claim 1, characterized in that, In step S1, the initialization and deployment steps of the filament material flow monitoring model based on the pruning network are as follows: On the cloud server, a pruning network is trained using source domain data as a cloud-side model. Each trainable weight parameter in the pruning network is associated with a learnable importance score. Based on the computing resource constraints of the target edge device, the cloud-side model is subjected to structured pruning to derive a compact edge model and deploy it to the edge device.
3. The cloud-edge collaborative learning method for large-scale model adaptation in complex production environments as described in claim 1, characterized in that... The initialization and deployment steps of the filament material flow monitoring model based on the pruning network further include: calculating a joint mask based on the maintained domain-specific score library, and sending the joint mask to the corresponding edge device; the joint mask is used to identify the parameter positions that are crucial to the observed domain in the passive domain adaptation step on the edge side.
4. The cloud-edge collaborative learning method for large-scale model cross-domain adaptation in complex production environments as described in claim 3, is characterized in that... The joint mask is a binary mask obtained by taking the union of all domain-specific masks seen in the domain; wherein each domain-specific mask is obtained by selecting the top layer with the highest importance score from each layer of its corresponding domain importance score vector. The parameter index is generated by %.
5. The cloud-edge collaborative learning method for large-scale model adaptation in complex production environments as described in claim 1, characterized in that, In step S2, the passive domain adaptation step of the edge detection model based on the neighborhood score mask is specifically performed on the edge device using locally collected unlabeled target domain data to perform passive domain adaptation on the edge model. During the adaptation process, the gradient mask issued by the cloud server is applied to selectively freeze the gradient of the model parameters, only updating parameters that are not important to the seen neighborhood, and accumulating the processed gradient.
6. The cloud-edge collaborative learning method for large-scale model adaptation in complex production environments as described in claim 5, characterized in that, In the passive domain adaptive step of the edge device, the operation of applying the gradient mask is formally represented as follows: ， (1) in, The original gradient obtained from backpropagation. Let ⊙ represent the joint mask, and let ⊙ denote the Hadamard product. This is the gradient after masking.
7. The large-scale cross-domain adaptation cloud-edge collaborative learning method for complex production environments according to claim 1, characterized in that, In step S3, the domain knowledge fusion of the cloud-side large model based on gradient masking is specifically achieved by receiving accumulated gradients from edge devices on the cloud-side server; filling the accumulated gradients to match the dimensions of the cloud-side model parameters, and then using them to update the weight parameters of the cloud-side model; and approximating the importance score based on the weight gradients to integrate new domain knowledge into the cloud-side model.
8. The cloud-edge collaborative learning method for large-scale model adaptation in complex production environments as described in claim 7, characterized in that, In the cloud-side large model knowledge fusion step, the filled gradient is used. Update cloud-side model weights The process is represented as: (2) in, The learning rate is used to approximately update the importance score based on the weight gradient. The process is represented as: (3) This approximation ignores terms involving high powers of the learning rate.
9. The large-scale cross-domain adaptation cloud-edge collaborative learning method for complex production environments according to claim 8, characterized in that, The cloud-side large model knowledge fusion step supports both synchronous and asynchronous modes: In asynchronous mode, the cloud-side server sequentially receives and processes gradients from a single edge device; In synchronous mode, the cloud-side server simultaneously receives gradients from multiple edge devices, and after handling gradient conflicts using gradient fusion technology, updates the cloud-side model all at once.
10. The large-scale cross-domain adaptation cloud-edge collaborative learning method for complex production environments according to claim 7, characterized in that, After the cloud-side large model knowledge fusion step is completed, the updated importance score vector is stored in the domain-specific score library; and the cloud-side model initialization and deployment steps are re-executed using the updated cloud-side model to generate a performance-enhanced edge model for deployment.