Edge-cloud collaborative learning method and system based on cloud model decomposition

Abstract

The present invention relates to an edge-cloud collaborative learning method and system based on cloud model decomposition, belonging to the technical fields of machine learning and data processing. The method includes: the cloud server performs model training to obtain a centralized cloud model, which is decomposed into several sub-models; each edge device determines a target sub-model and adjusts it to obtain a customized sub-model; each of the edge devices sends the model update parameters during the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters; the knowledge of the global model is migrated back to the edge device through the knowledge distillation technology; the edge device fine-tunes the global model according to the local dataset to achieve edge-cloud collaboration. By effectively exchanging knowledge between the edge device and the cloud, continuous learning and global optimization of the model are achieved, which is more flexible, scalable and adaptable.

Description

Technical Field

[0001] The present invention relates to the field of machine learning and data processing technology, and in particular to an edge-cloud collaborative learning method and system based on cloud model decomposition. Background Art

[0002] With the rapid development of mobile devices and Internet of Things technology, in the field of edge computing and federated learning, the data generated by massive distributed devices provides a rich data source for the training of machine learning models, which provides unprecedented opportunities for intelligent analysis and service deployment. However, these devices have significant limitations in computing power, storage space, and energy supply, which severely restricts their ability to run complex models. Although researchers have proposed a variety of model optimization and compression techniques to adapt to resource-constrained environments, there are still some problems that need to be solved in existing technologies.

[0003] First, most existing technologies focus on model optimization on a single device without fully considering collaboration and knowledge sharing between models running across multiple devices, which is a limiting factor in real-world applications because many tasks may require distributed processing across multiple devices. Second, existing technologies face challenges in processing non-independent and identically distributed (Non-IID) data. In a federated learning environment, different devices may have datasets with different characteristics, which may cause the model to perform poorly on some devices but well on others. Existing technologies generally do not effectively address this data heterogeneity problem, resulting in limited performance of global models. In addition, existing technologies are also deficient in model updating and maintenance. As data continues to change and new data is constantly generated, models need to be continuously updated to maintain their accuracy and effectiveness.

[0004] Therefore, existing model updating methods often have great limitations, are not flexible enough, and cannot adapt to the rapidly changing data environment, resulting in the degradation of model performance. Summary of the invention

[0005] Based on this, in order to solve the above technical problems, an edge-cloud collaborative learning method and system based on cloud model decomposition is provided, which is more flexible, scalable and adaptable.

[0006] An edge-cloud collaborative learning method based on cloud model decomposition, the method comprising:

[0007] The cloud server performs model training to obtain a centralized cloud model, and uses a model decomposition algorithm to decompose the cloud model into a plurality of sub-models;

[0008] Each edge device analyzes the local dataset respectively, determines the data characteristics, and determines the target sub-model corresponding to each edge device from each of the sub-models according to the data characteristics;

[0009] Each of the edge devices adjusts the respective target sub-model based on the local dataset to obtain a customized sub-model; the customized sub-model is independently trained on the edge device;

[0010] Each of the edge devices sends the model update parameters during the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters;

[0011] The knowledge of the global model is migrated back to the edge device through the knowledge distillation technique; the edge device fine-tunes the global model according to the local dataset to achieve edge-cloud collaboration.

[0012] In one embodiment, the cloud model is decomposed into several sub-models using a model decomposition algorithm, including:

[0013] Obtain a model decomposition algorithm, and determine a decomposition loss function based on the model decomposition algorithm;

[0014] Use the model decomposition algorithm to decompose the cloud model and minimize the decomposition loss function;

[0015] Determine several sub-models based on the minimized decomposition loss function to obtain a set of sub-models.

[0016] In one embodiment, determining several sub-models based on the minimized decomposition loss function to obtain a set of sub-models, including:

[0017] Determine the computing power and device requirements of each edge device, and select the sub-models to be combined from several sub-models according to the computing power and device requirements;

[0018] Combine each of the sub-models to be combined to obtain a set of sub-models corresponding to the edge device.

[0019] In one embodiment, each edge device analyzes the local dataset respectively, determines the data characteristics, and determines the target sub-model corresponding to each edge device from each of the sub-models, including:

[0020] Each edge device determines the data analysis conditions, and analyzes the local dataset according to the data analysis conditions to obtain the data characteristics;

[0021] Obtain the model performance data of each of the sub-models respectively, and determine the target sub-models corresponding to each of the edge devices from each of the sub-models according to the model performance data and data characteristics.

[0022] In one embodiment, each of the edge devices adjusts its respective target sub-model based on the local data set to obtain a customized sub-model, including:

[0023] Each of the edge devices generates a model adjustment strategy according to the model performance data and data characteristics;

[0024] Based on the model adjustment strategy, adjust the parameters of the target sub-model according to the data characteristics to obtain a customized sub-model.

[0025] In one embodiment, each of the edge devices sends the model update parameters during the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters, including:

[0026] Each of the edge devices trains the customized sub-model by using the local data set and collects the model update parameters during the training process;

[0027] Each of the edge devices sends the model update parameters to the cloud server;

[0028] The cloud server performs aggregation processing on each of the model update parameters and updates the global model based on the aggregated parameters.

[0029] In one embodiment, transfer the knowledge of the global model back to the edge device through knowledge distillation technology, including:

[0030] Determine the distillation parameters used in the distillation process and determine the distillation loss function;

[0031] Through knowledge distillation technology, calculate the objective function of knowledge distillation by using the distillation parameters and the distillation loss function;

[0032] Use the objective function to transfer the knowledge of the global model back to the edge device.

[0033] In one embodiment, the method further includes:

[0034] Obtain the cooperation parameters generated during the edge-cloud cooperation process, and perform performance evaluation on the customized sub-model and the global model according to the cooperation parameters to obtain an evaluation result;

[0035] Adjust the customized sub-model and the global model according to the evaluation result.

[0036] An edge-cloud collaborative learning system based on cloud model decomposition, the system includes:

[0037] A cloud server, which is used to perform model training to obtain a centralized cloud model, and uses a model decomposition algorithm to decompose the cloud model into several sub-models;

[0038] Each edge device is used to analyze the local data set respectively, determine the data characteristics, and determine the target sub-model corresponding to each edge device from each of the sub-models according to the data characteristics;

[0039] Each edge device is also used to adjust the respective target sub-model based on the local data set to obtain a customized sub-model; the customized sub-model is independently trained on the edge device;

[0040] Each edge device is also used to send the model update parameters during the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters;

[0041] An edge-cloud collaboration module, which is used to transfer the knowledge of the global model back to the edge device through knowledge distillation technology; the edge device fine-tunes the global model according to the local data set to achieve edge-cloud collaboration.

[0042] The above-mentioned edge-cloud collaborative learning method and system based on cloud model decomposition decompose a large cloud model into multiple modular sub-models, perform personalized combination according to the needs and resource limitations of edge devices, and allow edge devices to download and customize sub-models from the cloud according to their own local data distribution and resource status; through effective knowledge exchange between edge devices and the cloud, continuous learning and global optimization of the model are achieved; the limitations of the existing technology are overcome, and a more flexible, scalable and adaptable model continuous learning strategy is provided; it can effectively handle the collaboration and knowledge sharing problems between models running across multiple devices; it solves the data heterogeneity problem, and provides an effective model update and maintenance mechanism to adapt to the rapidly changing data environment. Brief Description of the Drawings

[0043] Figure 1 It is an application environment diagram of the edge-cloud collaborative learning method based on cloud model decomposition in an embodiment;

[0044] Figure 2 It is a flow schematic diagram of the edge-cloud collaborative learning method based on cloud model decomposition in an embodiment;

[0045] Figure 3 It is a structural block diagram of the edge-cloud collaborative learning system based on cloud model decomposition in an embodiment;

[0046] Figure 4 It is a schematic structural diagram of an edge-cloud collaborative learning system based on cloud model decomposition in another embodiment;

[0047] Figure 5 It is a schematic structural diagram of the process of the cloud model decomposition module in one embodiment;

[0048] Figure 6 It is a schematic structural diagram of the process of the personalized sub-model derivation module in one embodiment;

[0049] Figure 7 It is a schematic structural diagram of the process of the edge-cloud collaboration module in one embodiment;

[0050] Figure 8 It is a schematic structural diagram of the process of traffic prediction and scheduling applied in an intelligent transportation system in one embodiment;

[0051] Figure 9 A schematic structural diagram of the process of disease prediction and management applied in a health monitoring system in one embodiment;

[0052] Figure 10 It is an internal structure diagram of an edge device in one embodiment. Detailed implementation manners

[0053] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] The edge-cloud collaborative learning method based on cloud model decomposition provided by the embodiments of the present application can be applied to an application environment as Figure 1 shown. As Figure 1As shown, the application environment includes a cloud server 110 and various edge devices 120. Among them, the cloud server 110 is connected and communicates with various edge devices 120. The cloud server 110 performs model training to obtain a centralized cloud model, and uses a model decomposition algorithm to decompose the cloud model into several sub-models; each edge device 120 analyzes the local data set respectively to determine the data characteristics, and determines the target sub-model corresponding to each edge device 120 from each sub-model according to the data characteristics; each edge device 120 adjusts the respective target sub-model based on the local data set to obtain a customized sub-model; the customized sub-model independently completes training on the edge device 120; each edge device 120 sends the model update parameters during the model training process to the cloud server 110, and the cloud server 110 updates the global model according to the model update parameters; the knowledge of the global model is migrated back to the edge device 120 through knowledge distillation technology; the edge device 120 fine-tunes the global model according to the local data set to achieve edge-cloud collaboration. Among them, the edge device 120 can be but is not limited to various devices such as personal computers, laptops, smartphones, robots, and unmanned aerial vehicles.

[0055] In one embodiment, as Figure 2 shown, a method for edge-cloud collaborative learning based on cloud model decomposition is provided, including the following steps:

[0056] Step 202, the cloud server performs model training to obtain a centralized cloud model, and uses a model decomposition algorithm to decompose the cloud model into several sub-models.

[0057] The cloud server can collect model training data to train a centralized cloud model. Considering the computing power and resource limitations of edge devices, as well as the dynamically changing data environment, the cloud server can efficiently decompose the centralized cloud model into multiple sub-models for personalized learning on edge devices. Specifically, the cloud server can decompose a large cloud model into a series of modular sub-models, which are independent and combinable, and can be personalized combined according to the needs of different edge devices. This process can not only improve the scalability of the model, but also provide flexibility for edge devices, enabling them to select the most suitable sub-model according to their own computing power and resource limitations.

[0058] In one embodiment, a method for edge-cloud collaborative learning based on cloud model decomposition provided can also include the process of the cloud server dividing to obtain sub-models. The specific process includes: obtaining a model decomposition algorithm, determining a decomposition loss function based on the model decomposition algorithm; using the model decomposition algorithm to decompose the cloud model and minimizing the decomposition loss function; determining several sub-models based on the minimized decomposition loss function to obtain a sub-model set.

[0059] Among them, the model decomposition algorithm can decompose the large cloud model M cloud into multiple sub-models {M i}, where i = 1, 2,..., n, and each sub-model M i is responsible for processing specific data features or learning tasks. The goal of the decomposition algorithm is to minimize the information loss during the decomposition process while ensuring the composability and light weight of the sub-models. The decomposition algorithm can be expressed by the formula: ; where is the decomposition loss function, which is used to measure the difference between the set of decomposed sub-models and the original cloud model.

[0060] In one embodiment, a method for edge-cloud collaborative learning based on cloud model decomposition may further include a process of personalized combination of each sub-model. The specific process includes: determining the computing capabilities and device requirements of each edge device, and selecting the sub-models to be combined from several sub-models according to the computing capabilities and device requirements; combining each sub-model to be combined to obtain a set of sub-models corresponding to the edge device.

[0061] On the edge device, according to the characteristics of the local data and the computing capabilities of the device, a personalized combination strategy is provided in this embodiment for selecting and combining the most suitable sub-models from the set of decomposed sub-models. The goal of the personalized combination strategy is to maximize the performance of the model on local tasks while minimizing resource consumption. The combination strategy can be expressed by the formula: ; where M combined is the combined sub-model, is the combination loss function, and D local is the local data set on the edge device.

[0062] In this embodiment, in order to improve the efficiency of model decomposition, the computational and storage overheads during the decomposition process can be reduced through parallel processing and model compression, and the mathematical expression can be represented as: ; where O decomp is the overhead during the decomposition process, T decomp is the computing time, and S decomp is the storage space.

[0063] Considering the dynamics of the data environment, a dynamic model update mechanism is adopted in this embodiment, and the cloud model is self-updated according to the information fed back by the edge device. Among them, the mathematical expression of the dynamic update mechanism can be represented as: ; where is the updated cloud model, U is the update function, is the cloud model before update, is the model update uploaded by the edge device.

[0064] To reduce the communication overhead during the model decomposition process, in this embodiment, a communication efficiency optimization technique is adopted to reduce the transmission size of model parameters through model quantization and sparse coding. The optimization formula for communication efficiency can be expressed as: ; where C eff is the communication efficiency, n is the number of sub-models after decomposition, is the number of parameters of sub-model M i . The cloud model decomposition module can provide personalized and efficient sub-models for edge devices while ensuring that the cloud model can continuously adapt to environmental changes.

[0065] Step 204, each edge device analyzes the local dataset separately to determine the data characteristics, and determines the target sub-model corresponding to each edge device from each sub-model according to the data characteristics.

[0066] On the edge device, a personalized sub-model derivation process can be implemented. Specifically, the edge device downloads the corresponding modular sub-model from the cloud server according to its own data characteristics and computing resources, and performs local customization and optimization. By referring to the local independent training solution, it is ensured that the edge device can complete the training independently without relying on the global model knowledge, thereby improving the training efficiency and accuracy.

[0067] In one embodiment, a cloud-edge collaborative learning method based on cloud model decomposition may further include a sub-model selection process. The specific process includes: each edge device determines the data analysis conditions, analyzes the local dataset according to the data analysis conditions to obtain the data characteristics; respectively obtains the model performance data of each sub-model, and determines the target sub-model corresponding to each edge device from each sub-model according to the model performance data and the data characteristics.

[0068] Specifically, the edge device can first analyze the local dataset D local to determine its characteristics. Data characteristic analysis includes but is not limited to the distribution of data, class balance, and correlation with the cloud model. The data characteristics can be expressed by the formula: ; where P(x) is the probability distribution of data point x, N is the number of samples in the local dataset, and p(x i ) is the class probability distribution of the i-th sample.

[0069] In this embodiment, based on the results of the data characteristic analysis, the edge device selects the most suitable sub-model M i for further personalized customization. The sub-model selection process takes into account the performance, size, and computing requirements of the model. The selection criteria can be expressed by the formula: ; where is a selection loss function used to measure the sub - model M i and the local dataset D local matching degree.

[0070] Step 206: Each edge device adjusts its respective target sub - model based on the local dataset to obtain a customized sub - model; the customized sub - model is independently trained on the edge device.

[0071] In one embodiment, a cloud - edge collaborative learning method based on cloud - model decomposition may further include the process of sub - model customization. The specific process includes: Each edge device generates a model adjustment strategy according to model performance data and data characteristics; based on the model adjustment strategy, the target sub - model is parameter - adjusted according to data features to obtain a customized sub - model.

[0072] After selecting the sub - model, the edge device customizes the sub - model according to the characteristics of the local dataset. The customization process includes adjusting model parameters, optimizing the structure, and adjusting the feature extraction strategy. The goal of customization is to maximize the performance of the sub - model on local tasks. Specifically, the customization process can be expressed by the formula: ; where M customized is the customized sub - model, is the customization function that adjusts the selected sub - model M local according to the characteristics of the local dataset D selected for adjustment.

[0073] In this embodiment, after the customized sub - model is trained on the local dataset, its performance needs to be evaluated. The performance evaluation is carried out through a set of predefined metrics, such as accuracy, recall rate, and F1 - score. The evaluation results are fed back to the customization function for iterative optimization. The performance evaluation can be expressed by the formula: ; where is the performance metric of the sub - model, D test is the local test dataset, is the indicator function.

[0074] Step 208: Each edge device sends the model update parameters during the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters.

[0075] Through effective knowledge exchange between edge devices and cloud servers, continuous learning and global optimization of the model are achieved. Edge devices use local data to train sub-models and upload the updates to the cloud server. The cloud server then updates the global model according to the received updates using a customized aggregation strategy. In this process, a weighted knowledge transfer mechanism is adopted, allowing each client to selectively receive knowledge from other clients according to its own task requirements, optimizing the continuous learning ability and global optimization of the model.

[0076] In one embodiment, a cloud-edge collaborative learning method based on cloud model decomposition may further include the process of model training. The specific process includes: each edge device trains a customized sub-model using a local dataset and collects the model update parameters during the training process; each edge device sends the model update parameters to the cloud server; the cloud server aggregates the model update parameters and updates the global model based on the aggregated parameters.

[0077] In this embodiment, the edge device can use the local dataset D local to train the personalized sub-model M optimized . During the training process, the device updates the model parameters to adapt to local tasks. The objective function of local training can be expressed as: where are the model parameters, is the local training loss function.

[0078] After training, the edge device uploads the model update to the cloud. The cloud server collects the updates from all edge devices and uses an aggregation algorithm to update the global model M global , and the aggregation process can be expressed as: ; where is the aggregation function, are the model updates uploaded by all edge devices.

[0079] Step 210, transfer the knowledge of the global model back to the edge device through knowledge distillation technology; the edge device fine-tunes the global model according to the local dataset to achieve cloud-edge collaboration.

[0080] To solve the catastrophic forgetting problem, knowledge distillation technology is adopted in this embodiment. Specifically, in one embodiment, a cloud-edge collaborative learning method based on cloud model decomposition may further include the process of knowledge transfer. The specific process includes: determining the distillation parameters used in the distillation process and determining the distillation loss function; using knowledge distillation technology, calculating the objective function of knowledge distillation using the distillation parameters and the distillation loss function; using the objective function to transfer the knowledge of the global model back to the edge device.

[0081] The edge device migrates knowledge from the personalized sub - model to the global model through the distillation process. The objective function of knowledge distillation can be expressed as: ; where, are the parameters used in the distillation process, is the distillation loss function.

[0082] In one embodiment, a cloud - edge collaborative learning method based on cloud - model decomposition may further include a process of personalized adjustment of the model. The specific process includes: obtaining the collaborative parameters generated during the cloud - edge collaboration process, and evaluating the performance of the customized sub - model and the global model according to the collaborative parameters to obtain an evaluation result; adjusting the customized sub - model and the global model according to the evaluation result.

[0083] The global model M aggregated by the cloud server global may lose its adaptability to the local tasks of some edge devices. Therefore, a personalized adjustment mechanism is proposed in this embodiment, allowing edge devices to fine - tune the global model according to local data. Among them, the personalized adjustment can be expressed as: ; where, is the personalized adjustment function.

[0084] In order to reduce the communication overhead in the cloud - edge collaborative learning process, compression transmission is adopted in this embodiment. The goal of communication efficiency optimization is to minimize the amount of data transmitted while maintaining the model performance: where, is the model update of the i - th edge device, is the quantization function. Through the cloud - edge collaborative learning module, effective cooperation between edge devices and cloud servers can be achieved, which not only improves the global performance of the model but also maintains its adaptability to local tasks.

[0085] A method for edge-cloud collaborative learning based on cloud model decomposition provided in this application reduces the transmission requirements of raw data by performing model training and updating on edge devices, thereby reducing the risk of data leakage. In practical applications, such as smartphones and medical devices, this strategy significantly improves the security of user data. The cloud model decomposition technology allows edge devices to select the most suitable sub-models according to their own data characteristics and resource limitations. This method can significantly improve the adaptability and robustness of the model when facing data distribution changes. Through personalized sub-model derivation and edge-cloud collaborative learning, unnecessary model updates and data transmissions can be reduced, and the use of network bandwidth can be significantly reduced while maintaining the same model performance, reducing communication overhead. Edge-cloud collaborative learning realizes the rapid update of the model through effective knowledge exchange and aggregation strategies. Experiments on the Fashion-MNIST dataset prove that in the process of model training and updating, compared with the existing method FedProx-APD, it can significantly reduce the required time and computing resources, and the training time is shortened by about 10%. The knowledge distillation and personalized adjustment mechanisms adopted effectively alleviate the forgetting of the model's knowledge of previous tasks during the continuous learning process. Through edge-cloud collaborative learning, the model is allowed to perform continuous learning and optimization on edge devices, and the performance of the model on various tasks can be continuously improved, especially in the non-independent and identically distributed data environment. Allowing the cloud model to be dynamically adjusted according to the needs of edge devices reduces the need to customize and maintain independent models for each edge device.

[0086] It should be understood that although the steps in the above flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0087] In one embodiment, as Figure 1 shown, a system for edge-cloud collaborative learning based on cloud model decomposition is provided, including: a cloud server 110 and each edge device 120, wherein the cloud server 110 is connected and communicates with each edge device 120. Among them, a system for edge-cloud collaborative learning based on cloud model decomposition is provided, as Figure 3 shown, a cloud model decomposition module 310 is set in the cloud server, which is used to perform model training to obtain a centralized cloud model and decompose the cloud model into several sub-models by using a model decomposition algorithm;

[0088] A sub-model derivation module 320 is provided in the edge device, which is used to analyze the local data set respectively, determine the data characteristics, and determine the target sub-models corresponding to each edge device from each sub-model according to the data characteristics;

[0089] A sub-model customization module 330 is provided in the edge device, which is used to adjust the respective target sub-models based on the local data set to obtain customized sub-models; the customized sub-models are independently trained on the edge device;

[0090] A model update module 340 is provided in the edge device, which is used to send the model update parameters in the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters;

[0091] The edge-cloud collaboration module 350 is used to transfer the knowledge of the global model back to the edge device through the knowledge distillation technology; the edge device fine-tunes the global model according to the local data set to achieve edge-cloud collaboration.

[0092] In another embodiment, as Figure 4 shown, a kind of edge-cloud collaborative learning system based on cloud model decomposition may include: a cloud model decomposition module, a personalized sub-model derivation module, and an edge-cloud collaborative learning module, where:

[0093] The cloud model decomposition module is responsible for decomposing the large cloud model to generate a series of modular sub-models to meet the needs of different edge devices;

[0094] The personalized sub-model derivation module allows the edge device to download and customize the sub-model from the cloud according to its own data characteristics and resource limitations;

[0095] The edge-cloud collaborative learning module realizes continuous learning and global optimization of the model by exchanging knowledge between the edge device and the cloud.

[0096] In Figure 4 , each part uses arrows to represent the data flow and information transfer. The output of the cloud model decomposition module points to the personalized sub-model derivation module through an arrow, indicating the transfer of the modular sub-model; the output of the personalized sub-model derivation module is transferred to the edge-cloud collaborative learning module through an arrow, indicating the transfer of the personalized sub-model; the edge-cloud collaborative learning module is responsible for coordinating the knowledge exchange and model update between the edge device and the cloud to achieve edge-cloud collaborative learning.

[0097] In one embodiment, the cloud model decomposition module 310 is further configured to obtain a model decomposition algorithm, determine a decomposition loss function based on the model decomposition algorithm; decompose the cloud model using the model decomposition algorithm and minimize the decomposition loss function; determine a number of sub-models based on the minimized decomposition loss function to obtain a sub-model set.

[0098] In one embodiment, the cloud model decomposition module 310 is further configured to determine the computing capabilities and device requirements of each edge device, and select sub-models to be combined from a number of sub-models according to the computing capabilities and device requirements; combine the sub-models to be combined to obtain a sub-model set corresponding to the edge device.

[0099] As Figure 5 shown, in one embodiment, the cloud model decomposition module may include a cloud model, a model decomposition algorithm, a sub-model set, a personalized combination strategy, edge devices, performance evaluation and feedback, among which:

[0100] The cloud model represents the original large cloud model and serves as the starting point for the entire decomposition process;

[0101] The model decomposition algorithm is responsible for performing the model decomposition task and decomposing the cloud model into multiple sub-models;

[0102] The sub-model set is generated by the decomposition algorithm and contains multiple independent and combinable sub-models;

[0103] The personalized combination strategy selects and combines sub-models from the sub-model set according to the specific requirements of the edge device;

[0104] The edge device represents different edge devices, which receive the personalized combined sub-models and perform local task processing;

[0105] The performance evaluation and feedback is responsible for evaluating the performance of the decomposed model and providing feedback to optimize the model decomposition process.

[0106] In Figure 5 , each part represents the data flow and information transfer through arrows. The cloud model is decomposed into a sub-model set by the decomposition algorithm, and the sub-model set is customized according to the personalized combination strategy to meet the requirements of the edge device. The edge device transfers the performance data to the performance evaluation and feedback module, and then feeds back the evaluation results to the model decomposition algorithm, forming a closed-loop optimization process.

[0107] In one embodiment, the sub-model derivation module 320 is further configured to determine data analysis conditions for each edge device, analyze the local data set according to the data analysis conditions to obtain data characteristics; respectively obtain the model performance data of each sub-model, and determine the target sub-model corresponding to each edge device from each sub-model according to the model performance data and data characteristics.

[0108] In one embodiment, the sub-model customization module 330 is further configured to enable each edge device to generate a model adjustment strategy based on model performance data and data characteristics; and based on the model adjustment strategy, adjust the parameters of the target sub-model according to the data characteristics to obtain a customized sub-model.

[0109] In one embodiment, the sub-model derivation module and the sub-model customization module can jointly form a personalized sub-model derivation module. As Figure 6 shown, the personalized sub-model derivation module may include several parts: data characteristic analysis, sub-model selection, customized training, performance evaluation, resource adaptation optimization, sub-model deployment, and feedback loop. Among them:

[0110] Data characteristic analysis involves in-depth analysis of the local data set of the edge device to determine characteristics such as data distribution and class balance;

[0111] In sub-model selection, the edge device selects the most suitable sub-model for personalized customization based on the results of the data characteristic analysis in the data characteristic analysis;

[0112] Customized training is the further training process of the customized sub-model on the local data set to optimize the model performance;

[0113] Performance evaluation evaluates the performance of the customized sub-model to ensure that it meets the task requirements of the edge device;

[0114] In resource adaptation optimization, according to the resource status of the edge device, the model is further optimized to ensure the best performance of the model under limited resources;

[0115] Sub-model deployment involves deploying the personalized customized and optimized sub-model to the edge device;

[0116] The feedback loop is that the edge device feeds back the performance of the sub-model to the cloud for future model updates and optimizations.

[0117] In Figure 6 , the output of the data characteristic analysis points to the sub-model selection, indicating that the results of the data characteristic analysis are used to guide the selection of the sub-model. The output of the sub-model selection points to the customized training, indicating that the selected sub-model will enter the customized training stage. The output of the customized training points to the performance evaluation, which is used to evaluate the performance of the customized model. The output of the performance evaluation points to the resource adaptation optimization, indicating that resource adaptation optimization is performed according to the performance evaluation results. The output of the resource adaptation optimization points to the sub-model deployment, indicating that the optimized sub-model will be deployed. Finally, the output of the sub-model deployment points to the feedback loop, indicating that the deployed sub-model will enter the feedback loop to achieve continuous model improvement.

[0118] In one embodiment, the model update module 340 is further configured to enable each edge device to train the customized sub-model using the local data set, and collect the model update parameters during the training process; each edge device sends the model update parameters to the cloud server; the cloud server performs aggregation processing on each model update parameter, and updates the global model based on the aggregated parameters.

[0119] In one embodiment, the edge-cloud collaboration module 350 is further configured to determine the distillation parameters used during the distillation process, and determine the distillation loss function; through the knowledge distillation technology, use the distillation parameters and the distillation loss function to calculate the objective function of knowledge distillation; use the objective function to transfer the knowledge of the global model back to the edge device.

[0120] In one embodiment, the edge-cloud collaboration module 350 is further configured to obtain the collaboration parameters generated during the edge-cloud collaboration process, and perform performance evaluation on the customized sub-model and the global model according to the collaboration parameters to obtain an evaluation result; adjust the customized sub-model and the global model according to the evaluation result.

[0121] As Figure 7 shown, in one embodiment, the edge-cloud collaboration module may include several parts such as a local training unit, a knowledge uploading mechanism, a cloud aggregation unit, a knowledge distillation unit, a personalized adjustment unit, a communication efficiency optimization unit, and a performance evaluation and optimization unit, where:

[0122] The local training unit is responsible for training the personalized sub-model on the edge device using the local data set;

[0123] The knowledge uploading mechanism uploads the model updates obtained from training on the edge device to the cloud server;

[0124] The cloud aggregation unit aggregates the model updates from multiple edge devices on the cloud server to form a global model;

[0125] The knowledge distillation unit uses the knowledge distillation technology to transfer the knowledge of the cloud global model back to the edge device to help the sub-model on the edge device overcome the catastrophic forgetting problem;

[0126] The personalized adjustment unit allows the edge device to perform personalized adjustment on the globally aggregated global model on the cloud according to the local task requirements;

[0127] The communication efficiency optimization unit is responsible for optimizing the communication process between the edge and the cloud, reducing the data transmission volume and improving the transmission efficiency;

[0128] The performance evaluation and optimization unit performs performance evaluation on the entire edge-cloud collaborative learning process and performs optimization according to the evaluation result.

[0129] In Figure 7In it, the edge device first conducts local training on the local training unit, and then uploads the obtained updates to the cloud through the knowledge uploading mechanism. The cloud aggregation unit on the cloud server aggregates these updates to form a global model. The knowledge distillation unit of the knowledge distillation unit migrates the knowledge of the global model back to the edge device, and the personalized adjustment unit makes personalized adjustments to this knowledge to adapt to local tasks. The communication efficiency optimization unit optimizes the communication process between the edge and the cloud, while the performance evaluation and optimization unit evaluates and optimizes the entire process to ensure the effectiveness and efficiency of edge-cloud collaborative learning.

[0130] In one embodiment, as Figure 8 shown, a provided edge-cloud collaborative learning method based on cloud model decomposition can be applied to an intelligent transportation system, aiming to improve the accuracy of traffic flow prediction and the intelligent level of traffic scheduling. The specific implementation process is as follows:

[0131] Traffic data collection: Sensors deployed at key urban traffic nodes collect traffic flow data in real time, such as vehicle speed, density, and flow rate, etc.;

[0132] Data preprocessing: The edge device preprocesses the collected raw data, including data cleaning, standardization, etc.;

[0133] Feature engineering and model training: The cloud server performs feature engineering, extracts features helpful for traffic flow prediction, and trains an initial model;

[0134] Model decomposition and personalization: The cloud server decomposes the trained model into multiple sub-models and makes personalized adjustments according to the traffic characteristics of each region;

[0135] Sub-model distribution and local deployment: The personalized sub-models are distributed to the edge devices in the corresponding regions and deployed on the edge devices;

[0136] Real-time traffic flow prediction: The edge device uses the locally deployed sub-model to predict real-time traffic data;

[0137] Traffic scheduling decision-making: Based on the prediction results provided by the edge device, the cloud server formulates traffic scheduling decisions;

[0138] Scheduling strategy distribution and execution: The cloud server distributes the formulated traffic scheduling strategy to relevant traffic control units to execute operations such as traffic signal control;

[0139] Performance feedback and model iteration: Collect execution results and performance data, evaluate the effectiveness of the scheduling strategy, and use the feedback for model iteration and optimization.

[0140] In one embodiment, as Figure 9As shown, a cloud-edge collaborative learning method based on cloud model decomposition can be applied to a health monitoring system to improve the accuracy of disease prediction and the intelligence level of health management. The specific implementation process is as follows:

[0141] Physiological data collection: Collect users' physiological data, such as heart rate, blood pressure, and activity level, through wearable devices and medical sensors;

[0142] Data preprocessing and privacy protection: The edge device preprocesses the collected raw data and applies privacy protection technology;

[0143] Feature extraction and sub-model training: The cloud server extracts features helpful for disease prediction and trains on the sub-model;

[0144] Sub-model personalization and optimization: According to the user's health record and historical data, the cloud server adjusts the sub-model personally;

[0145] Sub-model distribution to edge devices: Distribute the personalized sub-model to the user's edge devices, such as smartphones or home medical devices;

[0146] Real-time health status monitoring and prediction: The edge device uses the sub-model to monitor the user's real-time physiological data and predict the disease risk;

[0147] Health advice and warning: Based on the prediction results, the cloud server generates health advice and disease warnings when necessary;

[0148] Health intervention strategy formulation: The cloud server formulates personalized health intervention and management strategies;

[0149] User feedback collection and model iteration: Collect the user's health feedback, evaluate the effect of the intervention strategy, and use the data for model iteration.

[0150] Through the cloud model decomposition module, a large cloud model is decomposed into multiple modular sub-models, which can be personalized combined according to the requirements and resource limitations of edge devices; the personalized sub-model derivation module allows edge devices to download and customize sub-models from the cloud according to their own local data distribution and resource conditions; the edge-cloud collaborative learning module realizes continuous learning and global optimization of the model by performing effective knowledge exchange between edge devices and the cloud. This application overcomes the limitations of the prior art and provides a more flexible, scalable, and adaptable model continuous learning strategy; through modular design and personalized sub-model derivation, it can effectively handle the collaboration and knowledge sharing problems between models running across multiple devices. In addition, through edge-cloud collaborative learning, the data heterogeneity problem is solved, and an effective model update and maintenance mechanism is provided to adapt to the rapidly changing data environment, which is applicable to various application scenarios such as intelligent transportation management, health care monitoring, and intelligent manufacturing, and has broad practical value and application prospects.

[0151] In one embodiment, an edge device is provided. The edge device can be a terminal, and its internal structure diagram can be as Figure 10 shown. The edge device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the edge device is used to provide computing and control capabilities. The memory of the edge device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the edge device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements an edge-cloud collaborative learning method based on cloud model decomposition. The display screen of the edge device can be a liquid crystal display screen or an electronic ink display screen. The input device of the edge device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad set on the outer shell of the edge device, or an external keyboard, a touchpad, or a mouse, etc.

[0152] Those skilled in the art can understand that Figure 10 the structure shown in

[0153] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the edge device to which the solution of this application is applied. The specific edge device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0154] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the edge-cloud collaborative learning method based on cloud model decomposition are implemented.

[0155] Those of ordinary skill in the art can understand that all or part of the processes in the above-described embodiment methods can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0156] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0157] The above-described embodiments merely represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. An edge-cloud collaborative learning method based on cloud model decomposition, characterized in that, The method includes: The cloud server performs model training to obtain a centralized cloud model, and uses a model decomposition algorithm to decompose the cloud model into several sub-models; Each edge device analyzes the local dataset respectively to determine data characteristics, and determines the target sub-model corresponding to each edge device from each of the sub-models according to the data characteristics; determines the computing power and device requirements of each edge device, and selects the sub-models to be combined from several sub-models according to the computing power and device requirements; combines each of the sub-models to be combined to obtain a sub-model set corresponding to the edge device; Each edge device adjusts the respective target sub-model based on the local dataset to obtain a customized sub-model; the customized sub-model is independently trained on the edge device; Each edge device sends the model update parameters during the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters, including: each edge device trains the customized sub-model by using the local dataset and collects the model update parameters during the training process; each edge device sends the model update parameters to the cloud server; the cloud server performs aggregation processing on each of the model update parameters and updates the global model based on the aggregated parameters; The knowledge of the global model is migrated back to the edge device through knowledge distillation technology; the edge device fine-tunes the global model according to the local dataset to achieve edge-cloud collaboration; The edge-cloud collaborative learning method based on cloud model decomposition is applied to an intelligent transportation system, and the specific implementation process is as follows: Traffic data collection: Sensors deployed at key urban traffic nodes collect traffic flow data in real time; Data preprocessing: The edge device preprocesses the collected raw data, including data cleaning and standardization; Feature engineering and model training: The cloud server performs feature engineering, extracts features helpful for traffic flow prediction, and trains an initial model; Model decomposition and personalization: The cloud server decomposes the trained model into multiple sub-models and adjusts them personally according to the traffic characteristics of each region; Sub-model distribution and local deployment: The personalized sub-models are distributed to the edge devices in the corresponding regions and deployed on the edge devices; Real-time traffic flow prediction: The edge device uses the locally deployed sub-model to predict real-time traffic data; Traffic scheduling decision-making: Based on the prediction results provided by the edge device, the cloud server formulates traffic scheduling decisions; Scheduling strategy distribution and execution: The cloud server distributes the formulated traffic scheduling strategy to relevant traffic control units to perform traffic signal control operations; Performance feedback and model iteration: Collect the execution results and performance data, evaluate the effect of the scheduling strategy, and use the feedback for model iteration and optimization.

2. The edge-cloud collaborative learning method based on cloud model decomposition according to claim 1, wherein Using a model decomposition algorithm to decompose the cloud model into several sub-models includes: Obtaining a model decomposition algorithm and determining a decomposition loss function based on the model decomposition algorithm; Decompose the cloud model using the model decomposition algorithm and minimize the decomposition loss function; Determine a number of sub-models based on the minimized decomposition loss function to obtain a set of sub-models.

3. The edge-cloud collaborative learning method based on cloud model decomposition according to claim 1, wherein Each edge device analyzes the local data set respectively to determine data characteristics, and determines a target sub-model corresponding to each edge device from each of the sub-models according to the data characteristics, including: Each edge device determines data analysis conditions and analyzes the local data set according to the data analysis conditions to obtain data characteristics; Respectively obtain the model performance data of each of the sub-models, and determine the target sub-model corresponding to each edge device from each of the sub-models according to the model performance data and the data characteristics.

4. The edge-cloud collaborative learning method based on cloud model decomposition according to claim 3, characterized in that, Each of the edge devices adjusts the model of its respective target sub-model based on the local data set to obtain a customized sub-model, including: Each of the edge devices generates a model adjustment strategy according to the model performance data and the data characteristics; Based on the model adjustment strategy, adjust the parameters of the target sub-model according to the data characteristics to obtain a customized sub-model.

5. The edge-cloud collaborative learning method based on cloud model decomposition according to claim 1, wherein Transfer the knowledge of the global model back to the edge device through knowledge distillation technology, including: Determine the distillation parameters used in the distillation process and determine the distillation loss function; Through knowledge distillation technology, calculate the objective function of knowledge distillation using the distillation parameters and the distillation loss function; Use the objective function to transfer the knowledge of the global model back to the edge device.

6. The edge-cloud collaborative learning method based on cloud model decomposition according to claim 1, characterized in that The method further includes: Obtain the collaboration parameters generated during the edge-cloud collaboration process, and perform performance evaluation on the customized sub-model and the global model according to the collaboration parameters to obtain an evaluation result; Adjust the customized sub-model and the global model according to the evaluation result.

7. An edge-cloud collaborative learning system based on cloud model decomposition, characterized in that The system includes: A cloud server for performing model training to obtain a centralized cloud model, and decomposing the cloud model into a number of sub-models using a model decomposition algorithm; Each edge device for respectively analyzing the local data set to determine data characteristics, and determining a target sub-model corresponding to each edge device from each of the sub-models according to the data characteristics; determining the computing power and device requirements of each of the edge devices, and selecting sub-models to be combined from a number of the sub-models according to the computing power and device requirements; combining each of the sub-models to be combined to obtain a set of sub-models corresponding to the edge device; Each edge device is further used to adjust the model of its respective target sub-model based on the local data set to obtain a customized sub-model; the customized sub-model independently completes training on the edge device; Each edge device is also used to send the model update parameters during the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters, including: each edge device trains the customized sub-model by using the local dataset and collects the model update parameters during the training process; each edge device sends the model update parameters to the cloud server; the cloud server performs aggregation processing on each of the model update parameters and updates the global model based on the aggregated parameters; The edge-cloud collaboration module is used to transfer the knowledge of the global model back to the edge device through the knowledge distillation technique; the edge device fine-tunes the global model according to the local dataset to achieve edge-cloud collaboration; The edge-cloud collaborative learning system based on cloud model decomposition is applied to the intelligent transportation system, and the specific implementation process is as follows: Traffic data collection: Sensors deployed at key urban traffic nodes collect traffic flow data in real time; Data preprocessing: The edge device preprocesses the collected raw data, including data cleaning and standardization; Feature engineering and model training: The cloud server performs feature engineering, extracts features helpful for traffic flow prediction, and trains the initial model; Model decomposition and personalization: The cloud server decomposes the trained model into multiple sub-models and adjusts them personally according to the traffic characteristics of each region; Sub-model distribution and local deployment: The personalized sub-models are distributed to the edge devices in the corresponding regions and deployed on the edge devices; Real-time traffic flow prediction: The edge device uses the locally deployed sub-model to predict the real-time traffic data; Traffic scheduling decision-making: Based on the prediction results provided by the edge device, the cloud server formulates traffic scheduling decisions; Scheduling strategy distribution and execution: The cloud server distributes the formulated traffic scheduling strategy to the relevant traffic control units to perform traffic signal control operations; Performance feedback and model iteration: Collect the execution results and performance data, evaluate the effect of the scheduling strategy, and use the feedback for model iteration and optimization.

8. The edge-cloud collaborative learning system based on cloud model decomposition according to claim 7, characterized in that, The cloud server is also used to: obtain a model decomposition algorithm, determine a decomposition loss function based on the model decomposition algorithm; decompose the cloud model by using the model decomposition algorithm and minimize the decomposition loss function; determine a number of sub-models based on the minimized decomposition loss function to obtain a sub-model set.

Images