Edge cloud collaborative learning method and system based on cloud model decomposition

Abstract

The invention relates to an edge cloud collaborative learning method and system based on cloud model decomposition. The method belongs to the technical field of machine learning and data processing, and comprises the following steps: a cloud server performs model training to obtain a centralized cloud model, and the centralized cloud model is decomposed into a plurality of sub-models; each edge device determines a target sub-model and adjusts the target sub-model to obtain a customized sub-model; all the edge devices send model updating parameters in the model training process to the cloud server, and the cloud server updates a global model according to the model updating parameters; migrating knowledge of the global model back to the edge device through a knowledge distillation technology; and the edge device performs fine adjustment on the global model according to the local data set to realize edge-cloud collaboration. Through effective knowledge exchange between the edge device and the cloud, continuous learning and global optimization of the model are realized, and the method is more flexible, expandable and high in adaptability.

Description

Technical Field

[0001] The present invention relates to the field of machine learning and data processing technology, and in particular to an edge-cloud collaborative learning method and system based on cloud model decomposition. Background Art

[0002] With the rapid development of mobile devices and Internet of Things technology, in the field of edge computing and federated learning, the data generated by massive distributed devices provides a rich data source for the training of machine learning models, which provides unprecedented opportunities for intelligent analysis and service deployment. However, these devices have significant limitations in computing power, storage space, and energy supply, which severely restricts their ability to run complex models. Although researchers have proposed a variety of model optimization and compression techniques to adapt to resource-constrained environments, there are still some problems that need to be solved in existing technologies.

[0003] First, most existing technologies focus on model optimization on a single device without fully considering collaboration and knowledge sharing between models running across multiple devices, which is a limiting factor in real-world applications because many tasks may require distributed processing across multiple devices. Second, existing technologies face challenges in processing non-independent and identically distributed (Non-IID) data. In a federated learning environment, different devices may have datasets with different characteristics, which may cause the model to perform poorly on some devices but well on others. Existing technologies generally do not effectively address this data heterogeneity problem, resulting in limited performance of global models. In addition, existing technologies are also deficient in model updating and maintenance. As data continues to change and new data is constantly generated, models need to be continuously updated to maintain their accuracy and effectiveness.

[0004] Therefore, existing model updating methods often have great limitations, are not flexible enough, and cannot adapt to the rapidly changing data environment, resulting in the degradation of model performance. Summary of the invention

[0005] Based on this, in order to solve the above technical problems, an edge-cloud collaborative learning method and system based on cloud model decomposition is provided, which is more flexible, scalable and adaptable.

[0006] An edge-cloud collaborative learning method based on cloud model decomposition, the method comprising: The cloud server performs model training to obtain a centralized cloud model, and uses a model decomposition algorithm to decompose the cloud model into a plurality of sub-models; Each edge device analyzes the local data set respectively to determine the data characteristics, and determines the target sub-model corresponding to each edge device from each sub-model according to the data characteristics; Each of the edge devices adjusts the target sub-model based on the local data set to obtain a customized sub-model; the customized sub-model is trained independently on the edge device; Each of the edge devices sends the model update parameters in the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters; The knowledge of the global model is migrated back to the edge device through knowledge distillation technology; the edge device fine-tunes the global model according to the local data set to achieve edge-cloud collaboration.

[0007] In one embodiment, a model decomposition algorithm is used to decompose the cloud model into several sub-models, including: Obtaining a model decomposition algorithm, and determining a decomposition loss function based on the model decomposition algorithm; Decomposing the cloud model using the model decomposition algorithm and minimizing the decomposition loss function; Based on the minimized decomposition loss function, several sub-models are determined to obtain a sub-model set.

[0008] In one embodiment, a plurality of sub-models are determined based on the minimized decomposition loss function to obtain a sub-model set, including: Determine the computing capability and device requirements of each of the edge devices, and select a sub-model to be combined from the plurality of sub-models according to the computing capability and device requirements; Each of the sub-models to be combined is combined to obtain a sub-model set corresponding to the edge device.

[0009] In one embodiment, each edge device analyzes a local data set respectively to determine data characteristics, and determines a target sub-model corresponding to each edge device from each sub-model according to the data characteristics, including: Each edge device determines a data analysis condition, and analyzes a local data set according to the data analysis condition to obtain data characteristics; The model performance data of each of the sub-models is obtained respectively, and the target sub-model corresponding to each of the edge devices is determined from each of the sub-models according to the model performance data and data characteristics.

[0010] In one embodiment, each of the edge devices adjusts the target sub-model based on the local data set to obtain a customized sub-model, including: Each of the edge devices generates a model adjustment strategy according to the model performance data and data characteristics; Based on the model adjustment strategy, parameters of the target sub-model are adjusted according to the data features to obtain a customized sub-model.

[0011] In one embodiment, each of the edge devices sends a model update parameter in a model training process to the cloud server, and the cloud server updates a global model according to the model update parameter, including: Each of the edge devices trains the customized sub-model by using the local data set, and collects model update parameters during the training process; Each of the edge devices sends the model update parameters to the cloud server; The cloud server aggregates the update parameters of each model and updates the global model based on the aggregated parameters.

[0012] In one embodiment, the knowledge of the global model is migrated back to the edge device through the knowledge distillation technology, including: Determine the distillation parameters used in the distillation process and determine the distillation loss function; By using the knowledge distillation technology, the objective function of the knowledge distillation is calculated using the distillation parameters and the distillation loss function; The knowledge of the global model is transferred back to the edge device using the objective function.

[0013] In one embodiment, the method further comprises: Acquire the coordination parameters generated in the edge-cloud coordination process, and perform performance evaluation on the customized sub-model and the global model according to the coordination parameters to obtain evaluation results; The customized sub-model and the global model are adjusted according to the evaluation results.

[0014] An edge-cloud collaborative learning system based on cloud model decomposition, the system comprising: A cloud server is used to perform model training to obtain a centralized cloud model, and decompose the cloud model into a plurality of sub-models using a model decomposition algorithm; Each edge device is used to analyze the local data set respectively, determine the data characteristics, and determine the target sub-model corresponding to each edge device from each sub-model according to the data characteristics; Each edge device is further used to adjust the target sub-model based on the local data set to obtain a customized sub-model; the customized sub-model is trained independently on the edge device; Each edge device is further used to send the model update parameters in the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters; The edge-cloud collaboration module is used to migrate the knowledge of the global model back to the edge device through knowledge distillation technology; the edge device fine-tunes the global model according to the local data set to achieve edge-cloud collaboration.

[0015] The above-mentioned edge-cloud collaborative learning method and system based on cloud model decomposition, by decomposing a large cloud model into multiple modular sub-models, performs personalized combination according to the needs and resource limitations of the edge devices, and allows the edge devices to download and customize the sub-models from the cloud according to their own local data distribution and resource conditions; through effective knowledge exchange between edge devices and the cloud, continuous learning and global optimization of the model are achieved; the limitations of the existing technology are overcome, and a more flexible, scalable and adaptable model continuous learning strategy is provided; it can effectively handle the collaboration and knowledge sharing problems between models running across multiple devices; it solves the problem of data heterogeneity, and provides an effective model update and maintenance mechanism to adapt to the rapidly changing data environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is an application environment diagram of an edge-cloud collaborative learning method based on cloud model decomposition in one embodiment; Figure 2 It is a flowchart of an edge-cloud collaborative learning method based on cloud model decomposition in one embodiment; Figure 3 It is a structural block diagram of an edge-cloud collaborative learning system based on cloud model decomposition in one embodiment; Figure 4 It is a structural diagram of an edge-cloud collaborative learning system based on cloud model decomposition in another embodiment; Figure 5 A schematic diagram of the structure of a cloud model decomposition module process in an embodiment; Figure 6 A schematic diagram of the structure of a personalized sub-model derivation module process in one embodiment; Figure 7 This is a schematic diagram of the process structure of the edge-cloud collaboration module in an embodiment; Figure 8 A schematic diagram of a flow structure of traffic prediction and scheduling applied in an intelligent transportation system in one embodiment; Fig. 9 A schematic diagram of the process structure of disease prediction and management applied in a health monitoring system in one embodiment; Fig.10 FIG. 4 is a diagram showing the internal structure of an edge device in one embodiment. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0018] The edge-cloud collaborative learning method based on cloud model decomposition provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Figure 1 As shown, the application environment includes a cloud server 110 and each edge device 120, wherein the cloud server 110 is connected and communicated with each edge device 120. The cloud server 110 performs model training to obtain a centralized cloud model, and uses a model decomposition algorithm to decompose the cloud model into several sub-models; each edge device 120 analyzes the local data set respectively, determines the data characteristics, and determines the target sub-model corresponding to each edge device 120 from each sub-model according to the data characteristics; each edge device 120 adjusts the model of its own target sub-model based on the local data set to obtain a customized sub-model; the customized sub-model is trained independently on the edge device 120; each edge device 120 sends the model update parameters in the model training process to the cloud server 110, and the cloud server 110 updates the global model according to the model update parameters; the knowledge of the global model is migrated back to the edge device 120 through the knowledge distillation technology; the edge device 120 fine-tunes the global model according to the local data set to achieve edge-cloud collaboration. Among them, the edge device 120 can be, but is not limited to, various personal computers, laptops, smart phones, robots, unmanned aerial vehicles and other devices.

[0019] In one embodiment, Figure 2 As shown, a method for edge-cloud collaborative learning based on cloud model decomposition is provided, including the following steps: In step 202, the cloud server performs model training to obtain a centralized cloud model, and uses a model decomposition algorithm to decompose the cloud model into several sub-models.

[0020] The cloud server can collect model training data to train a centralized cloud model. Considering the computing power and resource limitations of edge devices, as well as the dynamically changing data environment, the cloud server can efficiently decompose the centralized cloud model into multiple sub-models for personalized learning on edge devices. Specifically, the cloud server can decompose a large cloud model into a series of modular sub-models that are independent and combinable and can be personalized according to the needs of different edge devices. This process not only improves the scalability of the model, but also provides flexibility for edge devices to select the most appropriate sub-model based on their own computing power and resource limitations.

[0021] In one embodiment, a provided edge-cloud collaborative learning method based on cloud model decomposition may also include a process of dividing a cloud server into sub-models, and the specific process includes: obtaining a model decomposition algorithm, and determining a decomposition loss function based on the model decomposition algorithm; decomposing the cloud model using the model decomposition algorithm, and minimizing the decomposition loss function; determining a number of sub-models based on the minimized decomposition loss function to obtain a sub-model set.

[0022] Among them, the model decomposition algorithm can transform the large cloud model M cloud Decomposed into multiple sub-models {M i}, where i=1, 2, ..., n, each sub-model M i Responsible for processing specific data features or learning tasks. The goal of the decomposition algorithm is to minimize the information loss in the decomposition process while ensuring the composability and lightweight of the sub-models. The decomposition algorithm can be expressed as: ;in, is the decomposition loss function, which is used to measure the difference between the decomposed sub-model set and the original cloud model.

[0023] In one embodiment, a provided edge-cloud collaborative learning method based on cloud model decomposition may also include a process of personalized combination of each sub-model, and the specific process includes: determining the computing power and device requirements of each edge device, and selecting a sub-model to be combined from a number of sub-models based on the computing power and device requirements; combining the sub-models to be combined to obtain a sub-model set corresponding to the edge device.

[0024] On the edge device, according to the characteristics of local data and the computing power of the device, this embodiment provides a personalized combination strategy for selecting and combining the most appropriate sub-model from the decomposed sub-model set. The goal of the personalized combination strategy is to maximize the performance of the model on the local task while minimizing resource consumption. The combination strategy can be expressed as: ; Among them, M combined is the combined sub-model, is the combined loss function, D local It is a local dataset on the edge device.

[0025] In this embodiment, in order to improve the efficiency of model decomposition, the computation and storage overhead in the decomposition process can be reduced by parallel processing and model compression. The mathematical expression can be expressed as: ; Among them, O decomp is the cost of decomposition, T decomp is the computation time, S decomp It's storage space.

[0026] Considering the dynamic nature of the data environment, a dynamic model update mechanism is adopted in this embodiment, and the cloud model updates itself according to the information fed back by the edge device. The mathematical expression of the dynamic update mechanism can be expressed as: ;in, is the updated cloud model, U is the update function, This is the cloud model before the update. It is the model update uploaded by the edge device.

[0027] In order to reduce the communication overhead in the model decomposition process, in this embodiment, a communication efficiency optimization technology is used to reduce the transmission size of model parameters through model quantization and sparse coding. The optimization formula of communication efficiency can be expressed as: Among them, C eff is the communication efficiency, n is the number of sub-models after decomposition, It is the sub-model M i The cloud model decomposition module can provide personalized and efficient sub-models for edge devices while ensuring that the cloud model can continue to adapt to environmental changes.

[0028] In step 204, each edge device analyzes the local data set respectively, determines the data characteristics, and determines the target sub-model corresponding to each edge device from each sub-model according to the data characteristics.

[0029] On edge devices, a personalized sub-model derivation process can be implemented. Specifically, the edge device downloads the corresponding modular sub-model from the cloud server according to its own data characteristics and computing resources, and performs local customization and optimization. By referring to the local independent training solution, it ensures that the edge device can complete the training independently without relying on the global model knowledge, thereby improving the efficiency and accuracy of training.

[0030] In one embodiment, a provided edge-cloud collaborative learning method based on cloud model decomposition may also include a sub-model selection process, and the specific process includes: each edge device determines the data analysis conditions, and analyzes the local data set according to the data analysis conditions to obtain data characteristics; obtains the model performance data of each sub-model respectively, and determines the target sub-model corresponding to each edge device from each sub-model according to the model performance data and data characteristics.

[0031] Specifically, the edge device can first perform local data set D local Analyze to determine its characteristics. Data characteristics analysis includes but is not limited to data distribution, category balance, and correlation with cloud models. Data characteristics can be expressed as: ; where P(x) is the probability distribution of data point x, N is the number of samples in the local dataset, and p(xi ) is the category probability distribution of the i-th sample.

[0032] In this embodiment, based on the results of data characteristic analysis, the edge device selects the most appropriate sub-model M i Further customization is performed. The sub-model selection process takes into account the performance, size, and computational requirements of the model. The selection criteria can be expressed as: ;in, Is the selected loss function used to measure the sub-model M i With the local dataset D local degree of matching.

[0033] In step 206, each edge device adjusts its own target sub-model based on the local data set to obtain a customized sub-model; the customized sub-model is trained independently on the edge device.

[0034] In one embodiment, a provided edge-cloud collaborative learning method based on cloud model decomposition may also include a sub-model customization process, and the specific process includes: each edge device generates a model adjustment strategy based on model performance data and data characteristics; based on the model adjustment strategy, the parameters of the target sub-model are adjusted according to the data characteristics to obtain a customized sub-model.

[0035] After selecting the sub-model, the edge device customizes the sub-model according to the characteristics of the local data set. The customization process includes adjusting model parameters, structural optimization, and feature extraction strategy. The goal of customization is to maximize the performance of the sub-model on local tasks. Specifically, the customization process can be expressed as: ; Among them, M customized It is a customized sub-model. It is a customized function based on the local data set D local The characteristics of the selected sub-model M selected Make adjustments.

[0036] In this embodiment, after the customized sub-model is trained on the local data set, its performance needs to be evaluated. The performance evaluation is performed through a set of predefined indicators, such as accuracy, recall, and F1 score. The evaluation results are fed back to the customized function for iterative optimization. The performance evaluation can be expressed as: ;in, is the performance index of the sub-model, D test is a local test dataset, is the indicator function.

[0037] In step 208, each edge device sends the model update parameters in the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters.

[0038] Through effective knowledge exchange between edge devices and cloud servers, continuous learning and global optimization of the model are achieved. The edge device uses local data to train the sub-model and uploads the update to the cloud server. The cloud server uses a customized aggregation strategy based on the received updates to update the global model. In this process, a weighted knowledge transfer mechanism is adopted, allowing each client to selectively receive knowledge from other clients according to its own task requirements, optimizing the continuous learning ability and global optimization of the model.

[0039] In one embodiment, a provided edge-cloud collaborative learning method based on cloud model decomposition may also include a model training process, and the specific process includes: each edge device trains a customized sub-model by using a local data set, and collects model update parameters during the training process; each edge device sends the model update parameters to a cloud server; the cloud server aggregates each model update parameter, and updates the global model based on the aggregated parameters.

[0040] In this embodiment, the edge device can use the local data set D local For personalized sub-model M optimized During the training process, the device updates the model parameters to adapt to the local task. The objective function of local training can be expressed as: in, are model parameters, is the local training loss function.

[0041] After the training is completed, the edge device will update the model Upload to the cloud. The cloud server collects updates from all edge devices and uses an aggregation algorithm to update the global model M global , the aggregation process can be expressed as: ;in, is an aggregate function, It is the model update uploaded by all edge devices.

[0042] Step 210, migrate the knowledge of the global model back to the edge device through knowledge distillation technology; the edge device fine-tunes the global model according to the local data set to achieve edge-cloud collaboration.

[0043] In order to solve the problem of catastrophic forgetting, knowledge distillation technology is used in this embodiment. Specifically, in one embodiment, a cloud-edge collaborative learning method based on cloud model decomposition can also include a knowledge migration process, which includes: determining the distillation parameters used in the distillation process and determining the distillation loss function; using the knowledge distillation technology, using the distillation parameters and the distillation loss function to calculate the objective function of the knowledge distillation; using the objective function to migrate the knowledge of the global model back to the edge device.

[0044] The edge device transfers knowledge from the personalized sub-model to the global model through the distillation process. The objective function of knowledge distillation can be expressed as: ;in, is the parameter used in the distillation process, is the distillation loss function.

[0045] In one embodiment, a provided edge-cloud collaborative learning method based on cloud model decomposition may also include a personalized adjustment process of the model. The specific process includes: obtaining the collaborative parameters generated in the edge-cloud collaborative process, and performing performance evaluation on customized sub-models and global models based on the collaborative parameters to obtain evaluation results; and adjusting customized sub-models and global models based on the evaluation results.

[0046] The global model M after cloud server aggregation global The adaptability to local tasks of some edge devices may be lost. Therefore, this embodiment proposes a personalized adjustment mechanism to allow edge devices to fine-tune the global model based on local data, where the personalized adjustment can be expressed as: ;in, is a personalized adjustment function.

[0047] In order to reduce the communication overhead in the edge-cloud collaborative learning process, compressed transmission is used in this embodiment. The goal of communication efficiency optimization is to minimize the amount of transmitted data while maintaining model performance: in, is the model update of the ith edge device, is a quantization function. Through the edge-cloud collaborative learning module, effective collaboration between edge devices and cloud servers can be achieved, which not only improves the global performance of the model but also maintains its adaptability to local tasks.

[0048] A cloud-edge collaborative learning method based on cloud model decomposition provided in this application reduces the transmission requirements of raw data by training and updating the model on the edge device, thereby reducing the risk of data leakage. In practical applications, such as smartphones and medical devices, this strategy significantly improves the security of user data. Cloud model decomposition technology allows edge devices to select the most appropriate sub-model based on their own data characteristics and resource constraints. This method can significantly improve the adaptability and robustness of the model in the face of changes in data distribution. Through personalized sub-model derivation and edge-cloud collaborative learning, unnecessary model updates and data transmission can be reduced. Under the premise of maintaining the same model performance, the use of network bandwidth can be significantly reduced and communication overhead can be reduced. Edge-cloud collaborative learning achieves rapid model updates through effective knowledge exchange and aggregation strategies. Experiments on the Fashion-MNIST dataset have shown that in the process of model training and updating, compared with the existing method FedProx-APD, it can significantly reduce the required time and computing resources, and the training time is shortened by about 10%. The knowledge distillation and personalized adjustment mechanism adopted effectively alleviate the model's forgetting of previous task knowledge during continuous learning. Through edge-cloud collaborative learning, the model can be continuously learned and optimized on the edge device, which can continuously improve the performance of the model on various tasks, especially in non-independent and identically distributed data environments. It allows the cloud model to be dynamically adjusted according to the needs of the edge device, reducing the need to customize and maintain independent models for each edge device.

[0049] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0050] In one embodiment, Figure 1 As shown, an edge-cloud collaborative learning system based on cloud model decomposition is provided, including: a cloud server 110, and each edge device 120, wherein the cloud server 110 is connected and communicated with each edge device 120. Figure 3 As shown, a cloud model decomposition module 310 is provided in the cloud server, which is used to perform model training to obtain a centralized cloud model, and use a model decomposition algorithm to decompose the cloud model into several sub-models; The edge device is provided with a sub-model derivation module 320, which is used to analyze the local data set respectively, determine the data characteristics, and determine the target sub-model corresponding to each edge device from each sub-model according to the data characteristics; The edge device is provided with a sub-model customization module 330, which is used to adjust the respective target sub-models based on the local data set to obtain a customized sub-model; the customized sub-model is trained independently on the edge device; The edge device is provided with a model update module 340, which is used to send the model update parameters in the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters; The edge-cloud collaboration module 350 is used to migrate the knowledge of the global model back to the edge device through knowledge distillation technology; the edge device fine-tunes the global model according to the local data set to achieve edge-cloud collaboration.

[0051] In another embodiment, Figure 4 As shown, a provided edge-cloud collaborative learning system based on cloud model decomposition may include: a cloud model decomposition module, a personalized sub-model derivation module, and an edge-cloud collaborative learning module, wherein: The cloud model decomposition module is responsible for decomposing large cloud models and generating a series of modular sub-models to meet the needs of different edge devices; The personalized sub-model derivation module allows edge devices to download and customize sub-models from the cloud based on their own data characteristics and resource constraints; The edge-cloud collaborative learning module achieves continuous learning and global optimization of the model by exchanging knowledge between edge devices and the cloud.

[0052] exist Figure 4 In the figure, arrows are used to represent data flow and information transfer. The output of the cloud model decomposition module points to the personalized sub-model derivation module through an arrow, indicating the transfer of modular sub-models; the output of the personalized sub-model derivation module is passed to the edge-cloud collaborative learning module through an arrow, indicating the transfer of personalized sub-models; the edge-cloud collaborative learning module is responsible for coordinating knowledge exchange and model updates between edge devices and the cloud to achieve edge-cloud collaborative learning.

[0053] In one embodiment, the cloud model decomposition module 310 is also used to obtain a model decomposition algorithm, determine a decomposition loss function based on the model decomposition algorithm; decompose the cloud model using the model decomposition algorithm, and minimize the decomposition loss function; determine a number of sub-models based on the minimized decomposition loss function to obtain a sub-model set.

[0054] In one embodiment, the cloud model decomposition module 310 is also used to determine the computing power and equipment requirements of each edge device, and select a sub-model to be combined from a number of sub-models based on the computing power and equipment requirements; combine the sub-models to be combined to obtain a sub-model set corresponding to the edge device.

[0055] like Figure 5 As shown, in one embodiment, the cloud model decomposition module may include a cloud model, a model decomposition algorithm, a sub-model set, a personalized combination strategy, an edge device, and performance evaluation and feedback, wherein: The cloud model represents the original large cloud model, which serves as the starting point of the entire decomposition process; The model decomposition algorithm is responsible for performing the model decomposition task and decomposing the cloud model into multiple sub-models; The sub-model set is generated by the decomposition algorithm and contains multiple independent and combinable sub-models; The personalized combination strategy selects and combines sub-models from the sub-model set according to the specific needs of the edge device; The edge devices represent different edge devices, which receive the personalized combined sub-models and perform local task processing; Performance evaluation and feedback is responsible for evaluating the performance of the decomposed model and providing feedback to optimize the model decomposition process.

[0056] exist Figure 5 In the figure, arrows represent data flow and information transmission. The cloud model is decomposed into a set of sub-models through the decomposition algorithm, and the sub-model set is customized according to the personalized combination strategy to meet the needs of edge devices. The edge device passes the performance data to the performance evaluation and feedback module, and then feeds the evaluation results back to the model decomposition algorithm, forming a closed-loop optimization process.

[0057] In one embodiment, the sub-model derivation module 320 is also used for each edge device to determine data analysis conditions, and analyze the local data set according to the data analysis conditions to obtain data characteristics; obtain the model performance data of each sub-model respectively, and determine the target sub-model corresponding to each edge device from each sub-model according to the model performance data and data characteristics.

[0058] In one embodiment, the sub-model customization module 330 is also used for each edge device to generate a model adjustment strategy based on model performance data and data characteristics; based on the model adjustment strategy, the parameters of the target sub-model are adjusted according to the data characteristics to obtain a customized sub-model.

[0059] In one embodiment, the sub-model derivation module and the sub-model customization module can jointly form a personalized sub-model derivation module, such as Figure 6As shown in the figure, the personalized sub-model derivation module can include data feature analysis, sub-model selection, customized training, performance evaluation, resource adaptation optimization, sub-model deployment, and feedback loop, among which: Data feature analysis involves in-depth analysis of local data sets on edge devices to determine data distribution, category balance, and other characteristics; In sub-model selection, the edge device selects the most appropriate sub-model for personalized customization based on the data characteristic analysis results of data characteristic analysis; Customized training is the further training process of customized sub-models on local datasets to optimize model performance; Performance evaluation: Evaluate the performance of the customized sub-model to ensure that it meets the task requirements of the edge device; In resource adaptation optimization, the model is further optimized according to the resource status of the edge device to ensure the best performance of the model under limited resources; Sub-model deployment involves deploying the customized and optimized sub-model to edge devices; The feedback loop is where the edge device feeds the performance of the sub-model back to the cloud for future model updates and optimizations.

[0060] exist Figure 6 In the example, the output of data feature analysis points to sub-model selection, indicating that the data feature analysis results are used to guide the selection of sub-models. The output of sub-model selection points to customized training, indicating that the selected sub-model will enter the customized training phase. The output of customized training points to performance evaluation, which is used to evaluate the performance of customized models. The output of performance evaluation points to resource adaptation optimization, indicating that resource adaptation optimization is performed based on the performance evaluation results. The output of resource adaptation optimization points to sub-model deployment, indicating that the optimized sub-model will be deployed. Finally, the output of sub-model deployment points to the feedback loop, indicating that the deployed sub-model will enter the feedback loop to achieve continuous model improvement.

[0061] In one embodiment, the model update module 340 is also used for each edge device to train a customized sub-model by using a local data set, and collect model update parameters during the training process; each edge device sends the model update parameters to the cloud server; the cloud server aggregates the model update parameters and updates the global model based on the aggregated parameters.

[0062] In one embodiment, the edge-cloud collaboration module 350 is also used to determine the distillation parameters used in the distillation process and determine the distillation loss function; through the knowledge distillation technology, the objective function of the knowledge distillation is calculated using the distillation parameters and the distillation loss function; and the objective function is used to migrate the knowledge of the global model back to the edge device.

[0063] In one embodiment, the edge-cloud collaboration module 350 is also used to obtain the collaboration parameters generated in the edge-cloud collaboration process, and perform performance evaluation on the customized sub-model and the global model based on the collaboration parameters to obtain evaluation results; and adjust the customized sub-model and the global model based on the evaluation results.

[0064] like Figure 7 As shown, in one embodiment, the edge-cloud collaboration module may include a local training unit, a knowledge upload mechanism, a cloud aggregation unit, a knowledge distillation unit, a personalized adjustment unit, a communication efficiency optimization unit, and a performance evaluation and optimization unit, wherein: The local training unit is responsible for training the personalized sub-model using the local dataset on the edge device; The knowledge upload mechanism uploads the model updates trained on the edge device to the cloud server; The cloud aggregation unit aggregates the model updates from multiple edge devices on the cloud server to form a global model; The knowledge distillation unit uses knowledge distillation technology to migrate the knowledge of the global model on the cloud back to the edge device, helping the sub-model on the edge device overcome the catastrophic forgetting problem; The personalized adjustment unit allows edge devices to make personalized adjustments to the global model aggregated in the cloud according to local task requirements; The communication efficiency optimization unit is responsible for optimizing the communication process between edge and cloud, reducing the amount of data transmission and improving the transmission efficiency; The performance evaluation and optimization unit conducts performance evaluation on the entire edge-cloud collaborative learning process and optimizes it based on the evaluation results.

[0065] exist Figure 7 In the proposed method, the edge device first performs local training on the local training unit, and then uploads the training updates to the cloud through the knowledge upload mechanism. The cloud aggregation unit on the cloud server aggregates these updates to form a global model. The knowledge distillation unit migrates the knowledge of the global model back to the edge device, and the personalized adjustment unit personalizes the knowledge to adapt to the local task. The communication efficiency optimization unit optimizes the communication process between the edge and the cloud, while the performance evaluation and optimization unit evaluates and optimizes the entire process to ensure the effectiveness and efficiency of edge-cloud collaborative learning.

[0066] In one embodiment, Figure 8 As shown, the edge-cloud collaborative learning method based on cloud model decomposition can be applied to the intelligent transportation system to improve the accuracy of traffic flow prediction and the intelligence level of traffic scheduling. The specific implementation process is as follows: Traffic data collection: Sensors deployed at key urban traffic nodes collect real-time traffic flow data, such as vehicle speed, density, and flow; Data preprocessing: Edge devices preprocess the collected raw data, including data cleaning and standardization; Feature engineering and model training: The cloud server performs feature engineering, extracts features that are helpful for traffic flow prediction, and trains the initial model; Model decomposition and personalization: The cloud server decomposes the trained model into multiple sub-models and adjusts them according to the traffic characteristics of each area; Sub-model delivery and local deployment: deliver personalized sub-models to edge devices in the corresponding area and deploy them on the edge devices; Real-time traffic flow prediction: Edge devices use locally deployed sub-models to predict real-time traffic data; Traffic dispatch decision: Based on the prediction results provided by edge devices, the cloud server makes traffic dispatch decisions; Dispatching strategy issuance and execution: The cloud server sends the formulated traffic dispatching strategy to the relevant traffic control units to execute operations such as traffic signal control; Performance feedback and model iteration: Collect execution results and performance data, evaluate the effectiveness of scheduling strategies, and use feedback for model iteration and optimization.

[0067] In one embodiment, Fig. 9 As shown, the edge-cloud collaborative learning method based on cloud model decomposition can be applied to the health monitoring system to improve the accuracy of disease prediction and the intelligent level of health management. The specific implementation process is as follows: Physiological data collection: collect users’ physiological data, such as heart rate, blood pressure, and activity level, through wearable devices and medical sensors; Data preprocessing and privacy protection: Edge devices preprocess the collected raw data and apply privacy protection technology; Feature extraction and sub-model training: The cloud server extracts features that are helpful for disease prediction and trains them on the sub-model; Sub-model personalization and optimization: The cloud server personalizes and adjusts the sub-model based on the user's health records and historical data; Sub-model delivery to edge devices: Deliver personalized sub-models to users’ edge devices, such as smartphones or home medical devices. Real-time health status monitoring and prediction: Edge devices use sub-models to monitor users’ real-time physiological data and predict disease risks; Health advice and warnings: Based on the prediction results, the cloud server generates health advice and disease warnings when necessary; Health intervention strategy formulation: The cloud server formulates personalized health intervention and management strategies; User feedback collection and model iteration: Collect user health feedback, evaluate the effectiveness of intervention strategies, and use the data for model iteration.

[0068] Through the cloud model decomposition module, the large cloud model is decomposed into multiple modular sub-models, which can be personalized according to the needs and resource constraints of the edge device; the personalized sub-model derivation module allows the edge device to download and customize the sub-model from the cloud according to its own local data distribution and resource status; the edge-cloud collaborative learning module realizes the continuous learning and global optimization of the model through effective knowledge exchange between the edge device and the cloud. This application overcomes the limitations of the prior art and provides a more flexible, scalable and adaptable model continuous learning strategy; through modular design and personalized sub-model derivation, it can effectively handle the collaboration and knowledge sharing problems between models running across multiple devices. In addition, through edge-cloud collaborative learning, the problem of data heterogeneity is solved, and an effective model update and maintenance mechanism is provided to adapt to the rapidly changing data environment. It is suitable for a variety of application scenarios such as intelligent traffic management, health and medical monitoring, and intelligent manufacturing, and has broad practical value and application prospects.

[0069] In one embodiment, an edge device is provided. The edge device may be a terminal, and its internal structure diagram may be as follows: Fig.10 As shown. The edge device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the edge device is used to provide computing and control capabilities. The memory of the edge device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the edge device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method of edge-cloud collaborative learning based on cloud model decomposition is implemented. The display screen of the edge device can be a liquid crystal display or an electronic ink display screen, and the input device of the edge device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the outer shell of the edge device, or an external keyboard, touchpad or mouse, etc.

[0070] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the edge device to which the scheme of the present application is applied. The specific edge device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0071] In one embodiment, an edge device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, steps of an edge-cloud collaborative learning method based on cloud model decomposition are implemented.

[0072] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of an edge-cloud collaborative learning method based on cloud model decomposition are implemented.

[0073] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0074] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0075] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An edge-cloud collaborative learning method based on cloud model decomposition, characterized in that: The method comprises: The cloud server performs model training to obtain a centralized cloud model, and uses a model decomposition algorithm to decompose the cloud model into a plurality of sub-models; Each edge device analyzes the local data set respectively to determine the data characteristics, and determines the target sub-model corresponding to each edge device from each sub-model according to the data characteristics; Each of the edge devices adjusts the target sub-model based on the local data set to obtain a customized sub-model; the customized sub-model is trained independently on the edge device; Each of the edge devices sends the model update parameters in the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters; The knowledge of the global model is migrated back to the edge device through knowledge distillation technology; the edge device fine-tunes the global model according to the local data set to achieve edge-cloud collaboration.

2. The edge-cloud collaborative learning method based on cloud model decomposition according to claim 1 is characterized in that: The cloud model is decomposed into several sub-models using a model decomposition algorithm, including: Obtaining a model decomposition algorithm, and determining a decomposition loss function based on the model decomposition algorithm; Decomposing the cloud model using the model decomposition algorithm and minimizing the decomposition loss function; Based on the minimized decomposition loss function, several sub-models are determined to obtain a sub-model set.

3. The edge-cloud collaborative learning method based on cloud model decomposition according to claim 2 is characterized in that: Based on the minimized decomposition loss function, several sub-models are determined to obtain a sub-model set, including: Determine the computing capability and device requirements of each of the edge devices, and select a sub-model to be combined from the plurality of sub-models according to the computing capability and device requirements; Each of the sub-models to be combined is combined to obtain a sub-model set corresponding to the edge device.

4. The edge-cloud collaborative learning method based on cloud model decomposition according to claim 1 is characterized in that: Each edge device analyzes the local data set respectively to determine the data characteristics, and determines the target sub-model corresponding to each edge device from each sub-model according to the data characteristics, including: Each edge device determines a data analysis condition, and analyzes a local data set according to the data analysis condition to obtain data characteristics; The model performance data of each of the sub-models is obtained respectively, and the target sub-model corresponding to each of the edge devices is determined from each of the sub-models according to the model performance data and data characteristics.

5. The edge-cloud collaborative learning method based on cloud model decomposition according to claim 4 is characterized in that: Each of the edge devices performs model adjustment on the respective target sub-model based on the local data set to obtain a customized sub-model, including: Each of the edge devices generates a model adjustment strategy according to the model performance data and data characteristics; Based on the model adjustment strategy, parameters of the target sub-model are adjusted according to the data features to obtain a customized sub-model.

6. The edge-cloud collaborative learning method based on cloud model decomposition according to claim 1 is characterized in that: Each of the edge devices sends the model update parameters in the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters, including: Each of the edge devices trains the customized sub-model by using the local data set, and collects model update parameters during the training process; Each of the edge devices sends the model update parameters to the cloud server; The cloud server aggregates the update parameters of each model and updates the global model based on the aggregated parameters.

7. The edge-cloud collaborative learning method based on cloud model decomposition according to claim 1 is characterized in that: Migrating the knowledge of the global model back to the edge device through the knowledge distillation technology, including: Determine the distillation parameters used in the distillation process and determine the distillation loss function; By using the knowledge distillation technology, the objective function of the knowledge distillation is calculated using the distillation parameters and the distillation loss function; The knowledge of the global model is transferred back to the edge device using the objective function.

8. The edge-cloud collaborative learning method based on cloud model decomposition according to claim 1 is characterized in that: The method further comprises: Acquire the coordination parameters generated in the edge-cloud coordination process, and perform performance evaluation on the customized sub-model and the global model according to the coordination parameters to obtain evaluation results; The customized sub-model and the global model are adjusted according to the evaluation results.

9. An edge-cloud collaborative learning system based on cloud model decomposition, characterized in that: The system comprises: A cloud server is used to perform model training to obtain a centralized cloud model, and decompose the cloud model into a plurality of sub-models using a model decomposition algorithm; Each edge device is used to analyze the local data set respectively, determine the data characteristics, and determine the target sub-model corresponding to each edge device from each sub-model according to the data characteristics; Each edge device is further used to adjust the target sub-model based on the local data set to obtain a customized sub-model; the customized sub-model is trained independently on the edge device; Each edge device is further used to send the model update parameters in the model training process to the cloud server, and the cloud server updates the global model according to the model update parameters; The edge-cloud collaboration module is used to migrate the knowledge of the global model back to the edge device through knowledge distillation technology; the edge device fine-tunes the global model according to the local data set to achieve edge-cloud collaboration.

10. The edge-cloud collaborative learning system based on cloud model decomposition according to claim 9, characterized in that: The cloud server is also used to: obtain a model decomposition algorithm, determine a decomposition loss function based on the model decomposition algorithm; decompose the cloud model using the model decomposition algorithm and minimize the decomposition loss function; determine a number of sub-models based on the minimized decomposition loss function to obtain a sub-model set.

Images