Ray-based cloud-edge collaborative data analysis workflow optimization method and system
Patent Information
- Application Number
- CN202310526213.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-06
- Publication Date
- 2026-10-27
- Estimated Expiration
- 2043-05-06
AI Technical Summary
[0006]为了解决上述背景技术中存在的模型研发阶段从终端设备设计到云端分布式训练的操作复杂问题以及模型服务部署阶段模型压缩问题，本发明提供一种基于Ray的云边协同数据分析工作流优化方法及系统，其首先在模型研发阶段，采用分布式应用程序框架Ray实现终端设备到云端的无缝扩展，实现模型分布式训练的快速迭代；其次在服务部署阶段，采用知识蒸馏方法实现模型压缩，降低模型参数体积，更便于服务下游任务
[0043]为了解决目前将终端设备的模型训练代码扩展到云端，通常需要对代码进行较大调整，并需要掌握分布式训练方法，门槛高等问题，本发明利用Ray集成分布式并行、内存共享等方法，实现从终端设备到云端的无缝扩展机制，优化适配器，实现终端设备的模型训练在几乎不需要修改代码的情况下扩展到云端，从而提高模型优化效率，降低技术门槛。
Smart Images
Figure CN116681112B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data analysis workflow in deep learning, and particularly relates to a cloud-edge collaborative data analysis workflow optimization method and system based on Ray. Background Technology
[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.
[0003] With the rapid development of the digital economy and the large-scale implementation of edge computing, industry data has experienced explosive growth. Cloud-edge collaborative data computing has become a breakthrough for intelligent transformation, finding widespread application in education, healthcare, transportation, and marine sectors. Currently, model development is trending towards integrated and large-scale models, requiring distributed training in the cloud. However, these trained large models are often difficult to directly apply to downstream tasks.
[0004] Firstly, during the model development phase, when training pre-designed and tested models on terminal devices within a distributed cloud cluster, the main challenge is rewriting the code to support distributed storage or architecture. This is typically achieved through local editors, with remote connections to cloud servers during training. However, this requires algorithm engineers to understand distributed programming, undoubtedly increasing their workload. Secondly, during service deployment, cloud-trained models are often large in scale, making direct deployment to resource-constrained edge devices and downstream tasks difficult. Model compression is increasingly demonstrating its advantages in promoting the efficiency and deployment of machine learning applications. Knowledge distillation, as a model compression method, is widely used in computer vision and natural language processing tasks. Its working principle typically involves using a large model with a large parameter scale and higher accuracy as a "teacher" network, and a lighter model with fewer parameters as a student model to achieve knowledge transfer.
[0005] Currently, the following pain points are commonly encountered in the field of knowledge distillation: 1) The teacher model struggles to dynamically grasp the feature extraction capabilities of the student model. Traditional knowledge distillation methods typically rely on the student model to passively receive knowledge from the teacher model, without considering the student model's learning ability and performance. 2) The teacher model's parameters are often frozen during training, making updates impossible. Typically, the teacher model's parameters are frozen during training, making it difficult to adjust and update them based on the student model's state. Summary of the Invention
[0006] To address the operational complexity issues in the model development stage, from terminal device design to cloud-based distributed training, and the model compression problem in the model service deployment stage, this invention provides a Ray-based cloud-edge collaborative data analysis workflow optimization method and system. Firstly, in the model development stage, the distributed application framework Ray is used to achieve seamless expansion from terminal devices to the cloud, enabling rapid iteration of distributed model training. Secondly, in the service deployment stage, a knowledge distillation method is used to compress the model, reducing the size of model parameters and making it easier to serve downstream tasks.
[0007] To achieve the above objectives, the present invention adopts the following technical solution:
[0008] The first aspect of the present invention provides a Ray-based cloud-edge collaborative data analysis workflow optimization method.
[0009] Ray-based cloud-edge collaborative data analysis workflow optimization methods include:
[0010] Obtain the dataset, and based on Ray, use the dataset to train the first student model and the first teacher model on the terminal device to obtain the distillation training method.
[0011] Upload the dataset and distillation training method to the cloud, set the number of parallel workers and hyperparameters, train the second student model and the second teacher model in the cloud using the dataset and distillation training method, obtain the trained second student model, and deploy the trained second student model to the terminal;
[0012] The distillation training method includes: during training, replicating the student model into a secondary student model, introducing a decoupling loss function, and updating the secondary student model; during testing, using the loss result of the secondary student model as a feedback signal to perform a meta-update on the teacher model; passing parameters of the meta-updated teacher model to the student model in the same training batch, and obtaining the distilled student model after iterative knowledge transfer.
[0013] Furthermore, the decoupling loss function is:
[0014] Decoupled loss = αTCKD + PNCKD
[0015] Wherein, TCKD is the target class decoupling loss function, NCKD is the non-target class decoupling loss function, and α and β are hyperparameters.
[0016] Furthermore, the teacher model is meta-updated by calculating the second derivative and performing a gradient update operation.
[0017] Furthermore, the process of training the first student model and the first teacher model on the terminal device based on Ray and using the dataset includes:
[0018] The dataset is divided into training and validation sets and loaded into the Ray dataset interface;
[0019] Based on the first student model and the first teacher model, a Ray-style model training method is configured on the terminal device. The TorchTrainer component in the Ray Train library is used to manage the training process. The training functions and parameters of the first student model and the first teacher model are defined in the TorchTrainer. After setting the number of parallel workers to 1, the training set is used to train the first student model and the first teacher model on the terminal device.
[0020] The first student model was validated using a validation set.
[0021] Furthermore, the process of uploading the dataset and distillation training method to the cloud, setting the number of parallel workers and hyperparameters, and training the second student model and the second teacher model on the cloud using the dataset and distillation training method includes:
[0022] The dataset and distillation training method are uploaded to the cloud, the dataset is divided into training set and validation set, and loaded into the Ray dataset interface;
[0023] Based on the second student model and the second teacher model, a Ray-based model training method is configured in the cloud. The TorchTrainer component in the Ray Train library is used to manage the training process. The training functions and parameters of the second student model and the second teacher model are defined in the TorchTrainer. The number of parallel workers in the TorchTrainer is modified. The training set is used to train the second student model and the second teacher model on the terminal device.
[0024] The second student model was validated using a validation set.
[0025] A second aspect of the present invention provides a Ray-based cloud-edge collaborative data analysis workflow optimization system.
[0026] The Ray-based cloud-edge collaborative data analysis workflow optimization system includes: terminal devices, cloud, and terminals, wherein the cloud is connected to the terminal devices and terminals.
[0027] The terminal device is used to train the first student model and the first teacher model based on Ray and using a dataset to obtain a distillation training method, and to upload the dataset and the distillation training method to the cloud.
[0028] The cloud is used to set the number of parallel workers and hyperparameters, train the second student model and the second teacher model using dataset and distillation training methods, obtain the trained second student model, and deploy the trained second student model to the terminal;
[0029] The distillation training method includes: during training, replicating the student model as a secondary student model, introducing a decoupling loss function, and updating the secondary student model; during testing, using the loss result of the secondary student model as a feedback signal to perform a meta-update on the teacher model; passing parameters of the meta-updated teacher model to the student model in the same training batch, and obtaining the distilled student model after iterative knowledge transfer.
[0030] Furthermore, the decoupling loss function is:
[0031] Decoupled loss = αTCKD + PNCKD
[0032] Wherein, TCKD is the target class decoupling loss function, NCKD is the non-target class decoupling loss function, and α and β are hyperparameters.
[0033] Furthermore, the teacher model is meta-updated by calculating the second derivative and performing a gradient update operation.
[0034] Furthermore, the process of training the first student model and the first teacher model on the terminal device based on Ray and using the dataset includes:
[0035] The dataset is divided into training and validation sets and loaded into the Ray dataset interface;
[0036] Based on the first student model and the first teacher model, a Ray-style model training method is configured on the terminal device. The TorchTrainer component in the Ray Train library is used to manage the training process. The training functions and parameters of the first student model and the first teacher model are defined in the TorchTrainer. After setting the number of parallel workers to 1, the training set is used to train the first student model and the first teacher model on the terminal device.
[0037] The first student model was validated using a validation set.
[0038] Furthermore, the process of uploading the dataset and distillation training method to the cloud, setting the number of parallel workers and hyperparameters, and training the second student model and the second teacher model on the cloud using the dataset and distillation training method includes:
[0039] The dataset and distillation training method are uploaded to the cloud, the dataset is divided into training set and validation set, and loaded into the Ray dataset interface;
[0040] Based on the second student model and the second teacher model, a Ray-based model training method is configured in the cloud. The TorchTrainer component in the Ray Train library is used to manage the training process. The training functions and parameters of the second student model and the second teacher model are defined in the TorchTrainer. The number of parallel workers in the TorchTrainer is modified. The training set is used to train the second student model and the second teacher model on the terminal device.
[0041] The second student model was validated using a validation set.
[0042] Compared with the prior art, the beneficial effects of the present invention are:
[0043] To address the challenges of extending model training code from terminal devices to the cloud, which typically requires significant code modifications and mastery of distributed training methods, presenting a high barrier to entry, this invention utilizes Ray to integrate distributed parallelism and memory sharing methods. This enables a seamless extension mechanism from terminal devices to the cloud, optimizes the adapter, and allows model training on terminal devices to be extended to the cloud with minimal code modifications. This improves model optimization efficiency and lowers the technical barrier.
[0044] To address the issue of large cloud-trained models being unable to be directly deployed to resource-constrained edge computing environments, this invention first utilizes knowledge distillation based on meta-learning. By leveraging feedback from the student model, the "teaching ability" of the teacher model is enhanced, allowing for better knowledge transfer from the teacher model to the student model. Simultaneously, the distillation loss is optimized by incorporating the concept of decoupled loss. The traditional knowledge distillation loss function is decoupled into Target Class Knowledge Distillation (TCKD) and Non-Target Class Knowledge Distillation (NCKD). TCKD conveys knowledge about the "difficulty" of training samples, describing the ease or difficulty of identifying each sample; NCKD conveys knowledge between non-target logic, highlighting non-target (dark target) knowledge. Normally, these two are coupled and mutually restrictive. Experiments show that NCKD is the primary cause of distillation improvement, but it is in a suppressive state. Therefore, the traditional knowledge distillation loss function is modified by introducing two hyperparameters to decouple the two parts, achieving a true improvement in knowledge distillation. This enhances the accuracy and precision of the learning model, enabling the distilled student model to be deployed to edge computing environments. Attached Figure Description
[0045] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0046] Figure 1 This is a flowchart of a Ray-based cloud-edge collaborative data analysis workflow optimization method.
[0047] Figure 2 A step diagram for seamlessly extending Ray-based data analysis workflows.
[0048] Figure 3 The diagram shows the algorithm for the decoupled knowledge distillation method based on meta-learning.
[0049] Figure 4 The flowchart shows the loss function for traditional knowledge distillation.
[0050] Figure 5 Flowchart of the knowledge distillation loss function for decoupling. Detailed Implementation
[0051] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0052] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0053] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.
[0054] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and systems according to various embodiments of this disclosure. It should be noted that each block in a flowchart or block diagram may represent a module, segment, or portion of code, which may include one or more executable instructions for implementing the logical functions specified in the various embodiments. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, may be implemented using a dedicated hardware-based system that performs the specified functions or operations, or using a combination of dedicated hardware and computer instructions.
[0055] Terminology Explanation:
[0056] Ray: Ray is an open-source framework for building distributed applications. It can be used for distributed computing, distributed storage, task scheduling, and resource management.
[0057] Data analysis workflow: The data analysis workflow mainly includes data acquisition and preprocessing, exploratory data analysis, feature extraction and transformation, model training and tuning, model evaluation and validation, and model deployment and application. The data analysis workflow is an iterative process that requires continuous optimization and improvement to gradually enhance the model's performance and accuracy.
[0058] Knowledge distillation (KD) is a model compression technique that reduces model size and computational cost by transferring knowledge from a large, complex neural network to a small, simple neural network, while maintaining comparable performance. The basic idea of knowledge distillation is to use the predictions (soft labels) of a complex model (teacher network) as the training target for another simple model (student network).
[0059] Meta-learning, which can be understood as "learning-to-learn," refers to the process of improving learning algorithms across multiple learning stages. In the basic learning process, the internal (or lower-level / basic) learning algorithm solves the task defined by the dataset and the objective. In the meta-learning process, the external (or upper-level / meta) algorithm updates the internal learning algorithm, causing its learned model to improve upon the external objective. The core idea of meta-learning is to learn prior knowledge.
[0060] Example 1
[0061] This embodiment provides a Ray-based cloud-edge collaborative data analysis workflow optimization method, which includes:
[0062] Obtain the dataset, and based on Ray, use the dataset to train the first student model and the first teacher model on the terminal device to obtain the distillation training method.
[0063] Upload the dataset and distillation training method to the cloud, set the number of parallel workers and hyperparameters, train the second student model and the second teacher model in the cloud using the dataset and distillation training method, obtain the trained second student model, and deploy the trained second student model to the terminal;
[0064] The distillation training method includes: during training, replicating the student model into a secondary student model, introducing a decoupling loss function, and updating the secondary student model; during testing, using the loss result of the secondary student model as a feedback signal to perform a meta-update on the teacher model; passing parameters of the meta-updated teacher model to the student model in the same training batch, and obtaining the distilled student model after iterative knowledge transfer.
[0065] First, this embodiment discloses the process of seamlessly extending the training code of the terminal device to the cloud for distributed training in the Ray-based data analysis workflow.
[0066] Specifically, such as Figure 1 , Figure 2 As shown:
[0067] Local design phase of the model:
[0068] Step 1: Obtain the dataset. Due to limitations in terminal device resources and computing power, it is difficult to process all the data in the dataset. Therefore, a portion of the dataset can be selected for processing on the terminal device. The selected portion of the dataset is then divided into a training set and a validation set in an 8:2 ratio. The data is then adjusted to dictionary format and loaded into the Ray dataset interface.
[0069] Step 2: Define the logical structure of the model, write Ray-style model training single-machine code on the terminal device, use the TorchTrainer component in the RayTrain library to manage the training process through the Trainer, define the model training function and set various hyperparameters in the TorchTrainer, and start the model training operation on the terminal device after setting the number of parallel workers to 1.
[0070] Step 3: After the model is trained, validate the model on the validation set to check the model's convergence status and whether the model is overfitting, and save the model state after training and testing.
[0071] Model cloud training phase:
[0072] Step 1: Remotely connect to the cloud server, upload the entire dataset to the cloud server, then divide the dataset into training and validation sets in an 8:2 ratio, adjust the data to dictionary format, and load it into the Ray dataset interface.
[0073] Step 2: Take the trained and tested single-machine code for the terminal device. No code modification is required. Simply change the number of parallel workers in TorchTrainer. Then, Trainer will automatically create backend processors to perform distributed training, thereby achieving seamless expansion from the terminal device to the cloud. Set various hyperparameters in TorchTrainer to start the cloud-based distributed training operation.
[0074] Step 3: After training, test the model on the validation set to check the model's convergence status and whether the model is overfitting, and save the model state after training and testing.
[0075] Secondly, this embodiment discloses a decoupled knowledge distillation generation method based on meta-learning.
[0076] Specifically, such as Figure 3 , Figure 4 , Figure 5 As shown:
[0077] Step 1: Acquire images, perform preprocessing operations on the images, and select a training set and a validation set.
[0078] In the image preprocessing stage, the pixel range of [0,255] in the original image is normalized or standardized to control the pixel values within [0,1], resulting in the processed image.
[0079] Step 2: After selecting the teacher model T and the student model S, perform the meta-learning knowledge distillation step.
[0080] Before performing meta-learning knowledge distillation, the teacher and student models are selected. In this experiment, ResNet32 is chosen as the teacher model and ResNet8 as the student model. Then, the knowledge distillation operation is performed: first, the student model S is copied to S', and after updating student S', the experimental student model S' is obtained (e.g., ...). Figure 2MetaDistil step 1). Then, sampling is performed in a separate quizset to verify the performance of the student model S' on the test samples. First, the cross-entropy loss function of the student model S' is calculated, and this cross-entropy loss result is used as a feedback signal. The gradient with respect to the parameters of the teacher model T is calculated, and then a gradient update operation is performed using gradient descent to perform a meta-update on T (e.g., ...). Figure 2 MetaDistil step 2); then discard the student model S', and the teacher model T, after meta-updation, passes parameters to the student model S in the same training batch (e.g., Figure 2 MetaDistil step 3 in the middle.
[0081] Step 3: Optimize the distillation loss function by using a decoupled distillation loss method.
[0082] This embodiment introduces a decoupling loss to improve the transfer of knowledge from the teacher model to the student model. Typically, in traditional knowledge distillation, the softmax output probability of the teacher model at temperature T is denoted as P. t T First, adjust the temperature to T = i, and the teacher model outputs the probability P. i T The student model fits the teacher model and the true value respectively, and the probability distribution P generated by the SoftMax function for the student model at temperature T=i corresponds to the student model. i S And the student model generates probability distribution P1 using the SoftMax function at temperature T=1. S When temperature T = i, the loss function between the teacher model and the student model is denoted as Loss_kd. When temperature T = 1, the loss function between the student model and the true value is denoted as Loss_cls. This process is as follows: Figure 3 As shown.
[0083] However, after decomposition, the target class and non-target classes are usually coupled, with knowledge being transferred between non-target classes, when the weight... In smaller cases, it can inhibit the transfer of knowledge.
[0084] Representing KD loss: TCKD represents the similarity of binary probabilities between teachers and students in the target class. NCKD represents the similarity of binary probabilities between teachers and students in non-target classes. KD loss can be expressed as:
[0085]
[0086] The classic KD loss is reformulated as a weighted sum of two independent parts. A decoupling loss is introduced, with two hyperparameters α and β dynamically adjusting the weights of the target class decoupling loss (TCKD) and the non-target class decoupling loss (NCKD). This process is as follows: Figure 4 As shown.
[0087] The decoupling loss function is:
[0088] Decoupled loss=αTCKD+βNCKD (2)
[0089] Step 4: Set the parameters and perform the training operation.
[0090] By setting various parameters, including optimizing the learning rate to 0.4, setting the weight decay to 5e-4, setting the momentum to 0.9, and setting parameters during the distillation process (distillation temperature to 4, distillation weights), the distilled student model is obtained after iterative knowledge transfer.
[0091] Example 2
[0092] This embodiment provides a Ray-based cloud-edge collaborative data analysis workflow optimization system.
[0093] The Ray-based cloud-edge collaborative data analysis workflow optimization system includes: terminal devices, cloud, and terminals, wherein the cloud is connected to the terminal devices and terminals.
[0094] The terminal device is used to train the first student model and the first teacher model based on Ray and using a dataset to obtain a distillation training method, and to upload the dataset and the distillation training method to the cloud.
[0095] The cloud is used to set the number of parallel workers and hyperparameters, train the second student model and the second teacher model using dataset and distillation training methods, obtain the trained second student model, and deploy the trained second student model to the terminal;
[0096] The distillation training method includes: during training, replicating the student model as a secondary student model, introducing a decoupling loss function, and updating the secondary student model; during testing, using the loss result of the secondary student model as a feedback signal to perform a meta-update on the teacher model; passing parameters of the meta-updated teacher model to the student model in the same training batch, and obtaining the distilled student model after iterative knowledge transfer.
[0097] Specifically, the terminal device first acquires the dataset and divides it into training and validation parts. Since the dataset is large and the entire dataset cannot be run locally, a portion of the dataset can be used for training during the local design phase. Then, the training logic structure of the model is designed, the model training code is written, hyperparameters are set, and training operations are started locally. After that, validation is performed, and the model state after training and validation is saved.
[0098] In the cloud, the dataset is divided into training and validation parts. The training code of the locally tested model does not need to be changed. Only the number of parallel workers needs to be set to automatically change it to a form suitable for distributed training. Hyperparameters are set for distributed training in the cloud. After training, the model is validated and then the state of the model after training and validation is saved.
[0099] Distillation training methods include:
[0100] Step 1: Select the dataset and divide it into a training set, a validation set, a teacher model T, and a student model S;
[0101] Step 2: During the training process, the student model S is first copied to S', and a decoupling loss function is introduced. The student model S' is then updated on the training dataset by calculating the decoupling loss function and applying the backpropagation algorithm. This step is called the "teaching experiment". The student model S' is obtained after the "teaching experiment".
[0102] Step 3: Sample in the quiz set to verify the performance of the student model S' in the test samples. Use the loss result as a feedback signal to perform a gradient update operation by calculating the second derivative and using the gradient descent method to update the teacher model.
[0103] Step 4: Discard the student model S', and pass the parameters of the teacher model that has undergone meta-update to the student model S in the same training batch.
[0104] Step 5: After iterative knowledge transfer, the distilled student model is obtained, which will be deployed and released through Ray's data analysis workflow.
[0105] Before deployment on the terminal, the terminal first needs to deploy the model's operating environment and register various information of the terminal on the cloud server. Since the model trained in the cloud has a large number of parameters and computational load, and the terminal has limited resources and cannot be deployed, an optimized model compression method is used to compress the large model trained in the cloud to obtain a small model, and then the small model is deployed to the terminal device with limited resources with one click.
[0106] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A Ray-based cloud-edge collaborative data analysis workflow optimization method, characterized in that, include: Obtain the dataset, and based on Ray, use the dataset to train the first student model and the first teacher model on the terminal device to obtain the distillation training method. The process of training the first student model and the first teacher model on the terminal device using the dataset includes: The dataset is divided into training and validation sets and loaded into the Ray dataset interface; Based on the first student model and the first teacher model, a Ray-style model training method is configured on the terminal device. The TorchTrainer component in the Ray Train library is used to manage the training process. The training functions and parameters of the first student model and the first teacher model are defined in the TorchTrainer. The training set is used to train the first student model and the first teacher model on the terminal device. The first student model was validated using a validation set; Upload the dataset and distillation training method to the cloud, set the number of parallel workers and hyperparameters, train the second student model and the second teacher model in the cloud using the dataset and distillation training method, obtain the trained second student model, and deploy the trained second student model to the terminal; The process of training a second student model and a second teacher model in the cloud using datasets and distillation training methods includes: The dataset and distillation training method are uploaded to the cloud, the dataset is divided into training set and validation set, and loaded into the Ray dataset interface; Based on the second student model and the second teacher model, a Ray-based model training method is configured in the cloud. The TorchTrainer component in the RayTrain library is used to manage the training process. The training functions and parameters of the second student model and the second teacher model are defined in the TorchTrainer. The number of parallel workers in the TorchTrainer is modified. The training set is used to train the second student model and the second teacher model on the terminal device. The second student model was validated using a validation set; The distillation training method includes: during training, replicating the student model into a secondary student model, introducing a decoupling loss function, and updating the secondary student model; during testing, using the loss result of the secondary student model as a feedback signal to perform a meta-update on the teacher model; passing parameters of the meta-updated teacher model to the student model in the same training batch, and obtaining the distilled student model after iterative knowledge transfer.
2. The Ray-based cloud-edge collaborative data analysis workflow optimization method according to claim 1, characterized in that, The decoupling loss function is: Where TCKD is the target class decoupling loss function, NCKD is the non-target class decoupling loss function, and α and β are hyperparameters.
3. The Ray-based cloud-edge collaborative data analysis workflow optimization method according to claim 1, characterized in that, The teacher model is meta-updated by calculating the second derivative and performing gradient update operations.
4. A cloud-edge collaborative data analysis workflow optimization system based on Ray, characterized in that: include: The terminal device, the cloud, and the terminal, wherein the cloud is connected to the terminal device and the terminal; The terminal device is used to train the first student model and the first teacher model based on Ray and using a dataset to obtain the distillation training method, and upload the dataset and the distillation training method to the cloud. The process of training the first student model and the first teacher model on the terminal device using the dataset includes: The dataset is divided into training and validation sets and loaded into the Ray dataset interface; Based on the first student model and the first teacher model, a Ray-style model training method is configured on the terminal device. The TorchTrainer component in the Ray Train library is used to manage the training process. The training functions and parameters of the first student model and the first teacher model are defined in the TorchTrainer. The training set is used to train the first student model and the first teacher model on the terminal device. The first student model was validated using a validation set; The cloud is used to set the number of parallel workers and hyperparameters, train the second student model and the second teacher model using dataset and distillation training methods, obtain the trained second student model, and deploy the trained second student model to the terminal; The process of training a second student model and a second teacher model in the cloud using datasets and distillation training methods includes: The dataset and distillation training method are uploaded to the cloud, the dataset is divided into training set and validation set, and loaded into the Ray dataset interface; Based on the second student model and the second teacher model, a Ray-based model training method is configured in the cloud. The TorchTrainer component in the RayTrain library is used to manage the training process. The training functions and parameters of the second student model and the second teacher model are defined in the TorchTrainer. The number of parallel workers in the TorchTrainer is modified. The training set is used to train the second student model and the second teacher model on the terminal device. The second student model was validated using a validation set; The distillation training method includes: during training, replicating the student model as a secondary student model, introducing a decoupling loss function, and updating the secondary student model; during testing, using the loss result of the secondary student model as a feedback signal to perform a meta-update on the teacher model; passing parameters of the meta-updated teacher model to the student model in the same training batch, and obtaining the distilled student model after iterative knowledge transfer.
5. The Ray-based cloud-edge collaborative data analysis workflow optimization system according to claim 4, characterized in that, The decoupling loss function is: Where TCKD is the target class decoupling loss function, NCKD is the non-target class decoupling loss function, and α and β are hyperparameters.
6. The Ray-based cloud-edge collaborative data analysis workflow optimization system according to claim 4, characterized in that, The teacher model is meta-updated by calculating the second derivative and performing gradient update operations.