Large and small model mutual distillation method and system for autonomous traffic end-side cloud collaboration

Abstract

The invention discloses a large and small model mutual distillation method and system oriented to autonomous traffic end-side cloud collaboration, and relates to the technical field of intelligent traffic, and the method comprises the steps: completing task initialization according to equipment characteristics and scene demands in a system; according to task initialization, issuing and deploying a global small model, and executing local model training of edge-end side equipment; according to the training result of the local model, fused knowledge distillation on the cloud server is executed, and the cloud large model is finely adjusted; according to the cloud big model after fine tuning, executing guidance type knowledge distillation on the cloud server, and updating a global small model; judging whether a training termination condition is met or not, and if not, continuing training; and if yes, obtaining a performance bidirectional optimization large and small model. Through bidirectional knowledge distillation, mutual enhancement of the cloud large model and the terminal small model is realized, and a collaborative optimization mechanism of bidirectional distillation can enable the precision of each model to be superior to the effect of independent training.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, in particular to a method and system for mutual distillation of large and small models for autonomous transportation end-edge-cloud collaboration. Background Art

[0002] With the rapid development of artificial intelligence, deep learning, and big data technologies, autonomous transportation systems have gradually moved from conceptual exploration to practical application. Autonomous transportation systems are the next generation of intelligent transportation systems. Their core lies in the highly autonomous and collaborative nature of vehicles and road infrastructure. They enable real-time data collection, environmental perception, knowledge learning, and decision-making feedback in high-speed, dynamic environments to improve traffic efficiency and safety. In autonomous transportation systems, various traffic elements are interconnected to form a complex, multi-element collaborative system. Technologies such as edge computing are leveraged to bring AI capabilities down to the edge, improving system real-time performance and reliability to meet the demands of intelligent, autonomous, and sustainable urban transportation.

[0003] To support the high real-time and high reliability requirements of autonomous transportation systems, a three-layer collaborative architecture, end-edge-cloud, is becoming the mainstream solution. In this architecture, the end layer (such as autonomous vehicles and their onboard sensors) is primarily responsible for environmental data collection and local perception. The edge layer (such as roadside units (RSUs) or mobile edge computing servers) performs preliminary aggregation and processing of data and models from nearby terminals, providing low-latency services. The cloud layer deploys large-scale deep learning models, leveraging its powerful computing power for global model training and data analysis, providing decision support for the entire system. These layers collaborate through efficient communication and data exchange. This architecture combines the powerful computing power of the cloud with the real-time performance of the edge, effectively meeting the large-scale data processing and real-time perception requirements of autonomous transportation systems. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a method and system for mutual distillation of large and small models for autonomous transportation end-edge-cloud collaboration, so as to provide a model learning and collaborative optimization solution that supports the end-edge-cloud architecture within the autonomous transportation system, integrate computing power, data and other resources, and realize knowledge sharing and performance improvement among higher levels of the system.

[0005] One aspect of an embodiment of the present application provides a large-small model mutual distillation method for autonomous transportation end-edge-cloud collaboration, the method comprising the following steps:

[0006] Complete task initialization based on the device characteristics and scenario requirements within the system;

[0007] Initialize the task, issue and deploy a global small model, and perform local model training on edge devices.

[0008] Based on the results of the local model training, perform fusion knowledge distillation on the cloud server to fine-tune the cloud-based large model;

[0009] Based on the fine-tuned cloud-based large model, perform guided knowledge distillation on the cloud server to update the global small model;

[0010] Determine whether the training termination condition is met. If the training termination condition is not met, return to the steps of initializing according to the task, issuing and deploying the global small model, and executing the local model training of the edge-side device until the training termination condition is met; if the training termination condition is met, obtain the large and small models with bidirectional performance optimization.

[0011] In some embodiments, completing task initialization based on device characteristics and scenario requirements within the system includes the following steps:

[0012] Select a pre-trained large model based on the nature of the task and scenario requirements, and deploy it on a cloud server;

[0013] Based on the pre-trained large model, pruning and quantization are used to generate a small model that is adaptable to edge devices, and the parameters of the small model are initialized to serve as the global small model for the first round of training;

[0014] Create unlabeled proxy datasets based on public data;

[0015] Set training hyperparameters, including but not limited to learning rate, loss function, and training rounds.

[0016] In some embodiments, initializing the task, issuing and deploying a global small model, and performing local model training on an edge device include the following steps:

[0017] The cloud server selects the target edge device as the training client and delegates the task configuration and the current global small model parameters;

[0018] The training client downloads the task configuration and current global small model parameters as needed, and deploys the model training environment and the local small model;

[0019] The training client extracts a data subset suitable for the corresponding task based on the locally sensed and stored data, and trains and updates the local small model;

[0020] The training client extracts the updated local small model parameters and uploads them to the cloud server.

[0021] In some embodiments, performing fusion-based knowledge distillation on a cloud server based on the results of the local model training to fine-tune the cloud-based large model includes the following steps:

[0022] Collect local small model parameters uploaded by each training client;

[0023] Adopting a federated learning aggregation method to aggregate the collected parameters of multiple local small models and update the global small model. The federated learning aggregation method includes but is not limited to the FedAvg algorithm, the FedProx algorithm, or the MOON algorithm;

[0024] Based on the updated global small model, a fusion knowledge distillation method is used to fine-tune the cloud-based large model.

[0025] In some embodiments, the fusion-based knowledge distillation on the cloud server includes the following steps:

[0026] Define the main body of knowledge distillation; the global small model serves as the teacher model and the large model serves as the student model;

[0027] Randomly sample a subset based on the proxy data set to obtain a proxy data subset;

[0028] Calculating the main loss and the self-distillation loss based on the proxy data subset;

[0029] Calculating the total loss of fusion knowledge distillation according to the main loss and the self-distillation loss;

[0030] Based on the total loss of the fusion knowledge distillation, a preset optimizer and fine-tuning method are used to update the cloud-based large model.

[0031] In some embodiments, performing guided knowledge distillation on the cloud server based on the fine-tuned cloud-based large model to update the global small model includes the following steps:

[0032] Define the main body of knowledge distillation; the large model serves as the teacher model, the global small model serves as the student model, and the knowledge of the large model is extracted to update the global small model;

[0033] Randomly sample a subset based on the proxy data set to obtain a proxy data subset;

[0034] Clone the current global small model;

[0035] Calculating the output layer loss and the hidden layer loss based on the proxy data subset;

[0036] Calculating the total loss of guided knowledge distillation according to the output layer loss and the hidden layer loss;

[0037] updating the cloned global small model according to the total loss of the guided knowledge distillation;

[0038] Based on the updated cloned global small model, a meta-gradient is calculated, and the global small model is updated with the meta-gradient.

[0039] In some embodiments, the determining whether a training termination condition is met, if not, returning to the step of initializing the task, issuing and deploying a global small model, and executing local model training on edge devices until the training termination condition is met; if the training termination condition is met, obtaining a large and small model with bidirectional performance optimization, includes the following steps:

[0040] Check whether the current conditions meet the training termination conditions, the current conditions including but not limited to the training round, the large model performance index and the global small model performance index;

[0041] If the current condition meets the training termination condition, each client is notified of the training termination, and the fine-tuned large model and the trained global small model are converted into available services;

[0042] If the current conditions do not meet the training termination conditions, each client is notified to continue training, and the process returns to the steps of initializing the task, issuing and deploying the global small model, and executing the local model training of the edge device until the training termination conditions are met.

[0043] Another aspect of the present application further provides a large-small model mutual distillation system for autonomous transportation end-edge-cloud collaboration, the system comprising:

[0044] The first module is used to complete task initialization based on the device characteristics and scenario requirements within the system;

[0045] The second module is used to initialize the task, issue and deploy the global small model, and perform local model training on the edge device;

[0046] The third module is used to perform fusion knowledge distillation on the cloud server based on the results of the local model training and fine-tune the cloud large model;

[0047] The fourth module is used to perform guided knowledge distillation on the cloud server based on the fine-tuned cloud-based large model to update the global small model;

[0048] The fifth module is used to determine whether the training termination conditions are met. If the training termination conditions are not met, the module returns to the steps of initializing the task, issuing and deploying the global small model, and executing the local model training of the edge-side device until the training termination conditions are met. If the training termination conditions are met, a large and small model with bidirectional performance optimization is obtained.

[0049] Another aspect of the embodiments of the present application further provides an electronic device, including a processor and a memory;

[0050] The memory is used to store programs;

[0051] The processor executes the program to implement any of the above methods.

[0052] Another aspect of the embodiments of the present application further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement any of the above methods.

[0053] This application has at least the following beneficial effects:

[0054] This application can complete task initialization based on the device characteristics and scenario requirements within the system; according to task initialization, issue and deploy a global small model, and perform local model training on edge devices; according to the results of local model training, execute fusion knowledge distillation on the cloud server and fine-tune the cloud large model; according to the fine-tuned cloud large model, execute guided knowledge distillation on the cloud server and update the global small model; determine whether the training termination condition is met, and if not, return to the steps of issuing and deploying a global small model according to task initialization, and executing local model training on edge devices until the training termination condition is met; if the training termination condition is met, a large and small model with bidirectional performance optimization is obtained. This application realizes mutual enhancement of the cloud large model and the terminal small model through bidirectional knowledge distillation; the fusion distillation stage enables the large model to fuse information from multiple terminals to improve global performance; the guided distillation stage allows the terminal model to obtain the guiding knowledge of the large model and improve local reasoning accuracy; the collaborative optimization mechanism of bidirectional distillation can make the accuracy of each model better than the effect of individual training. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0056] Figure 1 A flow chart of a large-small model mutual distillation method for autonomous transportation end-edge-cloud collaboration provided in an embodiment of the present application;

[0057] Figure 2 An example flow chart of a large-small model mutual distillation method for autonomous transportation end-edge-cloud collaboration provided in an embodiment of the present application;

[0058] Figure 3 A schematic diagram of the large and small model inter-distillation method provided in an embodiment of the present application;

[0059] Figure 4 A result diagram of driver distraction detection provided by an embodiment of the present application;

[0060] Figure 5 This is a structural block diagram of the large and small model mutual distillation system for autonomous transportation end-edge-cloud collaboration provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0062] Before describing the embodiments of the present application in detail, some of the related technologies involved in the embodiments of the present application are first described as follows:

[0063] Explanation of terms:

[0064] 1. Autonomous Transportation Systems: The emerging trends in my country's transportation sector, characterized by surging passenger and freight demand, massive infrastructure and facilities development, and mutually reinforcing innovation in technology and services, have necessitated the development of autonomous transportation systems (ATS). Driven by emerging technologies such as big data, autonomous driving, vehicle-infrastructure collaboration, 5G, and Beidou navigation and positioning, these systems aim to be safer, more efficient, more convenient, greener, and more economical. Based on the business logic of autonomous perception, learning, decision-making, and response, they transport people and goods through self-organizing operations and autonomous services. Essentially, they aim to reduce human intervention in the transportation system and enhance its autonomy. These capabilities are primarily reflected in four key areas: proactive response to traffic demand, automated operation of transportation vehicles, proactive infrastructure management and control, and proactive adaptation to the external environment.

[0065] 2. End: Terminal devices primarily refer to the intelligent terminals deployed in vehicles, unmanned vehicles, or other transportation vehicles within autonomous transportation systems. These devices typically perform sensor acquisition, data preprocessing, and preliminary decision-making. However, due to hardware resource limitations, their computing power and storage capacity are far inferior to those of cloud servers. End devices play a role in data collection and local model updates within global model collaboration, and are also key nodes for real-time decision-making.

[0066] 3. Edge: Edge devices refer to edge computing platforms deployed near data generation locations, such as transportation hubs, roadsides, and traffic control centers. Edge servers possess higher computing power and storage resources than end devices. They can aggregate, preprocess, and perform local analysis on data from various terminals, and provide low-latency data transmission and coordinated scheduling with the cloud. In the edge-cloud architecture, edge nodes serve as the middle layer, carrying out important tasks such as data relay, preprocessing, caching, and secure encrypted transmission.

[0067] 4. Cloud: Cloud servers refer to central servers or data centers with massive storage and powerful computing capabilities. Cloud platforms run large-scale deep learning models for comprehensive global data analysis, knowledge extraction, and global model optimization. By collaborating with devices and edges, the cloud can conduct centralized training based on model parameters and local data uploaded by each terminal, continuously optimizing overall system performance.

[0068] 5. Large Model: This generally refers to a deep neural network model with a large number of parameters, complex structure, and superior performance. In this application, the large model is primarily deployed in the cloud and is responsible for capturing deep features and general knowledge from global data. The large model not only guides knowledge distillation for local models but also serves as the knowledge base for the entire system. Its superior generalization capabilities are key to achieving high-precision decision-making.

[0069] 6. Small Models: These refer to deep learning models designed to be lightweight and adaptable to resource-constrained environments. Small models typically have fewer parameters and require less computation, making them suitable for deployment on end devices such as vehicles. Despite their limited computing power, a bidirectional knowledge distillation mechanism with the large model can effectively improve the decision accuracy and robustness of small models, meeting the real-time and reliability requirements of autonomous transportation systems.

[0070] 7. Federated Learning: This is a distributed machine learning method whose core concept is to enable all devices to jointly train a global model without directly transmitting raw data. Each device trains the model locally using private data and only uploads model parameters or gradient information. The cloud aggregates and updates the global model, thus leveraging the diversity of data from each device while ensuring data privacy. In this paper, federated learning is used as an overall framework to achieve coordinated updates of models at the device, edge, and cloud levels.

[0071] 8. Knowledge distillation: This is a model compression and knowledge transfer technique that transfers soft labels or intermediate features from a teacher model to a student model, enabling the student model to achieve performance close to that of the teacher model with a lower number of parameters. During knowledge distillation, loss functions such as KL divergence and mean squared error are commonly used to measure the difference between the outputs of the teacher and student models.

[0072] Disadvantages of existing technology:

[0073] 1. Data silos and privacy protection challenges: Traditional centralized approaches require collecting data from multiple terminals for centralized processing to break down data silos. However, with increasing concerns about data security and user privacy, sensitive and valuable data used to train AI models has become difficult to obtain, making traditional solutions unfeasible and less practical for supporting intelligent services within autonomous transportation systems.

[0074] 2. Difficulty in applying and implementing large models: In autonomous transportation systems, achieving high-precision, highly autonomous services typically requires complex deep learning models. However, the limited computing power of onboard terminals and edge devices prevents the direct deployment and training of large models, while simple, lightweight models may not be able to meet diverse application requirements.

[0075] 3. Heterogeneous device adaptation: The computing power, storage capacity, and local data distribution of different nodes within autonomous transportation systems vary significantly. Traditional solutions assume that clients have homogeneous capabilities and data distribution, making it difficult to achieve unified training across heterogeneous environments, thus impacting training effectiveness.

[0076] The embodiment of the present application provides a method and system for the mutual distillation of large and small models for autonomous transportation end-edge-cloud collaboration, which relates to the field of intelligent transportation technology. The embodiment of the present application provides a method for the mutual distillation of large and small models for autonomous transportation end-edge-cloud collaboration architecture, which needs to be applied to terminals and servers at all levels, and can also be software running in terminals or servers. In some embodiments, the terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, and vehicle-mounted terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the method for the mutual distillation of large and small models for autonomous transportation end-edge-cloud collaboration architecture, etc., but is not limited to the above forms.

[0077] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0078] To address the challenges of existing technologies, this application proposes a method for cross-distillation of large and small models for autonomous transportation end-edge-cloud collaboration. This method builds a three-layer end-edge-cloud collaborative architecture and utilizes a bidirectional knowledge distillation mechanism between the client-side small model and the server-side large model to achieve collaborative training and optimization of large and small models, thereby jointly building an efficient end-edge-cloud collaborative system.

[0079] Reference Figure 1 The embodiment of the present application provides a large-small model mutual distillation method for autonomous transportation end-edge-cloud collaboration, which specifically includes the following steps S100 to S140:

[0080] S100: Complete task initialization based on the device characteristics and scenario requirements within the system;

[0081] S110: According to the task initialization, a global small model is issued and deployed, and local model training is performed on the edge device;

[0082] S120: Execute fusion knowledge distillation on the cloud server according to the result of the local model training to fine-tune the cloud large model;

[0083] S130: Execute guided knowledge distillation on the cloud server based on the fine-tuned cloud-based large model to update the global small model;

[0084] S140: Determine whether the training termination condition is met. If the training termination condition is not met, return to the step of initializing according to the task, issuing and deploying the global small model, and executing the local model training of the edge-side device until the training termination condition is met. If the training termination condition is met, a large and small model with bidirectional performance optimization is obtained.

[0085] Optionally, completing task initialization according to device characteristics and scenario requirements in the system includes the following steps:

[0086] Select a pre-trained large model based on the nature of the task and scenario requirements, and deploy it on a cloud server;

[0087] Based on the pre-trained large model, pruning and quantization are used to generate a small model that is adaptable to edge devices, and the parameters of the small model are initialized to serve as the global small model for the first round of training;

[0088] Create unlabeled proxy datasets based on public data;

[0089] Set training hyperparameters, including but not limited to learning rate, loss function, and training rounds.

[0090] Optionally, the initializing the task, issuing and deploying a global small model, and performing local model training on an edge device include the following steps:

[0091] The cloud server selects the target edge device as the training client and delegates the task configuration and the current global small model parameters;

[0092] The training client downloads the task configuration and current global small model parameters as needed, and deploys the model training environment and the local small model;

[0093] The training client extracts a data subset suitable for the corresponding task based on the locally sensed and stored data, and trains and updates the local small model;

[0094] The training client extracts the updated local small model parameters and uploads them to the cloud server.

[0095] Optionally, performing fusion-type knowledge distillation on the cloud server based on the result of the local model training to fine-tune the cloud-based large model includes the following steps:

[0096] Collect local small model parameters uploaded by each training client;

[0097] Adopting a federated learning aggregation method to aggregate the collected parameters of multiple local small models and update the global small model. The federated learning aggregation method includes but is not limited to the FedAvg algorithm, the FedProx algorithm, or the MOON algorithm;

[0098] Based on the updated global small model, a fusion knowledge distillation method is used to fine-tune the cloud-based large model.

[0099] Optionally, the fusion-based knowledge distillation on the cloud server includes the following steps:

[0100] Define the main body of knowledge distillation; the global small model serves as the teacher model and the large model serves as the student model;

[0101] Randomly sample a subset based on the proxy data set to obtain a proxy data subset;

[0102] Calculating the main loss and the self-distillation loss based on the proxy data subset;

[0103] Calculating the total loss of fusion knowledge distillation according to the main loss and the self-distillation loss;

[0104] Based on the total loss of the fusion knowledge distillation, a preset optimizer and fine-tuning method are used to update the cloud-based large model.

[0105] Furthermore, the calculation formula of the main loss includes:

[0106] ;

[0107] in, Indicates the main loss; Represents the KL divergence calculation function; Represents the Softmax calculation function; A sample of proxy data representing the input; represents the global small model parameters; represents the large model parameters; represents the distillation temperature; Represents the corresponding model output.

[0108] Furthermore, the calculation formula for the self-distillation loss includes:

[0109] ;

[0110] in, represents the loss from distillation; Represents the large model parameters of the previous round.

[0111] Furthermore, the calculation formula for the total loss of fusion knowledge distillation includes:

[0112] ;

[0113] in, represents the total loss of fusion knowledge distillation; Represents a self-distillation coefficient greater than 0.

[0114] Optionally, performing guided knowledge distillation on the cloud server based on the fine-tuned cloud-based large model to update the global small model includes the following steps:

[0115] Define the main body of knowledge distillation; the large model serves as the teacher model, the global small model serves as the student model, and the knowledge of the large model is extracted to update the global small model;

[0116] Randomly sample a subset based on the proxy data set to obtain a proxy data subset;

[0117] Clone the current global small model;

[0118] Calculating the output layer loss and the hidden layer loss based on the proxy data subset;

[0119] Calculating the total loss of guided knowledge distillation according to the output layer loss and the hidden layer loss;

[0120] updating the cloned global small model according to the total loss of the guided knowledge distillation;

[0121] Based on the updated cloned global small model, a meta-gradient is calculated, and the global small model is updated with the meta-gradient.

[0122] Furthermore, the calculation formula of the output layer loss includes:

[0123] ;

[0124] in, represents the output layer loss; Represents the global small model parameters of the clone.

[0125] Furthermore, the calculation formula of hidden layer loss includes:

[0126] ;

[0127] in, represents the hidden layer loss; Represents the mean square error calculation function; Represents the front of the cloned global small model Layer parameters; Representing the front of the large model Layer parameters; is a bridging matrix for dimension alignment.

[0128] Furthermore, the calculation formula for the total loss of guided knowledge distillation includes:

[0129] ;

[0130] in, represents the total loss of guided knowledge distillation; Represents a hidden layer coefficient greater than 0.

[0131] Furthermore, the calculation formula for updating the global small model with meta-gradient includes:

[0132] ;

[0133] in, Represents the updated global small model parameters; represents the learning rate; Represents the parameters of the cloned global small model after updating based on the total loss of guided knowledge distillation.

[0134] Optionally, the determining whether a training termination condition is met, if not, returning to the step of initializing the task, issuing and deploying a global small model, and executing local model training on edge devices until the training termination condition is met; if the training termination condition is met, obtaining a large and small model with bidirectional performance optimization, includes the following steps:

[0135] Check whether the current conditions meet the training termination conditions, the current conditions including but not limited to the training round, the large model performance index and the global small model performance index;

[0136] If the current condition meets the training termination condition, each client is notified of the training termination, and the fine-tuned large model and the trained global small model are converted into available services;

[0137] If the current conditions do not meet the training termination conditions, each client is notified to continue training, and the process returns to the steps of initializing the task, issuing and deploying the global small model, and executing the local model training of the edge device until the training termination conditions are met.

[0138] Next, the solution of the embodiment of the present application will be introduced and explained in detail with reference to specific application examples.

[0139] refer to Figure 2 The large-small model mutual distillation method for autonomous transportation end-edge-cloud collaboration provided in this embodiment includes the following steps:

[0140] S1: Analyze the nature of the task and the requirements of the scenario in which it is located, statistically analyze the characteristics of the devices in the system that can participate in the training task, prepare pre-trained large models and corresponding small models, create proxy datasets, and set hyperparameters such as learning rate, loss function, and training rounds;

[0141] S2: The cloud server selects a suitable target edge device as the training client, delegates the task configuration and the current global small model parameters, configures the task on the training client, deploys the local small model, trains the local small model based on local data, and uploads the model parameters to the cloud server.

[0142] S3: The cloud server collects and aggregates the parameters of the local small models uploaded by each training client. It uses a fusion-based knowledge distillation method, using the global small model as the teacher model and the large model as the student model. This method encourages the large model to imitate the behavior of the global small model and fine-tune the cloud-based large model.

[0143] S4: Based on the fine-tuned cloud-based large model in the previous step, a guided knowledge distillation method is used, with the large model as the teacher model and the global small model as the student model, to extract knowledge from the large model to update the global small model;

[0144] S5: Check whether the current situation meets the training termination conditions. If the training termination conditions are met, notify each client to terminate the training, and convert the fine-tuned large model and the trained global small model into available services; if the training termination conditions are not met, notify each client to continue training, that is, initialize according to the task, send and deploy the global small model, and execute local model training of the edge device.

[0145] like Figure 3As shown, this embodiment innovatively proposes a method for mutual distillation of large and small models. First, in the fusion knowledge distillation stage, the global small model is used as the teacher and the large model as the student. The proxy data is used to calculate the main loss and self-distillation loss. The weighted sum is then used to fine-tune the large model to enable it to absorb multi-source heterogeneous local information on the edge. Then, in the guided distillation knowledge distillation stage, the fine-tuned large model is used as the teacher and the global small model as the student. The proxy data is again used to calculate the output layer loss and hidden layer loss, and meta-learning is combined to update the global small model. Through these two closed-loop iterations, bidirectional knowledge sharing is achieved between the large and small models between the cloud and the edge, achieving complementary improvements in model generalization and personalized performance.

[0146] To summarize, this embodiment proposes a mutual distillation solution for large and small models for the autonomous transportation end-edge-cloud collaborative architecture. It realizes the collaborative training and iterative optimization of large cloud models and small edge models by constructing a three-layer end-edge-cloud collaborative system and combining fusion-type and guidance-type bidirectional knowledge distillation. It not only ensures the real-time performance and privacy security of edge-side devices, but also gives full play to the deep expression ability of cloud models, thereby achieving synchronous improvement in overall system performance and convergence speed.

[0147] Compared to existing technologies, this embodiment's large-to-small model mutual distillation method eliminates the bottleneck in knowledge transfer between large and small models, enabling resource-constrained edge devices to continuously benefit from the global generalization capabilities of the large cloud model. Meanwhile, the large cloud model can dynamically absorb diverse local knowledge from the edge. Its zero-end-side modification and compatibility with multiple federated aggregation algorithms and distillation strategies provide this method with enhanced privacy protection, scalability, and deployment flexibility, significantly enhancing the practical value and promotion potential of autonomous transportation systems in complex scenarios.

[0148] In the context of the Internet of Vehicles (IoV), taking the specific implementation case of driver distraction detection as an example, multiple autonomous vehicles are driving on urban roads, working in coordination with nearby roadside units and a cloud center. The implementation process of this embodiment is as follows:

[0149] S1: Scenario Modeling. Each vehicle is equipped with one or more high-definition cameras that capture real-time video streams of the driver's face and upper body. The driver is tested for distracted driving, with various types of distraction designed, including but not limited to making phone calls, chatting, and operating a mobile phone. The vehicle is connected to the roadside unit deployed on the roadside and the cloud center deployed on the central server through the on-board module.

[0150] S2: Task configuration. Use the public dataset SFDDD to simulate the vehicle camera acquisition environment; select the pre-trained ViT-L / 16 as the cloud-based large model and MobileNetV2 as the lightweight small model to be deployed on the client; set hyperparameters such as learning rate, distillation temperature, and training rounds;

[0151] S3: Model training. Each autonomous vehicle acts as a client, downloads and deploys the current global small model, trains it using the distraction detection samples it has collected, and uploads the model parameters. The cloud server collects the parameters of each client's local small model, uses the federated learning aggregation algorithm to update the global small model, and then uses fusion knowledge distillation and guided knowledge distillation to collaboratively optimize the cloud-based large model and the global small model. Iterate until the termination condition is reached.

[0152] S4: Model Deployment. The large model is retained in the cloud center for offline periodic analysis and updates. The final global small model is distributed to each autonomous vehicle and deployed in the onboard inference unit to run distraction detection locally. Bidirectional distillation is re-performed in the cloud as planned, and new models are distributed regularly to ensure continuous optimization and adaptability of the system during long-term deployment.

[0153] In order to conduct a comprehensive and fair comparison, the optimization scheme proposed in this application is tested under three federated learning aggregation algorithms, including FedAvg, FedProx, and MOON. The test indicators are distraction detection accuracy. The driver distraction detection results are as follows: Figure 4 As shown, the optimization effect brought by the method of this embodiment is illustrated, and the effectiveness of the method is proved.

[0154] In summary, this embodiment includes the following key technical solutions:

[0155] 1. Supporting Collaborative Optimization of the End-Edge-Cloud Architecture for Autonomous Transportation: This application constructs a three-layer architecture that closely collaborates: device terminals, roadside edge devices, and the cloud. The end-side is responsible for real-time perception and local lightweight model inference and training. The edge roadside unit is responsible for relaying multi-vehicle data, initial aggregation, and accelerated processing of latency-sensitive tasks. The cloud center concentrates on distillation and fine-tuning of large models and updating global small models. The three-layer collaboration enables on-demand resource scheduling, dynamic task offloading, and integrated sensing and computing, ensuring the real-time performance of the system while maximizing the utilization of computing power at all levels.

[0156] 2. Bidirectional knowledge distillation mechanism for large and small models based on proxy datasets and meta-learning: This mechanism leverages unlabeled public or synthetic proxy data in the cloud to drive bidirectional distillation. First, through "fusion distillation," the large cloud model absorbs the local knowledge aggregated by the small edge models. Then, through "guided distillation," meta-gradients are calculated using meta-learning, incorporating deep semantic cues from the large model into the global small model. This mechanism balances differences in model capacity and heterogeneous data distribution, using proxy data as a bridge and meta-gradients as a link to achieve complementary improvements and rapid convergence between large and small models.

[0157] 3. Stable support and continuous updates for real-time services: This method only requires the deployment of a lightweight, pruned or quantized small model on the device side, ensuring low-latency, high-throughput online inference capabilities. At the same time, the device, edge, and cloud periodically collaborate to update the model, using actual scenario data reported by edge devices to continuously supplement the proxy set and iterate bidirectional distillation, forming a closed-loop continuous optimization process. This ensures real-time and stable services while continuously improving the system's generalization performance in new scenarios and diverse driving behaviors.

[0158] The beneficial effects of this embodiment are as follows:

[0159] 1. Enhanced privacy protection: This embodiment only exchanges model information or soft labels between the client and the server, and does not upload the original training data. The private data of system nodes such as vehicles is always retained locally on the device, effectively avoiding the risk of privacy leakage and meeting the privacy protection requirements of autonomous transportation systems.

[0160] 2. Improved Model Performance: This embodiment utilizes bidirectional knowledge distillation to achieve mutual enhancement between the large cloud model and the small terminal model. The fusion distillation phase integrates information from multiple terminals into the large model, improving overall performance. The guided distillation phase allows the terminal model to obtain guidance from the large model, improving local inference accuracy. The collaborative optimization mechanism of bidirectional distillation improves the accuracy of each model compared to the results of individual training.

[0161] 3. Adapting to Heterogeneous Devices: This embodiment allows for the deployment of models of varying sizes based on the computing power of edge devices. Resource-constrained terminals can run small, lightweight models, while the cloud can run large, high-precision models, each fulfilling its own specific tasks. This heterogeneous model collaboration avoids the problem of a single model being unable to adapt to the needs of diverse devices, thereby increasing the system's flexibility and applicability.

[0162] 4. Improved Real-Time and Stability: The method of this embodiment performs distributed computing on edge devices, reducing reliance on the cloud and helping to meet the real-time requirements of autonomous transportation systems. Furthermore, the bidirectional distillation mechanism makes learning more stable for each model, making it less likely to fall into local optima, thereby improving the generalization and stability of the entire system in dynamic environments.

[0163] Reference Figure 5 , an embodiment of the present application provides a large-small model mutual distillation system for autonomous transportation end-edge-cloud collaboration, the system comprising:

[0164] The first module is used to complete task initialization based on the device characteristics and scenario requirements within the system;

[0165] The second module is used to initialize the task, issue and deploy the global small model, and perform local model training on the edge device;

[0166] The third module is used to perform fusion knowledge distillation on the cloud server based on the results of the local model training and fine-tune the cloud large model;

[0167] The fourth module is used to perform guided knowledge distillation on the cloud server based on the fine-tuned cloud-based large model to update the global small model;

[0168] The fifth module is used to determine whether the training termination conditions are met. If the training termination conditions are not met, the module returns to the steps of initializing the task, issuing and deploying the global small model, and executing the local model training of the edge-side device until the training termination conditions are met. If the training termination conditions are met, a large and small model with bidirectional performance optimization is obtained.

[0169] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0170] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flow chart of the present application are provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logical flows presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0171] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features described can be integrated into a single physical system and / or software module, or one or more functions and / or features can be implemented in separate physical systems or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present application. More specifically, given the properties, functions, and internal relationships of the various functional modules in the system disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present application as set forth in the claims using ordinary techniques without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the appended claims and their equivalents.

[0172] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0173] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, system, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions on an instruction execution system, system, or device). For purposes of this specification, a "computer-readable medium" can be any system that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, system, or device.

[0174] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic systems), a portable computer disk cartridge (magnetic systems), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic system, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0175] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0176] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0177] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and intent of the present application, and that the scope of the present application is defined by the claims and their equivalents.

[0178] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present application, and these equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A large-small model mutual distillation method for autonomous transportation end-edge-cloud collaboration, characterized by: The method comprises the following steps: Complete task initialization based on the device characteristics and scenario requirements within the system; Initialize the task, issue and deploy a global small model, and perform local model training on edge devices. Based on the results of the local model training, perform fusion knowledge distillation on the cloud server to fine-tune the cloud-based large model; Based on the fine-tuned cloud-based large model, perform guided knowledge distillation on the cloud server to update the global small model; Determine whether the training termination condition is met. If the training termination condition is not met, return to the steps of initializing according to the task, issuing and deploying the global small model, and executing the local model training of the edge-side device until the training termination condition is met; if the training termination condition is met, obtain the large and small models with bidirectional performance optimization.

2. The large-small model mutual distillation method for autonomous transportation end-edge-cloud collaboration according to claim 1 is characterized in that: The task initialization is completed according to the characteristics of the equipment in the system and the scene requirements, including the following steps: Select a pre-trained large model based on the nature of the task and scenario requirements, and deploy it on a cloud server; Based on the pre-trained large model, pruning and quantization are used to generate a small model that is adaptable to edge devices, and the parameters of the small model are initialized to serve as the global small model for the first round of training; Create unlabeled proxy datasets based on public data; Set training hyperparameters, including but not limited to learning rate, loss function, and training rounds.

3. The large-small model mutual distillation method for autonomous transportation end-edge-cloud collaboration according to claim 1 is characterized in that: Initializing the task, issuing and deploying a global small model, and performing local model training on edge devices include the following steps: The cloud server selects the target edge device as the training client and delegates the task configuration and the current global small model parameters; The training client downloads the task configuration and current global small model parameters as needed, and deploys the model training environment and the local small model; The training client extracts a data subset suitable for the corresponding task based on the locally sensed and stored data, and trains and updates the local small model; The training client extracts the updated local small model parameters and uploads them to the cloud server.

4. The large-small model mutual distillation method for autonomous transportation end-edge-cloud collaboration according to claim 1 is characterized in that: The method of performing fusion-type knowledge distillation on the cloud server based on the results of the local model training and fine-tuning the cloud-based large model includes the following steps: Collect local small model parameters uploaded by each training client; Adopting a federated learning aggregation method to aggregate the collected parameters of multiple local small models and update the global small model. The federated learning aggregation method includes but is not limited to the FedAvg algorithm, the FedProx algorithm, or the MOON algorithm; Based on the updated global small model, a fusion knowledge distillation method is used to fine-tune the cloud-based large model.

5. The large-small model mutual distillation method for autonomous transportation end-edge-cloud collaboration according to claim 1 is characterized in that: The fusion-based knowledge distillation on the cloud server includes the following steps: Define the main body of knowledge distillation; the global small model serves as the teacher model and the large model serves as the student model; Randomly sample a subset based on the proxy data set to obtain a proxy data subset; Calculating the main loss and the self-distillation loss based on the proxy data subset; Calculating the total loss of fusion knowledge distillation according to the main loss and the self-distillation loss; Based on the total loss of the fusion knowledge distillation, a preset optimizer and fine-tuning method are used to update the cloud-based large model.

6. The large-small model mutual distillation method for autonomous transportation end-edge-cloud collaboration according to claim 1 is characterized in that: The method of performing guided knowledge distillation on the cloud server based on the fine-tuned cloud-based large model to update the global small model includes the following steps: Define the main body of knowledge distillation; the large model serves as the teacher model, the global small model serves as the student model, and the knowledge of the large model is extracted to update the global small model; Randomly sample a subset based on the proxy data set to obtain a proxy data subset; Clone the current global small model; Calculating the output layer loss and the hidden layer loss based on the proxy data subset; Calculating the total loss of guided knowledge distillation according to the output layer loss and the hidden layer loss; updating the cloned global small model according to the total loss of the guided knowledge distillation; Based on the updated cloned global small model, a meta-gradient is calculated, and the global small model is updated with the meta-gradient.

7. The large-small model mutual distillation method for autonomous transportation end-edge-cloud collaboration according to claim 1 is characterized in that: The step of determining whether a training termination condition is met, and if not, returning to the step of initializing the task, issuing and deploying a global small model, and executing local model training on edge devices until the training termination condition is met; and obtaining a large and small model with bidirectional performance optimization if the training termination condition is met, includes the following steps: Check whether the current conditions meet the training termination conditions, the current conditions including but not limited to the training round, the large model performance index and the global small model performance index; If the current condition meets the training termination condition, each client is notified of the training termination, and the fine-tuned large model and the trained global small model are converted into available services; If the current conditions do not meet the training termination conditions, each client is notified to continue training, and the process returns to the steps of initializing the task, issuing and deploying the global small model, and executing the local model training of the edge device until the training termination conditions are met.

8. A large-small model mutual distillation system for autonomous transportation end-edge-cloud collaboration, characterized by: The system comprises: The first module is used to complete task initialization based on the device characteristics and scenario requirements within the system; The second module is used to initialize the task, issue and deploy the global small model, and perform local model training on the edge device; The third module is used to perform fusion knowledge distillation on the cloud server based on the results of the local model training and fine-tune the cloud large model; The fourth module is used to perform guided knowledge distillation on the cloud server based on the fine-tuned cloud-based large model to update the global small model; The fifth module is used to determine whether the training termination conditions are met. If the training termination conditions are not met, the module returns to the steps of initializing the task, issuing and deploying the global small model, and executing the local model training of the edge-side device until the training termination conditions are met. If the training termination conditions are met, a large and small model with bidirectional performance optimization is obtained.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images