Federal knowledge distillation method for heterogeneous data of Internet of Vehicles

Abstract

The invention belongs to the technical field of intelligent traffic communication, and discloses a federal knowledge distillation method for heterogeneous data of Internet of Vehicles, comprising the following steps: a client receives global model parameters issued by a server before the start of a current communication round; obtaining a local model of the client according to the global model parameter configuration, and obtaining a local labeled data set of the client to train the local model; on the basis of the trained local model, respectively calculating feature embedding mean values of all samples corresponding to each category to obtain local prototypes of the corresponding categories; the client uploads the local prototype and the local model parameters to the server; the server aggregates the local prototypes of the clients according to categories; the problem in the prior art that a federated learning system in which all nodes can effectively participate in and stably collaboratively converge is difficult to construct in an Internet of Vehicles dynamic environment in which data distribution and model structures are highly heterogeneous and network communication is interrupted and unstable is effectively solved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation communication technology, specifically relating to a federated knowledge distillation method for heterogeneous data in the Internet of Vehicles. Background Technology

[0002] With the rapid development of autonomous driving, vehicle-to-everything (V2X) communication technologies, vehicle terminals, roadside units (RSUs), and cloud platforms are constantly generating massive amounts of multimodal data, including traffic sign and lane line images, video streams, millimeter-wave radar point clouds, lidar point clouds, vehicle status information, and road environment data. This data is naturally distributed across various vehicles or roadside nodes, and due to constraints such as privacy compliance, data sovereignty, network bandwidth, and communication latency, it is difficult to directly upload it centrally to the cloud for unified model training. Federated learning addresses this by training models locally at each node, uploading only model parameters or intermediate statistics, thus achieving collaborative learning while protecting privacy.

[0003] However, the vehicular network (V2X) environment is more complex and dynamic than traditional mobile terminal federated learning. On one hand, the data exhibits significant non-independent and identically distributed characteristics statistically. Differences in driving paths, times, environmental conditions, and sensor configurations among different vehicles lead to high heterogeneity in data distribution, category ratios, and noise levels across various nodes. Traditional parameter averaging methods are prone to introducing bias, resulting in slow model convergence, decreased accuracy, and "client drift" due to inconsistent local update directions. On the other hand, the system and communication environment exhibit significant heterogeneity. Different vehicles and RSUs differ in computing power, memory size, energy consumption constraints, network bandwidth, and packet loss rates. The dynamic topology of the V2X further leads to unstable connections and low node participation rates. Especially in high-speed or network congestion scenarios, frequent client disconnections make it difficult for traditional synchronous aggregation mechanisms to proceed stably. Furthermore, actual deployments face the challenge of heterogeneous model structures. Different nodes may use models of different sizes or architectures due to differences in hardware and software configurations, and even output header structures may differ. Direct parameter aggregation is neither feasible nor conducive to stable results. While a unified model structure is one solution, it limits the performance potential or operational efficiency of some nodes. At the same time, privacy and security risks cannot be ignored. Even if the original data is not uploaded directly, the uploaded gradients, parameters or intermediate representations still pose a potential risk of information leakage. Therefore, it is necessary to design a more abstract and refined communication mechanism to enhance privacy protection capabilities.

[0004] To address the aforementioned issues, existing research has proposed various improvement schemes, such as employing proximal regularization, control variable correction, update normalization, contrastive learning constraints, and personalized layer splitting. There are also knowledge distillation-based methods that achieve model-independent collaboration through exchanging soft labels, or that utilize intra-class feature mean to express category semantics through prototype aggregation. However, most of these methods focus on single-dimensional optimization: regularization or correction methods struggle to effectively handle heterogeneous model structures; distillation methods typically rely on public datasets or additional generators; and simple prototype aggregation methods, lacking bidirectional knowledge transfer mechanisms, may still exhibit instability in scenarios with low participation rates and high drift. Therefore, existing technologies face the challenge of constructing a federated learning system where all nodes can effectively participate and stably converge in the dynamic environment of vehicular networks, where data distribution, model structure, and network communication are highly heterogeneous and intermittent. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a federated knowledge distillation method for heterogeneous data in the Internet of Vehicles (IoV). This method solves the problem that existing technologies struggle to construct a federated learning system in the dynamic environment of the IoV where data distribution, model structure, and network communication are highly heterogeneous and intermittent, making it difficult to effectively participate in and stably converge a federated learning system.

[0006] The objective of this invention can be achieved through the following technical solutions: A federated knowledge distillation method for heterogeneous data in the Internet of Vehicles includes the following steps: The client receives global model parameters sent by the server before the start of the current communication round; The local model on the client is obtained based on the global model parameter configuration, and the local labeled dataset on the client is used to train the local model. Based on the trained local model, the mean of the feature embeddings of all samples corresponding to each category is calculated to obtain the local prototype of the corresponding category. The client uploads the local prototype and local model parameters to the server; The server aggregates the local prototypes of each client by category to obtain a new round of global prototypes and statistics for each category, and updates the global model maintained by the server in combination with the local model parameters. Pseudo-samples are generated based on a new round of global prototypes and their statistics, and soft labels corresponding to the pseudo-samples are generated using the updated global model. The server will send a new round of global prototypes and soft tags to the client; The client receives a new round of global prototypes, constructs a prototype alignment loss by the distance between the local prototype and the new round of global prototypes, and uses soft labels to distill and fine-tune the local model's classification prediction head. The server uses pseudo-samples and soft labels to perform knowledge distillation training on the global model, optimizing the global model parameters for subsequent communication rounds.

[0007] Furthermore, the local model includes a base network for feature extraction and a prediction head network for classification; During local model training, the base network parameters are frozen or updated with a first learning rate, and the prediction head network parameters are updated with a second learning rate greater than the first learning rate.

[0008] Furthermore, based on the trained local model, the mean of the feature embeddings of all samples corresponding to each category is calculated to obtain the local prototype of the corresponding category. This includes the following steps: For each category appearing in the client's local labeled dataset, all samples of each category are input into the trained local model; The base network in the local model extracts the feature vectors of the samples, which are used as the feature embeddings of the samples in the feature space. Calculate the arithmetic mean of the feature embeddings of all samples in each category, and use it as the local prototype for the corresponding category.

[0009] Furthermore, the server aggregates the local prototypes of each client by category to obtain a new round of global prototypes and statistics for each category, and updates the global model maintained by the server in combination with the local model parameters. This process includes the following steps: For each category, collect the number of local prototypes and samples uploaded by each client for that category; Based on the number of samples for each category from each client, a weighted average is calculated on the collected local prototypes to obtain the global prototype corresponding to the category in the new round. Based on the local prototypes and weighted average results for each client for this category, the global prototype standard deviation for this category is calculated as a statistic. The server performs a weighted average of the local model parameters uploaded by each client to update the global model maintained by the server.

[0010] Furthermore, the global prototype in the new round The calculation formula is as follows: In the formula, Represents a collection of clients; Indicates client The actual set of categories included; Indicates client middle c Number of samples per category; Indicates client Uploadedc The local prototype corresponding to the category; Statistic The calculation formula is as follows: In the formula, This indicates the content uploaded by the client. c A collection of category local prototypes.

[0011] Furthermore, pseudo-samples are generated based on the new global prototype and its statistics, and soft labels corresponding to the pseudo-samples are generated using the updated global model. This process includes the following steps: For each category c The corresponding new round of global prototypes With statistics Generate pseudo samples The calculation formula is as follows: In the formula, This is the sampling scale coefficient; The generated pseudo samples Input into the updated global model; Obtain the output of the global model and process it using the softmax function to obtain pseudo-samples. Corresponding soft tags The specific calculation formula is as follows: In the formula, This is the output function of the updated global model.

[0012] Furthermore, the client receives a new round of global prototypes, constructs a prototype alignment loss based on the distance between the local prototype and the new round of global prototypes, and uses soft labels to distill and fine-tune the local model's classification prediction head, specifically including the following steps: Supervised loss is constructed based on the client's local labeled dataset. The specific expression is as follows: In the formula, Indicates client Local labeled dataset; x, y These represent local data and tags, respectively. Construct prototype alignment loss based on the distance between the local prototype and the new global prototype. The specific expression is as follows: In the formula, This is a local prototype for the client. For global prototype; Prediction head network predicts soft labels The degree of matching, constructing distillation loss The specific expression is as follows: In the formula, This represents the output function of the client's local model; This represents the set of categories participating in distillation; This represents the number of categories in the set of categories participating in distillation; Based on monitoring loss Prototype alignment loss and distillation loss Construct the total loss function for the client, with the following expression: In the formula, , Prototype alignment loss and distillation loss Corresponding to the preset weighting coefficients; The parameters of the prediction head network in the local model are updated by minimizing the total loss function of the client.

[0013] Furthermore, the server utilizes pseudo-samples and soft labels to perform knowledge distillation training on the global model, and the server's knowledge distillation loss function... The specific expression is as follows: In the formula, Indicates mean square error; Indicates KL divergence; , All are preset weighting coefficients; By minimizing the server knowledge distillation loss function Update the global model parameters.

[0014] The beneficial effects of this invention are: 1. This invention constructs and aggregates local prototypes to form a global prototype, and uses the global prototype to generate pseudo-samples and corresponding soft labels. Knowledge distillation is performed on the server side to update the global model. This enables the effective integration of multi-source distributed information without relying on real data in a highly heterogeneous vehicle network environment where both data and models are highly heterogeneous, significantly improving the semantic consistency and generalization performance of the global model. At the same time, through prototype alignment and soft label fine-tuning mechanisms, each client model is guided to converge toward the global semantic space, effectively alleviating the client drift problem caused by non-independent and identically distributed data, and enhancing the overall convergence stability and collaborative efficiency of the federated learning system in a dynamic and unstable communication environment. 2. By decoupling the model into a feature extraction base and a classification prediction head and adopting a differentiated update strategy, not only is the local convergence of the client model accelerated, but also a foundation is provided for achieving heterogeneous model collaboration among clients. This design allows different clients to achieve effective knowledge fusion and semantic unification through prototype alignment and soft label distillation while maintaining differences in the base or prediction head structure. This significantly enhances the deployment flexibility and system compatibility of the method in the complex heterogeneous environment of real-world vehicle networking. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the overall process of the federated knowledge distillation method of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] like Figure 1 As shown, a federated knowledge distillation method for heterogeneous data in the Internet of Vehicles includes the following steps: The client receives global model parameters sent by the server before the start of the current communication round; The local model on the client is obtained based on the global model parameter configuration, and the local labeled dataset on the client is used to train the local model. Based on the trained local model, the mean of the feature embeddings of all samples corresponding to each category is calculated to obtain the local prototype of the corresponding category. The client uploads the local prototype and local model parameters to the server; The server aggregates the local prototypes of each client by category to obtain a new round of global prototypes and statistics for each category, and updates the global model maintained by the server in combination with the local model parameters. Pseudo-samples are generated based on a new round of global prototypes and their statistics, and soft labels corresponding to the pseudo-samples are generated using the updated global model. The server will send a new round of global prototypes and soft tags to the client; The client receives a new round of global prototypes, constructs a prototype alignment loss by the distance between the local prototype and the new round of global prototypes, and uses soft labels to distill and fine-tune the local model's classification prediction head. The server uses pseudo-samples and soft labels to perform knowledge distillation training on the global model, optimizing the global model parameters for subsequent communication rounds. This invention constructs and aggregates local prototypes to form a global prototype, and uses the global prototype to generate pseudo-samples and corresponding soft labels. Knowledge distillation is performed on the server side to update the global model. This enables the effective integration of multi-source distributed information without relying on real data in a highly heterogeneous vehicle network environment where both data and models are highly heterogeneous, significantly improving the semantic consistency and generalization performance of the global model. At the same time, through prototype alignment and soft label fine-tuning mechanisms, each client model is guided to converge toward the global semantic space, effectively alleviating the client drift problem caused by non-independent and identically distributed data, and enhancing the overall convergence stability and collaborative efficiency of the federated learning system in a dynamic and unstable communication environment.

[0019] The local model consists of a base network for feature extraction and a prediction head network for classification; During local model training, the base network parameters are frozen or updated with a first learning rate, and the prediction head network parameters are updated with a second learning rate greater than the first learning rate.

[0020] Based on the trained local model, the mean of the feature embeddings of all samples corresponding to each category is calculated to obtain the local prototype of the corresponding category. The specific steps include: For each category appearing in the client's local labeled dataset, all samples of each category are input into the trained local model; The base network in the local model extracts the feature vectors of the samples, which are used as the feature embeddings of the samples in the feature space. Calculate the arithmetic mean of the feature embeddings of all samples in each category, and use it as the local prototype for the corresponding category.

[0021] The server aggregates the local prototypes of each client by category to obtain a new round of global prototypes and statistics for each category, and updates the global model maintained by the server based on the local model parameters. The specific steps include: For each category, collect the number of local prototypes and samples uploaded by each client for that category; Based on the number of samples for each category from each client, a weighted average is calculated on the collected local prototypes to obtain the global prototype corresponding to the category in the new round. Based on the local prototypes and weighted average results for each client for this category, the global prototype standard deviation for this category is calculated as a statistic. The server performs a weighted average of the local model parameters uploaded by each client to update the global model maintained by the server.

[0022] A new round of global prototypes The calculation formula is as follows: In the formula, Represents a collection of clients; Indicates client The actual set of categories included; Indicates client middle c Number of samples per category; Indicates client Uploaded c The local prototype corresponding to the category; Statistic The calculation formula is as follows: In the formula, This indicates the content uploaded by the client. c A collection of category local prototypes.

[0023] Based on the new global prototype and its statistics, pseudo-samples are generated, and the updated global model is used to generate soft labels corresponding to the pseudo-samples. The specific steps include: For each category c The corresponding new round of global prototypes With statistics Generate pseudo samples The calculation formula is as follows: In the formula, This is the sampling scale coefficient; The generated pseudo samples Input into the updated global model; Obtain the output of the global model and process it using the softmax function to obtain pseudo-samples. Corresponding soft tags The specific calculation formula is as follows: In the formula, This is the output function of the updated global model.

[0024] The client receives a new round of global prototypes, constructs a prototype alignment loss based on the distance between the local prototype and the new round of global prototypes, and uses soft labels to distill and fine-tune the local model's classification prediction head. Specifically, this includes the following steps: Supervised loss is constructed based on the client's local labeled dataset. The specific expression is as follows: In the formula, Indicates client Local labeled dataset; x, y These represent local data and tags, respectively. Construct prototype alignment loss based on the distance between the local prototype and the new global prototype. The specific expression is as follows: In the formula, This is a local prototype for the client. For global prototype; Prediction head network predicts soft labels The degree of matching, constructing distillation loss The specific expression is as follows: In the formula, This represents the output function of the client's local model; This represents the set of categories participating in distillation; This represents the number of categories in the set of categories participating in distillation; Based on monitoring loss Prototype alignment loss and distillation loss Construct the total loss function for the client, with the following expression: In the formula, , Prototype alignment loss and distillation loss Corresponding to the preset weighting coefficients; The parameters of the prediction head network in the local model are updated by minimizing the total loss function of the client.

[0025] The server uses pseudo-samples and soft labels to perform knowledge distillation training on the global model. The server's knowledge distillation loss function is... The specific expression is as follows: In the formula, Indicates mean square error; Indicates KL divergence; , All of these are preset weight coefficients, which can be set according to the task's preference for "regression consistency" and "distribution matching"; By minimizing the server knowledge distillation loss function Update the global model parameters.

[0026] Considering that different terminals in the Internet of Vehicles may have different computing capabilities and model structures, this invention provides a flexible heterogeneous collaboration solution, which mainly achieves structure-independent knowledge transfer through prototyping and knowledge distillation, specifically including the following two typical strategies: (1) Predicting head-heteromorphism Different clients can use prediction heads of different capacities or structures (e.g., fully connected layers of different widths or lightweight MLPs). In this case, the server can choose: Only aggregate Base: Distribute unified feature extractor parameters; Head maintains client-side personalization; Alternatively, aggregate Heads within the same structural subset: Aggregate Heads within client groups of the same structure. Groups do not directly aggregate Heads. Knowledge is transferred across structures through prototypes and soft tags.

[0027] Since soft-label distillation does not require the student model and teacher model to have the same structure, the client can still use it even if the Heads are different. Learn the overall knowledge.

[0028] (2) Base-hetero When client-side architectures differ (e.g., different backbone networks), direct parameter aggregation is not feasible. This invention can still achieve "structure-independent collaboration" through prototyping and distillation: the client uploads a feature space prototype (feature dimensions need to be aligned, which can be unified through a projection layer). The server only performs aggregation and pseudo-sample generation in a unified prototype space. The server maintains a universal teacher head or universal discriminator to generate soft labels, which are then distributed to each client for distillation and learning on their respective models. In this case, the projection layer of the prototype space can be provided by the server or learned locally by each client and optimized together with the prototype alignment terms.

[0029] To illustrate the deployability of this invention in a vehicle-to-everything (V2X) environment, this embodiment provides an explanatory analysis of communication and computing overhead.

[0030] (1) Upload cost Traditional FedAvg uploads complete model parameters in each round, with varying sizes and parameter counts. Proportional; however, when the model is large and there are many participating nodes, the communication burden becomes significant. Besides the uploadable parameters, the most crucial information in this invention is the prototype. Let the feature dimension be... The number of categories is The prototype upload volume is approximately (If the standard deviation is of the same order of magnitude). In practical classification tasks, Typically much smaller Especially when For millions When the number of prototypes reaches several hundred, the overhead of prototyping can significantly reduce communication pressure. Furthermore, when the network is poor, the strategy of "upload prototype only / aggregate base only" can be selected to further reduce the load.

[0031] (2) Disbursement of expenses The overhead of the server distributing global prototypes and statistics is approximately If soft tags are sent directly... Will generate Scale (one per pseudo-sample) (Dimensional distribution). To reduce the amount of data distributed, the following methods can be used: reduce... The solution involves sending only random seeds and temperature parameters, allowing the client to reproduce pseudo-samples and generate soft tags online; or quantizing / sparse encoding the soft tags. None of these changes alter the core concept of this invention.

[0032] (3) Calculate the cost The main overhead on the client side is local supervised training and prototype computation. Prototype computation involves calculating the mean and variance of the embeddings, which is far less computationally intensive than backpropagation. Lightweight distillation only updates the Head, requiring fewer iterations, making it suitable for resource-constrained nodes. Server-side distillation training is performed on pseudo-samples, with a controllable sample size and no reliance on loading real data, making it suitable for execution on cloud or edge servers.

[0033] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0034] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A federated knowledge distillation method for Internet of Vehicles (IoV) heterogeneous data, comprising: Includes the following steps: The client receives global model parameters sent by the server before the start of the current communication round; The local model on the client is obtained based on the global model parameter configuration, and the local labeled dataset on the client is used to train the local model. Based on the trained local model, the mean of the feature embeddings of all samples corresponding to each category is calculated to obtain the local prototype of the corresponding category. The client uploads the local prototype and local model parameters to the server; The server aggregates the local prototypes of each client by category to obtain a new round of global prototypes and statistics for each category, and updates the global model maintained by the server in combination with the local model parameters. Pseudo-samples are generated based on a new round of global prototypes and their statistics, and soft labels corresponding to the pseudo-samples are generated using the updated global model. The server will send a new round of global prototypes and soft tags to the client; The client receives a new round of global prototypes, constructs a prototype alignment loss by the distance between the local prototype and the new round of global prototypes, and uses soft labels to distill and fine-tune the local model's classification prediction head. The server uses pseudo-samples and soft labels to perform knowledge distillation training on the global model, optimizing the global model parameters for subsequent communication rounds.

2. The federated knowledge distillation method for Internet of Vehicles heterogeneous data according to claim 1, wherein, The local model consists of a base network for feature extraction and a prediction head network for classification; During local model training, the base network parameters are frozen or updated with a first learning rate, and the prediction head network parameters are updated with a second learning rate greater than the first learning rate.

3. The federated knowledge distillation method for Internet of Vehicles heterogeneous data according to claim 2, characterized in that, Based on the trained local model, the mean of the feature embeddings of all samples corresponding to each category is calculated to obtain the local prototype of the corresponding category. The specific steps include: For each category appearing in the client's local labeled dataset, all samples of each category are input into the trained local model; The base network in the local model extracts the feature vectors of the samples, which are used as the feature embeddings of the samples in the feature space; Calculate the arithmetic mean of the feature embeddings of all samples in each category, and use it as the local prototype for the corresponding category.

4. The federated knowledge distillation method for heterogeneous data in the Internet of Vehicles according to claim 3, characterized in that, The server aggregates the local prototypes of each client by category to obtain a new round of global prototypes and statistics for each category, and updates the global model maintained by the server based on the local model parameters. The specific steps include: For each category, collect the number of local prototypes and samples uploaded by each client for that category; Based on the number of samples for each category from each client, a weighted average is calculated on the collected local prototypes to obtain the global prototype corresponding to the category in the new round. Based on the local prototypes and weighted average results for each client for this category, the global prototype standard deviation for this category is calculated as a statistic. The server performs a weighted average of the local model parameters uploaded by each client to update the global model maintained by the server.

5. The federated knowledge distillation method for heterogeneous data in the Internet of Vehicles according to claim 4, characterized in that, A new round of global prototypes The calculation formula is as follows: In the formula, Represents a collection of clients; Indicates the client The actual set of categories included; Indicates client middle c Number of samples per category; Indicates client Uploaded c The local prototype corresponding to the category; Statistic The calculation formula is as follows: In the formula, This indicates the content uploaded by the client. c A collection of category local prototypes.

6. The federated knowledge distillation method for heterogeneous data in the Internet of Vehicles according to claim 5, characterized in that, Based on the new global prototype and its statistics, pseudo-samples are generated, and the updated global model is used to generate soft labels corresponding to the pseudo-samples. The specific steps include: For each category c The corresponding new round of global prototypes With statistics Generate pseudo samples The calculation formula is as follows: In the formula, This is the sampling scale coefficient; The generated pseudo samples Input into the updated global model; Obtain the output of the global model and process it using the softmax function to obtain pseudo-samples. Corresponding soft tags The specific calculation formula is as follows: In the formula, This is the output function for the updated global model.

7. The federated knowledge distillation method for heterogeneous data in the Internet of Vehicles according to claim 6, characterized in that, The client receives a new round of global prototypes, constructs a prototype alignment loss based on the distance between the local prototype and the new round of global prototypes, and uses soft labels to distill and fine-tune the local model's classification prediction head. Specifically, this includes the following steps: Supervised loss is constructed based on the client's local labeled dataset. The specific expression is as follows: In the formula, Indicates client Local labeled dataset; x, y These represent local data and tags, respectively. Construct prototype alignment loss based on the distance between the local prototype and the new global prototype. The specific expression is as follows: In the formula, This is a local prototype for the client. For global prototype; Prediction head network predicts soft labels The degree of matching, constructing distillation loss The specific expression is as follows: In the formula, This represents the output function of the client's local model; This represents the set of categories participating in distillation; This represents the number of categories in the set of categories participating in distillation; Based on monitoring loss Prototype alignment loss and distillation loss Construct the total loss function for the client, with the following expression: In the formula, , Prototype alignment loss and distillation loss Corresponding to the preset weighting coefficients; The parameters of the prediction head network in the local model are updated by minimizing the total loss function of the client.

8. The federated knowledge distillation method for heterogeneous data in the Internet of Vehicles according to claim 7, characterized in that, The server uses pseudo-samples and soft labels to perform knowledge distillation training on the global model. The server's knowledge distillation loss function is... The specific expression is as follows: In the formula, Indicates mean square error; Indicates KL divergence; , All are preset weighting coefficients; By minimizing the server knowledge distillation loss function Update the global model parameters.

Images