Federal mutual distillation method based on inactivation technology in Internet of Vehicles

Abstract

The invention relates to a federal mutual distillation method based on an inactivation technology in the Internet of Vehicles, and belongs to the technical field of mobile communication. Federal learning is a distributed machine learning method, local knowledge is aggregated into a global model through a sharing model, and data security is ensured. In an Internet of Vehicles scene, the communication overhead of the method is too high. Meanwhile, the local data distribution difference of vehicles is large, so that the global model performance is reduced when local knowledge is shared. In addition, the vehicle can be switched between RSU coverage areas in the driving process, and model transmission interruption may be caused. In order to solve the problem, the invention provides the federal mutual distillation method based on the inactivation technology in the Internet of Vehicles. According to the method, factors such as vehicle moving conditions and traffic conditions are considered, and RSU switching prediction is carried out by using a Markov chain; by screening misleading data, the accuracy of the model is improved; knowledge is transmitted through a knowledge mutual distillation method, and communication overhead is reduced; and introducing an inactivation technology to accelerate model training.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile communications and relates to a federal mutual distillation method based on deactivation technology in a vehicle network. Background Art

[0002] With the rapid development of intelligent connected vehicles, vehicles have the ability to exchange information with other vehicles and infrastructure through on-board communication systems. This allows vehicles to obtain detailed data about the surrounding environment and collaborate with other vehicles for path planning and autonomous driving, thereby greatly improving traffic efficiency and driving safety. However, the resulting massive data processing requests have become a problem that needs to be solved urgently. These data usually contain sensitive information, such as CAV location information, driving behavior, and road conditions. How to make full use of these data while protecting data privacy has become one of the hot issues in current research.

[0003] Federated Learning (FL), as a distributed machine learning framework, provides a new data processing method for CAVs. FL allows each CAV to train models locally and transmit the trained model parameters to a central server for aggregation, thereby avoiding centralized storage and processing of data and effectively protecting user privacy. However, the application of FL in the Internet of Vehicles faces the following significant challenges: high communication overhead, difficulty in adapting to heterogeneous models, and the risk of white-box attacks.

[0004] To address these challenges, the federated distillation method that combines federated learning with knowledge distillation has become an effective solution. In this method, the RSU and CAV train the teacher model and the student model respectively, and transfer knowledge through mutual distillation. This method can use the powerful computing power of the RSU to train a high-performance teacher model, and use the local data of the CAV to train the student model, and compare and optimize the output of the student model with the output of the teacher model. However, this method also has shortcomings. For example, the distribution difference of the local data of the CAV causes the trained model to be unable to accurately identify the corresponding regional data, and may output misleading knowledge, resulting in a decrease in global model performance. In addition, due to the mobility of the CAV, it needs to switch between different RSU coverage areas during driving. If the correct switch cannot be made, data transmission may be interrupted, affecting the training of the model.

[0005] Based on the above problems, the present invention designs a federated mutual distillation method based on deactivation technology in the Internet of Vehicles to reduce communication overhead and improve model accuracy. First, a switching prediction scheme based on Markov chain is proposed to predict the CAV accessed by RSU to reduce the impact of communication interruption on model training; secondly, a data selection method based on KuLSIF is proposed to screen out misleading data and improve model accuracy; finally, knowledge is transferred through knowledge mutual distillation, a federated distillation framework based on privacy protection is constructed, and deactivation technology is introduced to accelerate model training. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide a federated mutual distillation method based on deactivation technology in the Internet of Vehicles, which is used to solve the problems of high communication overhead, low model accuracy and difficulty in adapting heterogeneous models in the Internet of Vehicles.

[0007] To achieve the above object, the present invention provides a federated mutual distillation method based on deactivation technology in an Internet of Vehicles, comprising the following steps:

[0008] S1: Switching prediction scheme based on Markov chain;

[0009] S2: Data screening scheme based on KuLSIF;

[0010] S3: Gradient-based deactivation scheme;

[0011] S4: Federated knowledge mutual distillation scheme.

[0012] S1 is used to predict the RSU that the CAV will access and reduce communication interruptions. It provides a basis for S2 and S3, ensuring that the CAV can maintain a stable connection with the RSU and avoid data transmission interruptions that affect subsequent data screening and model training.

[0013] S2 is used to filter out data within the distribution to avoid the impact of misleading data on model training. It provides high-quality data for S3 to ensure the effectiveness of model training.

[0014] S3 is used to generate heterogeneous student sub-models to adapt to the local data distribution of different CAVs. It provides S4 with diverse student models to facilitate knowledge distillation and improve model performance.

[0015] S4 uses the powerful computing power of RSU to train a high-performance teacher model, and transfers knowledge to the student model through mutual distillation to improve model performance. The results of S1, S2, and S3 are integrated to ultimately improve the efficiency and accuracy of model training.

[0016] In the first aspect, in an embodiment of the present invention, in S1, a vehicle networking scenario in which N CAVs and M RSUs interact is established. Both CAVs and RSUs have computing and storage capabilities, and RSUs are connected to CAVs within their coverage areas through communication links. Due to the mobility of CAVs, they need to switch between different RSU coverage areas during driving. If the switching decision cannot be made in time, data transmission may be interrupted, affecting the training of the model. Therefore, a switching prediction scheme based on a Markov chain is considered, in which each CAV collects the mobility status, traffic conditions, and signal strength of the current round. Secondly, the probability of the CAV accessing the RSU is calculated. Based on the connection status of the CAV in the previous round and the access probability of the CAV in the current round, a Markov chain is used to predict the connection status of the CAV in the current round.

[0017] In the second aspect, an embodiment of the present invention proposes a data screening scheme based on KuLSIF in S2. The student model on the CAV side and the teacher model on the RSU side are initialized. The student model and the teacher model respectively adopt convolutional neural networks (CNN) with different architectures, and the teacher model is a CNN with more parameters and higher number of layers. CAV trains the student model based on local data, and then identifies the data through density ratio. Specifically, the optimal density ratio is estimated based on kernelized unconstrained least squares importance fitting (Kernelized Unconstrained Least-Squares Importance Fitting, KuLSIF), and a screening threshold is set. If the local data density ratio of CAV is greater than the threshold, it is judged as in-distribution data, otherwise it is out-of-distribution data. The CAV student model predicts the label of the in-distribution data in the regional data and uploads it to the RSU. The RSU aggregates the results uploaded by the CAV to obtain the predicted label of the regional data, and the RSU sends the aggregated regional data label to the corresponding CAV.

[0018] On the third aspect, an embodiment of the present invention proposes a gradient-based deactivation scheme in S3. The gradient update amplitude before and after the current round of student model training is calculated, and a discarding threshold is set to discard some student model parameters in units of neurons. Specifically, the student model parameters whose update amplitude is lower than the discarding threshold are discarded, and the remaining part of the model is retained as the student sub-model. This process is called "deactivation". Since the update amplitudes of various parameters of the student model are different, this will cause the student sub-model generated by CAV to be heterogeneous. Subsequently, CAV trains the student sub-model based on the regional data, and uploads the obtained output vector to the RSU. It should be noted that after a round of training is completed, the inactivated parameters will be reactivated with the original parameter values.

[0019] In the fourth aspect, an embodiment of the present invention proposes a federated knowledge mutual distillation scheme in S4. First, the RSU trains the teacher model based on the regional data, and sends the obtained output vector to each connected CAV. Then, the CAV calculates the task loss based on the regional data prediction label and output vector, and calculates the distillation loss based on the teacher model output vector and the student model output vector. Finally, the inactivated student model is updated based on the joint loss, and the joint loss is a linear combination of the task loss and the distillation loss. Similarly, the RSU follows the same steps to calculate the task loss, distillation loss and joint loss of the teacher model to update the teacher model.

[0020] The beneficial effects of the present invention are:

[0021] (1) Through the Markov chain-based handover prediction scheme, the RSUs accessed by the CAV are predicted in advance, reducing communication interruptions caused by handovers and thus reducing communication overhead. Through the gradient-based deactivation scheme, parameter deactivation is performed at the local CAV end to reduce the number of model parameters and the amount of communication data required for model transmission.

[0022] (2) Through the KuLSIF-based data screening solution, we can filter out the data within the distribution, avoid the impact of misleading data on model training, and improve the accuracy of the model. Through the federated knowledge mutual distillation solution, we can use the powerful computing power of RSU to train a high-performance teacher model, and pass the knowledge to the student model through mutual distillation to improve the performance of the student model.

[0023] (3) Through the gradient-based inactivation scheme, heterogeneous student sub-models are generated to adapt to the local data distribution of different CAVs, solving the problem that heterogeneous models are difficult to adapt. Through the federated knowledge mutual distillation scheme, the knowledge of the teacher model is transferred to the student model, helping the student model to quickly learn and adapt to different data distributions.

[0024] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0026] Figure 1 This is a diagram of the federated mutual distillation architecture based on deactivation technology;

[0027] Figure 2 Execute the flow chart for the federated inter-distillation algorithm based on deactivation technique. DETAILED DESCRIPTION

[0028] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0029] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0030] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0031] Figure 1 A possible structural diagram of a communication system involved in an embodiment of the present invention is shown. Figure 1 As shown in the figure, the network considers the Internet of Vehicles scenario where N CAVs and M RSUs interact. This scenario includes switching prediction, data screening, and knowledge mutual distillation based on inactivation. The RSU that the CAV can most likely connect to in the current round can be obtained through switching prediction; due to the distribution differences of CAV local data, the trained model cannot accurately identify the corresponding regional data, and may output misleading knowledge, resulting in a decrease in global model performance. The model accuracy is improved by screening misleading data; a privacy-preserving and adaptive federated distillation framework is constructed through knowledge mutual distillation based on inactivation, and model overfitting is reduced to accelerate model training.

[0032] 1. Switching prediction based on Markov chain

[0033] Since CAV switches between different RSU coverage areas during driving, the RSU connected to the CAV will change accordingly. If accurate switching predictions cannot be made in time, data transmission may be interrupted, affecting the performance of the model. To solve the above problems, a switching prediction based on Markov chain is proposed, which combines the mobility status, road conditions and signal strength of CAV. First, the RSU obtains the specific location coordinates of all CAVs within its coverage area, and uses the Markov chain to estimate the probability of the next round of connection status according to the mobility status, road conditions and signal strength of the CAV.

[0034] Assume that all CAVs are evenly distributed within the RSU coverage area at the beginning, and its coverage area is 500*500m. Use Markov chain to train RSU m The CAV mobility status within the coverage area is modeled. Specifically, Indicates the connection status at round k:

[0035]

[0036] in, Indicates that the kth iteration is located in RSU m The collection of vehicles covering the area. When it is 1, it means that in the kth round, v i with RSU m is connected, otherwise it is disconnected.

[0037] Assume that CAV and RSU in round k m The connection state is determined by the connection probability The probability is represented by v i It is estimated based on factors such as the current round's mobility, road conditions, and signal strength, and is specifically expressed as:

[0038]

[0039] Where, d r,i v i The distance to the RSU, the position coordinates of the RSU are expressed as (x R ,y R ), vehicle v i The position coordinates are expressed as (x i ,y i ), distance representation S r,i For vehicle v i The signal-to-interference-noise ratio (SINR) of the current RSU is usually approximated as the ratio of the received signal strength to the interference signal strength, which is expressed as:

[0040]

[0041] Among them, P r,i Indicates the transmission power, g r,i represents the channel gain, Represents the total interference from other RSUs, I r,i represents Gaussian white noise. T i leave Represented as vehicle v i If the CAV stays longer in the current RSU coverage area, it means that the probability of staying in the RSU in the next iteration is high. The residence time can be expressed as:

[0042] T i leave =T i l [1+a(H) b ] (4)

[0043]

[0044] Among them, T i l is the departure time / entry time, is the outgoing distance / entry time, c i Yes i speed, a and b are coefficients determined according to the actual traffic environment of the city, H represents the traffic status of the roads in the current area, Q is the traffic volume in the current area, and C is the actual traffic capacity of the current area. When H takes the value [0,1], the road is considered to be in a smooth state, and when H is greater than 1, the road is considered to be in a congested state.

[0045] According to the properties of Markov chain X t Indicates the state of the kth iteration of CAV, the next state X k The probability of occurrence depends only on the current state X t-1 Therefore, the connection of all CAVs to RSUs at the kth iteration is described as m Probability:

[0046]

[0047] In the above formula, σ is the connection threshold value. Since mobile CAVs arrive randomly, the number of CAVs in each round of RSU may be different. The performance of the global model improves with the increase of CAVs participating in the training. If the number of participating CAVs is small, the performance of the global model will decrease. i, if its connection probability is greater than the connection threshold, it is considered to be connected to the RSU at the kth iteration. At this time, the following three special cases should be noted: 1) If the CAV is located under the coverage of multiple RSUs at the same time, select the connection plan with a higher connection probability; 2) If the connection probabilities are the same and the connection plan of the previous round exists in the selection of the current round, then keep the connection plan to avoid frequent switching. 3) If the connection probabilities are the same and the connection plan of the previous round does not exist in the selection of the current round, then select the nearest RSU to connect based on the distance.

[0048] 2. Data screening method based on KuLSIF

[0049] The local data distribution of different CAVs is very different, which makes the trained local model unable to accurately predict samples outside its local distribution, resulting in poor performance of the aggregated global model. Therefore, we consider how to identify data outside of these distributions.

[0050] Assume V = [v1,…,v n ,…,v N ] represents all CAVs within the coverage area of ​​the RSU. Each CAV has its own local data D = [D1,…,D n ,…D N ] and unlabeled regional data U, which is derived from the data collected by RSU and sent to each CAV. i The data probability density function at is p i (x), define the regional data as uniformly distributed data, and its probability density function is u(x). By estimating the density ratio of CAV and RSU data, the integrated samples outside the CAV local data are called out-of-distribution samples. The density ratio is expressed as:

[0051]

[0052] Considering the statistical convergence property, KuLSIF is used to estimate the density ratio. The goal of KuLSIF is to find the optimal density ratio function through the following objective function: It is expressed as:

[0053]

[0054] From v i Local data D i and RSU collects regional data U in sample n i and n u data points, and the obtained sample set is recorded as S i and S u . β is a regularization parameter used to prevent overfitting. ||r i || Ri Yesi The norm of the density ratio estimation function usually uses a Gaussian kernel.

[0055] Since the data collected by CAVs is affected by various factors, such as different driving environments, different vehicle models, different road conditions, etc., resulting in large variations and diversities in the data, the actual collected regional data may not fully conform to the i distribution of local data. When the value of r i (x) approaches 0, it is considered an outlier data, and its prediction result will be avoided from being shared. For this scenario, by setting a screening threshold r * to determine whether a sample is an out-of-distribution sample. That is, when r i (x) ≥ r*, the sample is considered an in-distribution sample; conversely, when r i (x) < r*, the sample is considered an out-of-distribution sample. For out-of-distribution samples, the data screening will not upload the predicted value of the sample because the student model may not be able to make an accurate prediction for the sample. It should be noted that the screening threshold determines the number of screened samples. If r * is set too large, a large amount of regional data will be screened; if r * is set too small, out-of-distribution samples cannot be screened out. Therefore, the screening threshold needs to be set reasonably.

[0056] Subsequently, the CAV student model predicts the in-distribution data in the regional data and uploads it to the RSU. The RSU aggregates the results uploaded by the CAVs to obtain the predicted labels of the regional data in the k-th round, denoted as y i k . The RSU distributes the aggregated regional data labels to each connected CAV.

[0057] 3. Gradient-based inactivation

[0058] In the model training stage, due to the limited local computing power of CAVs, it is considered to train a student model with a simple structure and fewer parameters locally on the CAV, while the RSU has stronger computing power than the CAV and can undertake more complex processing tasks. Therefore, the RSU will undertake the task of training a teacher model with a more complex structure and more parameters, making full use of the high computing power of the RSU to process and analyze the collected data.

[0059] In the parameter initialization stage, the CAV and the RSU randomly initialize the student model and the teacher model respectively. According to the switching prediction method based on the Markov chain, the CAVs connected to the RSU train the student model based on local data Then, based on the gradient deactivation method, the update amplitude of the current round of student model training is calculated, and the student model parameters with update amplitudes lower than the discarding threshold are discarded, and the remaining models are retained as student sub-models. Specifically, for the student model, the update amplitude of the parameter n relative to the previous round is calculated:

[0060]

[0061] Set a discard threshold ε and discard the student model parameters whose update amplitude is lower than the discard threshold. Since the update amplitudes of the student model parameters are different, the student sub-models generated by CAV will be heterogeneous. Introduce the mask vector For each parameter, the mask is defined as follows:

[0062]

[0063] When the update amplitude of a parameter is lower than the discard threshold, it is considered that the update contribution of the parameter in the current round is small, and it is deactivated, that is, set Otherwise, keep the parameter. Finally, we get the student sub-model after inactivation In subsequent training, the student model only updates the parameters that are not inactivated. It should be noted that after a round of training, the inactivated parameters will be reactivated.

[0064] 4. Federated Knowledge Distillation

[0065] RSU will train teacher models based on regional data The output vector of the kth round is obtained The data is sent to each connected CAV. At the same time, the CAV trains the inactivated student model based on the regional data. The resulting output vector Upload to RSU.

[0066] RSU and CAV calculate the task loss based on the regional data prediction label and the corresponding output vector, which can be calculated by the following formula:

[0067]

[0068] Among them, c represents the label category. The output soft labels of the student model and the teacher model are the normalized model output vectors, expressed as:

[0069]

[0070] The distillation loss of the student model and the teacher model is calculated by the output soft labels of the two:

[0071]

[0072] Among them, ρ k represents the distillation intensity of the kth round of training. Finally, the inactivated student model and teacher model are updated based on the joint loss. The joint loss is a linear combination of the task loss and the distillation loss, which is expressed as the following formula:

[0073]

[0074] Among them, η represents the learning rate of training.

[0075] 5. System Process

[0076] Figure 2 The figure shows the execution flow chart of the heterogeneous adaptive federated mutual distillation algorithm based on deactivation. The specific steps are as follows:

[0077] Steps 201-205: Handover prediction based on Markov chain;

[0078] Step 201: Algorithm initialization;

[0079] Step 202: The RSU collects the movement status, road status, and signal strength of all CAVs within the coverage area;

[0080] Step 203: define all CAV connection states;

[0081] Step 204: Calculate the connection probability of a single CAV;

[0082] Step 205: Predict all CAVs connected to RSUs m probability;

[0083] Steps 206-213: Data screening based on KuLSIF;

[0084] Step 206: CAV and RSU collect data respectively;

[0085] Step 207: define a data density ratio;

[0086] Step 208: Calculate the optimal density estimation ratio function;

[0087] Step 209: Setting a data screening threshold;

[0088] Step 210: Filter out-of-distribution samples;

[0089] Step 211: Upload the predicted values ​​of the samples within the distribution;

[0090] Step 212: RSU aggregates and issues prediction labels;

[0091] Steps 213-220: gradient-based inactivation;

[0092] Step 213: Initialize the student model and the teacher model;

[0093] Step 214: CAV trains a student model based on local data;

[0094] Step 215: Calculate the update amplitude of the current student model intersecting the previous round

[0095] Step 216: Setting a discard threshold;

[0096] Step 217: inactivate some parameters to update the student model;

[0097] Steps 218-224: mutual distillation of federated knowledge;

[0098] Step 218: RSU trains the teacher model based on the regional data;

[0099] Step 219: CAV trains the inactivated student model based on the regional data;

[0100] Step 220: The student model and the teacher model calculate the corresponding task loss;

[0101] Step 221: The student model and the teacher model calculate the corresponding distillation loss;

[0102] Step 222: The student model and the teacher model calculate the corresponding joint loss;

[0103] Step 223: Update the student model and the teacher model respectively;

[0104] Step 224: the student model restores the original structure by discarding the threshold;

[0105] Step 225: If the stop condition is reached, execute step 426, otherwise repeat 202;

[0106] Step 226: The algorithm ends.

[0107] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0108] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), including a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network side device, etc.) to execute the method of switching cells described in multiple embodiments of the present invention.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A federated mutual distillation method based on deactivation technology in an Internet of Vehicles, characterized by: The following steps are involved: S1 switching prediction based on Markov chain; S2 Data screening based on KuLSIF; S3 gradient-based deactivation; S4 federated knowledge mutual distillation.

2. According to claim 1, a federated mutual distillation method based on deactivation technology in a vehicle network is characterized in that: The S1 includes: S11: Collect the mobility status, traffic status and signal strength of the current round; S12: Calculate the probability of a connected and automated vehicle (CAV) connecting to an RSU; S13: Based on the connection status of the CAV in the previous round and the access probability of the CAV in the current round, a Markov chain is used to predict the connection status of the CAV in the current round.

3. The method for federated mutual distillation based on deactivation technology in a vehicle network according to claim 1, characterized in that: The S2 includes: S21: Initialize the student model of the CAV end and the teacher model of the RSU end. The student model and the teacher model use convolutional neural networks (CNN) with different architectures. The teacher model is a CNN with more parameters and higher layers. S22: CAV trains the student model based on local data and then identifies the data through density ratio; S23: Estimate the optimal density ratio based on kernelized unconstrained least-squares importance fitting (KuLSIF) and set the screening threshold; S24: If the local data density ratio of the CAV is greater than the threshold, it is determined to be in-distribution data, otherwise it is out-distribution data; S25: The CAV student model predicts the labels of the in-distribution data in the regional data and uploads them to the RSU; S26: The RSU aggregates the results uploaded by the CAV to obtain a predicted label of the regional data, and the RSU sends the aggregated regional data label to the corresponding CAV.

4. The method for federated mutual distillation based on deactivation technology in a vehicle network according to claim 1, characterized in that: The S3 includes: S31: Calculate the gradient update amplitude before and after the current round of student model training, and set the discard threshold; S32: discard the student model parameters whose update amplitude is lower than the discarding threshold, and keep the remaining part of the model as the student sub-model; S33: Introduce the mask vector. For each parameter, the mask is defined as follows: When the update amplitude of a certain parameter is lower than the discarding threshold, it is considered that the update contribution of the parameter in the current round is small and it is inactivated; Otherwise, the parameter is retained, and the student sub-model after inactivation is finally obtained; S34: In subsequent training, the student model is updated only for the parameters that are not inactivated; S35: After a round of training, the inactivated parameters will be reactivated.

5. The method for federated mutual distillation based on deactivation technology in a vehicle network according to claim 1, characterized in that: The S4 includes: S41: The RSU trains the teacher model based on the regional data and sends the obtained output vector to each connected CAV; S42: CAV calculates the task loss based on the predicted labels and output vectors of the regional data; S43: CAV calculates the distillation loss based on the teacher model output vector and the student model output vector; S44: Update the inactivated student model based on the joint loss, where the joint loss is a linear combination of the task loss and the distillation loss. S45: RSU follows the same steps to calculate the task loss, distillation loss, and joint loss of the teacher model to update the teacher model.

Images