Federated learning method and system based on two-way feedback knowledge distillation and differential privacy

Abstract

The present invention belongs to the field of intelligent transportation, and proposes a federated learning method and system based on two-way feedback knowledge distillation and differential privacy. By designing a two-way feedback mechanism, dynamic collaborative optimization of the client and server is realized. On the client, by designing a knowledge distillation framework, local model features in a non-independent and identically distributed data environment are extracted respectively, and the privacy of traffic data is protected by combining an adaptive differential privacy algorithm. The strength of knowledge distillation and privacy protection is dynamically adjusted based on the two-way feedback mechanism, significantly improving the balance between model performance and privacy protection. On the server side, an adaptive knowledge distillation fusion strategy based on two-way feedback is designed to perform global model optimization, thereby improving the prediction accuracy and communication efficiency in complex traffic scenarios. The present invention not only effectively avoids user privacy leakage, but also improves the accuracy of the federated learning model and its adaptability to complex traffic scenarios.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation, and in particular to a federated learning method and system based on two-way feedback knowledge distillation and differential privacy. Background Art

[0002] With the rapid development of intelligent transportation, the Internet of Vehicles (IoV), as an emerging network paradigm, is gradually becoming an important part of smart cities. By intelligently connecting vehicles, infrastructure, and cloud servers, IoV enables real-time data sharing and analysis, thereby improving traffic efficiency and enhancing driving safety. However, this highly interconnected environment also brings severe data privacy and security challenges.

[0003] In existing technologies, the Internet of Vehicles (IoV) typically adopts distributed machine learning methods represented by federated learning, which completes model training locally and uploads model parameters to a central server for global model aggregation. However, existing federated learning methods face two key problems. On the one hand, the large amount of traffic data (such as location, speed, and driving behavior) generated by vehicles and devices in the IoV is highly sensitive. If it is directly transmitted to the cloud for model training, it may lead to user privacy leakage and violate increasingly stringent data protection regulations. On the other hand, traditional federated learning alleviates some privacy issues through distributed training, but its model accuracy is insufficient when processing non-independent and identically distributed data, and the trade-off between privacy protection and performance optimization has not been fully resolved.

[0004] Therefore, how to design a federated learning method that can protect data privacy while improving model accuracy and adapting to complex traffic scenarios has become an urgent problem to be solved. Summary of the Invention

[0005] Based on this, the present invention proposes a federated learning method and system based on bidirectional feedback knowledge distillation and differential privacy. By designing a bidirectional feedback mechanism, dynamic collaborative optimization of the client and server is realized. On the client, by designing a knowledge distillation framework, local model features in non-independent and identically distributed data environments are extracted respectively, and the privacy of traffic data is protected by combining the adaptive differential privacy algorithm. The strength of knowledge distillation and privacy protection is dynamically adjusted based on the bidirectional feedback mechanism, which significantly improves the balance between model performance and privacy protection. On the server side, an adaptive knowledge distillation fusion strategy based on bidirectional feedback is designed to perform global model optimization, thereby improving the prediction accuracy and communication efficiency in complex traffic scenarios. The present invention not only effectively avoids user privacy leakage, but also improves the accuracy of the federated learning model and its adaptability to complex traffic scenarios.

[0006] This paper proposes a federated learning method based on bidirectional feedback knowledge distillation and differential privacy, including:

[0007] The IoV client obtains local traffic data to generate model gradient optimization parameters based on the knowledge distillation algorithm;

[0008] performing privacy enhancement processing on the model gradient optimization parameters according to an adaptive differential privacy algorithm to obtain a basic local model, wherein the adaptive differential privacy algorithm and the knowledge distillation algorithm are based on a bidirectional feedback mechanism, wherein the bidirectional feedback mechanism is an interactive adjustment loop between the adaptive differential privacy algorithm and the knowledge distillation algorithm, and the bidirectional feedback mechanism includes a client evaluation mechanism and a server evaluation mechanism, and performs dynamic collaborative optimization of the client and the server according to the client evaluation mechanism and the server evaluation mechanism;

[0009] Optimizing a two-way feedback mechanism based on supervised learning, wherein the two-way feedback mechanism optimization is based on evaluation indicators and state variables of the IoV client and IoV server. The evaluation indicators and state variables are dynamically adjusted by a multi-layer perceptron for the weights of the evaluation indicators of the IoV client and server in the two-way feedback mechanism;

[0010] The IoV server obtains the basic local models of multiple IoV clients and performs weighted average aggregation, and then performs adaptive knowledge distillation loss optimization on the IoV server according to the two-way feedback mechanism to obtain a global optimization model, and dynamically prunes and quantizes the global optimization model. The dynamic pruning and quantization are driven by knowledge distillation, and a parameter importance evaluation matrix is constructed through the knowledge distillation loss function on the IoV server. The parameter importance evaluation matrix is used to guide the pruning rate and quantization of parameters in the global optimization model, and the pruning rate and quantization bit width are dynamically adjusted according to the server-side evaluation indicators in the two-way feedback mechanism, and the quantization step size is dynamically adjusted based on the changes in the knowledge distillation loss on the IoV server.

[0011] In summary, according to the above-mentioned federated learning method based on bidirectional feedback knowledge distillation and differential privacy, a bidirectional feedback mechanism is designed to realize dynamic collaborative optimization of the client and server. On the client side, a knowledge distillation framework is designed to extract local model features in non-independent and identically distributed data environments, and the traffic data privacy is protected by combining the adaptive differential privacy algorithm. The strength of knowledge distillation and privacy protection is dynamically adjusted based on the bidirectional feedback mechanism, which significantly improves the balance between model performance and privacy protection. On the server side, an adaptive knowledge distillation fusion strategy based on bidirectional feedback is designed to perform global model optimization, thereby improving the prediction accuracy and communication efficiency in complex traffic scenarios. The present invention not only effectively avoids user privacy leakage, but also improves the accuracy of the federated learning model and its adaptability to complex traffic scenarios.Specifically, the Internet of Vehicles client obtains local traffic data to generate model gradient optimization parameters according to the knowledge distillation algorithm, which better captures the characteristics of the global data distribution. At the same time, the student model of the Internet of Vehicles client can better adapt to the local specific data characteristics of the client by optimizing the loss related to the hard labels in the local traffic data, reducing the loss of generalization performance caused by non-independent and identically distributed data, thereby avoiding the model bias caused by uneven data distribution. The model gradient optimization parameters are privacy-enhanced according to the adaptive differential privacy algorithm to obtain the basic local model. The adaptive differential privacy algorithm and the knowledge distillation algorithm are based on a two-way feedback mechanism. The two-way feedback mechanism is an interactive adjustment loop between the adaptive differential privacy algorithm and the knowledge distillation algorithm. The two-way feedback mechanism includes a client evaluation mechanism and a server evaluation mechanism. The client and the server are dynamically coordinated and optimized according to the client evaluation mechanism and the server evaluation mechanism. The adaptive differential privacy algorithm can tailor the privacy protection level according to the specific application environment and requirements, reasonably allocate the privacy budget, and avoid the problem of user privacy leakage. The two-way feedback mechanism realizes dynamic parameter adjustment and continuous performance optimization while protecting user privacy, combined with the basic weight coefficient and the flexibility. A sensitivity adjustment factor is used, while also considering the relative ratio of cross-entropy loss to knowledge distillation loss. This further achieves adaptive optimization and performs a two-way feedback mechanism optimization based on supervised learning. The two-way feedback mechanism optimization is based on evaluation indicators and state variables on the IoV client and IoV server. These evaluation indicators and state variables are dynamically adjusted using a multi-layer perceptron to adjust the weights of the evaluation indicators on the IoV client and server sides in the two-way feedback mechanism. The IoV server obtains the basic local models of multiple IoV clients and performs weighted average aggregation. Adaptive knowledge distillation loss optimization is then performed on the IoV server side based on the two-way feedback mechanism to obtain a global optimization model. Dynamic pruning and quantization of the global optimization model are then performed. This dynamic pruning and quantization is driven by knowledge distillation. A parameter importance evaluation matrix is constructed using the knowledge distillation loss function on the IoV server side. The parameter importance evaluation matrix is used to guide the pruning rate and quantization of parameters in the global optimization model. The pruning rate and quantization bit width are dynamically adjusted based on the server-side evaluation indicators in the two-way feedback mechanism. The quantization step size is dynamically adjusted based on changes in the knowledge distillation loss on the IoV server side. This invention not only effectively prevents user privacy leaks but also enhances the prediction accuracy of the overall model and its adaptability to complex traffic scenarios.

[0012] Furthermore, the IoV client obtains local traffic data to generate model gradient optimization parameters according to the knowledge distillation algorithm, specifically including:

[0013] The Internet of Vehicles client obtains local traffic data;

[0014] generating a learning hard label based on the local traffic data;

[0015] Inputting the local traffic data into a teacher model of the Internet of Vehicles client to generate learning soft labels;

[0016] Inputting the local traffic data, the learned hard labels, and the learned soft labels into a student model of an Internet of Vehicles client to obtain model gradient optimization parameters according to a knowledge distillation algorithm;

[0017] The outputs of the Internet of Vehicles client teacher model and the Internet of Vehicles client student model are softened and converted into probability outputs. The probability outputs are as follows:

[0018] ,

[0019]

[0020] in, represents the prediction probability of the IoV client teacher model, represents the predicted probability of the student model of the Internet of Vehicles client, Indicates the index number of the sample, represents the output activation function, represents the output of the IoV client teacher model, represents the output of the student model of the Internet of Vehicles client, Indicates the current The temperature parameters of the wheel, represents the reference temperature parameter, is the temperature regulation coefficient, Indicates client evaluation feedback;

[0021] The loss of the knowledge distillation algorithm for the Internet of Vehicles client is as follows:

[0022] ,

[0023] ,

[0024] ,

[0025] in, represents the total loss of the Internet of Vehicles client, represents the cross entropy loss of the Internet of Vehicles client, represents the knowledge distillation loss of the Internet of Vehicles client, represents the cross entropy loss function, represents learning hard labels, represents the number of samples in the local traffic data, Indicates the index ordinal number of the target category, represents the KL divergence.

[0026] Furthermore, the step of performing privacy enhancement processing on the model gradient optimization parameters according to the adaptive differential privacy algorithm specifically includes:

[0027] Gaussian noise is added to the model gradient optimization parameters according to the adaptive differential privacy algorithm, and the Gaussian noise is adaptively adjusted. The specific algorithm for adding Gaussian noise is as follows:

[0028] ,

[0029] ,

[0030]

[0031] in, represents the model gradient after adding Gaussian noise, represents the model gradient, represents the batch size, Indicates that the batch size is Randomly select a batch from represents the model gradient after clipping, represents Gaussian noise, Indicates the current The noise parameters of the wheel, represents the reference noise parameter, represents the noise adjustment coefficient, represents the clipping threshold, represents the noise gain factor, Indicates client evaluation feedback, and They represent the consumed differential privacy budget and total privacy budget respectively.

[0032] Furthermore, the two-way feedback mechanism specifically includes:

[0033] The two-way feedback mechanism includes client evaluation mechanism and server evaluation mechanism;

[0034] The client evaluation mechanism includes an adaptive differential privacy-knowledge distillation feedback path and a knowledge distillation-adaptive differential privacy feedback path;

[0035] The adaptive differential privacy-knowledge distillation feedback path dynamically adjusts the temperature parameter of the knowledge distillation algorithm according to the performance impact of the adaptive differential privacy algorithm. The goal of adjusting the temperature parameter is to maximize the adaptation to the non-independent and identically distributed (NIID) distribution.

[0036] The knowledge distillation-adaptive differential privacy feedback path adjusts the Gaussian noise parameters of the adaptive differential privacy algorithm based on the performance impact of the knowledge distillation algorithm. The goal of adjusting the Gaussian noise parameters is to ensure client data privacy.

[0037] The specific algorithm of the client evaluation mechanism is as follows:

[0038]

[0039]

[0040]

[0041]

[0042]

[0043] in, Indicates client evaluation feedback, 、 、 、 Respectively represent the weights of different evaluation indicators of the client, represents the gradient change evaluation index, represents the loss change evaluation indicator, represents the training time efficiency, represents the privacy utility balance indicator, represents the L2 norm of the gradient of the current communication round, represents the moving average of the L2 norm of the gradient in the previous communication round, represents the loss ratio, 、 Respectively represent the actual training time and estimated training time of the client. and denote the consumed differential privacy budget and total privacy budget respectively, represents the maximum value of the noise parameter, represents the noise parameter of the t-1 round;

[0044] The specific algorithm of the server evaluation mechanism is as follows:

[0045]

[0046]

[0047]

[0048]

[0049] in, Indicates server evaluation feedback, 、 、 Respectively represent the weights of different evaluation indicators on the server side, represents the model accuracy index, represents the communication efficiency index, represents the client engagement evaluation indicator, Indicates the accuracy of the global model in the current round in the validation set, Indicates the target accuracy, Indicates the current communication round, represents the total number of communication rounds, Indicates the number of clients participating in the current round, represents the total number of Internet of Vehicles clients for federated learning, Indicates the ordinal number of the Internet of Vehicles client. Indicates the maximum value of the client evaluation feedback.

[0050] Furthermore, the step of obtaining the basic local models of multiple Internet of Vehicles clients and performing weighted average aggregation on the Internet of Vehicles server side specifically includes:

[0051] The IoV server obtains the basic local models of multiple IoV clients and performs weighted average aggregation. The specific algorithm of the weighted average aggregation is as follows:

[0052] ,

[0053] in, represents the global model parameters of the weighted average aggregation, Indicates the current communication round, Represents the basic local model parameters of the Internet of Vehicles client, Indicates the The data weight of each client, represents the total number of Internet of Vehicles clients for federated learning, Indicates the ordinal number of the Internet of Vehicles client.

[0054] Furthermore, the step of optimizing the adaptive knowledge distillation loss on the IoV server side according to the two-way feedback mechanism specifically includes:

[0055] On the IoV server side, adaptive knowledge distillation loss optimization is performed based on a two-way feedback mechanism;

[0056] The adaptive knowledge distillation loss optimization consists of a weighted combination of cross-entropy loss and knowledge distillation loss. The weight coefficients in the weighted combination are dynamically adjusted based on server evaluation feedback. The dynamic adjustment is based on the basic weight coefficient and the sensitivity adjustment factor to adjust the relative ratio of cross-entropy loss to knowledge distillation loss to complete adaptive knowledge distillation. The temperature parameter of the adaptive knowledge distillation is dynamically adjusted based on the evaluation feedback and the initial threshold. The client selection threshold is adaptively updated and selected based on the evaluation feedback and the initial threshold according to an exponential function. The client selection threshold represents the upper limit of the number of Internet of Vehicles clients selected by the Internet of Vehicles server for weighted average aggregation;

[0057] The specific algorithm of adaptive knowledge distillation loss on the IoV server side is as follows:

[0058] ,

[0059]

[0060] in, represents the total loss of the Internet of Vehicles server, represents the cross entropy loss of the Internet of Vehicles server, represents the knowledge distillation loss of the Internet of Vehicles server, represents the adaptive weight coefficient of knowledge distillation loss, represents the basic weight coefficient, represents the sensitivity adjustment factor, Indicates server evaluation feedback;

[0061] Feedback adjustment is performed on the adaptive knowledge distillation of the IoV server and the selection threshold of the IoV client. The specific algorithm of the feedback adjustment is as follows:

[0062]

[0063]

[0064] in, Indicates the current The temperature parameters of the wheel, represents the reference temperature parameter, is the temperature regulation coefficient, Indicates the selection threshold of the Internet of Vehicles client, represents the initial selection threshold, Indicates the threshold adjustment coefficient.

[0065] Furthermore, the step of obtaining the global optimization model further includes:

[0066] The IoV server calculates the privacy overhead and checks the privacy budget;

[0067] It is determined whether the privacy overhead is greater than the privacy budget. If the privacy overhead is greater than the privacy budget, the training cycle of the global optimization model is terminated, and the client and the server are adjusted separately according to the two-way feedback mechanism.

[0068] This paper proposes a federated learning system based on bidirectional feedback knowledge distillation and differential privacy, including:

[0069] The client distillation module is used by the IoV client to obtain local traffic data and generate model gradient optimization parameters based on the knowledge distillation algorithm;

[0070] A differential privacy module, configured to perform privacy-enhancing processing on the model gradient optimization parameters according to an adaptive differential privacy algorithm to obtain a basic local model, wherein the adaptive differential privacy algorithm and the knowledge distillation algorithm are based on a two-way feedback mechanism, which is an interactive adjustment loop between the adaptive differential privacy algorithm and the knowledge distillation algorithm. The two-way feedback mechanism includes a client evaluation mechanism and a server evaluation mechanism, and performs dynamic collaborative optimization of the client and the server according to the client evaluation mechanism and the server evaluation mechanism;

[0071] A bidirectional feedback optimization module, configured to optimize a bidirectional feedback mechanism based on supervised learning. The bidirectional feedback mechanism optimization is based on evaluation indicators and state variables of the IoV client and IoV server. The evaluation indicators and state variables are dynamically adjusted using a multi-layer perceptron for the bidirectional feedback mechanism to adjust the weights of the evaluation indicators of the IoV client and server.

[0072] The server-side distillation module is used to obtain the basic local models of multiple Internet of Vehicles clients on the Internet of Vehicles server side and perform weighted average aggregation, and then perform adaptive knowledge distillation loss optimization on the Internet of Vehicles server side according to the two-way feedback mechanism to obtain a global optimization model, and dynamically prune and quantize the global optimization model. The dynamic pruning and quantization are driven by knowledge distillation, and a parameter importance evaluation matrix is constructed through the knowledge distillation loss function on the Internet of Vehicles server side. The parameter importance evaluation matrix is used to guide the pruning rate and quantization of parameters in the global optimization model, and dynamically adjust the pruning rate and quantization bit width according to the server-side evaluation indicators in the two-way feedback mechanism, and dynamically adjust the quantization step size based on changes in the knowledge distillation loss on the Internet of Vehicles server side.

[0073] The present invention also provides a storage medium, which stores one or more programs, and when the programs are executed by a processor, implements the above-mentioned federated learning method based on bidirectional feedback knowledge distillation and differential privacy.

[0074] The present invention further provides a computer device, comprising a memory and a processor, wherein:

[0075] The memory is used to store computer programs;

[0076] When the processor is used to execute the computer program stored in the memory, it implements the above-mentioned federated learning method based on bidirectional feedback knowledge distillation and differential privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 This is a flowchart of the federated learning method based on bidirectional feedback knowledge distillation and differential privacy proposed in the first embodiment of the present invention;

[0078] Figure 2 This is a schematic diagram of the structure of a federated learning system based on bidirectional feedback knowledge distillation and differential privacy proposed in the second embodiment of the present invention;

[0079] Figure 3 This is a diagram of the federated learning framework of the federated learning method based on bidirectional feedback knowledge distillation and differential privacy proposed in the first embodiment of the present invention;

[0080] Figure 4 A diagram of the bidirectional feedback mechanism of the federated learning method based on bidirectional feedback knowledge distillation and differential privacy proposed in the first embodiment of the present invention;

[0081] Figure 5 This is a diagram of the knowledge distillation framework for the Internet of Vehicles client based on the federated learning method with bidirectional feedback knowledge distillation and differential privacy proposed in the first embodiment of the present invention.

[0082] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0083] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. The drawings illustrate several embodiments of the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present invention.

[0084] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be an intermediate element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only.

[0085] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0086] See also Figure 1 , which is a flowchart of a federated learning method based on bidirectional feedback knowledge distillation and differential privacy proposed in the first embodiment of the present invention. The federated learning method based on bidirectional feedback knowledge distillation and differential privacy includes steps S01 to S04, wherein:

[0087] Step S01: The IoV client obtains local traffic data to generate model gradient optimization parameters according to the knowledge distillation algorithm;

[0088] It should be noted that the knowledge distillation framework of the Internet of Vehicles client in this embodiment is specifically referred to in Figure 5 , the IoV client obtains local traffic data;

[0089] generating a learning hard label based on the local traffic data;

[0090] Inputting the local traffic data into a teacher model of the Internet of Vehicles client to generate learning soft labels;

[0091] Inputting the local traffic data, the learned hard labels, and the learned soft labels into a student model of an Internet of Vehicles client to obtain model gradient optimization parameters according to a knowledge distillation algorithm;

[0092] The outputs of the Internet of Vehicles client teacher model and the Internet of Vehicles client student model are softened and converted into probability outputs. The probability outputs are as follows:

[0093] ,

[0094]

[0095] in, represents the prediction probability of the IoV client teacher model, represents the predicted probability of the student model of the Internet of Vehicles client, Indicates the index number of the sample, represents the output activation function, represents the output of the IoV client teacher model, represents the output of the student model of the Internet of Vehicles client, Indicates the current The temperature parameters of the wheel, represents the reference temperature parameter, is the temperature regulation coefficient, Indicates client evaluation feedback;

[0096] The loss of the knowledge distillation algorithm for the Internet of Vehicles client is as follows:

[0097] ,

[0098] ,

[0099] ,

[0100] in, represents the total loss of the Internet of Vehicles client, represents the cross entropy loss of the Internet of Vehicles client, represents the knowledge distillation loss of the Internet of Vehicles client, represents the cross entropy loss function, represents learning hard labels, represents the number of samples in the local traffic data, Indicates the index ordinal number of the target category, represents the KL divergence.

[0101] Step S02: Perform privacy enhancement processing on the model gradient optimization parameters according to the adaptive differential privacy algorithm to obtain a basic local model;

[0102] It should be noted that the adaptive differential privacy algorithm and the knowledge distillation algorithm in this embodiment are based on a two-way feedback mechanism, which is an interactive adjustment cycle between the adaptive differential privacy algorithm and the knowledge distillation algorithm;

[0103] Gaussian noise is added to the model gradient optimization parameters according to the adaptive differential privacy algorithm, and the Gaussian noise is adaptively adjusted. The specific algorithm for adding Gaussian noise is as follows:

[0104] ,

[0105] ,

[0106]

[0107] in, represents the model gradient after adding Gaussian noise, represents the model gradient, represents the batch size, Indicates that the batch size is Randomly select a batch from represents the model gradient after clipping, represents Gaussian noise, Indicates the current The noise parameters of the wheel, represents the reference noise parameter, represents the noise adjustment coefficient, represents the clipping threshold, represents the noise gain factor, Indicates client evaluation feedback, and Represent the consumed differential privacy budget and total privacy budget respectively;

[0108] For details on the bidirectional feedback mechanism in this embodiment, please refer to Figure 4 ;

[0109] The two-way feedback mechanism includes client evaluation mechanism and server evaluation mechanism;

[0110] The client evaluation mechanism includes an adaptive differential privacy-knowledge distillation feedback path and a knowledge distillation-adaptive differential privacy feedback path;

[0111] The adaptive differential privacy-knowledge distillation feedback path dynamically adjusts the temperature parameter of the knowledge distillation algorithm according to the performance impact of the adaptive differential privacy algorithm. The goal of adjusting the temperature parameter is to maximize the adaptation to the non-independent and identically distributed (NIID) distribution.

[0112] The knowledge distillation-adaptive differential privacy feedback path adjusts the Gaussian noise parameters of the adaptive differential privacy algorithm based on the performance impact of the knowledge distillation algorithm. The goal of adjusting the Gaussian noise parameters is to ensure client data privacy.

[0113] The specific algorithm of the client evaluation mechanism is as follows:

[0114]

[0115]

[0116]

[0117]

[0118]

[0119] in, Indicates client evaluation feedback, 、 、 、 Respectively represent the weights of different evaluation indicators of the client, represents the gradient change evaluation index, represents the loss change evaluation indicator, represents the training time efficiency, represents the privacy utility balance indicator, represents the L2 norm of the gradient of the current communication round, represents the moving average of the L2 norm of the gradient in the previous communication round, represents the loss ratio, 、 Respectively represent the actual training time and estimated training time of the client. and denote the consumed differential privacy budget and total privacy budget respectively, represents the maximum value of the noise parameter, represents the noise parameter of the t-1 round;

[0120] The specific algorithm of the server evaluation mechanism is as follows:

[0121]

[0122]

[0123]

[0124]

[0125] in, Indicates server evaluation feedback, 、 、 Respectively represent the weights of different evaluation indicators on the server side, represents the model accuracy index, represents the communication efficiency index, represents the client engagement evaluation indicator, Indicates the accuracy of the global model in the current round in the validation set, Indicates the target accuracy, Indicates the current communication round, represents the total number of communication rounds, Indicates the number of clients participating in the current round, represents the total number of Internet of Vehicles clients for federated learning, Indicates the ordinal number of the Internet of Vehicles client. Indicates the maximum value of the client evaluation feedback.

[0126] Step S03: Optimizing the two-way feedback mechanism based on supervised learning;

[0127] It should be noted that the bidirectional feedback mechanism optimization described in this embodiment is based on the evaluation indicators and state variables of the Internet of Vehicles client and the Internet of Vehicles server. The evaluation indicators and state variables are used to dynamically adjust the evaluation indicator weights of the Internet of Vehicles client and the server in the bidirectional feedback mechanism through a multi-layer perceptron. The multi-layer perceptron maps the input evaluation indicators and state variables to the optimal weights through learnable parameters to achieve dynamic adjustment of the evaluation indicator weights. The specific algorithm for dynamic adjustment of the evaluation indicator weights is as follows:

[0128]

[0129]

[0130] 、 Represent the weights of the client-side and server-side evaluation indicators, 、 Represent the supervised learning multi-layer perceptrons on the client and server sides respectively, 、 Represent the state variables of the client and server respectively, 、 Represent the learnable parameters of the supervised learning multilayer perceptron on the client and server sides respectively, 、 Represent the evaluation indicators of the client and server respectively.

[0131] Step S04: The IoV server obtains the basic local models of multiple IoV clients and performs weighted average aggregation. Then, the IoV server performs adaptive knowledge distillation loss optimization based on the two-way feedback mechanism to obtain a global optimization model, and dynamically prunes and quantizes the global optimization model.

[0132] It should be noted that in this embodiment, the Internet of Vehicles server obtains the basic local models of multiple Internet of Vehicles clients and performs weighted average aggregation. The specific algorithm of the weighted average aggregation is as follows:

[0133] ,

[0134] in, represents the global model parameters of the weighted average aggregation, Indicates the current communication round, Represents the basic local model parameters of the Internet of Vehicles client, Indicates the The data weight of each client, represents the total number of Internet of Vehicles clients for federated learning, Indicates the ordinal number of the Internet of Vehicles client;

[0135] The weighted average aggregation algorithm in this embodiment is based on the federated learning framework. For the specific architecture of the federated learning framework, please refer to Figure 3 ;

[0136] On the IoV server side, adaptive knowledge distillation loss optimization is performed based on a two-way feedback mechanism;

[0137] The adaptive knowledge distillation loss optimization consists of a weighted combination of cross-entropy loss and knowledge distillation loss. The weight coefficients in the weighted combination are dynamically adjusted based on server evaluation feedback. The dynamic adjustment is based on the basic weight coefficient and the sensitivity adjustment factor to adjust the relative ratio of cross-entropy loss to knowledge distillation loss to complete adaptive knowledge distillation. The temperature parameter of the adaptive knowledge distillation is dynamically adjusted based on the evaluation feedback and the initial threshold. The client selection threshold is adaptively updated and selected based on the evaluation feedback and the initial threshold according to an exponential function. The client selection threshold represents the upper limit of the number of Internet of Vehicles clients selected by the Internet of Vehicles server for weighted average aggregation;

[0138] The specific algorithm of adaptive knowledge distillation loss on the IoV server side is as follows:

[0139] ,

[0140]

[0141] in, represents the total loss of the Internet of Vehicles server, represents the cross entropy loss of the Internet of Vehicles server, represents the knowledge distillation loss of the Internet of Vehicles server, represents the adaptive weight coefficient of knowledge distillation loss, represents the basic weight coefficient, represents the sensitivity adjustment factor, Indicates server evaluation feedback;

[0142] Feedback adjustment is performed on the adaptive knowledge distillation of the IoV server and the selection threshold of the IoV client. The specific algorithm of the feedback adjustment is as follows:

[0143]

[0144]

[0145] in, Indicates the current The temperature parameters of the wheel, represents the reference temperature parameter, is the temperature regulation coefficient, Indicates the selection threshold of the Internet of Vehicles client, represents the initial selection threshold, represents the threshold adjustment coefficient;

[0146] The dynamic pruning and quantization described in this embodiment are driven by knowledge distillation. A parameter importance evaluation matrix is constructed using the knowledge distillation loss function on the IoV server. The parameter importance evaluation matrix is used to guide the pruning rate and quantization of parameters in the global optimization model. The pruning rate and quantization bit width are dynamically adjusted based on the server-side evaluation indicators in the bidirectional feedback mechanism. The quantization step size is dynamically adjusted based on changes in the knowledge distillation loss on the IoV server. Pruning is used to remove parameters that have little contribution or impact on the knowledge distillation process. Quantization is used to convert model parameters from high precision to low precision to reduce storage and transmission requirements.

[0147] The specific algorithms for dynamic pruning and quantization in this embodiment are as follows:

[0148]

[0149]

[0150]

[0151]

[0152] in, represents the parameter importance evaluation matrix, represents the knowledge distillation loss of the Internet of Vehicles server, represents the squared expected value of the global optimization model parameters, represents the pruning rate, represents the maximum pruning rate, represents the sensitivity of pruning rate, represents the model accuracy index, represents the communication efficiency index, represents the client engagement evaluation indicator, represents the global optimization model parameters, Indicates the quantization bit width, represents the quantization step size, represents the basic quantization step size, Indicates the current communication round;

[0153] The IoV server calculates the privacy overhead and checks the privacy budget;

[0154] It is determined whether the privacy overhead is greater than the privacy budget. If the privacy overhead is greater than the privacy budget, the training cycle of the global optimization model is terminated, and the client and the server are adjusted separately according to the two-way feedback mechanism.

[0155] In summary, according to the above-mentioned federated learning method based on bidirectional feedback knowledge distillation and differential privacy, a bidirectional feedback mechanism is designed to realize dynamic collaborative optimization of the client and server. On the client side, a knowledge distillation framework is designed to extract local model features in non-independent and identically distributed data environments, and the traffic data privacy is protected by combining the adaptive differential privacy algorithm. The strength of knowledge distillation and privacy protection is dynamically adjusted based on the bidirectional feedback mechanism, which significantly improves the balance between model performance and privacy protection. On the server side, an adaptive knowledge distillation fusion strategy based on bidirectional feedback is designed to perform global model optimization, thereby improving the prediction accuracy and communication efficiency in complex traffic scenarios. The present invention not only effectively avoids user privacy leakage, but also improves the accuracy of the federated learning model and its adaptability to complex traffic scenarios.Specifically, the Internet of Vehicles client obtains local traffic data to generate model gradient optimization parameters according to the knowledge distillation algorithm, which better captures the characteristics of the global data distribution. At the same time, the student model of the Internet of Vehicles client can better adapt to the local specific data characteristics of the client by optimizing the loss related to the hard labels in the local traffic data, reducing the loss of generalization performance caused by non-independent and identically distributed data, thereby avoiding the model bias caused by uneven data distribution. The model gradient optimization parameters are privacy-enhanced according to the adaptive differential privacy algorithm to obtain the basic local model. The adaptive differential privacy algorithm and the knowledge distillation algorithm are based on a two-way feedback mechanism. The two-way feedback mechanism is an interactive adjustment loop between the adaptive differential privacy algorithm and the knowledge distillation algorithm. The two-way feedback mechanism includes a client evaluation mechanism and a server evaluation mechanism. The client and the server are dynamically coordinated and optimized according to the client evaluation mechanism and the server evaluation mechanism. The adaptive differential privacy algorithm can tailor the privacy protection level according to the specific application environment and requirements, reasonably allocate the privacy budget, and avoid the problem of user privacy leakage. The two-way feedback mechanism realizes dynamic parameter adjustment and continuous performance optimization while protecting user privacy, combined with the basic weight coefficient and the flexibility. A sensitivity adjustment factor is used, while also considering the relative ratio of cross-entropy loss to knowledge distillation loss. This further achieves adaptive optimization and performs a two-way feedback mechanism optimization based on supervised learning. The two-way feedback mechanism optimization is based on evaluation indicators and state variables on the IoV client and IoV server. These evaluation indicators and state variables are dynamically adjusted using a multi-layer perceptron to adjust the weights of the evaluation indicators on the IoV client and server sides in the two-way feedback mechanism. The IoV server obtains the basic local models of multiple IoV clients and performs weighted average aggregation. Adaptive knowledge distillation loss optimization is then performed on the IoV server side based on the two-way feedback mechanism to obtain a global optimization model. Dynamic pruning and quantization of the global optimization model are then performed. This dynamic pruning and quantization is driven by knowledge distillation. A parameter importance evaluation matrix is constructed using the knowledge distillation loss function on the IoV server side. The parameter importance evaluation matrix is used to guide the pruning rate and quantization of parameters in the global optimization model. The pruning rate and quantization bit width are dynamically adjusted based on the server-side evaluation indicators in the two-way feedback mechanism. The quantization step size is dynamically adjusted based on changes in the knowledge distillation loss on the IoV server side. This invention not only effectively prevents user privacy leaks but also enhances the prediction accuracy of the overall model and its adaptability to complex traffic scenarios.

[0156] See also Figure 2 , which is a schematic diagram of the structure of a federated learning system based on bidirectional feedback knowledge distillation and differential privacy proposed in the second embodiment of the present invention. The system includes:

[0157] The client distillation module 10 is used for the IoV client to obtain local traffic data to generate model gradient optimization parameters according to the knowledge distillation algorithm;

[0158] A differential privacy module 20 is configured to perform privacy enhancement processing on the model gradient optimization parameters according to an adaptive differential privacy algorithm to obtain a basic local model. The adaptive differential privacy algorithm and the knowledge distillation algorithm are based on a two-way feedback mechanism. The two-way feedback mechanism is an interactive adjustment loop between the adaptive differential privacy algorithm and the knowledge distillation algorithm. The two-way feedback mechanism includes a client evaluation mechanism and a server evaluation mechanism, and dynamic collaborative optimization of the client and the server is performed according to the client evaluation mechanism and the server evaluation mechanism.

[0159] a bidirectional feedback optimization module 30 for optimizing a bidirectional feedback mechanism based on supervised learning, wherein the bidirectional feedback mechanism optimization is based on evaluation indicators and state variables of the IoV client and IoV server, and wherein the evaluation indicators and state variables are dynamically adjusted by a multi-layer perceptron for weighting the evaluation indicators of the IoV client and server in the bidirectional feedback mechanism;

[0160] The server-side distillation module 40 is used to obtain the basic local models of multiple Internet of Vehicles clients on the Internet of Vehicles server side and perform weighted average aggregation, and then perform adaptive knowledge distillation loss optimization on the Internet of Vehicles server side according to the two-way feedback mechanism to obtain a global optimization model, and dynamically prune and quantize the global optimization model. The dynamic pruning and quantization are driven by knowledge distillation, and a parameter importance evaluation matrix is constructed through the knowledge distillation loss function on the Internet of Vehicles server side. The parameter importance evaluation matrix is used to guide the pruning rate and quantization of parameters in the global optimization model, and dynamically adjust the pruning rate and quantization bit width according to the server-side evaluation indicators in the two-way feedback mechanism, and dynamically adjust the quantization step size based on the changes in the knowledge distillation loss on the Internet of Vehicles server side.

[0161] The present invention also proposes a computer storage medium on which one or more programs are stored, which, when executed by a processor, implement the above-mentioned federated learning method based on bidirectional feedback knowledge distillation and differential privacy.

[0162] The present invention also proposes a computer device comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the above-mentioned federated learning method based on bidirectional feedback knowledge distillation and differential privacy.

[0163] Those skilled in the art will appreciate that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or in conjunction with such instruction execution system, apparatus, or device. For purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device.

[0164] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting, or processing it in another suitable manner as necessary, and then storing it in a computer memory.

[0165] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the aforementioned embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following technologies known in the art may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0166] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0167] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A federated learning method based on bidirectional feedback knowledge distillation and differential privacy, characterized by: include: The IoV client obtains local traffic data to generate model gradient optimization parameters based on the knowledge distillation algorithm; performing privacy enhancement processing on the model gradient optimization parameters according to an adaptive differential privacy algorithm to obtain a basic local model, wherein the adaptive differential privacy algorithm and the knowledge distillation algorithm are based on a bidirectional feedback mechanism, wherein the bidirectional feedback mechanism is an interactive adjustment loop between the adaptive differential privacy algorithm and the knowledge distillation algorithm, and the bidirectional feedback mechanism includes a client evaluation mechanism and a server evaluation mechanism, and performs dynamic collaborative optimization of the client and the server according to the client evaluation mechanism and the server evaluation mechanism; The two-way feedback mechanism specifically includes: The two-way feedback mechanism includes client evaluation mechanism and server evaluation mechanism; The client evaluation mechanism includes an adaptive differential privacy-knowledge distillation feedback path and a knowledge distillation-adaptive differential privacy feedback path; The adaptive differential privacy-knowledge distillation feedback path dynamically adjusts the temperature parameter of the knowledge distillation algorithm according to the performance impact of the adaptive differential privacy algorithm. The goal of adjusting the temperature parameter is to maximize the adaptation to the non-independent and identically distributed (NIID) distribution. The knowledge distillation-adaptive differential privacy feedback path adjusts the Gaussian noise parameters of the adaptive differential privacy algorithm based on the performance impact of the knowledge distillation algorithm. The goal of adjusting the Gaussian noise parameters is to ensure client data privacy. The specific algorithm of the client evaluation mechanism is as follows: in, Indicates client evaluation feedback, 、 、 、 Respectively represent the weights of different evaluation indicators of the client, represents the gradient change evaluation index, represents the loss change evaluation indicator, represents the training time efficiency, represents the privacy utility balance indicator, represents the L2 norm of the gradient of the current communication round, represents the moving average of the L2 norm of the gradient in the previous communication round, represents the loss ratio, 、 Respectively represent the actual training time and estimated training time of the client. and denote the consumed differential privacy budget and total privacy budget respectively, represents the maximum value of the noise parameter, represents the noise parameter of the t-1 round; The specific algorithm of the server evaluation mechanism is as follows: in, Indicates server evaluation feedback, 、 、 Respectively represent the weights of different evaluation indicators on the server side, represents the model accuracy index, represents the communication efficiency index, represents the client engagement evaluation indicator, Indicates the accuracy of the global model in the current round in the validation set, Indicates the target accuracy, Indicates the current communication round, represents the total number of communication rounds, Indicates the number of clients participating in the current round, represents the total number of Internet of Vehicles clients for federated learning, Indicates the ordinal number of the Internet of Vehicles client. Indicates the maximum value of the client evaluation feedback; Optimizing a two-way feedback mechanism based on supervised learning, wherein the two-way feedback mechanism optimization is based on evaluation indicators and state variables of the IoV client and IoV server. The evaluation indicators and state variables are dynamically adjusted by a multi-layer perceptron for the weights of the evaluation indicators of the IoV client and server in the two-way feedback mechanism; The IoV server obtains the basic local models of multiple IoV clients and performs weighted average aggregation, and then performs adaptive knowledge distillation loss optimization on the IoV server according to the two-way feedback mechanism to obtain a global optimization model, and dynamically prunes and quantizes the global optimization model. The dynamic pruning and quantization are driven by knowledge distillation, and a parameter importance evaluation matrix is constructed through the knowledge distillation loss function on the IoV server. The parameter importance evaluation matrix is used to guide the pruning rate and quantization of parameters in the global optimization model, and the pruning rate and quantization bit width are dynamically adjusted according to the server-side evaluation indicators in the two-way feedback mechanism, and the quantization step size is dynamically adjusted based on the changes in the knowledge distillation loss on the IoV server.

2. The federated learning method based on bidirectional feedback knowledge distillation and differential privacy according to claim 1 is characterized in that: The IoV client obtains local traffic data to generate model gradient optimization parameters according to the knowledge distillation algorithm, specifically including: The Internet of Vehicles client obtains local traffic data; generating a learning hard label based on the local traffic data; Inputting the local traffic data into a teacher model of the Internet of Vehicles client to generate learning soft labels; Inputting the local traffic data, the learned hard labels, and the learned soft labels into a student model of an Internet of Vehicles client to obtain model gradient optimization parameters according to a knowledge distillation algorithm; The outputs of the Internet of Vehicles client teacher model and the Internet of Vehicles client student model are softened and converted into probability outputs. The probability outputs are as follows: ， in, represents the prediction probability of the IoV client teacher model, represents the predicted probability of the student model of the Internet of Vehicles client, Indicates the index number of the sample, represents the output activation function, represents the output of the IoV client teacher model, represents the output of the student model of the Internet of Vehicles client, Indicates the current The temperature parameters of the wheel, represents the reference temperature parameter, is the temperature regulation coefficient, Indicates client evaluation feedback; The loss of the knowledge distillation algorithm for the Internet of Vehicles client is as follows: ， ， ， in, represents the total loss of the Internet of Vehicles client, represents the cross entropy loss of the Internet of Vehicles client, represents the knowledge distillation loss of the Internet of Vehicles client, represents the cross entropy loss function, represents learning hard labels, represents the number of samples in the local traffic data, Indicates the index ordinal number of the target category, represents the KL divergence.

3. The federated learning method based on bidirectional feedback knowledge distillation and differential privacy according to claim 1, characterized in that: The step of performing privacy enhancement processing on the model gradient optimization parameters according to the adaptive differential privacy algorithm specifically includes: Gaussian noise is added to the model gradient optimization parameters according to the adaptive differential privacy algorithm, and the Gaussian noise is adaptively adjusted. The specific algorithm for adding Gaussian noise is as follows: ， ， in, represents the model gradient after adding Gaussian noise, represents the model gradient, represents the batch size, Indicates that the batch size is Randomly select a batch from represents the model gradient after clipping, represents Gaussian noise, Indicates the current The noise parameters of the wheel, represents the reference noise parameter, represents the noise adjustment coefficient, represents the clipping threshold, represents the noise gain factor, Indicates client evaluation feedback, and They represent the consumed differential privacy budget and total privacy budget respectively.

4. The federated learning method based on bidirectional feedback knowledge distillation and differential privacy according to claim 1, characterized in that: The step of obtaining the basic local models of multiple Internet of Vehicles clients and performing weighted average aggregation on the Internet of Vehicles server specifically includes: The IoV server obtains the basic local models of multiple IoV clients and performs weighted average aggregation. The specific algorithm of the weighted average aggregation is as follows: ， in, represents the global model parameters of the weighted average aggregation, Indicates the current communication round, Represents the basic local model parameters of the Internet of Vehicles client, Indicates the The data weight of each client, represents the total number of Internet of Vehicles clients for federated learning, Indicates the ordinal number of the Internet of Vehicles client.

5. The federated learning method based on bidirectional feedback knowledge distillation and differential privacy according to claim 1, characterized in that: The step of optimizing the adaptive knowledge distillation loss on the IoV server side based on the two-way feedback mechanism specifically includes: On the IoV server side, adaptive knowledge distillation loss optimization is performed based on a two-way feedback mechanism; The adaptive knowledge distillation loss optimization consists of a weighted combination of cross-entropy loss and knowledge distillation loss. The weight coefficients in the weighted combination are dynamically adjusted based on server evaluation feedback. The dynamic adjustment is based on the basic weight coefficient and the sensitivity adjustment factor to adjust the relative ratio of cross-entropy loss to knowledge distillation loss to complete adaptive knowledge distillation. The temperature parameter of the adaptive knowledge distillation is dynamically adjusted based on the evaluation feedback and the initial threshold. The client selection threshold is adaptively updated and selected based on the evaluation feedback and the initial threshold according to an exponential function. The client selection threshold represents the upper limit of the number of Internet of Vehicles clients selected by the Internet of Vehicles server for weighted average aggregation; The specific algorithm of adaptive knowledge distillation loss on the IoV server side is as follows: ， in, represents the total loss of the Internet of Vehicles server, represents the cross entropy loss of the Internet of Vehicles server, represents the knowledge distillation loss of the Internet of Vehicles server, represents the adaptive weight coefficient of knowledge distillation loss, represents the basic weight coefficient, represents the sensitivity adjustment factor, Indicates server evaluation feedback; Feedback adjustment is performed on the adaptive knowledge distillation of the IoV server and the selection threshold of the IoV client. The specific algorithm of the feedback adjustment is as follows: in, Indicates the current The temperature parameters of the wheel, represents the reference temperature parameter, is the temperature regulation coefficient, Indicates the selection threshold of the Internet of Vehicles client, represents the initial selection threshold, Indicates the threshold adjustment coefficient.

6. The federated learning method based on bidirectional feedback knowledge distillation and differential privacy according to claim 1, characterized in that: The step of obtaining the global optimization model further includes: The IoV server calculates the privacy overhead and checks the privacy budget; It is determined whether the privacy overhead is greater than the privacy budget. If the privacy overhead is greater than the privacy budget, the training cycle of the global optimization model is terminated, and the client and the server are adjusted separately according to the two-way feedback mechanism.

7. A federated learning system based on bidirectional feedback knowledge distillation and differential privacy, characterized by: include: The client distillation module is used by the IoV client to obtain local traffic data and generate model gradient optimization parameters based on the knowledge distillation algorithm; A differential privacy module, configured to perform privacy-enhancing processing on the model gradient optimization parameters according to an adaptive differential privacy algorithm to obtain a basic local model, wherein the adaptive differential privacy algorithm and the knowledge distillation algorithm are based on a two-way feedback mechanism, which is an interactive adjustment loop between the adaptive differential privacy algorithm and the knowledge distillation algorithm. The two-way feedback mechanism includes a client evaluation mechanism and a server evaluation mechanism, and performs dynamic collaborative optimization of the client and the server according to the client evaluation mechanism and the server evaluation mechanism; The two-way feedback mechanism specifically includes: The two-way feedback mechanism includes client evaluation mechanism and server evaluation mechanism; The client evaluation mechanism includes an adaptive differential privacy-knowledge distillation feedback path and a knowledge distillation-adaptive differential privacy feedback path; The adaptive differential privacy-knowledge distillation feedback path dynamically adjusts the temperature parameter of the knowledge distillation algorithm according to the performance impact of the adaptive differential privacy algorithm. The goal of adjusting the temperature parameter is to maximize the adaptation to the non-independent and identically distributed (NIID) distribution. The knowledge distillation-adaptive differential privacy feedback path adjusts the Gaussian noise parameters of the adaptive differential privacy algorithm based on the performance impact of the knowledge distillation algorithm. The goal of adjusting the Gaussian noise parameters is to ensure client data privacy. The specific algorithm of the client evaluation mechanism is as follows: in, Indicates client evaluation feedback, 、 、 、 Respectively represent the weights of different evaluation indicators of the client, represents the gradient change evaluation index, represents the loss change evaluation indicator, represents the training time efficiency, represents the privacy utility balance indicator, represents the L2 norm of the gradient of the current communication round, represents the moving average of the L2 norm of the gradient in the previous communication round, represents the loss ratio, 、 Respectively represent the actual training time and estimated training time of the client. and denote the consumed differential privacy budget and total privacy budget respectively, represents the maximum value of the noise parameter, represents the noise parameter of the t-1 round; The specific algorithm of the server evaluation mechanism is as follows: in, Indicates server evaluation feedback, 、 、 Respectively represent the weights of different evaluation indicators on the server side, represents the model accuracy index, represents the communication efficiency index, represents the client engagement evaluation indicator, Indicates the accuracy of the global model in the current round in the validation set, Indicates the target accuracy, Indicates the current communication round, represents the total number of communication rounds, Indicates the number of clients participating in the current round, represents the total number of Internet of Vehicles clients for federated learning, Indicates the ordinal number of the Internet of Vehicles client. Indicates the maximum value of the client evaluation feedback; A bidirectional feedback optimization module, configured to optimize a bidirectional feedback mechanism based on supervised learning. The bidirectional feedback mechanism optimization is based on evaluation indicators and state variables of the IoV client and IoV server. The evaluation indicators and state variables are dynamically adjusted using a multi-layer perceptron for the bidirectional feedback mechanism to adjust the weights of the evaluation indicators of the IoV client and server. The server-side distillation module is used to obtain the basic local models of multiple Internet of Vehicles clients on the Internet of Vehicles server side and perform weighted average aggregation, and then perform adaptive knowledge distillation loss optimization on the Internet of Vehicles server side according to the two-way feedback mechanism to obtain a global optimization model, and dynamically prune and quantize the global optimization model. The dynamic pruning and quantization are driven by knowledge distillation, and a parameter importance evaluation matrix is constructed through the knowledge distillation loss function on the Internet of Vehicles server side. The parameter importance evaluation matrix is used to guide the pruning rate and quantization of parameters in the global optimization model, and dynamically adjust the pruning rate and quantization bit width according to the server-side evaluation indicators in the two-way feedback mechanism, and dynamically adjust the quantization step size based on changes in the knowledge distillation loss on the Internet of Vehicles server side.

8. A storage medium, characterized in that: The storage medium stores one or more programs, which, when executed by the processor, implement the federated learning method based on bidirectional feedback knowledge distillation and differential privacy as described in any one of claims 1 to 6.

9. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein: The memory is used to store computer programs; When the processor is used to execute the computer program stored in the memory, it implements the federated learning method based on bidirectional feedback knowledge distillation and differential privacy described in any one of claims 1-6.

Images