Federal prototype learning method fusing personalization and generalization in Internet of Vehicles

Abstract

The invention relates to a federal prototype learning method fusing personalization and generalization in the Internet of Vehicles, and belongs to the technical field of mobile communication. For the problems of low model precision, poor fairness and fuzzy class boundary caused by data heterogeneity of the Internet of Vehicles, a double-layer architecture based on a DAG block chain is proposed to realize model security sharing, and the contribution degree distribution of a model is optimized by measuring the influence between vehicles through a dynamic hierarchical aggregation algorithm. And a federal prototype learning algorithm is designed to fuse generalization prototype and local prototype features. And a feature enhancement layer dynamic contrast learning mechanism is adopted, so that the model synchronously captures public knowledge, collaborative knowledge and personalized features, and the problem of unbalance between personalization and generalization is effectively solved. According to the method, the training efficiency, the classification precision and the fairness of the intelligent driving decision model are remarkably improved, and the method has outstanding advantages in the aspects of data safety, class boundary definition and asynchronous cooperative training.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile communications and relates to a federated prototype learning method integrating personalization and generalization in a vehicle-to-everything (V2X) network. Background Art

[0002] In recent years, connected and automated vehicles (CAVs) have emerged as a key vertical application of the sixth generation (6G) of mobile communications, generating an unprecedented amount of sensing data in real time at multiple geographically dispersed locations through in-vehicle sensors and communication technologies. Traditional machine learning can effectively support the centralized training of intelligent driving models such as pedestrian detection, traffic flow prediction, and path planning through the bottom-up collection of large-scale sensing data by roadside units (RSUs) or base stations (BSs). However, the centralized machine learning approach has a single point of failure problem and cannot meet the requirements of user privacy and data security.

[0003] Federated learning (FL), as a distributed learning paradigm that exchanges models rather than raw data, has become an effective solution for protecting user privacy. In addition, blockchain technology enables RSUs or BSs to jointly maintain a traceable and immutable distributed ledger to achieve data security while preventing single point of failure problems. However, due to the dynamic and complex changes in the V2X environment and different user driving preferences, the collected data from different vehicles exhibits obvious non-independent and identically distributed (Non-IID) characteristics, resulting in a reduction in the accuracy of traditional FL models. To address this issue, personalized federated learning (PFL) has emerged, aiming to train personalized models adapted to the local data distribution for different CAVs. However, existing PFL methods typically treat each model as an indivisible whole and measure the similarity of CAVs by calculating the distance between the entire model parameters or loss values. This approach ignores the functional differences of different neural layers, leading to inaccurate model contribution allocation.

[0004] Meanwhile, each CAV cannot access other vehicle data and does not even require a public dataset. Therefore, the model will inevitably lose some important global information during local training and may even overfit the local data, resulting in limited model performance and multi-vehicle cooperation effects. Therefore, introducing additional common knowledge in model training is beneficial to enhancing both the personalization ability and generalization ability of the model. Prototype Learning (PL), as an emerging machine learning technology, can effectively inject common knowledge into the FL model by learning a representative lightweight "prototype" for each category to represent the common semantic information of that category. However, existing PL methods usually use the weighted average of all sample features as the class prototype. This averaging method cannot accurately represent the complex Non-IID data information. At the same time, samples of different categories may be similarly distributed or even overlap in the feature space, resulting in poor distinguishability between different prototypes and a decline in the quality of common knowledge. In addition, affected by the class overlap problem, the decision boundary of the model is prone to becoming blurred, leading to a further reduction in model accuracy.

[0005] Based on the above problems, the present invention designs a federated prototype learning method that combines personalization and generalization, which can alleviate the class overlap problem, improve model accuracy, and vehicle fairness. In this method, each CAV has a Local Driving Model (LDM) and a Prototype Learning Model (PLM). The LDM is divided into three parts: a globally shared feature extractor, a feature enhancement layer that absorbs prototype knowledge, and a locally retained decision head. The PLM is used to autonomously learn generalization prototypes with good distinguishability to provide high-quality common knowledge. The CAVs will sequentially perform feature extractor aggregation, generalization prototype training, and local contrast learning, and finally generate high-quality local models. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a federated prototype learning method that combines personalization and generalization in a vehicle networking system, which is used to solve the problems of low data security, loss of common knowledge, poor vehicle fairness, and serious class overlap in data heterogeneous scenarios.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] In the first aspect, according to the characteristics of the vehicle networking scenario and the requirements of data privacy protection, the embodiments of the present invention use DAG blockchain technology to realize the asynchronous training and secure sharing of personalized models. The method includes the following steps:

[0009] S1: A two-layer vehicle networking architecture based on DAG blockchain;

[0010] S2: Influence-based Dynamic Hierarchical Aggregation Algorithm;

[0011] S3: Generalized Prototype Learning Based on Contrastive Training;

[0012] S4: Feature-Enhanced Federated Prototype Learning Algorithm.

[0013] In a second aspect, in S1 of the embodiments of the present invention, a two-layer vehicle-to-everything (V2X) network architecture based on DAG blocks is established, including an edge layer and a vehicle layer. All RSUs in the edge layer jointly maintain a DAG blockchain network and store FL model data in the blockchain in the form of transactions, thereby providing security and anti-tampering for data sharing. The vehicle layer consists of randomly moving CAVs, which are responsible for jointly completing the FL training task. Among them, each CAV locally has an LDM for driving decision-making and a PLM for prototype knowledge learning and LDM training guidance.

[0014] The RSU is equipped with a micro edge server, which has rich computing and communication resources and can provide high computing power and low latency services for CAVs. The CAV is equipped with advanced on-vehicle sensors, and its sensed data is used for collaborative training and driving decision-making. The different road environments where CAVs are located make their sensed data exhibit Non-IID characteristics. Therefore, there is usually an obvious deviation between the local optimization direction of the local model and the global optimization direction, resulting in a decrease in the accuracy of the aggregated global model. Therefore, the PFL method is used to train different personalized local models for different CAVs, thereby transforming the data heterogeneity problem into an individualized information advantage.

[0015] In a third aspect, in S2 of the embodiments of the present invention, an Influence-oriented Dynamic Layer-wise Aggregation (IDLA) algorithm is designed to aggregate highly personalized initial LDMs for CAVs in each round of global training. Since different layers of the neural network model may have different utilities, it is difficult for the overall metric of the model to accurately reflect the hierarchical functional differences. Therefore, hierarchical aggregation is used instead of conventional model aggregation to achieve more accurate personalized training. At the same time, the conventional fixed aggregation weights or multi-sensitive parameter adjustment methods are difficult to adapt to different data distributions. Therefore, the present invention fully exploits the useful contributions of different models by quantifying the hierarchical influence among CAVs and dynamically guides CAVs to aggregate cross-domain knowledge from similar models in a personalized manner.

[0016] Fourthly, in S3 of the embodiments of the present invention, a PLM is introduced for each CAV, and a contrastive training method is adopted to guide the PLM to autonomously learn high-quality generalization prototypes. In order to accurately represent the common information of each category, the generalization prototypes generated by the PLM need to meet two conditions: firstly, each generalization prototype should be aligned with the local prototypes of the same category as much as possible to represent the global information of this category; secondly, each generalization prototype should be as far away from the local prototypes of different categories as possible to ensure the clarity of the prototype boundary. Based on this, the PLM uses contrastive learning to jointly optimize the prototype distance within the same category and the prototype distance between different categories, so as to continuously learn generalization prototypes with good representativeness and distinguishability, thereby providing common knowledge for subsequent LDM training.

[0017] Fifthly, in S4 of the embodiments of the present invention, a Feature-enhanced Federated Prototype Learning (FFPL) algorithm that combines personalization and generalization is proposed to achieve high-quality training of personalized LDM. First, the LDM dynamically extracts common knowledge from the generalization prototypes and improves the clarity of class boundaries. The LDM guides each sample feature to be close to the generalization prototypes of the same category and far from the generalization prototypes of different categories, and dynamically adjusts the prototype guidance ratio according to the quality of the prototype knowledge. Secondly, the LDM learns collaborative knowledge from the local prototypes and takes into account the fine-grained intra-class differences. The intra-class similarity between CAVs is measured by calculating the distance between the local prototypes of the same category, and similar local prototypes are encouraged to be close to each other while dissimilar local prototypes maintain a distance, thereby improving the personalization ability of the model. Finally, the LDM learns personalized task knowledge from local data through supervised training to meet the needs of different CAVs. In summary, by integrating common knowledge, collaborative knowledge, and personalized knowledge, both the local personalized learning goal and the multi-vehicle collaborative training goal are satisfied, thereby improving the model accuracy and vehicle fairness.

[0018] The beneficial effects of the present invention are as follows: As a distributed learning paradigm, Federated Learning (FL) protects user privacy by interacting with models rather than raw data. However, the complex changes in the Internet of Vehicles (IoV) environment lead to high data heterogeneity, poor vehicle fairness, and blurred classification boundaries, thereby reducing the speed and accuracy of model training. In addition, models that only have access to local data will lose some common information during local training, resulting in a single global model being difficult to simultaneously meet personalized learning goals and collaborative learning goals. To solve the above problems, the present invention first constructs a two-layer IoV architecture based on a Directed Acyclic Graph (DAG) blockchain, including an edge layer and a vehicle layer, which are responsible for the secure sharing of model data and the training and verification of personalized models respectively. Secondly, an influence-based dynamic hierarchical aggregation algorithm is proposed, which abandons the traditional fixed aggregation weights or dependence on multiple sensitive parameters, and dynamically optimizes the hierarchical contribution degree distribution of different models by finely measuring the influence between vehicles. Finally, a feature-enhanced federated prototype learning algorithm is designed. By introducing a prototype learner to autonomously learn generalization prototypes with high distinguishability, and based on generalization prototypes, local prototypes, and local tasks, dynamic contrast learning is used to guide the model to simultaneously capture personalized feature knowledge, collaborative feature knowledge, and common feature knowledge, thereby improving model accuracy, vehicle fairness, and class boundary clarity. This solution is oriented to the IoV scenario with data heterogeneity, realizes asynchronous model training with high security, high precision, high fairness, and high personalization, and can effectively assist intelligent driving decisions.

[0019] Other advantages, objectives, and features of the present invention will be described to some extent in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the following specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail preferably with reference to the accompanying drawings, where:

[0021] Figure 1 It is a two-layer IoV architecture diagram based on a DAG blockchain;

[0022] Figure 2 It is a schematic diagram of generalization prototype learning based on contrast training;

[0023] Figure 3 It is an implementation idea diagram of a feature-enhanced federated prototype learning algorithm;

[0024] Figure 4The execution flowchart of the federated prototype learning method that integrates personalization and generalization in the vehicle network. Specific implementation manners

[0025] The following uses specific specific examples to illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concept of the present invention. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0026] Among them, the attached drawings are only for illustrative purposes, showing only schematic diagrams, not physical diagrams, and cannot be understood as a limitation to the present invention; in order to better illustrate the embodiments of the present invention, some components in the attached drawings will be omitted, enlarged or reduced, which does not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the attached drawings may be omitted.

[0027] In the attached drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicating the orientation or positional relationship, it is based on the orientation or positional relationship shown in the attached drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the attached drawings are only for illustrative purposes and cannot be understood as a limitation to the present invention. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0028] Figure 1 A possible structural schematic diagram of the communication system involved in the embodiments of the present invention is shown. This network considers a two-layer structure, including an edge layer and a vehicle layer. The edge layer is a blockchain based on the DAG structure, where the blockchain network is jointly maintained by RSUs close to the road, and the model data is recorded and shared in the form of Tangle transactions. The vehicle layer consists of CAVs equipped with multiple sensors, having good sensing capabilities and computing resources, and communicating with the RSU through a wireless link.

[0029] 1. Two-layer vehicle network architecture based on DAG blockchain

[0030] As Figure 1 shown, consider a two-layer vehicle network system empowered by a DAG blockchain. The detailed functions are as follows:

[0031] Edge layer: This layer aggregates a large number of RSUs and idle device units through optical fibers and wireless links, and can provide rich computing, storage, and communication resources. Among them, all RSUs jointly maintain a DAG blockchain, which is responsible for verifying, recording, and globally sharing the model data uploaded by CAVs in the form of transactions. The DAG blockchain contains confirmed transactions and unconfirmed transactions. Each transaction consists of a transaction header and a transaction body. The transaction header contains information such as version number, timestamp, hash value, and model test accuracy, and the transaction body stores the model data. Each transaction points to M previous forward transactions, indicating that the model in this transaction has collaborated with the models in the M previous forward transactions and verified the legality of the M previous forward transactions. In addition, a new transaction that has verified the previous forward transaction but has not been verified itself is called a Tip.

[0032] CAV layer: This layer consists of CAVs participating in federated training. Each CAV locally has training data and test data, and is equipped with a PLM for learning generalization prototypes and an LDM for assisting intelligent driving decisions. In particular, each LDM is divided into three parts: a shared feature extractor, a learnable feature enhancement layer, and a personalized decision head. Since shallow networks can capture general low-level features, the feature extractor is globally shared through the DAG blockchain to extract common knowledge. Since deep networks can capture differentiated high-level features, the decision head is retained locally to learn personalized knowledge. The feature enhancement layer incorporates common knowledge and personalized knowledge into the initial sample features and transforms them into more distinguishable feature representations. Denote the set of CAVs as where N represents the total number of CAVs.

[0033] CAVs and RSUs communicate with each other, and the federated training task is jointly completed by both the vehicle and the road parties. The detailed communication process is as follows:

[0034] (1) System initialization

[0035] First, CAVs and VSPs register with RSUs and provide identity authentication information, such as features like device ID and device type. After RSUs verify the registration information, they issue Software Development Kit (SDK) certificates to authorize access to the DAG blockchain. Then, the VSP publishes the genesis transaction of the DAG blockchain through RSUs, declaring FL task information, including identity recognition, model accuracy requirements, initial model structure, total number of categories C, etc. Finally, each CAV receives the task information and randomly initializes the LDM, PLM, and C generalization prototypes.

[0036] (2) Feature extractor and local prototype download

[0037] At the beginning of each round of global training, CAV v nFirst, all Tips of the DAG blockchain are downloaded through the neighboring RSU and their legitimacy is verified, and then the feature extractor, local prototype and other information are extracted from them. After that, in order to select the feature extractor adapted to the local data, CAV v n Each feature extractor is concatenated with the feature enhancement layer and the local decision head into a cascaded LDM, and the accuracy of the LDM is evaluated using local test data. Finally, the tips corresponding to the top M LDMs with the highest accuracy are selected.

[0038] (3) Local model update

[0039] During each round of global training, CAV v n Use IDLA algorithm to extract M features Perform personalized aggregation to obtain the initial LDM for this round of training Then, CAV v n Extract M local prototypes corresponding to CAVs from the selected Tips, and use the FFPL algorithm to transform PLMφ n (t) is updated to φ n (t+1), and generate C new generalized prototypes that are more globally representative. Finally, CAV v n The initial LDM is transformed using the new generalized prototype and similar local prototypes. Training a new personalized LDM w n (t+1).

[0040] (4) Feature extractor and local prototype upload

[0041] After local training is completed, CAV v n The local prototype, new feature extractor, hash value of the selected M tips and LDM test accuracy are packaged into a new transaction for uploading. n may be associated with a new RSU. Therefore, CAV v n First, a model upload request is initiated to the new RSU. After that, the RSU verifies the validity of the new transaction and runs the PoW algorithm to add transaction header information, including version number, timestamp, hash value, etc. Finally, the transaction is broadcast to all RSUs and added to the DAG blockchain as a Tip, waiting for download and verification by other CAVs.

[0042] (5) Training mission termination

[0043] To adapt the model to the dynamic changes of the environment, the training task will continue. Specifically, VSP monitors the model training in the DAG blockchain through RSUs and controls the suspension and restart of continuous training according to the preset upper and lower limits of model accuracy. When the model accuracy converges to the preset upper limit, the training task is suspended. When the model accuracy does not meet the requirements or drops to the preset lower limit, return to step 2 to enter the next round of training.

[0044] 2. Influence-based Dynamic Hierarchical Aggregation Algorithm

[0045] Observing the human collaboration environment, it can be seen that the more similar to the local data distribution, the greater the positive impact on local performance. Therefore, this section proposes an influence-based dynamic hierarchical aggregation algorithm, called IDLA. This algorithm quantitatively simulates the hierarchical influence between local models and dynamically guides CAVs to absorb cross-domain knowledge from similar models in a personalized manner. Specifically, inspired by the leave-one-out method, the hierarchical influence between CAVs is measured by retaining the parameters of a single-layer model, and a hierarchical influence matrix is modeled to guide the personalized aggregation of multiple feature extractors, so as to customize a better initial LDM for each round of training. The specific steps are as follows:

[0046] At the initial stage of the t-th round of global training, CAV v n selects M feature extractors from the Tips. All CAVs' feature extractors have L layers of parameters, so any feature extractor can be expanded as where represents the l-th layer parameter of the feature extractor θ n,m (t). In addition, the feature extractor θ n,m (t) comes from CAV v n,m , where θ n,m (t) may be the local historical model, that is, v n,m =v n . Based on the leave-one-out method, observe the performance change of the LDM of CAV v n after removing the parameters of a single layer, and then measure the influence of the parameters of this layer in the low-dimensional feature space on CAV v n .

[0047] First, remove the l-th layer parameters of a single feature extractor, and then perform average aggregation on the M feature extractors to obtain a temporary feature extractor. Then, splice the temporary feature extractor with the local feature enhancement layer and the local decision head to obtain a temporary LDM. At this time, by calculating the prediction loss of the temporary LDM on the local dataset, it can be reflected whether the parameters of this layer of extractor have a positive impact on CAV v n . Specifically, the larger the prediction loss, the greater the role played by the parameters of this layer of extractor in the local task. On the contrary, when the prediction loss is small, it indicates that the parameters of this layer of extractor have little impact on the model performance.

[0048] Based on the above analysis, by normalizing the prediction losses of the parameters of each layer, the parameters of a single-layer extractor can be measured for CAVv n The hierarchical impact caused is as follows:

[0049]

[0050] where represents the temporary LDM prediction loss after excluding the parameters of a single layer , and represents the magnitude of the hierarchical impact.

[0051] After that, the contribution degrees of M feature extractors are allocated according to the hierarchical influence. For CAV v n The single-layer parameters with greater influence are given greater aggregation weights, so as to generate a better initial feature space for this round of training. The aggregation of the parameters of the l-th layer is as follows:

[0052]

[0053] where represents the parameters of the l-th layer feature extractor after aggregation, and the complete aggregation extractor can be expressed as By splicing this aggregation extractor with the feature enhancement layer and the local decision head, the personalized initial LDM for this round of training can be obtained, which is expressed as

[0054] In particular, in this algorithm, the aggregation weights on different CAVs are not fixed and unified, but customized for each hierarchical aggregation weight according to the influence of different feature extractors on each CAV, so as to achieve dynamic learning between similar CAVs in different data distributions. In addition, this algorithm does not need to rely on additional sensitive hyperparameters, and the aggregation method is simpler and more general.

[0055] 3. Generalized Prototype Learning Based on Contrastive Training

[0056] Existing PL methods usually take the weighted average of all sample features as the prototype of this class, which ignores the overlap problem of sample distributions of different classes in the feature space and the data heterogeneity problem, resulting in poor discriminability and insufficient global representativeness of each class prototype. In addition, the weighted average method requires CAVs to upload detailed class distribution information as weights to the central server, which will leak sensitive user information. Therefore, the present invention introduces a PLM for each CAV, and learns representative generalized prototypes through contrastive training from the host, and its implementation idea is as Figure 2 shown, and the specific steps are as follows:

[0057] In the t-th round of global training, CAV v n First, screen M Tips from the DAG blockchain and then extract M CAVs from them The generated local prototypes. Assume CAV v n,m The local dataset of contains a total of C n,m categories. Then, the local prototype of a single category c ∈ [C n,m can be expressed as The set of local prototypes for all categories can be expressed as Following the conventional PL method, CAV v n,m Takes the average value of all sample features in category c n,m as the local prototype of this category.

[0058] To accurately represent the common information of each category, the generalized prototypes generated by the PLM should satisfy two conditions: First, each generalized prototype should be aligned with the local prototypes of the same category as much as possible to represent the global information of this category; Second, each generalized prototype should be as far away from the local prototypes of different categories as possible to ensure the clarity of the prototype boundary.

[0059] Therefore, the PLM uses the local prototypes of the M CAVs and the local prototypes of CAV v n as training data, and jointly optimizes the prototype distance between the same categories and the prototype distance between different categories through contrastive learning, so as to learn C generalized prototypes, which are expressed as Among them, the contrastive training loss of a single generalized prototype is as follows: In the above formula,

[0060]

[0061] is independently generated by the PLM and corresponds to the generalized prototype of category c. The set is composed of CAV v and the CAVs in the M CAVs n that contain category c. The set A(c) contains all categories except category c.

[0062] Subsequently, sum the contrastive training losses of the C generalized prototypes to obtain the total training loss of the PLM, which is expressed as CAV v n Updates the PLM based on the contrastive training loss using the Stochastic Gradient Descent (SGD) method as follows:

[0063] ​

[0064] Among them, φ n (t) represents the model parameters of the LDM, and η P represents the learning rate.

[0065] Finally, with the change of local prototypes in each round, the PLM will be continuously updated, so as to autonomously optimize C new generalized prototypes, which are represented as

[0066] 4. Feature-Enhanced Federated Prototype Learning Algorithm

[0067] Under the PFL framework with strict privacy requirements, CAVs cannot access the data of other vehicles and even do not need a public dataset, and some important global feature information will inevitably be lost in local training, and even overfit the local data features, resulting in a decline in model performance. In fact, the perception data collected by each CAV contains both global feature information and personalized feature information, but it is difficult for a single model to capture these two mutually restrictive features at the same time. Therefore, the present invention designs a feature-enhanced federated prototype learning algorithm, namely FFPL, to guide the LDM to learn common knowledge, collaborative knowledge and personalized knowledge at the same time, so as to improve the model accuracy and vehicle fairness. The implementation idea of this algorithm is as Figure 3 shown, and the specific steps are as follows:

[0068] (1) Generalized prototypes provide common knowledge

[0069] To prevent the LDM from losing useful global class information in local training, a feature enhancement layer Ψ n (t) is added between the feature extractor θ and the decision head n (t) to enhance the common information of the output features of the extractor with the help of generalized prototypes. Inspired by the contrastive loss, the feature enhancement layer Ψ n (t) guides each sample feature to align with the generalized prototypes of the same class, while moving away from the generalized prototypes of other classes, thereby improving the distinguishability of different classes in local training. Therefore, the common knowledge provided by the generalized prototypes can be defined as follows:

[0070]

[0071] Among them, D n represents the total number of samples of CAV v n , and sim(·) represents the cosine similarity.

[0072] (2) Local prototypes provide collaborative knowledge

[0073] Consider the CAV cognitive differences within a single class. In the actual environment, different users also have diverse cognitions of the same class. Taking dog recognition as an example, due to differences in personal preferences and environmental backgrounds, the dog data collected by different CAV groups will have different inclination angles, resulting in obvious differences in multiple feature dimensions such as coat color, body shape, and posture. Inspired by this, the present invention further considers finer-grained intra-class differences. While aggregating the features of similar samples, it allows dissimilar CAVs to maintain a certain distance in the feature space to improve the model accuracy and personalization level.

[0074] Specifically, each local prototype can represent the local information of the data of this class for the current CAV by aggregating the sample features of a single CAV. The closer the local prototypes of different CAVs are, the more similar the cognition of this class by this CAV group is. Strengthening their collaboration is beneficial to improving the accuracy and convergence speed of the model. Based on this, the similarity of CAVs is evaluated by calculating the distance between different local prototypes under the same class.

[0075] First, represent the set of CAVs jointly composed of CAV v n and M CAVs as In the data heterogeneous scenario, each CAV often only holds a part of the classes, so there will be missing local prototypes for several classes. To ensure the consistency of the similarity measurement criteria for all classes, a prototype filling technique is introduced, using the generalized prototypes generated by the PLM as the local prototypes for the missing classes to ensure that each CAV contains the local prototypes of all classes.

[0076] Subsequently, calculate the feature similarity between all pairs of local prototypes under the same class and normalize it to obtain the similarity weight as follows:

[0077]

[0078] In the above formula, is the local prototype of CAV v n after filling. The weight represents the feature similarity between CAV v n and CAV on class c, and its value range is [0, 1].

[0079] Next, in order to strengthen the collaboration of similar CAVs and retain the intra-class differences, the feature enhancement layer Ψ n (t) uses contrastive learning to guide each sample feature to be closer to the more similar local prototypes and at the same time increase the distance between dissimilar local prototypes. Finally, the collaborative knowledge Can be defined as:

[0080]

[0081] Among them, the weight Encourages more similar CAVs to contribute more intra-class knowledge to each other.

[0082] (3) Training the personalized LDM

[0083] The FFPL algorithm extracts common knowledge through various prototypes and local tasks Collaborative knowledge And personalized knowledge Enables the LDM to map various features to an enhanced space with good discriminability, thereby alleviating class overlap and improving the model accuracy. Based on this, three types of knowledge are fused to obtain the high-quality knowledge of the LDM As follows:

[0084]

[0085] Among them, the personalized knowledge Is the cross-entropy training loss of the LDM On the local dataset. λ p Represents the dynamic guidance ratio of the prototype knowledge, which dynamically adjusts the contribution degree of the prototype knowledge according to the training quality of the LDM and the PLM, as follows:

[0086]

[0087] In the above formula, λ p ∈[0, 0.5]. The training loss of the LDM And the training loss of the PLM The smaller they are, the stronger the feature capture ability of the model, then the better the prototype quality, and the larger the guidance ratio λ p The larger. In addition, the loss threshold δ ∈ [0, +∞) is used to measure the quality of the model. When It means that the training quality of the LDM and the PLM is not good, and λ p Will always be less than 0.5; when It means that the quality of the LDM and the PLM meets the standard, and λ p Will reach the maximum value of 0.5.

[0088] Next, based on the fused knowledge The aggregated LDM is updated using the SGD method As:

[0089]

[0090] Among them, w n(t + 1) is the CAV v n The personalized LDM in this round of global training.

[0091] Finally, the CAV v n Extracts a new feature extractor θ n (t + 1) from the new personalized LDM w n (t + 1), and calculates the new local prototypes corresponding to each category c Packages and shares them to the upper - layer DAG blockchain.

[0092] 5. System Flowchart

[0093] Figure 4 The following is the execution flowchart of the federated prototype learning method that combines personalization and generalization in the vehicle - to - everything network:

[0094] S501: System initialization. The VSP publishes initial task information through the RSU, and the DAG blockchain generates a genesis transaction based on the initial task information. The CAV registers as a legal training vehicle and receives the initial task information, including the initial PLM model, the initial LDM model, the FL model accuracy requirement, etc.

[0095] S502: The CAV obtains the current Tips from the DAG blockchain, extracts the feature extractor from them, and concatenates it with the local feature enhancement layer and the local decision head to form a cascaded LDM. Subsequently, the CAV uses the local dataset to test the accuracy of the cascaded LDM and selects the Tips corresponding to the top M LDMs with the highest accuracy.

[0096] S503: The CAV extracts M feature extractors from the selected Tips and performs the influence - based dynamic hierarchical aggregation algorithm, i.e., IDLA. First, the CAV sequentially removes the single - layer extractor parameters, and then averages and aggregates the M feature extractors into a temporary LDM. Then, the CAV uses the local data to test the prediction loss of the temporary LDM and normalizes the prediction losses of each layer to obtain the hierarchical influence. Finally, the CAV guides the hierarchical aggregation of the M feature extractors based on the influence, and obtains the personalized initial LDM for this round of training, where the larger the influence of a layer parameter, the greater the aggregation weight assigned to it.

[0097] S504: The CAV extracts the local prototypes of M CAVs from the selected Tips and uses them together with its own local prototypes as the training data for the PLM. The PLM is updated in a contrastive training manner to autonomously generate C high - quality generalization prototypes. Each generalization prototype will be close to the local prototypes of the same category and far from the local prototypes of different categories.

[0098] S505: The LDM extracts common knowledge from the generalized prototype to guide each sample feature to approach the local prototypes of the same category while moving away from the local prototypes of different categories.

[0099] S506: The LDM learns collaborative knowledge from local prototypes, measures the intra-class similarity of CAVs by calculating the distances between local prototypes of the same category, and encourages similar local prototypes to approach each other while dissimilar local prototypes move away from each other.

[0100] S507: The LDM performs supervised training on local data and uses its cross-entropy training loss as personalized knowledge.

[0101] S508: The CAV fuses common knowledge, collaborative knowledge, and personalized knowledge, and trains the aggregated LDM generated by the IDLA algorithm into a new personalized LDM.

[0102] S509: The CAV splits a new feature extractor from the new LDM, calculates new local prototypes, and packages information such as the hash values of the selected Tips and the LDM test accuracy, and uploads them to the associated RSU. The RSU verifies the validity of the transaction uploaded by the CAV, and after successful verification, broadcasts the new transaction to all RSUs for DAG blockchain update.

[0103] S510: The VSP continuously monitors the changes in model accuracy in the DAG blockchain, and controls the suspension and restart of FL training according to the preset upper and lower limits of accuracy, so as to adapt to the dynamically changing vehicle networking environment.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A federated prototype learning method that integrates personalization and generalization in the Internet of Vehicles, characterized by: The following steps are involved: S1: Based on the DAG blockchain architecture, a two-layer federated network consisting of a global model layer and a local model layer is constructed to initialize the global generalized prototype and the local personalized model of the vehicle node; S2: The vehicle node performs model training based on the local data set, encrypts the local gradient update value and uploads it to the blockchain network; S3: Calculate the influence weights between nodes through a dynamic hierarchical aggregation algorithm, and perform hierarchical aggregation optimization on global model parameters based on the weight values; S4: Generate generalized prototype features that integrate public knowledge at the global model layer, and generate local prototype features that contain personalized features at the local model layer; S5: Construct a feature enhancement layer to implement dynamic contrastive learning and jointly optimize the representation space of generalized prototype and local prototype features.

2. The method for federated prototype learning integrating personalization and generalization in the Internet of Vehicles according to claim 1 is characterized by: The dynamic hierarchical aggregation algorithm in S3 includes: Calculate the cosine similarity of the model update trajectory between nodes as the initial influence coefficient; Dynamically adjust influence weights through a sliding time window mechanism; A hierarchical aggregation tree is established to prioritize the aggregation of nodes above the weight threshold.

3. The method for federated prototype learning integrating personalization and generalization in the Internet of Vehicles according to claim 1 is characterized by: The method for generating the generalized prototype feature in S4 includes: Decouple the global model parameters to obtain domain-invariant features; Enhance the category discrimination of features by maximizing mutual information constraints; A multi-head attention mechanism is used to fuse the common knowledge of different vehicle nodes.

4. The method for federated prototype learning integrating personalization and generalization in the Internet of Vehicles according to claim 1 is characterized by: The dynamic contrastive learning in S5 specifically includes: Construct positive and negative sample pairs to enhance the clarity of the boundaries between classes; Design an adaptive temperature coefficient to control the contrast learning intensity; The prototype feature memory is maintained through a momentum update mechanism.

5. A federated learning system implementing the method according to any one of claims 1 to 4, characterized in that: include: Blockchain network module, used to implement distributed ledger storage and model parameter sharing based on DAG structure; The local training module is deployed on the vehicle terminal to perform personalized model training and gradient encryption; Dynamic aggregation module, equipped with hierarchical aggregation algorithm and influence assessment unit; Prototype management module, including generalized prototype generator and local prototype optimizer; Feature enhancement module, integrating contrastive learning unit and feature fusion controller.

6. The federated learning system according to claim 5, characterized in that: The dynamic aggregation module further comprises: The similarity calculation unit uses the improved Jaccard similarity algorithm to evaluate the node contribution; Weight adjustment unit, integrating LSTM network to predict weight change trend; The security verification unit is configured with a zero-knowledge proof protocol to verify the node identity.

7. The federated learning system according to claim 5, characterized in that: The prototype management module includes: Feature decoupling unit, which uses adversarial domain adaptation network to separate public features from private features; Attention fusion unit, deployed with deformable convolution to enhance spatial feature extraction capability; The memory unit maintains the prototype feature library through the nearest neighbor retrieval algorithm.

8. The federated learning system according to claim 5, characterized in that: The feature enhancement module also includes: Sample enhancement unit, configured with data shuffling strategy and feature interpolation algorithm; Loss calculation unit, integrating Focal Loss and Triplet Loss hybrid loss functions; The feature alignment unit uses the optimal transmission theory to achieve cross-domain feature alignment.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 4 when executing the program.

10. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images