Federal learning method, system, device and medium oriented to Internet of Vehicles data isomerism

By adopting similarity-based regional division and hypernetwork hierarchical personalized federated learning methods in Internet of Vehicles federated learning, the reduction of model accuracy and communication congestion caused by data heterogeneity is solved, and the model training speed and accuracy are improved.

CN118918554AActive Publication Date: 2024-11-08CHONGQING UNIV

Patent Information

Application Number
CN202410972857.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-08
Estimated Expiration
2044-07-19

AI Technical Summary

Technical Problem

When faced with the heterogeneity of Internet of Vehicles data, existing federated learning methods lead to reduced model accuracy and slowed down in training speed, and communication congestion problems exist in personalized federated learning.

Method used

Two-stage method is adopted: first, through similarity-based area division, intelligent connected vehicles are divided into areas with similar data distribution, and then a layered personalized federated learning method based on hypernetwork is adopted, using the three edge servers and cloud center servers. Layer architecture predicts personalized aggregation weights through hypernetwork models, aggregates personalized vehicle models, alleviates communication congestion and improves model training accuracy.

Benefits of technology

It effectively alleviates the impact of data heterogeneity on model training, improves model training speed and accuracy, reduces communication congestion, and realizes rapid personalized fine-tuning of personalized models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918554A_ABST
    Figure CN118918554A_ABST
Patent Text Reader

Abstract

The invention relates to a federated learning method, system and device for Internet of Vehicles data isomerism and a medium, and belongs to the technical field of Internet of Vehicles and the technical field of distributed machine learning. The method can automatically and adaptively divide a large area where all intelligent connected automobiles participating in neural network model training are located into a plurality of sub-areas, so that the distribution of data held by the intelligent connected automobiles in the same area is similar, and the influence of data isomerism on model training is relieved. Meanwhile, on the basis of region division, the system can realize a terminal (intelligent network connection automobile)-edge (edge server)-cloud (cloud center server) three-layer communication architecture cooperative training neural network model. Through training the super network model, the system predicts a personalized aggregation weight for each intelligent network connection automobile, a personalized area model is realized for each area while a personalized vehicle model is realized, the accuracy of the model is improved, the model training speed is accelerated, and the communication congestion problem is relieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet of Vehicles and distributed machine learning, and relates to a federated learning method, system, device and medium for heterogeneous Internet of Vehicles data. Background Art

[0002] With the rapid development of Internet of Vehicles technology, more and more intelligent connected vehicles are deployed in vehicle networks, which connect physical and digital transportation and provide comprehensive services for traffic participants. In this context, each intelligent connected vehicle is a producer and owner of data, and various multi-source spatiotemporal data are growing rapidly. Driven by big data and computing power, machine learning has become one of the important ways to achieve intelligent driving. Machine learning plays an important role in tasks such as omnidirectional perception, deep scene understanding, and intelligent behavior decision-making. However, learning intelligent driving models from data scattered across various vehicle devices faces many difficulties. In order to solve the problem of decentralized training, the mainstream solutions are centralized learning and federated learning. Centralized learning collects data generated on different on-board devices to a cloud server (or cloud server cluster) for model training, which may result in high data transmission costs and data privacy leakage. As a new type of distributed machine learning technology, federated learning allows on-board computing devices to train local models using gradient descent or stochastic gradient descent methods on local data sets, and then send the local models to cloud servers. The cloud server aggregates local models into a global shared model so that the model contains the empirical knowledge learned by all local models. This process makes the data available but not visible, technically achieves user privacy protection, and fully taps the value of data collected by a large number of vehicles in real time.

[0003] Existing federated learning methods are mainly divided into two categories: single-model federated learning mechanism and multi-model federated learning mechanism. In single-model federated learning, all on-board computing devices train local models on local data sets and then send the local models to the cloud server. The cloud server is responsible for aggregating all local models into a single global model. In multi-model federated learning, the cloud server collaborates with all on-board computing devices to train multiple personalized local models, which can meet the personalized driving needs of users.

[0004] However, in the implementation of the above process, it was found that the multi-source data collected by different on-board sensors have complex data heterogeneity due to differences in collection time and location, which is manifested in the large differences in the distribution of data collected by different on-board sensors. The above-mentioned federated learning did not solve the problem of data heterogeneity during the design process. Complex data heterogeneity will significantly reduce the performance of a single global model and slow down the speed of model training. Secondly, traditional personalized federated learning often adopts a two-layer federated learning architecture, namely the on-board computing device-cloud center server architecture. In the context of large-scale Internet of Vehicles federated learning, the two-layer federated learning architecture faces serious communication congestion problems. Vehicle network congestion will increase the communication overhead of on-board computing devices to upload or download models, further slowing down the speed of model training and reducing the accuracy of the model. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a federated learning method, system, device and medium for heterogeneous Internet of Vehicles data, which can solve the problems of reduced model accuracy and slow training speed caused by data heterogeneity in existing federated learning methods, as well as the communication congestion problem in personalized federated learning.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] The federated learning method for heterogeneous Internet of Vehicles data provided by the present invention includes two stages: the first stage is a region division method based on similarity, and the second stage is a hierarchical personalized federated learning method based on a hypernetwork.

[0008] Specifically include:

[0009] The first aspect of the present invention provides a similarity-based region division method, which is applied to an intelligent connected vehicle and a cloud center server, comprising:

[0010] Before starting federated learning training, the intelligent connected vehicle uses the onboard sensors to collect sample data, the onboard camera to collect image data, and the positioning sensor to collect location information; the image data collected by the onboard camera will be stored in the local storage device as the local training data set of the intelligent connected vehicle; the intelligent connected vehicle calculates the label information of the encrypted image data and the location information of the vehicle through the homomorphic encryption algorithm, and uploads the encrypted data to the cloud center server; after receiving the upload request of the label information and location information of all intelligent connected vehicles, the cloud center server uses the regional division algorithm to divide each of the intelligent connected vehicles into a matching area;

[0011] After receiving metadata information such as tag information and location information uploaded by all intelligent connected vehicles, the cloud center server executes the following partitioning algorithm to obtain a regional partitioning result: the cloud center server creates a regional partitioning process, decrypts the encrypted metadata information through a homomorphic encryption algorithm, and randomly selects metadata information of K intelligent connected vehicles as K initialized regional centers; for the remaining unselected intelligent connected vehicles, the regional partitioning process of the cloud center server calculates the distance or similarity between the metadata information of the remaining unselected intelligent connected vehicles and the metadata information of the K regional centers, and assigns the unselected intelligent connected vehicles to the region represented by the regional center with the smallest distance or the largest similarity; the cloud center server calculates the average value of the metadata information of the intelligent connected vehicles covered in each region, takes it as the new regional center, and continues to repeat the above partitioning steps until the regional center no longer changes or reaches the specified stop condition; the regional partitioning result includes K regions, each of which is a subset of intelligent connected vehicles;

[0012] The cloud center server calculates the encrypted form of the regional division result through a homomorphic encryption algorithm, and sends the encrypted data to the intelligent connected vehicles; the division method divides the intelligent connected vehicles with similar label information or similar location information into the same geographical area; the collaborative regional division between the intelligent connected vehicles and the cloud center server makes the data distribution between the intelligent connected vehicles in the same area similar, effectively improving the model training accuracy and training speed.

[0013] The second aspect of the present invention provides a hierarchical personalized federated learning method based on a super network, including: after regional division, deploying an edge server and a personalized regional model in each of the regions; the cloud center server is responsible for managing all edge servers and building a personalized regional model for each region; each edge server schedules the intelligent networked vehicles in a region to collaboratively train vehicle models, and builds personalized vehicle models for the intelligent networked vehicles in the region; the intelligent networked vehicles download vehicle models from the edge servers and train the vehicle models on local training data sets. Each edge server and the cloud center server use a communication terminal to transmit model parameters, and each intelligent networked vehicle and the edge server in the region use a communication terminal to transmit model parameters; the super network is a multi-layer perceptron network model, composed of trainable embedding vectors and network parameters, responsible for learning personalized model aggregation weights.

[0014] The third aspect of the present invention provides a hypernetwork-based hierarchical personalized federated learning method, which is applied to intelligent connected vehicles, including: after the current communication round starts, for each of the intelligent connected vehicles, the intelligent connected vehicle downloads the personalized vehicle model initialized for the current communication round from the edge server in the region to which it belongs; the on-board local computing device creates a training process and performs a fixed k1 number of local trainings on a full batch training data set or a random batch training data set; during each local training process, the computing device of the intelligent connected vehicle optimizes the local model using a preset optimization algorithm (including gradient descent or random gradient descent, etc.). After the local training is completed, the intelligent connected vehicle calculates the local model update between the trained local model and the initialized personalized vehicle model, and uploads the local model update and the trained local model parameters to the edge server.

[0015] The fourth aspect of the present invention provides a hierarchical personalized federated learning method based on a hypernetwork, which is applied to an edge server, including: before the start of a federated learning communication round, for each edge server, the edge server creates a scheduling process to schedule all intelligent connected vehicles in the area to collaboratively participate in federated learning training; the scheduling process creates a hypernetwork model for each intelligent connected vehicle managed by the edge server, and initializes the embedding vector and hypernetwork model parameters of the hypernetwork model; the scheduling process creates a vehicle model for each intelligent connected vehicle managed by the edge server, downloads a personalized regional model from a cloud center server, and uses the personalized regional model parameters to initialize the parameters of the vehicle model;

[0016] In the first round of communication or the initial round of federated learning, after the edge server receives a request from each intelligent connected vehicle to download a personalized vehicle model, the scheduling process inputs the embedding vector into the super network model matching the intelligent connected vehicle; the super network model predicts and calculates the personalized aggregation weight vector; the scheduling process aggregates the vehicle models of all intelligent connected vehicles in the area managed by the edge server based on the personalized aggregation weight vector to obtain a personalized vehicle model; the edge server sends the personalized vehicle model to the intelligent connected vehicle;

[0017] Before starting the second and subsequent rounds of federated learning, the edge server will receive and process the requests for local model updates and local model parameters uploaded by the smart connected vehicles in the previous round; the scheduling process uses the local model updates as pseudo-gradient information and uses gradient descent to optimize the hypernetwork model parameters and embedding vectors matched by the smart connected vehicle. When the scheduling process responds to the requests for local model updates uploaded by all smart connected vehicles and updates the parameters and embedding vectors of all hypernetwork models, a new communication round will be started;

[0018] In the second communication round and subsequent communication rounds of federated learning, after receiving a request from each intelligent connected vehicle to download a personalized vehicle model, the scheduling process inputs the updated embedding vector into the super network model matching the intelligent connected vehicle; the super network model predicts and calculates a personalized aggregation weight vector; the scheduling process aggregates the vehicle models of all intelligent connected vehicles in the area managed by the edge server based on the personalized aggregation weight vector to obtain a personalized vehicle model; the edge server sends the personalized vehicle model to the intelligent connected vehicle;

[0019] When the federated learning executes k2 communication rounds, the scheduling process aggregates the vehicle models in the area managed by the edge server according to the pre-configured aggregation strategy to obtain the regional model. The scheduling process calculates the regional model update between the aggregated regional model and the personalized regional model recently downloaded from the cloud center server, and sends the regional model update and regional model parameters to the cloud center server. Preferably, the edge server downloads the optimized personalized regional model from the cloud center server, reinitializes the parameters of the vehicle model, and starts the next round of training.

[0020] The fifth aspect of the present invention provides a hierarchical personalized federated learning method based on a hypernetwork, which is applied to a cloud center server, including: before the start of federated learning, the cloud center server will create a central scheduling process. The central scheduling process creates a hypernetwork model for each edge server and initializes the embedding vector and model parameters of the hypernetwork model. The central scheduling process creates a regional model for each edge server and initializes the regional model parameters. When the federated learning executes k2 communication rounds, the cloud center server will receive a request from the edge server to upload a regional model update and a regional model parameter request, and the central scheduling process will use the regional model update as a pseudo gradient and use a gradient descent method to update the embedding vector and model parameters of the hypernetwork model matched by the edge server. In the current communication round, after the cloud center server receives a download request for a personalized regional model from the edge server, the central scheduling process inputs the embedding vector matched by the edge server into the hypernetwork model; the hypernetwork model predicts and calculates a personalized aggregation weight vector; the central scheduling process aggregates all edge server uploaded regional models based on the personalized aggregation weight vector to obtain the personalized regional model of the edge server; the central scheduling process sends the personalized regional model to the edge server.

[0021] The sixth aspect of the present invention provides a computer device and a non-volatile readable storage medium, including: the computer device covers sensors, processors, memories and communication terminals in intelligent connected vehicles, edge servers and cloud center servers; the sensors sense the surrounding environment and collect local training data sets, the processors execute computer instructions stored in the memories, and the communication terminals include any one of 2G, 3G, 4G and 5G; preferably, selecting a 5G communication terminal can reduce communication costs and improve model training speed; the non-volatile readable storage medium includes a computer program and model parameters, and the computer processor can control the non-volatile readable storage medium to implement the similarity-based region division method and the hypernetwork-based hierarchical personalized federated learning method of the first to fifth aspects above.

[0022] The beneficial effects of the present invention are:

[0023] The present invention provides a similarity-based regional division method, which divides the intelligent connected vehicles into multiple regions based on the label information and location information of the data collected by the intelligent connected vehicles, so that the data collected by the intelligent connected vehicles in the same region are distributed similarly, thereby alleviating the influence of data heterogeneity on the accuracy of model training; on the basis of regional division, the present invention provides a hierarchical personalized federated learning method based on a hypernetwork, which adopts a three-layer communication architecture of intelligent connected vehicle-edge server-cloud center server, and can train a matching hypernetwork model for each intelligent connected vehicle, use the hypernetwork model to predict personalized aggregation weights, and aggregate personalized vehicle models for each intelligent connected vehicle; through stratification, it can train a matching hypernetwork model for each region, use the hypernetwork model to predict personalized aggregation weights, and aggregate personalized regional models for each region, thereby effectively alleviating communication congestion and improving the model training speed and the accuracy of the model; the personalized regional model can be used to initialize the vehicle model, thereby enabling the intelligent connected vehicle to fine-tune the model on the local data set to achieve rapid personalization.

[0024] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0026] Figure 1 A flowchart of a federated learning method for heterogeneous Internet of Vehicles data provided in the first embodiment of the present invention;

[0027] Figure 2 A system framework diagram of a federated learning system for heterogeneous Internet of Vehicles data provided in the first embodiment of the present invention;

[0028] Figure 3 A flowchart of a region division algorithm based on similarity provided in Embodiment 2 of the present invention;

[0029] Figure 4 A schematic diagram of a similarity-based area division subsystem structure provided in Embodiment 2 of the present invention;

[0030] Figure 5 A schematic diagram of the structure of a hierarchical personalized federated learning subsystem based on a hypernetwork is provided for Embodiment 3, Embodiment 4 and Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0031] The technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0032] Embodiment 1:

[0033] Reference Figure 1 , which is a flow chart of a federated learning method for heterogeneous Internet of Vehicles data provided by the first embodiment of the present invention;

[0034] The first embodiment of the present invention provides a federated learning method for heterogeneous Internet of Vehicles data, which is applied to intelligent connected vehicles, edge servers, and cloud center servers, and includes steps S11 to S15:

[0035] Step S11, the intelligent networked vehicle uploads the label information and vehicle location information of the data sample to the cloud center server, and the cloud center server executes a region division algorithm to divide the large area where all the intelligent networked vehicles are located into several small areas, and the intelligent networked vehicles with similar data distribution are gathered in the same area;

[0036] Step S12, the intelligent connected vehicle downloads a personalized vehicle model from the edge server in the region to which it belongs, performs k1 gradient descents on the local training data set, and sends the local model update and local model parameters to the edge server;

[0037] Step S13, after the edge server receives the local model update uploaded by the intelligent connected vehicle, the scheduling process uses the local model update as a pseudo-gradient to update the embedding vector and model parameters of the super network model, uses the updated embedding vector as the input of the super network and predicts the personalized aggregation weight vector; the scheduling process uses the personalized aggregation weight vector to aggregate the local models uploaded by the other intelligent connected vehicles in this area to obtain the personalized vehicle model of the current intelligent connected vehicle;

[0038] Step S14, when executing k2 communication rounds, the scheduling process aggregates the vehicle models in the area managed by the edge server according to the pre-configured aggregation strategy to obtain the regional model. The scheduling process sends the regional model update and regional model parameters to the cloud center server. Preferably, the edge server will download the optimized personalized regional model from the cloud center server, reinitialize the parameters of the vehicle model, and start the next round of training;

[0039] Step S15, after the cloud center server receives the regional model update uploaded by the edge server, the central scheduling process uses the regional model update as a pseudo-gradient to update the embedding vector and model parameters of the super network model, inputs the updated embedding vector into the super network model and predicts a personalized aggregation weight vector; the central scheduling process aggregates the regional models uploaded by all edge servers according to the personalized aggregation weight vector to obtain the personalized regional model of the edge server;

[0040] Reference Figure 2 , Figure 2 The framework diagram of the federated learning system for heterogeneous Internet of Vehicles data provided by the first embodiment of the present invention is shown. The system performs the following steps: Figure 1 Step S11 to step S15; the system framework designed in this embodiment consists of three layers: intelligent networked vehicles, edge servers and cloud center servers. Among them, each intelligent networked vehicle is equipped with sensors to collect training data sets 208 and store the training data in local non-volatile memory; the edge server creates and runs the scheduling process 205, the vehicle model personalization process 206, and the regional model aggregation process 207 when the system starts, reads data from the memory 204 and writes the program execution results to the memory 204; the intelligent networked vehicle can communicate the neural network model parameters with the edge server through the base station 209, and the specific communication terminal can be one of 2G, 3G, 4G, and 5G; the cloud center server creates and runs the regional division process 201, the central scheduling process 202, and the regional model personalization process 203 when it starts, reads data from the memory 204 and writes the program execution results to the memory 204.

[0041] Embodiment 2:

[0042] This embodiment 2 is a detailed supplement to step S11 of embodiment 1. Figure 3 , which is a flow chart of a region division method based on similarity provided in Embodiment 2 of the present invention. Embodiment 2 of the present invention provides a region division method based on similarity, including steps S21 to S26:

[0043] Step S21: Intelligent networked vehicles collect sample label information vectors and position information vector Calculate the encrypted tag information vector L through the homomorphic encryption algorithm i The position information vector V i , sending the encrypted data to the cloud center server; where, represents the number of categories represented by the mth label in all samples collected by the intelligent connected car numbered i, and Respectively represent the location coordinates of the intelligent connected car;

[0044] Step S22: The cloud center server uses a homomorphic encryption algorithm to encrypt the encrypted tag information vector L i The position information vector V i Decryption, the label information vector L i Quantified as label richness vector The label information vector L i The quantization method of the features represented by the mth dimension is:

[0045]

[0046] in, All intelligent connected cars are scattered in M ​​cities, and the subscript j represents the jth city among the M cities; Indicates the number of categories represented by the mth label in all samples collected by the intelligent connected car numbered i; represents the average number of categories represented by the mth label in the samples collected from all intelligent connected vehicles in the jth city (number / vehicle); It represents the relative richness of the mth category of the samples collected by the intelligent connected car numbered i;

[0047] Step S23, the cloud center server randomly selects label richness vectors and location information vectors of K intelligent network-connected vehicles as initialized regional centers;

[0048] Step S24, the cloud center server calculates the distances between the remaining intelligent networked vehicles and the K regional centers, and the intelligent networked vehicles are allocated to the area represented by the regional center with the smallest distance; wherein the distance between the intelligent networked vehicle numbered i and the regional center numbered u is calculated as follows:

[0049] Dist(i,u)=||V i -V u ||2+γ||C i -C u ||2

[0050] Among them, λ can be set to [0, 1] according to the specific task to be performed;

[0051] Step S25: The cloud center server calculates the average value of the label richness vector and the location information vector of the smart connected vehicles covered in each area, and uses it as the new regional center; repeat steps S24 to S25 until the regional center is stable or the maximum number of iterations is reached;

[0052] Step S26: The cloud center server uses a homomorphic encryption algorithm to encrypt the regional division results. Encryption, broadcast the encrypted data to all smart connected vehicles; smart connected vehicles decrypt the encrypted regional division results through homomorphic encryption algorithm; represents the set of all intelligent connected vehicles in the kth region;

[0053] In specific implementation, step S23 may use the KMeans++ algorithm to select K regional centers, so as to avoid the random selection algorithm from selecting outliers as regional centers, so as to improve the effect of regional division; the encryption algorithm may select the Paillier encryption algorithm, etc.;

[0054] Reference Figure 4 , which is a schematic diagram of the subsystem framework of the similarity-based regional division method provided in the second embodiment of the present invention, and shows a round of communication between the intelligent networked vehicle and the cloud center server; wherein, each intelligent networked vehicle will collect sample data in a certain urban area, and the circle represents the sample points collected in the area. The dotted arrow to the right indicates that the intelligent networked vehicle counts the label information of all samples and calculates the vehicle location information; the solid arrow to the right indicates that the intelligent networked vehicle uploads the encrypted label information and vehicle location information; the solid arrow pointing downward indicates that the cloud center server executes the regional division algorithm based on the label information and vehicle location information, and distributes the intelligent networked vehicle to multiple areas; the solid arrow to the left indicates that the cloud center server broadcasts the regional division result; through the coordinated regional division between the intelligent networked vehicle and the cloud center server, the intelligent networked vehicles with similar data distribution can be gathered in the same area to execute the federated learning training model, alleviating the impact of data heterogeneity on the model accuracy.

[0055] Embodiment three:

[0056] This third embodiment is a detailed supplement to step S12 in the first embodiment; the third embodiment of the present invention provides a hierarchical personalized federated learning method based on a hypernetwork, which is applied to an intelligent connected vehicle, including steps S31 to S33:

[0057] Step S31, after the current communication round starts, for each of the intelligent networked vehicles, the intelligent networked vehicle numbered i downloads the initialized personalized vehicle model w of the current communication round from the edge server in the region to which it belongs. i

[0058] Step S32: The onboard local computing device creates a training process and Perform a fixed k1 times of local training on the , and optimize the following local training objectives,

[0059]

[0060] in, is the size of the full batch training dataset, x (i,j) is the jth sample data of the intelligent connected car numbered i, y (i,j) is the sample label of the jth sample data;

[0061] Step S33, during each local training process, the computing device of the intelligent connected vehicle uses a gradient update Optimize local models;

[0062] Step S34: After the local training is completed, the intelligent networked vehicle calculates a local model update Δw between the trained local vehicle model and the initialized vehicle model. i , and update the local model by Δw i and the trained local model parameters w i Upload to the edge server.

[0063] In specific implementation, depending on the different tasks of intelligent connected vehicle training, the loss function l can be specified as cross entropy loss, mean square error loss, or other loss functions;

[0064] Reference Figure 5 , which is a schematic diagram of the structure of the hierarchical personalized federated learning subsystem based on the hypernetwork. Figure 5 Sub-figure (1) shows the process of training the vehicle model 302 of the intelligent connected vehicle on the local training data set 301.

[0065] Embodiment 4:

[0066] This fourth embodiment is a detailed supplement to step S13 and step S14 in the first embodiment. The fourth embodiment of the present invention provides a hypernetwork-based hierarchical personalized federated learning method, which is applied to an edge server and includes steps S41 to S44;

[0067] Step S41, before the federated learning communication round begins, for each edge server, the edge server creates a scheduling process; the scheduling process creates a hypernetwork model h for each intelligent connected vehicle (taking the intelligent connected vehicle numbered i as an example) managed by the edge server (taking the edge server numbered k as an example) i (v i ψ i), and initialize the embedding vector v of the hypernetwork model i and the hypernetwork model parameter ψ i The scheduling process creates a vehicle model for each smart connected vehicle managed by the edge server Download the personalized regional model from the cloud center server and use the personalized regional model parameters to initialize the parameters of the vehicle model;

[0068] Step S42: After receiving the request from each intelligent connected vehicle to download the personalized vehicle model, the edge server schedules the embedding vector v i Input to the hypernetwork model h that matches the intelligent connected car i (v i ψ i ); The hypernetwork model predicts and calculates the personalized aggregation weight vector in The scheduling process aggregates the vehicle models of all intelligent connected vehicles in the area managed by the edge server based on the personalized aggregation weight vector to obtain a personalized vehicle model. The specific personalized aggregation method is as follows:

[0069]

[0070] in, represents a set of vehicle models uploaded by intelligent network-connected vehicles in the region received by the edge server numbered k. The edge server sends the personalized vehicle model to the intelligent network-connected vehicle;

[0071] Step S43: the edge server receives and processes the local model update request uploaded by the smart connected vehicle in the previous communication round; the scheduling process updates Δw with the local model i The pseudo gradient information is used to optimize the hypernetwork model parameters and embedding vectors matched by the intelligent connected vehicle by gradient descent. When the scheduling process responds to the local model update requests uploaded by all intelligent connected vehicles and updates the parameters and embedding vectors of all hypernetwork models, a new communication round is started;

[0072] Step S44, when the federated learning executes k2 communication rounds, the scheduling process aggregates the vehicle models in the area managed by the edge server (taking the edge server numbered k as an example) according to the pre-configured aggregation strategy to obtain the regional model, and adopts the following aggregation strategy:

[0073]

[0074] in, represents the average model of the vehicle models in the area managed by the edge server numbered k, g(·)=e -(·)is a penalty function. The greater the norm distance between the vehicle model and the average model, the greater the penalty and the smaller the aggregation weight. The scheduling process calculates the regional model update between the aggregated regional model and the personalized regional model recently downloaded from the cloud center server, and sends the regional model update and regional model parameters to the cloud center server. Preferably, the edge server downloads the optimized personalized regional model from the cloud center server, reinitializes the parameters of the vehicle model, and starts the next round of training.

[0075] In specific implementation, other aggregation strategies may be used to replace the aggregation strategy in step S44, or other penalty functions may be used to replace the penalty function in step S44.

[0076] Reference Figure 5 , which is a schematic diagram of the structure of the hierarchical personalized federated learning subsystem based on the hypernetwork. Figure 5 Sub-figure (2) of shows the execution process of the edge server. The edge server updates the embedding vector 304 and the hypernetwork model parameter 303, and inputs the embedding vector 304 into the hypernetwork model 303. The edge server uses the hypernetwork model to predict the personalized aggregation weight, aggregates other vehicle models through the personalized aggregation weight, and obtains a personalized vehicle model.

[0077] Embodiment five:

[0078] This fifth embodiment is a detailed supplement to step S15 in the first embodiment. The fifth embodiment of the present invention provides a hypernetwork-based hierarchical personalized federated learning method, which is applied to a cloud center server and includes steps S51 to S53;

[0079] Step S51: before the federated learning starts, the cloud center server will create a central scheduling process. The central scheduling process creates a hypernetwork model h for each edge server (taking the edge server numbered k as an example). k (v k ψ k ), and initialize the embedding vector v of the hypernetwork model k and the model parameter ψ k The central scheduling process creates a regional model for each edge server and initializes the regional model parameters

[0080] Step S52: In the current communication round, after the cloud center server receives a download request for the personalized regional model from the edge server, the central scheduling process sends the embedding vector v matched by the edge server to the cloud center server. k Input to the hypernetwork model h k (v k ψ k ); The hypernetwork model predicts and calculates the personalized aggregation weight vector in The central scheduling process aggregates all edge server uploaded regional models according to the personalized aggregation weight vector to obtain the personalized regional model of the edge server in the following manner:

[0081]

[0082] in, It means that the cloud center server receives the set of regional models uploaded by K edge servers; the central scheduling process sends the personalized regional model to the edge server.

[0083] Step S53, when the federated learning executes k2 communication rounds each time, the cloud center server receives the request for uploading regional model updates and regional model parameters from the edge server. The central scheduling process uses the regional model updates as pseudo-gradients and uses gradient descent to update the embedding vectors and model parameters of the hypernetwork model matched by the edge server.

[0084] Reference Figure 5 , which is a schematic diagram of the structure of the hierarchical personalized federated learning subsystem based on the hypernetwork. Figure 5 Sub-figure (3) shows the execution process of the cloud center server.

[0085] Embodiment six:

[0086] Embodiment 6 of the present invention provides a computer device and a non-volatile readable storage medium, wherein the computer device includes a sensor, a processor, a memory, and a communication terminal, and the computer non-volatile storage medium includes a stored computer program;

[0087] The sensor is a detection device that can sense the information being measured and can transform the sensed information into electrical signals or other required forms of information output according to certain rules to meet the requirements of information transmission, processing, storage, display, recording and control. It can be a position sensor (Crankshaft Position Sensor, GPS), an RGB camera (RGB Camera), an inertial sensor (IMU Sersor), or an odometer sensor, etc.

[0088] The processor may be a central processing unit (CPU), a graphics processing unit (GPU) or some other general-purpose processor; the processor is capable of executing the federated learning method for heterogeneous Internet of Vehicles data implemented by the computer program of the above embodiment, for example Figure 1 Steps S11 to S15 in .

[0089] The memory can be used to store the computer program, training data and model parameters described in the above embodiment, and the processor can execute the computer program in the memory and access the training data in the memory to perform the model training process; the memory includes a main memory and an auxiliary memory; wherein the main memory may be a RAM memory, a flash memory or other form of storage medium, mainly used to store the program and data being executed by the processor during the implementation process; the auxiliary memory may be a disk, a removable hard disk or other form of memory, mainly used to store inactive programs and data during the implementation process, such as data collected by the corresponding sensor;

[0090] The computer non-volatile storage medium includes a stored computer program; the computer program may be a source code program, an executable file or other forms of machine instruction sequences; the computer non-volatile storage medium refers to any recording medium that can carry a computer program, such as a USB flash drive, an optical disk, or a computer memory, etc.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A federated learning method for heterogeneous Internet of Vehicles data, involving intelligent connected vehicles, edge servers and cloud center servers, characterized in that: The method consists of two stages: the first stage is a region division method based on similarity, and the second stage is a hierarchical personalized federated learning method based on a hypernetwork, which specifically includes the following steps: S1. The intelligent connected vehicle uploads the label information and vehicle location information of the data sample to the cloud center server. The cloud center server executes the area division algorithm to divide the large area where all intelligent connected vehicles are located into several small areas. S2, the intelligent connected car downloads the personalized vehicle model from the edge server in its area, performs k1 gradient descents on the local training data set, and sends the local model update and local model parameters to the edge server; S3. After the edge server receives the local model update uploaded by the intelligent connected vehicle, the scheduling process uses the local model update as a pseudo-gradient to update the embedding vector and model parameters of the super network model, uses the updated embedding vector as the input of the super network and predicts the personalized aggregation weight vector; the scheduling process uses the personalized aggregation weight vector to aggregate the local models uploaded by the other intelligent connected vehicles in this area to obtain the personalized vehicle model of the current intelligent connected vehicle; S4. Every time k2 communication rounds are executed, the scheduling process aggregates the vehicle models in the area managed by the edge server according to the pre-configured aggregation strategy to obtain a regional model; the scheduling process sends the regional model update and regional model parameters to the cloud center server; S5. After the cloud center server receives the regional model update uploaded by the edge server, the central scheduling process uses the regional model update as a pseudo-gradient to update the embedding vector and model parameters of the super network model, inputs the updated embedding vector into the super network model and predicts a personalized aggregation weight vector; the central scheduling process aggregates the regional models uploaded by all edge servers based on the personalized aggregation weight vector to obtain the personalized regional model of the edge server.

2. The method for federated learning of heterogeneous Internet of Vehicles data according to claim 1, characterized in that: In step S1, the intelligent connected vehicle calculates the label information of the image data and the location information of the vehicle in encrypted form through a homomorphic encryption algorithm, and uploads the encrypted data to the cloud center server.

3. The method for federated learning of heterogeneous Internet of Vehicles data according to claim 2, characterized in that: In step S1, the cloud center server creates a regional division process, decrypts the encrypted metadata information through a homomorphic encryption algorithm, and randomly selects metadata information of K smart connected vehicles as K initialized regional centers.

4. The method for federated learning of heterogeneous Internet of Vehicles data according to claim 3, characterized in that: For the remaining unselected intelligent connected vehicles, the regional division process of the cloud center server calculates the distance or similarity between it and the metadata information of the K regional centers, and allocates the unselected intelligent connected vehicles to the area represented by the regional center with the smallest distance or the largest similarity.

5. The method for federated learning of heterogeneous Internet of Vehicles data according to claim 4, characterized in that: In step S2, for each intelligent connected vehicle, the intelligent connected vehicle downloads the initialized personalized vehicle model of the current communication round from the edge server in the region to which it belongs; the on-board local computing device creates a training process and performs a fixed number of local training on the full batch training data set or the random batch training data set; During each local training process, the computing device of the intelligent connected vehicle optimizes the local model using a preset optimization algorithm; after the local training is completed, the intelligent connected vehicle calculates the local model update between the trained local model and the initialized personalized vehicle model, and uploads the local model update and the trained local model parameters to the edge server.

6. The method for federated learning of heterogeneous Internet of Vehicles data according to claim 5, characterized in that: Before the start of the federated learning communication round, for each edge server, the edge server creates a scheduling process to schedule all intelligent connected vehicles in the area to collaboratively participate in the federated learning training; the scheduling process creates a super network model for each intelligent connected vehicle managed by the edge server, and initializes the embedding vector and super network model parameters of the super network model; the scheduling process creates a vehicle model for each intelligent connected vehicle managed by the edge server, downloads a personalized regional model from the cloud center server, and uses the personalized regional model parameters to initialize the parameters of the vehicle model; After the edge server receives a request from each intelligent connected vehicle to download a personalized vehicle model, the scheduling process inputs the embedding vector into a super network model that matches the intelligent connected vehicle; the super network model predicts and calculates a personalized aggregation weight vector; the scheduling process aggregates the vehicle models of all intelligent connected vehicles in the area managed by the edge server based on the personalized aggregation weight vector to obtain a personalized vehicle model; The edge server sends the personalized vehicle model to the intelligent connected vehicle; The edge server receives and processes the local model update and local model parameter requests uploaded by the smart connected car in the previous communication round; the scheduling process uses the local model update as pseudo-gradient information and optimizes the super network model parameters and embedding vectors matched by the smart connected car by gradient descent; when the scheduling process responds to the local model update requests uploaded by all smart connected cars and updates all the super network model parameters and embedding vectors, a new communication round is started; Every time the federated learning executes k2 communication rounds, the scheduling process aggregates the vehicle models in the area managed by the edge server according to the pre-configured aggregation strategy to obtain the regional model; the scheduling process calculates the regional model update between the aggregated regional model and the personalized regional model most recently downloaded from the cloud center server, and sends the regional model update and regional model parameters to the cloud center server.

7. The method for federated learning of heterogeneous Internet of Vehicles data according to claim 6, characterized in that: Before the federated learning starts, the cloud center server creates a central scheduling process; the central scheduling process creates a super network model for each edge server and initializes the embedding vector and model parameters of the super network model; the central scheduling process creates a regional model for each edge server and initializes the regional model parameters; When the federated learning executes k2 communication rounds each time, the cloud center server receives the request for uploading regional model updates and regional model parameters from the edge server. The central scheduling process uses the regional model updates as pseudo-gradients and uses gradient descent to update the embedding vectors and model parameters of the hypernetwork model matched by the edge server. In the current communication round, after the cloud center server receives a download request for the personalized regional model from the edge server, the central scheduling process inputs the embedding vector matched by the edge server into the hypernetwork model; the hypernetwork model predicts and calculates the personalized aggregation weight vector; the central scheduling process aggregates the regional models uploaded by all edge servers according to the personalized aggregation weight vector to obtain the personalized regional model of the edge server; The central scheduling process sends the personalized regional model to the edge server.

8. A federated learning system for heterogeneous Internet of Vehicles data, characterized by: The system adopts the method as claimed in any one of claims 1 to 7.

9. A federated learning device for heterogeneous Internet of Vehicles data, characterized by: The device adopts the method as claimed in any one of claims 1 to 7.

10. A computer device and a non-volatile readable storage medium, characterized in that: The computer device includes sensors, processors, memories and communication terminals in intelligent connected vehicles, edge servers and cloud center servers; the sensors sense the surrounding environment and collect local training data sets, and the processors execute computer instructions stored in the memories; the non-volatile readable storage medium includes computer programs and model parameters, and the computer processor can control the computer device where the non-volatile readable storage medium is located to execute the similarity-based region division method and the hypernetwork-based hierarchical personalized federated learning method as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Personalized federal learning method and device based on super network and block chain

    CN115545223A

  • Hierarchical federal learning system based on super network

    CN116681125A

  • Personalized federal learning generalization method, device and application based on attention mechanism

    CN117010484A

  • Federated learning lower-side cooperative channel adaptive gradient compression method

    WO2024050659A1

Cited By

  • Alliance-based Internet of Vehicles federal learning method, system and device

    CN121935758A

  • An alliance-based federated learning method, system and device for internet of vehicles

    CN121935758B