Federal learning application method suitable for vehicle Internet of Things

Abstract

The invention discloses a federal learning application method suitable for a vehicle internet of things, and belongs to the technical field of vehicle internet mobile edge computing, and the method comprises the steps: S1, dividing roads in a city into a plurality of servers; s2, setting a base station in the server as the first brain of the service area, and detecting vehicles entering and exiting the coverage area of the first brain of the service area; s3, the vehicle of the specific server reports to the corresponding service area for the first time at regular intervals; s4, the vehicle performs federated learning based on the vision data acquired by the vehicle; s5, repeating the step S4 until the vehicle fog node leaves or performs a plurality of alternating current cycles; and S6, at the end of each communication period, the vehicle fog node sets the number of training times in the next communication period based on the performance of the previous round of the client vehicle. By adopting the federated learning application method suitable for the vehicle Internet of Things, application frameworks of different models are provided for different environments, so that the availability and robustness of the federated learning application of the vehicle Internet of Things are improved.

Description

Technical Field

[0001] The present invention relates to the field of mobile edge computing technology for vehicle networking, and in particular to a federated learning application method suitable for vehicle Internet of Things. Background Art

[0002] With the advancement of assisted driving and autonomous driving technologies, more and more sensors are installed on vehicles. Among them, visible light sensors or dashcams can generate information-rich visual data. By analyzing these visual data through mass perception, urban road conditions can be perceived in a low-cost and timely manner, such as identifying dangerous driving behaviors and locating parking spaces. However, uploading such a large amount of visual data to the cloud for centralized processing may bring huge bandwidth challenges and cause privacy concerns among car owners.

[0003] Federated Learning (FL) is a distributed machine learning method that allows models to be trained on multiple edge devices or local servers without centralizing the data to a central server. In this way, federated learning can use decentralized computing resources and data for model training while protecting data privacy. As a field that is highly dependent on machine learning, the Internet of Vehicles (IoV) can provide data privacy protection while reducing communication overhead and optimizing the training of complex models. In the IoV environment, federated learning trains models on edge vehicles and then aggregates the updated model parameters to vehicle fog nodes or regional servers, which can both protect the privacy of vehicle users and improve data processing efficiency.

[0004] In the field of edge computing of the Internet of Vehicles, computing tasks are usually assigned to vehicle fog nodes and base station servers in the area, which can effectively reduce the delay and bandwidth pressure caused by visual data transmission. However, the amount of data obtained by fog nodes and base stations alone is limited. This problem can be effectively solved by collecting data from running vehicles as client vehicles. However, the visual data of client vehicles usually has a certain degree of privacy. Directly uploading this data and training models will lead to certain privacy and legal risks, and will also cause a lot of bandwidth waste. By introducing the paradigm of federated learning, a large amount of visual data can be fully utilized to improve the performance of edge computing applications in the Internet of Vehicles while avoiding the privacy leakage of client vehicles. In addition, road visual data is usually non-independent and identically distributed, and there are significant differences in the composition of image members in different road sections. Traditional federated learning schemes aggregate all model training increments without distinction. However, the visual data of urban roads usually has the characteristics of non-independent and identically distributed, and the scenes of different roads are very different. Direct aggregation not only consumes more computing resources, but also leads to a decrease in training effect. Summary of the invention

[0005] The purpose of the present invention is to provide a federated learning application method suitable for vehicle Internet of Things, so as to solve the privacy issues existing in cloud computing of visual data in the vehicle Internet of Things environment and the low training efficiency of traditional federated learning schemes.

[0006] To achieve the above object, the present invention provides a federated learning application method applicable to vehicle Internet of Things, comprising the following steps:

[0007] S1. Divide the roads in the city into several servers according to their types and surrounding environments;

[0008] S2, setting the base station in the server as the service area leader, and detecting vehicles entering and leaving the coverage area of ​​the service area leader through the existing cellular network registration mechanism;

[0009] S3, the vehicles located at a specific server will periodically report their driving direction, location and available capacity to their corresponding service area heads;

[0010] S4: The vehicle performs federated learning based on the visual data it obtains.

[0011] S5, repeat step S4 until the vehicle fog node leaves or after several communication cycles;

[0012] S6. At the end of each communication cycle, the vehicle fog node sets the number of training times for the next communication cycle based on the performance of the client vehicle in the previous round.

[0013] Preferably, step S4 is specifically:

[0014] S41, the client vehicle and the vehicle fog node register with the service area leader and obtain the initial model of the area;

[0015] S42. In one communication cycle, the client vehicle trains the initial model several times according to its own hardware level and the collected visual data;

[0016] S43, the client vehicle sends the trained model to the vehicle fog node, and the vehicle fog node aggregates the received model;

[0017] S44. The model after aggregation will be used as the new initial model and distributed by the vehicle fog node to the client vehicle for the next round of training.

[0018] Preferably, step S41 specifically includes: setting the initial model to The vehicle is based on the resolution r of its own image data k and training round n k Register with the service area leader.

[0019] Preferably, the training of the initial model in step S42 is: training the initial model from three aspects: recognition frame, recognition object and classification.

[0020] Preferably, the recognition box loss function is:

[0021]

[0022] Among them, IoU(ω) is the predicted bounding box b(ω) and the real bounding box b gt The intersection of 2 b(ω) and b gt The Euclidean distance between 2 (ω) is the diagonal length of the closed rectangle of the intersection of the recognition box; υ(ω) is the aspect ratio parallax; α(ω)*υ(ω) is the influencing factor of image resolution.

[0023] Preferably, the loss function for identifying an object is:

[0024]

[0025] Where N is the number of bounding boxes; y i is the basic truth label, indicating whether the value of the object in the i-th bounding box exists; represents the predicted probability of containing an object in the i-th bounding box; σ is an activation function used to map the output result to the probability of (0,1).

[0026] Preferably, the loss function for classification is:

[0027]

[0028] Where C is the number of classes; y ic Indicates whether the object in the i-th box is of category c; is the probability of predicting that the i-th box is of type c.

[0029] Preferably, the total loss function is the sum of the recognition box loss function, the recognition object loss function and the classification loss function, specifically:

[0030]

[0031] Preferably, step S43 is specifically as follows:

[0032] Vehicle k updates the model according to a specific visual class optimization learning algorithm to obtain the model And calculate its increment for the previous round model, there is The increment aggregated at the vehicle fog node is Among them, D k For a specific local dataset, the expression is x i represents the i-th picture; Y i is a set of annotations, consisting of Y i ={((b j1 ,b j2 ,b j3 ,b j3 ),c j )|j∈J i}, are the bounding box coordinates and categories; the overall model optimization is expressed as Specifically, it minimizes the loss values ​​of the recognition box, recognition object and classification of the aggregation model.

[0033] Preferably, in step S5, step S4 is repeated until the vehicle fog node leaves or after several communication cycles, and the trained model is used to replace the initial model at the head of the service area.

[0034] Therefore, the present invention adopts the above-mentioned federated learning application method suitable for vehicle Internet of Things, realizes an application framework that provides different models for different environments, thereby improving the availability and robustness of the federated learning application of vehicle Internet of Things.

[0035] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flowchart of a federated learning application method applicable to vehicle Internet of Things of the present invention;

[0037] Figure 2 A comparison chart of the recall rates of the aggregation models of the embodiment of the present invention under three different environments in the test data set;

[0038] Figure 3 This is a comparison chart of the accuracy of the aggregation model of the embodiment of the present invention under three different environments under the test data set. DETAILED DESCRIPTION

[0039] Example

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0041] like Figure 1As shown, a federated learning application method suitable for vehicle Internet of Things, which is suitable for the federated learning application of vehicle Internet of Things, can avoid the leakage of vehicle user privacy to the greatest extent by providing different preliminary test models for different road sections, and can effectively improve the training efficiency and reduce the communication bandwidth requirement based on the federated learning method; the method includes the following steps:

[0042] S1. Divide the roads in the city into several servers according to their types and surrounding environments;

[0043] S2, setting the base station in the server as the service area leader, and detecting vehicles entering and leaving the coverage area of ​​the service area leader through the existing cellular network registration mechanism;

[0044] S3, the vehicles located at a specific server will periodically report their driving direction, location and available capacity to their corresponding service area heads;

[0045] S4. The vehicle performs federated learning based on the visual data it acquires. Specifically:

[0046] S41, the client vehicle and the vehicle fog node register with the service area leader and obtain the initial model of the area; that is, the initial model is set to The vehicle is based on the resolution r of its own image data k and training round n k Register with the service area leader.

[0047] S42. In one communication cycle, the client vehicle performs several trainings on the initial model based on its own hardware level and the collected visual data, including training the initial model from three aspects: recognition frame, recognition object and classification.

[0048] The recognition box loss function is:

[0049]

[0050] Among them, IoU(ω) is the predicted bounding box b(ω) and the real bounding box b gt The intersection of 2 b(ω) and b gt The Euclidean distance between 2 (ω) is the diagonal length of the closed rectangle of the intersection of the recognition box; υ(ω) is the aspect ratio parallax; α(ω)*υ(ω) is the influencing factor of image resolution.

[0051] The loss function for identifying an object is:

[0052]

[0053] Where N is the number of bounding boxes; yi is the basic truth label, indicating whether the value of the object in the i-th bounding box exists; represents the predicted probability of containing an object in the i-th bounding box; σ is an activation function used to map the output result to the probability of (0,1).

[0054] The loss function for classification is:

[0055]

[0056] Where C is the number of classes; y ic Indicates whether the object in the i-th box is of category c; is the probability of predicting that the i-th box is of type c.

[0057] The total loss function is the sum of the recognition box loss function, the recognition object loss function and the classification loss function, specifically:

[0058]

[0059] S43, the client vehicle sends the trained model to the vehicle fog node, and the vehicle fog node aggregates the received model; the details are as follows:

[0060] Vehicle k updates the model according to a specific visual class optimization learning algorithm to obtain the model And calculate its increment for the previous round model, there is The increment aggregated at the vehicle fog node is Among them, D k For a specific local dataset, the expression is x i represents the i-th picture; Y i is a set of annotations, consisting of Y i ={((b j1 ,b j2 ,b j3 ,b j3 ),c j )|j∈J i}, are the bounding box coordinates and categories; the overall model optimization is expressed as Specifically, it minimizes the loss values ​​of the recognition box, recognition object and classification of the aggregation model.

[0061] S44. The model after aggregation will be used as the new initial model and distributed by the vehicle fog node to the client vehicle for the next round of training.

[0062] S5, repeat step S4 until the vehicle fog node leaves or after several communication cycles, and use the trained model to replace the initial model on the service area leader;

[0063] S6. At the end of each communication cycle, the vehicle fog node sets the number of training times for the next communication cycle based on the performance of the client vehicle in the previous round.

[0064] Figure 2 This is a comparison chart of the recall rates of the aggregation models in three different environments under the test data set. It can be seen that in this embodiment, the model effect trained by federated learning based on environmental perception in different environments is significantly higher than the model effect of indiscriminate aggregation. As can be seen from the figure, the gap is most obvious in the main road environment, with the maximum gap close to 36%, which shows that the model trained by our framework has a higher target recognition number.

[0065] Figure 3 This is a comparison chart of the accuracy of the aggregation models in three different environments under the test data set. It can be seen that under this method, the model trained by federated learning based on environmental perception in different environments is significantly better than the model that aggregates without distinction. As can be seen from the figure, the gap is most obvious in the intersection environment, with the maximum gap close to 27%, which shows that the model trained by our method has better target recognition accuracy.

[0066] Therefore, the present invention adopts the above-mentioned federated learning application method suitable for vehicle Internet of Things, which solves the problems of low training efficiency and high recognition error rate caused by the heterogeneity of visual data in the vehicle Internet of Things environment, and realizes an application framework that provides different models for different environments, thereby improving the availability and robustness of the vehicle Internet of Things federated learning application.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A federated learning application method suitable for vehicle Internet of Things, characterized in that: The following steps are involved: S1. Divide the roads in the city into several servers according to their types and surrounding environments; S2, setting the base station in the server as the service area leader, and detecting vehicles entering and leaving the coverage area of ​​the service area leader through the cellular network registration mechanism; S3, the vehicles located at a specific server will periodically report their driving direction, location and available capacity to their corresponding service area heads; S4: The vehicle performs federated learning based on the visual data it obtains. S5, repeat step S4 until the vehicle fog node leaves or after several communication cycles; S6. At the end of each communication cycle, the vehicle fog node sets the number of training times for the next communication cycle based on the performance of the client vehicle in the previous round.

2. A federated learning application method suitable for vehicle Internet of Things according to claim 1, characterized in that: Step S4 is specifically as follows: S41, the client vehicle and the vehicle fog node register with the service area leader and obtain the initial model of the area; S42. In one communication cycle, the client vehicle trains the initial model several times according to its own hardware level and the collected visual data; S43, the client vehicle sends the trained model to the vehicle fog node, and the vehicle fog node aggregates the received model; S44. The model after aggregation will be used as the new initial model and distributed by the vehicle fog node to the client vehicle for the next round of training.

3. A federated learning application method applicable to vehicle Internet of Things according to claim 2, characterized in that: Step S41 specifically includes: setting the initial model to The vehicle is based on the resolution r of its own image data k and training round n k Register with the service area leader.

4. The method for applying federated learning to vehicle Internet of Things according to claim 2, characterized in that: The initial model is trained in step S42 as follows: the initial model is trained from three aspects: recognition frame, recognition object and classification.

5. The method for applying federated learning to vehicle Internet of Things according to claim 4, characterized in that: The recognition box loss function is: Among them, IoU(ω) is the predicted bounding box b(ω) and the real bounding box b gt The intersection of 2 b(ρ) and b gt The Euclidean distance between 2 (ω) is the diagonal length of the closed rectangle of the intersection of the recognition box; v(ω) is the aspect ratio disparity; α(ω)*υ(ω) is the influencing factor of image resolution.

6. The method for applying federated learning to vehicle Internet of Things according to claim 4, characterized in that: The loss function for identifying an object is: Where N is the number of bounding boxes; y i is the basic truth label, indicating whether the value of the object in the i-th bounding box exists; represents the predicted probability of the object contained in the i-th bounding box; σ is an activation function used to map the output result to the probability of (0,1).

7. The method for applying federated learning to vehicle Internet of Things according to claim 4, characterized in that: The loss function for classification is: Where C is the number of classes; y ic Indicates whether the object in the i-th box is of category c; is the probability of predicting that the i-th box is of type c.

8. The method for applying federated learning to vehicle Internet of Things according to claim 4, characterized in that: The total loss function is the sum of the recognition box loss function, the recognition object loss function and the classification loss function, specifically:

9. The method for applying federated learning to vehicle Internet of Things according to claim 4, characterized in that: Step S43 is specifically as follows: Vehicle k updates the model according to a specific visual class optimization learning algorithm to obtain the model And calculate its increment for the previous round model, there is The increment aggregated on the vehicle fog node is Among them, D k For a specific local dataset, the expression is x i represents the i-th picture; Y i is a set of annotations, consisting of Y i ={((b j1 ,b j2 ,b j3 ,b j3 ),c j )|j∈J i }, are the bounding box coordinates and categories; the overall model optimization is expressed as Specifically, it minimizes the loss values ​​of the recognition box, recognition object and classification of the aggregation model.

10. The method for applying federated learning to the vehicle Internet of Things according to claim 1, characterized in that: In step S5, step S4 is repeated until the vehicle fog node leaves or after several communication cycles, and the trained model is used to replace the initial model on the service area head.

Images