A federated learning method for Internet of Vehicles based on active learning and semi-supervised learning

Abstract

The present invention discloses a method for federated learning of Internet of Vehicles based on active learning and semi-supervised learning, which belongs to the field of federated learning of Internet of Vehicles. The method includes training a global model based on labeled data on a cloud server side, and updating global model parameters; distributing global model parameters to autonomous vehicles within the signal coverage of each roadside unit through a plurality of roadside units; updating the parameters of the vehicle side model according to the global model parameters and the unlabeled data based on the unlabeled data of the autonomous vehicle; sending the parameters of the vehicle side model of each autonomous vehicle to the cloud server side through the roadside unit; randomly determining an aggregated client set, and aggregating the parameters of the vehicle side model of each autonomous vehicle in the aggregated client set on the cloud server side to obtain global model parameters, and copying the global model to the teacher-student network on the cloud server side until a communication round is reached. The present invention solves the problem that the effectiveness of traditional federated learning is too low when applied to Internet of Vehicles.

Description

Technical Field

[0001] The present invention belongs to the field of vehicle network federated learning, and in particular relates to a vehicle network federated learning method based on active learning and semi-supervised learning. Background Art

[0002] The convergence of Internet of Vehicles (IoV) and machine learning (ML) is revolutionizing the automotive industry. IoV refers to the interconnection of vehicles, infrastructure, and the Internet to enable information exchange and provide a wide range of services to drivers and passengers. As a component of IoV, Connected and Automated Vehicle (CAV) combines vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication capabilities, using cameras, sensors, and other data sources to identify and locate various objects within the vehicle or entities in the vehicle environment. The application of connected autonomous vehicles aims to improve the overall safety, efficiency, and autonomy of the vehicle. By accurately detecting and identifying objects in real time, connected autonomous vehicles can make intelligent decisions and take appropriate actions to prevent collisions, bypass obstacles, and improve the overall driving experience.

[0003] RGB images are one of the most critical data modalities for autonomous vehicles when utilizing vehicles for perception tasks. However, from a user’s perspective, privacy issues are critical for image-based systems due to the intrusive design of centralized machine learning. Federated learning addresses the privacy issues inherent in image-based systems, as only model parameters or gradient information are transmitted, while sensitive data remains stored locally in the autonomous vehicle. In addition, model-based federated learning avoids the problem of excessive consumption of communication resources by image and video data sources, leading to a more efficient communication learning framework. However, deploying federated learning in connected vehicles with autonomous vehicles as the main entity poses multiple challenges. These challenges include the low accuracy of federated learning models, the lack of effective selection and aggregation methods, the lack of real-world simulators, online training strategies, and pre-trained base models, etc.

[0004] To address these challenges, people are working on machine learning architectures. First, semi-supervised learning improves detection accuracy by leveraging labeled and unlabeled data, which is particularly valuable when labeled data is scarce or difficult to obtain. Some studies focus on combining semi-supervised learning with federated learning to achieve online training of semi-supervised learning or obtain pre-trained models. However, most of these works are based on simple datasets and tasks, with limited research on object detection and no description of the deployment process in actual scenarios of the Internet of Vehicles. Second, the efficiency and model performance of federated learning are improved by changing the aggregation method and client selection. For example, the KL divergence (Kullback-Leibler divergence) of customer engagement and user data is used as a reference to improve federated learning performance. In addition, some studies have prioritized customers that are easily underestimated or ignored, thereby mitigating bias in the federated learning model and improving model performance. However, considering the needs of autonomous vehicle users for object detection in the context of the Internet of Vehicles, these works still lack refinement.

[0005] Therefore, a new federated learning architecture that can handle large amounts of real data and complex tasks for large-scale Internet of Vehicles scenarios becomes the key to the problem.

[0006] Federated learning has been widely used in tasks related to object recognition. For example, studies have shown that the federated learning framework can achieve higher detection accuracy than centralized learning and gossip-decentralized methods under adverse weather conditions in urban environments. This work demonstrates the effectiveness of collaborative training, but ignores the challenges posed by heterogeneous data distribution. In addition, existing research highlights the potential to significantly improve the performance of federated learning through strategies such as multi-level resource allocation and wise vehicle selection. An existing technical solution uses lidar technology to deploy a decentralized federated learning method for object classification on autonomous vehicles, while promoting federated collaboration through V2V communication. The results highlight that federated learning is more effective than self-learning techniques. In summary, federated learning technology has certain advantages in privacy, collaborative learning, and reduced bandwidth networks.

[0007] Active learning has gained tremendous traction, especially in conjunction with federated learning, embodied in two main concepts: applying active learning on the client side to address data challenges in federated learning, or using active learning in model selection or aggregation.

[0008] One existing technical solution uses active learning for client data selection and demonstrates the feasibility of combining active learning with federated learning for training target detection models. One existing technical solution introduces a federated learning method that considers model uncertainty and actively selects clients based on uncertainty values ​​in each round of training. One existing technical solution integrates active learning and sampling strategies into the federated learning framework and proposes an active learning method based on client collaboration to obtain global model instances that maximize the improvement of federated learning. In addition, one existing technical solution incorporates active learning into traditional federated learning, uses Gaussian processes to model client loss correlation, proposes a new client selection strategy, and improves the convergence speed of federated learning. However, most existing methods are based on data feature screening models from a single evaluation perspective, and their integration with target detection tasks is still limited. Discussions on model adaptation and training lack comprehensive demonstration and experiments.

[0009] In the field of semi-supervised learning, recent works have extended its application to object detection, mainly using the teacher-student architecture. Existing technical solutions introduce the application of this framework for object detection. However, the use of a static labeling strategy in this work limits its performance. Subsequent studies have addressed these limitations through methods such as exponential sliding average (EMA), focal loss, and active learning, effectively improving the performance. Most of these methods use a centralized architecture. In addition, some works focus on combining semi-supervised learning with federated learning using a teacher-student architecture. However, most of these works currently use simple tasks (such as image classification) as examples, which cannot be used as typical examples for autonomous vehicle applications. Summary of the invention

[0010] In view of the above-mentioned deficiencies in the prior art, the present invention provides a vehicle network federated learning method based on active learning and semi-supervised learning, which solves the problem of low effectiveness of traditional federated learning when applied to vehicle networks.

[0011] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a vehicle network federated learning method based on active learning and semi-supervised learning, comprising the following steps:

[0012] S1. Setting communication rounds, storing the labeled historical vehicle driving image data on the cloud server, i.e., labeled data, and training the global model based on the labeled data to update the global model parameters;

[0013] S2. Distribute the global model parameters to the autonomous vehicles within the signal coverage of each roadside unit through several roadside units;

[0014] S3, obtaining unlabeled data based on image data during the driving process of the autonomous driving car, and updating parameters of the vehicle side model according to the global model parameters and the unlabeled data;

[0015] S4. Sending the parameters of the vehicle side model of each autonomous driving vehicle to the cloud server through the road side unit;

[0016] S5. Randomly determine the aggregated client set, and aggregate the parameters of the vehicle-side models of each autonomous driving vehicle in the aggregated client set on the cloud server side to obtain the global model parameters, copy the global model to the teacher-student network on the cloud server side, and return to step S2 until the communication round is reached.

[0017] The beneficial effects of the present invention are as follows: the present invention proposes a teacher-student architecture that combines a method designed for semi-supervised online learning using unlabeled data, selectively filters models based on data richness, diversity, and model loss criteria, and finally proposes an advanced active aggregation framework that aims to achieve more effective aggregation from the perspective of user data, thereby improving the generalization and performance of federated learning, and applying the proposed new federated learning method to the Internet of Vehicles, thereby improving the efficiency of traditional federated learning when applied to the Internet of Vehicles.

[0018] Furthermore, in step S1, the marked data is stored on the cloud server, and the global model parameters are updated based on the marked data, specifically:

[0019] A101. On the cloud server side, the Faster R-CNN network is used as the baseline detection network to build a teacher-student network on the cloud server side. The teacher network on the cloud server side is the global model.

[0020] A102. Storing labeled data on the cloud server:

[0021] D l ={X l ,Y l}

[0022] Among them, D l is the labeled data; X l is the data set; Y l The set of labels collected for the roadside units;

[0023] A103. Obtain the current sample from the data set, use it to train the teacher-student network on the cloud server, update the parameters of the teacher-student network, obtain the teacher network parameters, which are the global model parameters, and repeat step A102 until all samples in the data set have completed training.

[0024] The beneficial effects of the above further scheme are: correctly initializing the student and teacher models on the server side is a key step in training, because subsequent stages rely on the teacher model to generate pseudo labels to train the student model; separating supervised and unsupervised learning and incremental updates can prevent the training vehicle from reverse-engineering the data through gradient changes, thereby enhancing security.

[0025] Furthermore, the step S3 is specifically as follows:

[0026] S301, in the self-driving car, using the Faster R-CNN network as the baseline detection network to build a teacher-student network of the self-driving car, i.e., a vehicle-side model;

[0027] S302, collecting image data during the driving process of the autonomous driving vehicle, obtaining unlabeled data after reaching the cache capacity of the autonomous driving vehicle, and clearing the cache;

[0028] S303, updating the teacher network of the vehicle side model based on the global model parameters;

[0029] S304, performing two levels of data enhancement on the unlabeled data to obtain weakly enhanced training data and strongly enhanced training data;

[0030] S305, input the weakly enhanced training data into the teacher network of the vehicle-side model to obtain the pseudo label of the data:

[0031]

[0032] in, is the pseudo label of the data; f() is the pseudo label calculation function; γ() is the weak data enhancement; x u,m is the unlabeled data of the mth autonomous vehicle; θ t,m are the teacher network parameters of the mth autonomous vehicle;

[0033] S306, inputting the strongly enhanced data into the student network of the vehicle-side model to obtain a predicted label of the data;

[0034] S307. Calculate the loss function of the vehicle-side model based on the pseudo-label of the data and the predicted label of the data:

[0035]

[0036]

[0037] Among them, L unsup is the loss function of the vehicle side model; N u is the number of samples in the unlabeled data; i is the sample number in the unlabeled data; L cls() is the classification loss function; is the i-th unlabeled data sample; is the pseudo label of the i-th unlabeled data sample; is the classification loss for the region proposal network; Classification loss for interest proposal leader;

[0038] S308. Update the teacher network parameters and the student network parameters respectively according to the loss function of the vehicle-side model:

[0039]

[0040]

[0041] in, is the updated student network parameter; is the current student network parameter; η is the learning rate; Under the current student network parameters; is the updated teacher network parameter; α is the hyperparameter of the teacher-student model aggregation; are the parameters of the current teacher network.

[0042] The beneficial effects of the above further scheme are: enhancing the robustness of the student model and smoothing the prediction results to achieve the effect of improving learning efficiency.

[0043] Furthermore, step S4 specifically includes each autonomous driving vehicle sending the student network parameters in the vehicle-side model to the road-side unit within the communication range, and each road-side unit sending the student network parameters in all vehicle-side models to the cloud server.

[0044] The beneficial effects of the above further scheme are: achieving model aggregation of federated learning, aggregating to obtain a robust and well-generalized global model, and transmitting only the student model can effectively achieve updates and reduce the communication burden of the federated teacher-student architecture.

[0045] Furthermore, the S5 is specifically:

[0046] S501, determining the aggregated client set capacity according to the computing load of the cloud server, and randomly acquiring the autonomous driving vehicles participating in the aggregation based on the aggregated client capacity to obtain the aggregated client set;

[0047] S502. Obtain the evaluation vector of the vehicle side model of each autonomous driving vehicle in the aggregated client set through the road side unit:

[0048]

[0049]

[0050]

[0051]

[0052] in, is the evaluation vector of the vehicle side model of the m'th autonomous driving vehicle in the aggregated client set; m' is the number of the autonomous driving vehicle in the aggregated client set; N is the information evaluation of the vehicle side model of the m'th autonomous driving vehicle in the aggregated client set; u is the number of samples in the unlabeled data; i is the sample number in the unlabeled data; j is the bounding box number of the i-th sample predicted by the teacher network; is the total number of bounding boxes of the i-th sample predicted by the teacher network; confidence() is the highest confidence score of the bounding box predicted by the teacher network; θ t,m' b is the teacher network parameter of the vehicle-side model of the m'th autonomous vehicle in the aggregated client set; j is the anchor box of the i-th sample; is the diversity evaluation of the vehicle side model of the m'th autonomous vehicle in the aggregated client set; c j is the predicted category of the jth bounding box of the i-th sample predicted by the teacher network; is the absolute value; is the loss assessment of the vehicle side model of the m'th autonomous vehicle in the aggregated client set; e is a natural constant; is the loss function; x u,m' is the unlabeled data of the m'th autonomous driving car in the aggregated client set; θ s,m' is the student network parameters of the vehicle-side model of the m'th autonomous vehicle in the aggregated client set; is the pseudo label of the data;

[0053] S503. Obtain active aggregation parameters of each autonomous driving vehicle in the aggregated client set according to the evaluation vector of the vehicle side model of each autonomous driving vehicle in the aggregated client set:

[0054]

[0055]

[0056] in, is the active aggregation parameter of the m'th autonomous driving car in the aggregation client set; a is the information evaluation weight; for Normalized data; b is the diversity assessment weight; for Normalized data; c is the loss assessment weight; for Normalized data; is the normalized data; q is the parameter reference identifier; info is the information evaluation identifier; dive is the diversity evaluation identifier; loss is the loss evaluation identifier; is the data before normalization; is the maximum value of the data before normalization;

[0057] S504, based on the student network parameters and active aggregation parameters of the vehicle-side models of each autonomous driving vehicle in the aggregation client set, a global model is obtained; the expression of the global model parameters is:

[0058]

[0059] Among them, θ g is the global model parameter; M is the aggregate client set capacity;

[0060] S505. Copy the global model to the teacher-student network on the cloud server side and return to step S2 until the communication round is reached.

[0061] The beneficial effect of the above further scheme is: efficient aggregation of the global model is achieved through the active aggregation method, so that the client model with better effect obtains a larger weight.

[0062] Furthermore, the objective function of the global model in step S504 is:

[0063]

[0064]

[0065] in, In order to make the objective function L(θ g ) the smallest global model parameter value θ; L(θ g ) is the objective function of the global model; p m' is the polymerization ratio; L m' (θ) is the loss function of the m'th autonomous driving car in the aggregated client set; m' is the number of the autonomous driving car in the aggregated client set; S t A collection of aggregated clients; is the expected value calculation; l m (θ g ; x) is the loss function of the m'th autonomous driving car in the aggregated client set; θ g is the global model parameter; x is the unlabeled raw data of the m'th autonomous driving car in the aggregated client set.

[0066] The beneficial effect of the above further scheme is: the optimization target is determined by establishing a global loss function, which is unified with the model aggregation scheme.

[0067] Furthermore, the loss function of the global model in step S504 is expressed as:

[0068] L=L sup +λL unsup

[0069]

[0070] Among them, L is the loss function of the global model; L sup is the supervised learning loss calculated on the cloud server; L unsup is the loss function of the vehicle-side model; λ is the unsupervised learning loss weight; N l is the amount of labeled data; i' is the order of labeled data; L cls () is the classification loss function; is the i'th labeled data; is the confidence label of the i'th labeled data; L reg () is the bounding box regression network layer loss; The offset label for the i'th labeled data.

[0071] The beneficial effect of the above further solution is: ensuring that the client performs effective training locally. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 The figure is a flow chart of the method of the present invention.

[0073] Figure 2 Schematic diagram of the comparison between Example 2 of the present invention and traditional federated learning and the influence of various parameters on the results. DETAILED DESCRIPTION

[0074] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0075] Example 1

[0076] like Figure 1 As shown, in one embodiment of the present invention, a vehicle network federated learning method based on active learning and semi-supervised learning includes the following steps:

[0077] S1. Setting communication rounds, storing the labeled historical vehicle driving image data on the cloud server, i.e., labeled data, and training the global model based on the labeled data to update the global model parameters;

[0078] S2. Distribute the global model parameters to the autonomous vehicles within the signal coverage of each roadside unit through several roadside units;

[0079] S3, obtaining unlabeled data based on image data during the driving process of the autonomous driving car, and updating parameters of the vehicle side model according to the global model parameters and the unlabeled data;

[0080] S4. Sending the parameters of the vehicle side model of each autonomous driving vehicle to the cloud server through the road side unit;

[0081] S5. Randomly determine the aggregated client set, and aggregate the parameters of the vehicle-side models of each autonomous driving vehicle in the aggregated client set on the cloud server side to obtain the global model parameters, copy the global model to the teacher-student network on the cloud server side, and return to step S2 until the communication round is reached.

[0082] In this embodiment, compared with the traditional federated learning algorithm, the present invention improves the detection effect of small and medium scales, speeds up the training speed of federated learning, and achieves an ideal effect.

[0083] In this embodiment, the present invention adopts a three-layer architecture consisting of a connected and automated vehicle (CAV), a road side unit (RSU) and a cloud server as the basic framework for the training and application of the Internet of Vehicles extended model. As the client of the Internet of Vehicles, the CAV uses data to train the distributed model. The RSU serves as a relay communication node and a secondary storage node. At the same time, the cloud server is responsible for initial training, model aggregation and distribution. This system architecture will be fully explained below.

[0084] First, the server is in a central position. The semi-supervised federated learning architecture proposed in this paper adopts the labels-at-server method, that is, the processing of data labels or tags is concentrated on the server side, and the final model obtained by server aggregation is regarded as the final output. Specifically, for trust considerations, the labeled data D required for semi-supervised learning l ={X l ,Y l} is stored on the server side, where X l represents the data set sample, Y l These labeled data are used to pre-train the server-side model θ g. Due to resource constraints, the labeled data is annotated by trusted personnel on the server side. In addition, due to computing power constraints, the amount of labeled data is greatly limited compared to the subsequent use of unlabeled data from real scenarios. Ensuring that the student and teacher models are properly initialized on the server side is a critical step in training, as subsequent stages rely on the teacher model to generate pseudo labels to train the student model. The intentional separation of supervised and unsupervised learning and incremental updates can prevent the training vehicle from reverse-engineering the data through gradient changes, thereby enhancing safety.

[0085] On the client side, i.e., the CAV side, it is assumed that the CAV can only collect unlabeled data to train the vehicle-side model. Considering security and privacy issues, vehicles are not allowed to transmit or label data, and only federated learning models are transmitted between vehicles and RSUs. Assume that all models share the same model architecture, represented by f:(x,θ)→f(x,θ), which maps the input x and parameter θ to a vector on a K-dimensional simplex, for example, so that the softmax function is applied to the model output.

[0086] The third layer of the architecture is the roadside unit, whose task is to solve the communication challenges between CAV and the cloud server and collect data for labeling by the cloud server. Since the parameter size of the visualization model is usually large and the clients in the connected vehicle scenario are mobile, the challenges in the transmission process cannot be ignored. Therefore, scenarios involving the mobility of autonomous vehicles and congested scenarios with limited storage and communication resources will be discussed.

[0087] First, consider the feasibility of communication. According to actual vehicle networking communication tests using real RSUs on urban roads, the effective coverage radius of RSU on straight roads without obstacles is 200 to 300 meters. Consider a scenario where a car passes this RSU at a speed of 50 kilometers per hour. Considering that the size of the trained model is about 377MB, the transmission bandwidth of the RSU must exceed 10.48Mbps, a requirement that meets realistic expectations. Based on empirical test data, the designed model size meets the communication constraints of the vehicle networking under current typical mobile scenarios. In the future landscape of the proliferation of 5G roadside base stations and the potential integration of 6G technology, the prospect of uploading and downloading larger models within the vehicle networking becomes feasible.

[0088] The second issue is related to storage resource constraints caused by the number and size of models. In the crowded scenarios of the Internet of Vehicles, when models are deployed in a distributed manner and aggregated using federated learning, the numerous parameters in these models pose a significant challenge to the computing power of the server. A strategic approach can be implemented to correct this problem, in which the edge devices of the base station are used for model selection and storage. Subsequently, the server can adopt a random selection mechanism among the participating base stations to determine the models that are eligible for aggregation. The server can then select the participating models for aggregation by adopting a random selection mechanism among the participating base stations. This random selection and distributed storage prevents highly active clients with a single dataset from disproportionately affecting the global model. In addition, this approach also reduces the storage pressure on the central server.

[0089] In step S1, the marked data is stored on the cloud server, and the global model parameters are updated based on the marked data, specifically:

[0090] A101. On the cloud server side, the Faster R-CNN network is used as the baseline detection network to build a teacher-student network on the cloud server side. The teacher network on the cloud server side is the global model.

[0091] A102. Storing labeled data on the cloud server:

[0092] D l ={X l ,Y l}

[0093] Among them, D l is the labeled data; X l is the data set; Y l The set of labels collected for the roadside units;

[0094] A103. Obtain the current sample from the data set, use it to train the teacher-student network on the cloud server, update the parameters of the teacher-student network, obtain the teacher network parameters, which are the global model parameters, and repeat step A102 until all samples in the data set have completed training.

[0095] The step S3 is specifically as follows:

[0096] S301, in the self-driving car, using the Faster R-CNN network as the baseline detection network to build a teacher-student network of the self-driving car, i.e., a vehicle-side model;

[0097] S302, collecting image data during the driving process of the autonomous driving vehicle, obtaining unlabeled data after reaching the cache capacity of the autonomous driving vehicle, and clearing the cache;

[0098] S303, updating the teacher network of the vehicle side model based on the global model parameters;

[0099] S304, performing two levels of data enhancement on the unlabeled data to obtain weakly enhanced training data and strongly enhanced training data;

[0100] S305, input the weakly enhanced training data into the teacher network of the vehicle-side model to obtain the pseudo label of the data:

[0101]

[0102] in, is the pseudo label of the data; f() is the pseudo label calculation function; γ() is the weak data enhancement; x u,m is the unlabeled data of the mth autonomous vehicle; θ t,m are the teacher network parameters of the mth autonomous vehicle;

[0103] S306, inputting the strongly enhanced data into the student network of the vehicle-side model to obtain a predicted label of the data;

[0104] S307. Calculate the loss function of the vehicle-side model based on the pseudo-label of the data and the predicted label of the data:

[0105]

[0106]

[0107] Among them, L unsup is the loss function of the vehicle side model; N u is the number of samples in the unlabeled data; i is the sample number in the unlabeled data; L cls () is the classification loss function; is the i-th unlabeled data sample; is the pseudo label of the i-th unlabeled data sample; is the classification loss for the region proposal network; Classification loss for interest proposal leader;

[0108] S308. Update the teacher network parameters and the student network parameters respectively according to the loss function of the vehicle-side model:

[0109]

[0110]

[0111] in, is the updated student network parameter; is the current student network parameter; η is the learning rate; Under the current student network parameters; is the updated teacher network parameter; α is the hyperparameter of the teacher-student model aggregation; are the parameters of the current teacher network.

[0112] The step S4 specifically includes each autonomous driving vehicle sending the student network parameters in the vehicle-side model to the road-side unit within the communication range, and each road-side unit sending the student network parameters in all vehicle-side models to the cloud server.

[0113] In this embodiment, it is necessary to prioritize privacy factors while collaboratively training a global model by utilizing individual units within the scope of the federated learning framework. Considering the actual situation of the Internet of Vehicles, a strategy of partial device participation is adopted, and the output of all devices is not required. The server determines the threshold based on its computing load and capacity.

[0114] Assuming each client has a local dataset, the aggregation step performs:

[0115]

[0116] in, is the global model parameter of the t+Eth step; S t For the aggregation client; The vehicle side model parameters of the t+Eth step length; p m' is the ratio of aggregation;

[0117] The expectation is equal to 1, and the global objective function is defined as:

[0118]

[0119] According to the distribution of local data, the local objective function at the client is defined as:

[0120]

[0121] In the object detection task, since the trained model is a black box, the correspondence between data distribution and model accuracy cannot be accurately modeled. However, it can be confirmed that the diversity and richness of data are crucial for training object detection models. They not only enhance the generalization ability of the model, making it more robust in various unseen situations, but also enhance the robustness of the model, making it more tolerant to interference such as noise and occlusion. In addition, rich and diverse training data helps to enhance the distinguishability between categories, improve detection accuracy, and enhance the model's ability to handle uncertainty, so that it can better adapt to complex real-world scenarios with blurred object boundaries and partial occlusion. In summary, the diversity and richness of data is one of the key factors to ensure the excellent training performance of object detection models.

[0122] In this embodiment, since the present invention includes semi-supervised learning, it is necessary to solve the convergence problem that occurs when integrating semi-supervised learning with federated learning, which means that the convergence problem of the client needs to be analyzed.

[0123] The training process of the teacher-student architecture is essentially to solve an optimization problem:

[0124]

[0125] Θ is the domain of network parameters; L s is the loss function of the student model; x is the input; is a pseudo-label. f(x u，m ,θ s ) is the predicted value of the student model, then the optimization problem of deep learning model training is recorded as:

[0126]

[0127] Since unlabeled data will continuously flow into the driverless car during vehicle operation, a sample collection cache is established according to the batch size. When the cache capacity is reached, the accumulated samples are fed into the network, and then the cache is cleared to seamlessly accommodate subsequent batches. It is worth noting that the collected samples can be discarded in time after network input, avoiding the need for repeated training, and the epoch is set to 1. During the operation of the autonomous vehicle, contemporary data has a greater correlation with the mainstream behavior state of users than historical data. Therefore, the reasonable practice of discarding old data is consistent with the dynamics of user behavior during vehicle operation.

[0128] For the nth batch, when the batch size is n, the optimization problem in the case of online learning is:

[0129]

[0130] l() is the batch loss; L n (θ) is the total loss.

[0131] The S5 is specifically:

[0132] S501, determining the aggregated client set capacity according to the computing load of the cloud server, and randomly acquiring the autonomous driving vehicles participating in the aggregation based on the aggregated client capacity to obtain the aggregated client set;

[0133] S502. Obtain the evaluation vector of the vehicle side model of each autonomous driving vehicle in the aggregated client set through the road side unit:

[0134]

[0135]

[0136]

[0137]

[0138] in, is the evaluation vector of the vehicle side model of the m'th autonomous driving vehicle in the aggregated client set; m' is the number of the autonomous driving vehicle in the aggregated client set; N is the information evaluation of the vehicle side model of the m'th autonomous driving vehicle in the aggregated client set; u is the number of samples in the unlabeled data; i is the sample number in the unlabeled data; j is the bounding box number of the i-th sample predicted by the teacher network; is the total number of bounding boxes of the i-th sample predicted by the teacher network; confidence() is the highest confidence score of the bounding box predicted by the teacher network; θ t,m' b is the teacher network parameter of the vehicle-side model of the m'th autonomous vehicle in the aggregated client set; j is the anchor box of the i-th sample; is the diversity evaluation of the vehicle side model of the m'th autonomous vehicle in the aggregated client set; c j is the predicted category of the jth bounding box of the i-th sample predicted by the teacher network; is the absolute value; is the loss assessment of the vehicle side model of the m'th autonomous vehicle in the aggregated client set; e is a natural constant; is the loss function; x u,m' is the unlabeled data of the m'th autonomous driving car in the aggregated client set; θ s,m' is the student network parameters of the vehicle-side model of the m'th autonomous vehicle in the aggregated client set; is the pseudo label of the data;

[0139] S503. Obtain active aggregation parameters of each autonomous driving vehicle in the aggregated client set according to the evaluation vector of the vehicle side model of each autonomous driving vehicle in the aggregated client set:

[0140]

[0141]

[0142] in, is the active aggregation parameter of the m'th autonomous driving car in the aggregation client set; a is the information evaluation weight; for Normalized data; b is the diversity assessment weight; for Normalized data; c is the loss assessment weight; for Normalized data; is the normalized data; q is the parameter reference identifier; info is the information evaluation identifier; dive is the diversity evaluation identifier; loss is the loss evaluation identifier; is the data before normalization; is the maximum value of the data before normalization;

[0143] S504, based on the student network parameters and active aggregation parameters of the vehicle-side models of each autonomous driving vehicle in the aggregation client set, a global model is obtained; the expression of the global model parameters is:

[0144]

[0145] Among them, θ g is the global model parameter; M is the aggregate client set capacity;

[0146] S505. Copy the global model to the teacher-student network on the cloud server side and return to step S2 until the communication round is reached.

[0147] In this embodiment, when traditional federated learning is used for the Internet of Vehicles, the main method of aggregation relies on the amount of data as a benchmark. Subsequently, a satisfactory model is obtained. Although the model may not be optimal, through iterative refinement, the model can generally meet the specified requirements. In a federated learning environment, complex network structures introduce complexity. The persistence of traditional strategies may significantly increase the number of iterations, thereby endangering the effectiveness of the model. Solving this challenge requires effectively designing a network architecture suitable for training under resource constraints. In response to this dilemma, we draw inspiration from the principles of active learning and introduce the innovative concept of active federated learning. This new approach aims to improve the proficiency of the federated learning global model in extracting relevant information from data and identifying more valuable users.

[0148] The traditional active learning method is to select difficult and information-rich data instances for manual annotation based on the characteristics of the data instances, thereby reducing the annotation workload. However, this approach increases risks in scenarios involving privacy-sensitive information. The main idea behind the active federated learning framework proposed in this invention is to regard the vehicle as a multi-head data detector in a distributed model training center. Enable the vehicle to judge the data source of the model training and adjust the aggregation weights. This approach reflects the idea of ​​federated learning, where models can transmit and share knowledge information learned through model aggregation.

[0149] The global model can summarize and integrate the knowledge of local devices in the distributed network through parameter aggregation. The model of each local device learns specific patterns, features, or contextual information by training local data. Through parameter aggregation, this local knowledge is transferred to the global model, enriching and improving the expressiveness of the global model.

[0150] The present invention discusses the importance of data sources and client models for improving the performance and aggregation of global models in federated learning. In addition, three indicators that affect the aggregation weight are introduced below: information, diversity, and loss valuation.

[0151] (1) Information

[0152] Information is a basic metric used to quantify the amount of information in an unlabeled image in the context of semi-supervised object detection. In some classification tasks, it is often evaluated by predicting entropy, which indicates the complexity of the task, while object detection requires a different approach. In object detection, richer information indicates the presence of more diverse visual concepts in the image, allowing the model to obtain a wider range of detection patterns.

[0153] (2) Diversity

[0154] Diversity is defined as the cardinality of the predicted class of the jth bounding box. Diversity evaluates the breadth of feature space coverage of the training data for client-side semi-supervised learning. In the actual application scenarios of the Internet of Vehicles, there may be minority classification samples. For semi-supervised learning, minority classification samples are incorrectly predicted as the majority class. In the later stage of training, plausible pseudo-labels may be added to update model parameters, resulting in poor performance. Therefore, we tend to choose trained models from data sources with high diversity.

[0155] (3) Loss valuation

[0156] The loss function value of the client participating in the training is one of the reference weights used in active federated learning. The calculation of the loss function value has been obtained during the model training process, so no additional computational overhead is incurred. It is worth noting that in the part involving vehicles, unlabeled data is used to update semi-supervised learning. Unlabeled data does not provide additional supervision information; it is used to expand the distribution area of ​​each class in the feature space, ultimately improving the generalization performance of the model. In addition, if there is an extreme classification imbalance and weak class separation, the loss of minority class data points will be significantly higher than the loss of majority class data points.

[0157] The objective function of the global model in step S504 is:

[0158]

[0159]

[0160] in, In order to make the objective function L(θ g ) the smallest global model parameter value θ; L(θ g ) is the objective function of the global model; p m' is the polymerization ratio; L m' (θ) is the objective function of the m'th autonomous driving car in the aggregated client set; m' is the number of the autonomous driving car in the aggregated client set; S t A collection of aggregated clients; is the expected value calculation; l m (θ g ; x) is the loss function of the m'th autonomous driving car in the aggregated client set; θ g is the global model parameter; x is the unlabeled raw data of the m'th autonomous driving car in the aggregated client set.

[0161] The expression of the loss function of the global model in step S504 is:

[0162] L=L sup +λL unsup

[0163]

[0164] Among them, L is the loss function of the global model; L sup is the supervised learning loss calculated on the cloud server; L unsup is the loss function of the vehicle-side model; λ is the unsupervised learning loss weight; N l is the amount of labeled data; i' is the order of labeled data; L cls () is the classification loss function; is the i'th labeled data; is the confidence label of the i'th labeled data; L reg () is the bounding box regression network layer loss; The offset label for the i'th labeled data.

[0165] Example 2

[0166] Considering the limited computing resources, ResNet-50 is used as the benchmark network of Faster RCNN for target detection, and the implementation and hyperparameter settings are the same as detectron2. In order to simulate the deployment in the Internet of Vehicles, it is not trained on multiple cards, but on a single RTX 3090 (24GB) graphics card, and the CPU used in the experiment is a 15vCPU AMD EPYC 7742 64-core processor.

[0167] For semi-supervised learning, the model is first pre-trained using a small amount of labeled data; no federation or semi-supervised learning is used in the pre-training stage, the initial model parameters are completely randomized, and the metric of the initial training is 0. The number of pre-training steps set in the experiment is 8k. After pre-training, federated semi-supervised learning is used, and the optimizer used is SGD. The general training setting is 2-10 clients, and the total training step size is 270k each time. The total training step size is equal to the number of federated communication rounds multiplied by the number of local training rounds on the client. During the training process, the learning rate is adjusted by adjusting the model according to milestones and different learning rate adjustment factors. The warm-up phase (the first 1000 rounds) allows the learning rate to be gradually increased at the beginning of training to help the model converge better

[0168] Similar to previous semi-supervised work, random horizontal flipping is used as weak teacher augmentation. In contrast, strong augmentations for the student include horizontal flipping, color jittering, grayscale, Gaussian blur, and clipping. Threshold = 0.7 is used to filter out low-quality pseudo labels. If the threshold is too small, it will only blindly encourage the global model to select customers with more labels, resulting in a decrease in the effectiveness of the model and may cause a large number of misclassifications. However, if the threshold is too large, it will bring new problems to the model. In the early stages of training, the model cannot train an accurate model, so the confidence score reaches the threshold requirement, resulting in few or even no labels. At this point, training is only about loss. In fact, only when the local model reaches a certain result will the global model consider the information richness and diversity of the images used for training. In the early stages, the global model selects a model with a smaller loss to obtain better results.

[0169] In all experiments, the hyperparameter of EMA is set to 0.9996, and the hyperparameter of the unsupervised loss in the loss function is set to 4.0. Since the experiment needs to evaluate a dataset to complete the federated active learning process, when evaluating the model after a round of training, the client is set to predict the proportion of the dataset that needs to be sampled. Other important parameters are shown in Table 1.

[0170] Table 1 Simulation parameter settings

[0171]

[0172] Below, the proposed method is first compared with the latest centralized active learning algorithm to illustrate the effectiveness of the federated learning method in training models while protecting privacy. Then, the proposed method is compared with traditional algorithms, and the results show that the proposed method has advantages in accuracy, generalization, and convergence. Insights are provided by visualizing the training process and intuitively comparing pseudo-labeled images. Finally, ablation experiments are performed to discuss the necessity of the adopted parameters. The results show that the parameter combination based on data evaluation and model performance is effective.

[0173] (1)Comparison with the state-of-the-art.

[0174] The present invention is compared with existing advanced algorithms, most of which are based on teacher-student semi-supervised learning architectures, which are centralized learning methods. In general, federated learning will reduce the training effect, which is usually seen as a balance between privacy and data security and model effectiveness. The initial goal is to make the effect of active federated semi-supervised learning close to that of centralized semi-supervised learning. After 200,000 rounds of training, we obtained the results shown in Table 2. In this case, "Active-Teacher" represents the state-of-the-art (SOTA) method, and the metric we use is mAP (50:95), "Supervised" refers to the performance of the model trained only with labeled data, Δ: AP gain over supervised performance.

[0175] Table 2 Comparison of the proposed AFSSL with other methods on MS-COCO val2017

[0176]

[0177] As shown in the table, good results were achieved with limited labeled data, which is of great significance for practical application scenarios. It should be noted here that when the proportion of labeled data is small, the model effect is easy to remain unchanged without further optimization. This may be due to poor model performance after the pre-training stage or too small learning rate. This experiment verifies that the algorithm proposed in this invention can achieve performance close to that of a centralized algorithm while ensuring privacy, indicating that federated learning can be used as an effective learning paradigm in the Internet of Vehicles.

[0178] (2)Comparison with traditional federated learning.

[0179] In the initial analysis, the AP values ​​obtained by the models generated by the two different learning methods at different training stages were comprehensively compared, using specific labeled scale data. The evaluation indicators include mAP (50:95),

[0180] AP50, AP75, APs, APm, and Apl. The subsequent results (see the table below) show that active federated semi-supervised learning can quickly reach the desired accuracy level compared to traditional federated semi-supervised learning.

[0181] In addition, if Figure 2 As shown, the proposed method shows superior performance compared with traditional methods, especially in scenarios involving small and medium scales. This consistency with the application requirements of the Internet of Vehicles is noteworthy. Considering the dynamic characteristics of the Internet of Vehicles, users must actively perceive long-range and micro objects to avoid potential accidents or preemptively send road safety information warnings with quantifiable confidence.

[0182] Table 3 Comparison results of the two methods on small and medium scales

[0183]

[0184] (3) Visualization of pseudo labels

[0185] To demonstrate the generalization of the model, performance on the test set is shown and the pseudo labels are visualized.

[0186] This visualization intuitively illustrates that the model trained with the new method performs better on a test set based on real-world scenarios. It can be seen that the traditional method has limited recognition capabilities for small targets at a distance and is not as good as the proposed algorithm in complex situations. We speculate that active federated learning aggregation tends to train models on data with richer semantic information and greater diversity. We believe that this approach helps to quickly acquire knowledge from the new model, giving it an advantage in applications such as detecting small-scale objects in complex traffic scenes and images.

[0187] (4) Ablation experiment

[0188] In the ablation experiment, the necessity of using these parameters is discussed. It can be seen that the best effect is achieved when the three parameters are used in combination, while the effect is not ideal when only the loss parameter is used. It is assumed that, in the case of imbalanced data types and distributions, the loss of each client only reflects the effectiveness of the model trained on its local dataset. When models from different clients are aggregated, if there is a significant difference between the test set of the global model and the training set of the client, the test performance may degrade. On the other hand, if the evaluation is based solely on the data distribution, it may lead to training difficulties and poor model performance of the dominant model, thereby slowing down the overall training progress.

[0189] Table 4 Comparison of ablation test results under different parameters

[0190]

Claims

1. A method for federated learning of Internet of Vehicles based on active learning and semi-supervised learning, characterized in that: The following steps are involved: S1. Setting communication rounds, storing the labeled historical vehicle driving image data on the cloud server, i.e., labeled data, and training the global model based on the labeled data to update the global model parameters; S2. Distribute the global model parameters to the autonomous vehicles within the signal coverage of each roadside unit through several roadside units; S3, obtaining unlabeled data based on image data during the driving process of the autonomous driving car, and updating parameters of the vehicle side model according to the global model parameters and the unlabeled data; S4. Sending the parameters of the vehicle side model of each autonomous driving vehicle to the cloud server through the road side unit; S5, randomly determine an aggregated client set, and aggregate the parameters of the vehicle-side models of each autonomous driving vehicle in the aggregated client set on the cloud server side to obtain global model parameters, copy the global model to the teacher-student network on the cloud server side, and return to step S2 until the communication round is reached; S5 is specifically: S501, determining the aggregated client set capacity according to the computing load of the cloud server, and randomly acquiring the autonomous driving vehicles participating in the aggregation based on the aggregated client capacity to obtain the aggregated client set; S502. Obtain the evaluation vector of the vehicle side model of each autonomous driving vehicle in the aggregated client set through the road side unit: in, is the evaluation vector of the vehicle side model of the m'th autonomous driving vehicle in the aggregated client set; m' is the number of the autonomous driving vehicle in the aggregated client set; N is the information evaluation of the vehicle side model of the m'th autonomous driving vehicle in the aggregated client set; u is the number of samples in the unlabeled data; i is the sample number in the unlabeled data; j is the bounding box number of the i-th sample predicted by the teacher network; is the total number of bounding boxes of the i-th sample predicted by the teacher network; confidence() is the highest confidence score of the bounding box predicted by the teacher network; θ t,m' b is the teacher network parameter of the vehicle-side model of the m'th autonomous vehicle in the aggregated client set; j is the anchor box of the i-th sample; is the diversity evaluation of the vehicle side model of the m'th autonomous vehicle in the aggregated client set; c j is the predicted category of the jth bounding box of the i-th sample predicted by the teacher network; || is the absolute value; is the loss assessment of the vehicle side model of the m'th autonomous vehicle in the aggregated client set; e is a natural constant; is the loss function; x u,m' is the unlabeled data of the m'th autonomous driving car in the aggregated client set; θ s,m' is the student network parameters of the vehicle-side model of the m'th autonomous vehicle in the aggregated client set; is the pseudo label of the data; S503. Obtain active aggregation parameters of each autonomous driving vehicle in the aggregated client set according to the evaluation vector of the vehicle side model of each autonomous driving vehicle in the aggregated client set: in, is the active aggregation parameter of the m'th autonomous driving car in the aggregation client set; a is the information evaluation weight; for Normalized data; b is the diversity assessment weight; for Normalized data; c is the loss assessment weight; for Normalized data; is the normalized data; q is the parameter reference identifier; info is the information evaluation identifier; dive is the diversity evaluation identifier; loss is the loss evaluation identifier; is the data before normalization; is the maximum value of the data before normalization; S504, based on the student network parameters and active aggregation parameters of the vehicle-side models of each autonomous driving vehicle in the aggregation client set, a global model is obtained; the expression of the global model parameters is: Among them, θ g is the global model parameter; M is the aggregate client set capacity; S505. Copy the global model to the teacher-student network on the cloud server side and return to step S2 until the communication round is reached.

2. The vehicle network federated learning method based on active learning and semi-supervised learning according to claim 1 is characterized in that: In step S1, the marked data is stored on the cloud server, and the global model parameters are updated based on the marked data, specifically: A101. On the cloud server side, the Faster R-CNN network is used as the baseline detection network to build a teacher-student network on the cloud server side. The teacher network on the cloud server side is the global model. A102. Storing labeled data on the cloud server: D l ={X l ,Y l } Among them, D l is the labeled data; X l is the data set; Y l The set of labels collected for the roadside units; A103. Obtain the current sample from the data set, use it to train the teacher-student network on the cloud server, update the parameters of the teacher-student network, obtain the teacher network parameters, which are the global model parameters, and repeat step A102 until all samples in the data set have completed training.

3. The vehicle network federated learning method based on active learning and semi-supervised learning according to claim 1 is characterized in that: The step S3 is specifically as follows: S301, in the self-driving car, using the Faster R-CNN network as the baseline detection network to build a teacher-student network of the self-driving car, i.e., a vehicle-side model; S302, collecting image data during the driving process of the autonomous driving vehicle, obtaining unlabeled data after reaching the cache capacity of the autonomous driving vehicle, and clearing the cache; S303, updating the teacher network of the vehicle side model based on the global model parameters; S304, performing two levels of data enhancement on the unlabeled data to obtain weakly enhanced training data and strongly enhanced training data; S305, input the weakly enhanced training data into the teacher network of the vehicle-side model to obtain the pseudo label of the data: in, is the pseudo label of the data; f() is the pseudo label calculation function; γ() is the weak data enhancement; x u,m is the unlabeled data of the mth autonomous vehicle; θ t,m are the teacher network parameters of the mth autonomous vehicle; S306, inputting the strongly enhanced data into the student network of the vehicle-side model to obtain a predicted label of the data; S307. Calculate the loss function of the vehicle-side model based on the pseudo-label of the data and the predicted label of the data: Among them, L unsup is the loss function of the vehicle side model; N u is the number of samples in the unlabeled data; i is the sample number in the unlabeled data; L cls () is the classification loss function; is the i-th unlabeled data sample; is the pseudo label of the i-th unlabeled data sample; is the classification loss for the region proposal network; Classification loss for interest proposal leader; S308. Update the teacher network parameters and the student network parameters respectively according to the loss function of the vehicle-side model: in, is the updated student network parameter; is the current student network parameter; η is the learning rate; Under the current student network parameters; is the updated teacher network parameter; α is the hyperparameter of the teacher-student model aggregation; are the parameters of the current teacher network.

4. The vehicle network federated learning method based on active learning and semi-supervised learning according to claim 3 is characterized in that: The step S4 specifically includes each autonomous driving vehicle sending the student network parameters in the vehicle-side model to the road-side unit within the communication range, and each road-side unit sending the student network parameters in all vehicle-side models to the cloud server.

5. The vehicle network federated learning method based on active learning and semi-supervised learning according to claim 1 is characterized in that: The objective function of the global model in step S504 is: in, In order to make the objective function L(θ g ) the smallest global model parameter value θ; L(θ g ) is the objective function of the global model; p m' is the polymerization ratio; L m' (θ) is the loss function of the m'th autonomous driving car in the aggregated client set; m' is the number of the autonomous driving car in the aggregated client set; S t A collection of aggregated clients; is the expected value calculation; l m (θ g ; x) is the loss function of the m'th autonomous driving car in the aggregated client set; θ g is the global model parameter; x is the unlabeled raw data of the m'th autonomous driving car in the aggregated client set.

6. The vehicle network federated learning method based on active learning and semi-supervised learning according to claim 1 is characterized in that: The expression of the loss function of the global model in step S504 is: L=L sup +λL unsup Among them, L is the loss function of the global model; L sup is the supervised learning loss calculated on the cloud server; L unsup is the loss function of the vehicle-side model; λ is the unsupervised learning loss weight; N l is the amount of labeled data; i' is the order of labeled data; L cls () is the classification loss function; is the i'th labeled data; is the confidence label of the i'th labeled data; L reg () is the bounding box regression network layer loss; The offset label for the i'th labeled data.

Images