A heterogeneous federated learning convergence optimization method for vehicle perception

Abstract

The present application relates to a kind of heterogeneous federal learning convergence optimization methods for vehicle perception, belong to vehicle perception and federal learning technical field, solve the existing federal learning in vehicle perception scene because of the heterogeneity of each vehicle node perception data and lead to model training convergence slow, the problem such as the decline of perception accuracy and high communication cost.The method includes: each vehicle node downloads the global basic layer parameter of previous communication round and the personalized layer parameter after fusion, uses local training data set to train, obtains the basic layer parameter and personalized layer parameter of current communication round;Current communication round personalized layer parameter and historical communication round personalized layer parameter are weighted and fused, and updated personalized layer parameter is obtained;Each vehicle node uploads the basic layer parameter of current communication round and updated personalized layer parameter to cloud server for fusion, and obtains the global basic layer parameter of current communication round and the corresponding personalized layer parameter after fusion of each vehicle node.

Description

Technical Field

[0001] This invention relates to the field of vehicle perception and federated learning technology, and in particular to a heterogeneous federated learning convergence optimization method for vehicle perception. Background Technology

[0002] In vehicle perception scenarios, federated learning is a distributed machine learning paradigm whose core idea is "the data doesn't move, the model moves." It allows the same perception model to be collaboratively trained on multiple vehicle nodes holding local data without uploading the raw sensor data to a cloud server, thus protecting data privacy and reducing transmission bandwidth. However, in practical applications, due to differences in the driving areas and road environments of each vehicle, as well as variations in the perspective and perception capabilities of the sensors on each vehicle, the training data collected by each vehicle node exhibits non-independent and non-uniformly distributed characteristics, i.e., perception data heterogeneity. This manifests in various forms, such as differences in scene distribution across different driving areas, differences in data distribution under different lighting and weather conditions, and an imbalance in the number of samples collected by each vehicle.

[0003] Existing federated learning methods rely on traditional (weighted) average aggregation methods, requiring different layers of the neural network model to use the same aggregation weights. This coarse-grained aggregation approach ignores the differentiated representation capabilities of different network layers in vehicle perception tasks and cannot effectively adapt to the non-independent and identically distributed characteristics of vehicle node perception data. This slows down the convergence speed of federated learning methods, reduces the accuracy of vehicle models, and increases the communication overhead between vehicle nodes and cloud servers. On the other hand, traditional model aggregation methods ignore historical model parameters from the local training process of federated learning. These historical model parameters still contain a large amount of experiential knowledge about dynamic traffic scenarios from each vehicle node in past training rounds. Therefore, the heterogeneity of perception data and the coarse-grained aggregation method together lead to slow convergence and decreased perception accuracy of traditional federated learning algorithms in vehicle perception scenarios. Summary of the Invention

[0004] Based on the above analysis, the embodiments of the present invention aim to provide a heterogeneous federated learning convergence optimization method for vehicle perception, in order to solve the problems of slow model training convergence, decreased perception accuracy, and high communication costs caused by the heterogeneity of perception data of each vehicle edge node in the existing federated learning in vehicle perception scenarios.

[0005] The objective of this invention is mainly achieved through the following technical solutions: On the one hand, this invention provides a heterogeneous federated learning convergence optimization method for vehicle perception, comprising: Each vehicle node acquires road perception data and constructs a local training dataset; Each vehicle node downloads the global base layer parameters of the previous communication round and the fused personalized layer parameters of the previous communication round corresponding to each vehicle node from the cloud server, and trains the local model using the local training dataset to obtain the base layer parameters and personalized layer parameters of the current communication round; wherein, each vehicle node has the same model structure. The personalized layer parameters of the current communication round and the personalized layer parameters of the historical communication rounds are weighted and fused to obtain the updated personalized layer parameters of each vehicle node. Each vehicle node uploads the basic layer parameters and the updated personalized layer parameters of the current communication round to the cloud server for fusion, thereby obtaining the global basic layer parameters of the current communication round and the fused personalized layer parameters corresponding to each vehicle node.

[0006] Furthermore, based on a preset number of network layers, the model of each vehicle node is divided into a basic layer and a personalized layer, with the corresponding model parameters being the basic layer parameters and the personalized layer parameters, respectively.

[0007] Furthermore, when training the local model for each vehicle node, the sum of the original loss function of each vehicle node and the L2 regularization term of the base layer parameters is used as the training optimization objective.

[0008] Furthermore, the optimization objective is expressed as: ; in, This represents the model parameters of the i-th vehicle node in the t-th communication round; Let i represent the loss function for the i-th vehicle node; Indicates the penalty coefficient; This represents the base layer parameters of the i-th vehicle node in the t-th communication round; This represents the global base layer parameter in the (t-1)th communication round.

[0009] Furthermore, based on the model loss values ​​of a preset number of historical communication rounds prior to the current communication round for each vehicle node, the personalized layer parameters of the current communication round and the personalized layer parameters of each historical communication round are weighted and fused using the following formula to obtain the updated personalized layer parameters for each vehicle node: ; ; in, This represents the updated personalized layer parameters obtained by the i-th vehicle node in the t-th communication round; Indicates the size of the time window; This represents the fusion weight of the i-th vehicle node in the t-th communication round; This represents the loss value of the i-th vehicle node in the t-th communication round; This represents the maximum value of the i-th vehicle node within the time window of the t-th communication round; This represents the minimum value of the i-th vehicle node within the time window of the t-th communication round.

[0010] Furthermore, each vehicle node uploads the basic layer parameters of the current communication round and the updated personalized layer parameters to the cloud server for parameter fusion between vehicle nodes, including: Based on the number of training datasets for each vehicle node, the basic layer parameters are weighted and fused to obtain global basic layer parameters; Based on the similarity value of the gradient of the personalized layer parameters of each vehicle node in the same network layer, the personalized layer parameters are fused in a layered weighted manner to obtain the fused personalized layer parameters corresponding to each vehicle node.

[0011] Furthermore, the base layer parameters are weighted and fused using the following formula to obtain the global base layer parameters: ; in, Represents the global base layer parameters for the t-th communication round; N represents the number of vehicle nodes; This represents the number of training datasets for the i-th vehicle node; This represents the base layer parameters obtained by the i-th vehicle node in the t-th communication round.

[0012] Furthermore, based on the similarity values ​​of the gradients of the personalized layer parameters of each vehicle node within the same network layer, the personalized layer parameters are subjected to hierarchical weighted fusion to obtain the fused personalized layer parameters corresponding to each vehicle node, including: Based on the personalized layer parameters of each vehicle node, the sub-parameters of each network layer of the personalized layer of each vehicle node are obtained. For each network layer, the cosine similarity between the gradients of the sub-parameters of any two vehicle nodes in the corresponding network layer is calculated, and used as the gradient similarity value of the two vehicle nodes in the corresponding network layer. Based on the gradient similarity values ​​of each vehicle node and other vehicle nodes in each network layer, a hierarchical aggregation weight matrix is ​​constructed for each vehicle node and other vehicle nodes. Based on the hierarchical aggregation weight matrix, the sub-parameters of other vehicle nodes in each network layer of each vehicle node are weighted and summed, and the weighted summation result is superimposed on the sub-parameters of the corresponding network layer of each vehicle node to obtain the updated sub-parameters of each vehicle node in each network layer. Based on the updated sub-parameters of each vehicle node in each network layer, the fused personalized layer parameters corresponding to each vehicle node are obtained.

[0013] Furthermore, the gradient similarity values ​​of any two vehicle nodes at each network layer can be obtained using the following formula: ; in, This represents the gradient similarity value between the l-th network layer of the i-th vehicle node and the j-th vehicle node at the t-th communication round; and These represent the sub-parameters of the l-th network layer of the i-th vehicle node and the j-th vehicle node in the t-th communication round, respectively. This represents the gradient function.

[0014] Furthermore, the hierarchical aggregation weight matrix is ​​expressed as: ; ; in, This represents the hierarchical aggregation weight matrix of the i-th vehicle node in the t-th communication round; The weights of the l-th network layer for the i-th and j-th vehicle nodes in the t-th communication round are represented by N; N represents the number of vehicle nodes; and L represents the number of personalization layers.

[0015] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects: 1. This invention divides the vehicle node model into a basic layer and a personalized layer, and introduces an L2 regularization term for the basic layer parameters on top of the original loss function during model training. This allows each vehicle node to continuously learn general perception features in its basic layer and adapt to the unique features of the region where each vehicle is located during local training. This effectively mitigates the impact of heterogeneity in training data on model training and significantly improves the model's recognition accuracy and generalization ability in various scenarios.

[0016] 2. This invention integrates the personalized layer parameters accumulated by each vehicle node in historical training rounds into the personalized layer parameter updates of each round through a loss value weighted fusion method, which can accelerate model convergence and reduce communication overhead.

[0017] 3. When aggregating parameters of various vehicle nodes on a cloud server, this invention performs weighted fusion of basic layer parameters to obtain unified global basic layer parameters, calculates the gradient similarity between any two vehicle nodes according to network layers for personalized layer parameters, and performs weighted aggregation of the same network layer parameters of vehicle nodes based on gradient similarity. This reduces negative migration between dissimilar vehicle nodes, enabling each vehicle node to obtain personalized layer parameters that best suit its own data distribution. This significantly improves the local model adaptation accuracy of each vehicle node in data heterogeneity scenarios and the overall performance of the federated learning system.

[0018] 4. This invention transmits model parameters only between each vehicle node and the cloud server. The original vehicle sensor data is stored locally on the vehicle node and does not participate in cross-node transmission. Through distributed collaborative training between the cloud server and vehicle edge nodes, the transfer and fusion of cross-vehicle perception knowledge can be achieved while protecting data privacy, providing an efficient and secure training solution for distributed model training in vehicle perception scenarios.

[0019] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0020] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0021] Figure 1 This is a flowchart illustrating a heterogeneous federated learning convergence optimization method for vehicle perception in an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating the process of fusing personalized layer parameters for each vehicle node in an embodiment of the present invention. Detailed Implementation

[0022] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0023] One specific embodiment of the present invention discloses a convergence optimization method for heterogeneous federated learning oriented towards vehicle perception, applied to a federated learning system including a cloud server and several vehicle nodes. Federated learning is a distributed machine learning framework whose core idea is "data remains stationary, model moves." In federated learning, each participating node stores its training data locally and only uploads model parameters or gradient information to a central server. The central server aggregates and distributes the updated global model. This approach achieves multi-node collaborative training while protecting data privacy.

[0024] like Figure 1 As shown, the method includes the following steps S1-S4: Step S1: Each vehicle node acquires road perception data and constructs a local training dataset.

[0025] Specifically, each vehicle node is equipped with a variety of sensors, including but not limited to forward / surround view cameras, lidar, or millimeter-wave radar. During vehicle operation, the sensors collect road environment data. The cameras are used to collect image data around the vehicle; the lidar is used to collect three-dimensional point cloud data around the vehicle node; and the millimeter-wave radar is used to collect directional information such as speed and azimuth of targets around the vehicle node.

[0026] It should be noted that the sensors mounted on the vehicle node and the types of data they collect are only one specific implementation method provided for the vehicle target detection and recognition task in this embodiment. Those skilled in the art will understand that when the perception task of the vehicle node changes, the type of onboard sensor, the sensor deployment method, and the corresponding perception data used can be adjusted and replaced according to the specific task requirements, and these adjustments do not depart from the protection scope of this invention.

[0027] Furthermore, the road perception data acquired by the sensors mounted on the vehicle nodes is preprocessed, including but not limited to image size normalization, data cleaning, noise reduction, and image enhancement.

[0028] The preprocessed road perception data is labeled with targets in the data manually or semi-automatically according to the current perception task of the vehicle node, resulting in detection boxes and category labels. Based on the road perception data and the corresponding detection boxes and category labels, a local training dataset for each vehicle node is obtained. It is understood that the local training dataset for each vehicle node is always stored locally and the original data is not transmitted externally; it is only used for local model training at each vehicle node.

[0029] Furthermore, based on the training dataset of each vehicle node, federated learning is performed on the cloud server and each vehicle node until the global model of the cloud server satisfies a preset convergence condition; wherein, the preset convergence condition may be that the number of communication rounds reaches a preset maximum communication round threshold. During the learning process of each communication round, the following steps S2-S4 are executed: Step S2: Each vehicle node downloads the global base layer parameters of the previous communication round and the fused personalized layer parameters of the previous communication round corresponding to each vehicle node from the cloud server, and trains the local model using the local training dataset to obtain the base layer parameters and personalized layer parameters of the current communication round; wherein, each vehicle node has the same model structure.

[0030] In this embodiment, all vehicle nodes use the same neural network structure, which can be, for example, a target detection model built on a convolutional neural network.

[0031] Furthermore, the vehicle node model is divided into a base layer and a personalized layer according to a preset number of network layers. The base layer includes several convolutional and pooling layers near the input, responsible for extracting common visual features from the input perceptual data, such as edges, corners, and textures. These features have high similarity among different vehicle nodes. The personalized layer is the network structure outside the base layer. Specifically, it can be several fully connected layers and a classification or regression head near the output, used for high-level semantic reasoning based on the common features, outputting prediction results for specific perceptual tasks. The parameters of these layers are closely related to the distribution of local perceptual data for each vehicle node, and there are significant differences between different vehicle nodes. More specifically, the preset number of network layers can be determined based on the total number of layers in the neural network model used. For example, when the local model of the vehicle node is ResNet34, it includes a total of 34 network layers. The first 17 layers can be used as the base layer, and the last 17 layers as the personalized layer. In this case, the preset number of network layers is 17. It is understood that those skilled in the art can set the preset number of network layers according to the actual perceptual task requirements and model structure.

[0032] The model parameters corresponding to the base layer are base layer parameters; the model parameters corresponding to the personalization layer are personalization layer parameters.

[0033] Furthermore, each vehicle node downloads the global base layer parameters of the previous communication round and the fused personalized layer parameters of the previous communication round corresponding to each vehicle node from the cloud server.

[0034] Specifically, the cloud server stores the global basic layer parameters obtained from the aggregation of the previous communication round and the fused personalized layer parameters of each vehicle node corresponding to the previous communication round. The global basic layer parameters are obtained by weighted fusion of the basic layer parameters uploaded by each vehicle node in the cloud server after the previous communication round, representing the general parameters of all participating vehicle nodes in the general feature extraction layer. The fused personalized layer parameters are obtained by aggregating each vehicle node separately in the cloud server after the previous communication round, representing the personalized parameters of each vehicle node after the previous communication round.

[0035] It should be noted that, for the first round of communication, the cloud server can randomly generate global model parameters and personalized layer parameters for each vehicle node.

[0036] Furthermore, the local training dataset is loaded to perform multiple batches of iterative training on the local model. When training the local model for each vehicle node, the sum of the original loss function of each vehicle node and the L2 regularization term of the base layer parameters is used as the training optimization objective, and the model parameters are updated using gradient backpropagation.

[0037] Specifically, in order to maintain the learning ability of the base layer of each vehicle node to the general perception features of the vehicle during local training, and to avoid the base layer parameters of each vehicle node from shifting due to fitting their own local data distribution, this embodiment introduces an L2 regularization term for the base layer parameters in the optimization objective of local training. The optimization objective is expressed as: ; in, This represents the model parameters of the i-th vehicle node in the t-th communication round, including the basic layer parameters and the personalized layer parameters of the vehicle node; Let i represent the loss function for the i-th vehicle node; Indicates the penalty coefficient; This represents the base layer parameters of the i-th vehicle node in the t-th communication round; This represents the global base layer parameter in the (t-1)th communication round.

[0038] It should be noted that the loss function is related to the perception task performed by the vehicle node. For example, when the task performed by the vehicle node is target detection, the loss function can be cross-entropy loss. Those skilled in the art can select an appropriate original loss function according to the actual perception task type.

[0039] The penalty coefficient is used to control the weight of the L2 regularization term in the overall optimization objective. In this embodiment, its value range can be from 0.01 to 0.1. When the value is relatively large, the base layer parameters of each vehicle node are forcibly constrained to be near the global base layer parameters; when the penalty coefficient is relatively large... When the value is relatively small, the basic layer parameters of each vehicle node can be better adapted to the local data distribution.

[0040] Step S3: Perform weighted fusion of the personalized layer parameters of the current communication round and the personalized layer parameters of the historical communication rounds to obtain the updated personalized layer parameters of each vehicle node.

[0041] Specifically, traditional federated learning methods only use the model parameters trained in the current round for aggregation and updating in each round of communication. However, in vehicle perception scenarios, the local data distribution of each vehicle node has a certain continuity and correlation over time. Based on this, this embodiment achieves effective fusion of historical experience by weighted fusion of personalized layer parameters from the current round and historical rounds.

[0042] Furthermore, based on the model loss values ​​of a preset number of historical communication rounds prior to the current communication round for each vehicle node, the personalized layer parameters of the current communication round and the personalized layer parameters of each historical communication round are weighted and fused using the following formula to obtain the updated personalized layer parameters for each vehicle node: ; ; in, This represents the updated personalized layer parameters obtained by the i-th vehicle node in the t-th communication round; Indicates the size of the time window; This represents the fusion weight of the i-th vehicle node in the t-th communication round; This represents the loss value of the i-th vehicle node in the t-th communication round; This represents the maximum value of the i-th vehicle node within the time window of the t-th communication round, and its calculation method is as follows: ; This represents the minimum value of the i-th vehicle node within the time window of the t-th communication round, and its calculation method is as follows: .

[0043] Specifically, a time window size τ (τ is a positive integer) is predefined to represent the number of historical communication rounds involved in the fusion. In this embodiment, the value range of τ is set to [5, 10]. For example, when τ = 5, it means that the personalized layer parameters of the current round and the previous 5 historical rounds are fused. More specifically, when t = 1, the weighted fusion operation is not performed, and the personalized layer parameters of the current round are directly used as the updated personalized layer parameters; when 2 ≤ t ≤ τ, that is, when the current communication round has not accumulated enough historical rounds, τ = t-1 is set, which means that the fusion starts from the first communication round and goes up to the previous communication round, that is, the number of historical rounds involved in the fusion is t-1 rounds.

[0044] It should be noted that, in this embodiment, the loss value of the i-th vehicle node in the t-th communication round is... Let be the loss function value obtained by vehicle node i after completing all training in the current batch in the t-th communication round.

[0045] Understandably, through the weighted fusion strategy of personalized layer parameters from the above historical rounds, each vehicle node can inherit the experience and knowledge of the best model in the historical training rounds, avoiding the negative impact of poor training performance in a certain round on the overall convergence, thus accelerating the convergence speed of the model. At the same time, the weight calculation is based on the loss value of each vehicle node during local training, without the need for additional communication overhead.

[0046] Step S4: Each vehicle node uploads the basic layer parameters and the updated personalized layer parameters of the current communication round to the cloud server for fusion, so as to obtain the global basic layer parameters of the current communication round and the fused personalized layer parameters corresponding to each vehicle node.

[0047] Specifically, each vehicle node uploads its basic layer parameters and personalized layer parameters to the cloud server via a wireless communication network, and then merges the basic layer parameters and personalized layer parameters of each vehicle node.

[0048] Furthermore, based on the number of training datasets for each vehicle node, the base layer parameters are weighted and fused to obtain global base layer parameters.

[0049] Specifically, the base layer parameters represent general perception features. These features have high similarity and transferability among different vehicle nodes. Therefore, in this embodiment, the base layer parameters of all vehicle nodes are globally weighted and fused to obtain global base layer parameters that represent the general perception capabilities of all participating vehicles.

[0050] Furthermore, the base layer parameters are weighted and fused using the following formula to obtain the global base layer parameters: ; in, Represents the global base layer parameters for round t; N represents the number of vehicle nodes; This represents the number of training datasets for the i-th vehicle node; This represents the base layer parameters obtained by the i-th vehicle node in the t-th round of training.

[0051] It is understandable that weighted fusion based on the amount of training data for each vehicle node indicates that vehicle nodes with larger amounts of training data have learned the common features more fully and can be given higher fusion weights.

[0052] Furthermore, based on the similarity value of the gradient of the personalized layer parameters of each vehicle node in the same network layer, the personalized layer parameters are fused in a layered weighted manner to obtain the fused personalized layer parameters corresponding to each vehicle node.

[0053] Specifically, since the personalization layer is used to adapt to the local data distribution of each vehicle node, there are significant differences in the personalization layer parameters between different vehicle nodes. If a parameter fusion method similar to that of the basic layer is used, the aggregation result will not be able to effectively adapt to the local data distribution of any single vehicle node, thereby reducing the perception accuracy of each vehicle node.

[0054] Furthermore, such as Figure 2 As shown, the personalized layer parameters of each vehicle node are weighted and fused in a layered manner based on the similarity value of the gradient of the personalized layer parameters in the same network layer to obtain the fused personalized layer parameters corresponding to each vehicle node, including steps S41-S45: Step S41: Based on the personalized layer parameters of each vehicle node, obtain the sub-parameters of each network layer of the personalized layer of each vehicle node.

[0055] Specifically, the personalized layer parameters uploaded by each vehicle node are decomposed according to its network layer structure to obtain the sub-parameters of each network layer, represented as follows: ;in, Let L represent the personalized layer parameters of the i-th vehicle node in the t-th communication round; L represents the total number of network layers in the personalized layer.

[0056] Step S42: For each network layer, calculate the cosine similarity between the gradients of the sub-parameters of any two vehicle nodes in the corresponding network layer, and use this as the gradient similarity value of the two vehicle nodes in the corresponding network layer. The formula is as follows: ; in, This represents the gradient similarity value between the l-th network layer of the i-th vehicle node and the j-th vehicle node at the t-th communication round; and These represent the sub-parameters of the l-th network layer of the i-th vehicle node and the j-th vehicle node in the t-th communication round, respectively. Represents the gradient function; This represents the vector dot product operation; This represents the L2 norm of a vector.

[0057] Specifically, in the above calculation formula, the molecule The denominator represents the degree of similarity between two gradient vectors along their directions. This means normalizing the magnitudes of the two gradient vectors so that the range of similarity values ​​is not affected by the magnitude of the absolute gradient. Since the result of cosine similarity calculation is between [-1, 1], this embodiment needs to use the gradient similarity value as a weight for subsequent calculations. The gradient similarity value should avoid being negative; therefore, 1 is added to the cosine similarity value to make the gradient similarity value... The value range is [0,2].

[0058] Step S43: Based on the gradient similarity values ​​of each vehicle node and other vehicle nodes in each network layer, construct a hierarchical aggregated weight matrix of each vehicle node and other vehicle nodes.

[0059] Specifically, based on the gradient similarity values ​​obtained in step S42, a hierarchical aggregation weight matrix is ​​constructed for each vehicle node. This is used to represent which network layer knowledge each vehicle node receives from other vehicle nodes, specifically represented as: ; The matrix has a dimension of L×(N-1); N represents the number of vehicle nodes; and the matrix contains... The row corresponds to the first Layers are network layers; columns correspond to other vehicle nodes.

[0060] In the matrix This represents the i-th vehicle node and the j-th vehicle node. The weight of a layer in the network at the t-th communication round is given by the formula; ; Specifically, the above formula represents the expression for the i-th vehicle node at the i-th time node. The gradient similarity values ​​of the i-th vehicle node with all other vehicle nodes in the network layer are normalized, so that the gradient similarity values ​​of the i-th vehicle node in the i-th layer are normalized. The sum of the aggregate weights for all other vehicle nodes on the network layer is 1. Among them, the vehicle node with the larger gradient similarity value has a higher aggregate weight, indicating that the node has a more similar data distribution on the corresponding network layer and should receive more knowledge transfer; conversely, the vehicle node with the smaller gradient similarity value has a lower aggregate weight, in order to avoid negative interference between dissimilar vehicle nodes.

[0061] Step S44: Based on the hierarchical aggregation weight matrix, the sub-parameters of other vehicle nodes in each network layer of each vehicle node are weighted and summed, and the weighted summation result is superimposed on the sub-parameters of the corresponding network layer of each vehicle node to obtain the updated sub-parameters of each vehicle node in each network layer.

[0062] Specifically, the update sub-parameters of each network layer for each vehicle node are calculated using the following formula:

[0063] in, This represents the update sub-parameter of the l-th network layer for the i-th vehicle node in the t-th communication round.

[0064] Step S45: Based on the updated sub-parameters of each vehicle node in each network layer, obtain the fused personalized layer parameters corresponding to each vehicle node.

[0065] Specifically, the updated sub-parameters of each vehicle node in each network layer obtained in step S44 are combined according to the original network layer structure to obtain the personalized layer parameters of each vehicle node after fusion, denoted as: This personalized layer parameter serves as the initial personalized layer parameter for the corresponding vehicle node in the next round of communication, enabling cross-vehicle knowledge transfer and continuous optimization.

[0066] In summary, the heterogeneous federated learning convergence optimization method for vehicle perception according to the embodiments of the present invention has the following beneficial effects: 1. This invention divides the vehicle node model into a base layer and a personalized layer, and introduces an L2 regularization term for the base layer parameters on top of the original loss function during model training. This allows each vehicle node to continuously learn general perception features in its base layer and adapt to the unique features of the region where each vehicle is located during local training. This effectively mitigates the impact of heterogeneity in training data on model training and significantly improves the model's recognition accuracy and generalization ability in various scenarios.

[0067] 2. In this embodiment of the invention, the personalized layer parameters accumulated by each vehicle node in historical training rounds are integrated into the personalized layer parameter updates of each round through a loss value weighted fusion method. This can accelerate model convergence and reduce communication overhead.

[0068] 3. When aggregating parameters of various vehicle nodes on a cloud server, this invention performs weighted fusion of basic layer parameters to obtain unified global basic layer parameters, calculates the gradient similarity between any two vehicle nodes according to network layers for personalized layer parameters, and performs weighted aggregation of the same network layer parameters of vehicle nodes based on gradient similarity. This reduces negative migration between dissimilar vehicle nodes, enabling each vehicle node to obtain personalized layer parameters that best suit its own data distribution. This significantly improves the local model adaptation accuracy of each vehicle node in data heterogeneity scenarios and the overall performance of the federated learning system.

[0069] 4. In this embodiment of the invention, model parameters are transmitted only between each vehicle node and the cloud server. The original vehicle sensor data is stored locally on the vehicle node and does not participate in cross-node transmission. Through distributed collaborative training between the cloud server and vehicle edge nodes, the transfer and fusion of cross-vehicle perception knowledge can be achieved while protecting data privacy, providing an efficient and secure training solution for distributed model training in vehicle perception scenarios.

[0070] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0071] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A heterogeneous federated learning convergence optimization method for vehicle perception, characterized in that, include: Each vehicle node acquires road perception data and constructs a local training dataset; Each vehicle node downloads the global base layer parameters of the previous communication round and the fused personalized layer parameters of the previous communication round corresponding to each vehicle node from the cloud server, and trains the local model using the local training dataset to obtain the base layer parameters and personalized layer parameters of the current communication round; wherein, each vehicle node has the same model structure. The personalized layer parameters of the current communication round and the personalized layer parameters of the historical communication rounds are weighted and fused to obtain the updated personalized layer parameters of each vehicle node. Each vehicle node uploads the basic layer parameters and the updated personalized layer parameters of the current communication round to the cloud server for fusion, thereby obtaining the global basic layer parameters of the current communication round and the fused personalized layer parameters corresponding to each vehicle node.

2. The method according to claim 1, characterized in that, Based on a preset number of network layers, the model of each vehicle node is divided into a basic layer and a personalized layer, with the corresponding model parameters being the basic layer parameters and the personalized layer parameters, respectively.

3. The method according to claim 2, characterized in that, When training the local model for each vehicle node, the sum of the original loss function of each vehicle node and the L2 regularization term of the base layer parameters is used as the training optimization objective.

4. The method according to claim 3, characterized in that, The optimization objective is expressed as: ； in, This represents the model parameters of the i-th vehicle node in the t-th communication round; Let i represent the loss function for the i-th vehicle node; Indicates the penalty coefficient; This represents the base layer parameters of the i-th vehicle node in the t-th communication round; This represents the global base layer parameter in the (t-1)th communication round.

5. The method according to claim 2, characterized in that, Based on the model loss value of a preset number of historical communication rounds prior to the current communication round for each vehicle node, the personalized layer parameters of the current communication round and the personalized layer parameters of each historical communication round are weighted and fused using the following formula to obtain the updated personalized layer parameters of each vehicle node: ； ； in, This represents the updated personalized layer parameters obtained by the i-th vehicle node in the t-th communication round; Indicates the size of the time window; This represents the fusion weight of the i-th vehicle node in the t-th communication round; This represents the loss value of the i-th vehicle node in the t-th communication round; This represents the maximum value of the i-th vehicle node within the time window of the t-th communication round; This represents the minimum value of the i-th vehicle node within the time window of the t-th communication round.

6. The method according to any one of claims 1-5, characterized in that, Each vehicle node uploads the basic layer parameters and the updated personalized layer parameters of the current communication round to the cloud server for parameter fusion between vehicle nodes, including: Based on the number of training datasets for each vehicle node, the basic layer parameters are weighted and fused to obtain global basic layer parameters; Based on the similarity value of the gradient of the personalized layer parameters of each vehicle node in the same network layer, the personalized layer parameters are fused in a layered weighted manner to obtain the fused personalized layer parameters corresponding to each vehicle node.

7. The method according to claim 6, characterized in that, The global base layer parameters are obtained by weighting and fusing the base layer parameters using the following formula: ； in, Represents the global base layer parameters for the t-th communication round; N represents the number of vehicle nodes; This represents the number of training datasets for the i-th vehicle node; This represents the base layer parameters obtained by the i-th vehicle node in the t-th communication round.

8. The method according to claim 6, characterized in that, The personalized layer parameters of each vehicle node are weighted and fused hierarchically based on the similarity value of their gradients within the same network layer to obtain the fused personalized layer parameters for each vehicle node, including: Based on the personalized layer parameters of each vehicle node, the sub-parameters of each network layer of the personalized layer of each vehicle node are obtained. For each network layer, the cosine similarity between the gradients of the sub-parameters of any two vehicle nodes in the corresponding network layer is calculated, and used as the gradient similarity value of the two vehicle nodes in the corresponding network layer. Based on the gradient similarity values ​​of each vehicle node and other vehicle nodes in each network layer, a hierarchical aggregation weight matrix is ​​constructed for each vehicle node and other vehicle nodes. Based on the hierarchical aggregation weight matrix, the sub-parameters of other vehicle nodes in each network layer of each vehicle node are weighted and summed, and the weighted summation result is superimposed on the sub-parameters of the corresponding network layer of each vehicle node to obtain the updated sub-parameters of each vehicle node in each network layer. Based on the updated sub-parameters of each vehicle node in each network layer, the fused personalized layer parameters corresponding to each vehicle node are obtained.

9. The method according to claim 8, characterized in that, The gradient similarity values ​​of any two vehicle nodes at each network layer can be obtained using the following formula: ； in, This represents the gradient similarity value between the l-th network layer of the i-th vehicle node and the j-th vehicle node at the t-th communication round; and These represent the sub-parameters of the l-th network layer of the i-th vehicle node and the j-th vehicle node in the t-th communication round, respectively. This represents the gradient function.

10. The method according to claim 9, characterized in that, The hierarchical aggregation weight matrix is ​​represented as follows: ； ； in, This represents the hierarchical aggregation weight matrix of the i-th vehicle node in the t-th communication round; The weights of the l-th network layer for the i-th and j-th vehicle nodes in the t-th communication round are represented by N; N represents the number of vehicle nodes; and L represents the number of personalization layers.

Images