Personalized federal learning method based on image perception of graph attention network and application

Abstract

The invention discloses a personalized federated learning method for image perception based on a graph attention network and application, belongs to the technical field of crossing of federated learning and a graph neural network, and aims to solve the problems of coarse granularity of model aggregation and neglect of a client graph structure in existing personalized federated learning. According to the method, a graph attention network is deployed at a server side, a client side uploads local model parameters, the server flattens the parameters and performs layer-by-layer normalization to generate node features, and a full-connection client side graph is constructed; node features are projected to a high-dimensional space through GAT, attention scores among clients are calculated and normalized, multi-attention head results are fused to generate a distribution matrix, and individual model parameters of the clients are obtained through aggregation; and the client calculates loss by using a local test set and feeds back the loss to the server, and the server updates GAT parameters after aggregating the loss, so that end-to-end optimization is realized. The method is suitable for edge device cooperative training and privacy protection type personalized model construction.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of federated learning, graph neural networks and personalized machine learning. Specifically, it relates to a personalized federated learning method based on graph attention network (GAT), which is suitable for distributed model training in non-independent and identically distributed (Non-IID) data scenarios and can be applied to scenarios such as collaborative training of edge devices (smartphones, IoT terminals) and construction of privacy-preserving personalized models. Background Technology

[0002] Federated learning (FL) achieves data privacy protection through distributed collaborative training, but in real-world scenarios, client data generally exhibits non-IID characteristics (such as class distribution shifts and unbalanced data volume), making it impossible for traditional global models to adapt to the needs of various clients. Personalized federated learning (PFL) has emerged to address this issue.

[0003] Existing PFL methods neglect fine-grained control of model aggregation: most methods (such as Per-FedAvg and pFedHN) focus only on "model personalization generation" (meta-learning, hypernetworks), but ignore the "client model aggregation rules" at the core of federated learning; existing aggregation methods (such as CFL clustering and FedAMP attention) only achieve coarse-grained collaboration (fixed clusters, simple similarity weights), and cannot dynamically capture the relationships between clients; on the other hand, existing federated learning methods do not utilize the graph structure between clients: the similarity of client model parameters can reflect the similarity of data distribution (e.g., clients that "all contain animal images" have more similar parameters), but existing methods (such as SFL* using GCN) cannot dynamically distinguish the importance of clients, and GCN treats all residents equally, resulting in poor adaptability.

[0004] The aforementioned shortcomings cause the performance of existing PFL methods to plummet in extreme Non-IID scenarios (such as ill-conditioned distributions and low βDirichlet distributions), necessitating a method that can dynamically optimize aggregation rules using graph structures. Summary of the Invention

[0005] Purpose of the invention: To address the problems of "coarse-grained aggregation and unutilized client associations" in existing PFL methods, this invention provides a personalized federated learning method (pFedGAT) based on graph attention networks. By modeling clients as graph nodes and dynamically learning attention weights through GAT, and combining client loss feedback to achieve end-to-end optimization, this invention improves the performance and adaptability of personalized models in Non-IID scenarios.

[0006] Technical Solution: A personalized federated learning method for image perception based on graph attention networks. This method includes deploying a graph attention network on the server side to capture the latent graph structure among clients, thereby achieving fine-grained aggregation rule optimization for the local models of each client, and combining client loss feedback to achieve end-to-end optimization. It also dynamically calculates client collaboration weights to generate a personalized model for each client. The method includes the following steps:

[0007] (1) Client graph construction: In the t-th round of communication, each client uploads the locally trained model parameters to the server. The server flattens and normalizes the parameters to obtain the node features of the client and constructs a fully connected graph topology based on this.

[0008] (2) Attention weight allocation: The server projects node features to a high-dimensional space through a graph attention network, calculates and normalizes the attention scores between clients, fuses the results of multiple attention heads to generate an allocation matrix, and aggregates the personalized model parameters of each client based on the allocation matrix.

[0009] (3) End-to-end optimization: The server distributes the personalized model parameters to each client. The client calculates the test loss using the local test set and uploads it to the server. The server aggregates the total loss and updates the graph attention network parameters through gradient descent to complete one round of communication.

[0010] (4) Repeat steps (1)-(3) until the preset communication rounds, and output the personalized model of each client.

[0011] Furthermore, the process of generating node features on the client side is as follows:

[0012] Let the model parameters of client i in round t be... The Flatten function will be used to... Flattened into a one-dimensional vector, and then standardized using a LayerNorm layer, the node features are obtained. , is represented as:

[0013] ,

[0014] in, This represents a d-dimensional matrix representing the node features, where d is the dimension of the flattened model parameters;

[0015] The process of constructing a fully connected graph topology is as follows:

[0016] Treat each client as a graph node, with the following node characteristics: The initial adjacency matrix A has all entries set to 1, forming a fully connected graph.

[0017] Furthermore, the specific process of step (2) includes:

[0018] High-dimensional feature projection is achieved through a shared weight matrix. Node features Projecting onto a high-dimensional space yields a high-dimensional feature matrix. ,in The output dimension after projection;

[0019] Attention score calculation involves calculating an initial attention score for each client pair. The mathematical expression for this score is:

[0020] ;

[0021] in, This indicates that 'a' is a dimension of... The matrix, It is an activation function. Let i and j represent the indices of two different clients, and T represent the transpose of the matrix. and Let these represent the high-dimensional feature matrices of the i-th and j-th clients in the t-th round, respectively. This indicates vector concatenation;

[0022] Attention score normalization is achieved by normalizing the original attention score using the Softmax function to obtain the attention coefficient. N represents the total number of clients, and k represents the k-th attention head;

[0023] Multi-attention head fusion and parameter aggregation employs K independent attention heads for parallel computation, with each attention head outputting an attention matrix. , Allocation matrix The mathematical expression for the average of the results from each attention point is as follows:

[0024] ,

[0025] The personalized model parameters for client i are obtained based on the aggregation of the allocation matrix:

[0026] ,

[0027] In the formula, This represents the weight distribution between client i and client j in round t. This represents the model parameters of client j in round t. and Let represent the attention score between client i and client j at the k-th attention head in round t, and let represent the attention matrix at the k-th attention head in round t, respectively.

[0028] Furthermore, the specific process of end-to-end optimization described in step (3) includes:

[0029] Client-side loss calculation: The client receives personalized model parameters. Using the local test set Evaluate personalized model parameters Calculate the test loss and will Uploaded to the server;

[0030] Total Loss Aggregation: The server aggregates the test losses from all clients to obtain the total loss. ;

[0031] Graph Attention Network Parameter Update: The server updates the learnable parameters W and a of the graph attention network using gradient descent.

[0032] Furthermore, the method includes collaborative training scenarios applied to medical image data and image perception training during autonomous driving processes.

[0033] Based on the application of the above method, this invention provides a federated learning system for collaborative training of medical images. This system performs personalized aggregation and training of local models using the aforementioned graph attention network-based personalized federated learning method for medical images. The system includes:

[0034] Client module: Each medical institution acts as a client, deploying local models for lesion detection, organ segmentation, or disease-aided diagnosis. These models are trained on local medical image data and are periodically uploaded with model parameters and validation loss.

[0035] The server module treats each client as a node, constructs a graph attention network, learns the similarity between institutions based on the uploaded model parameters, assigns higher aggregation weights to institutions with similar data distributions and lower weights to institutions with large differences, and performs end-to-end optimization of the aggregator based on the validation loss to achieve adaptive personalized aggregation.

[0036] Furthermore, each client performs model training based on local medical image data, uses stochastic gradient descent optimization algorithm to update parameters, and uploads the model parameters and validation loss to the server after completing local training. The server then aggregates and distributes the updated model to achieve multi-round personalized collaborative training.

[0037] Furthermore, the server and each client are connected via a secure communication protocol. The model parameters uploaded by the client are compressed and encrypted before transmission to ensure the privacy and security of medical image data and model parameters.

[0038] After collaborative training is completed, each client uses the generated personalized local model to perform lesion detection, organ segmentation, and disease-assisted diagnosis on medical images, thereby improving diagnostic accuracy and the generalization performance of models across different institutions.

[0039] Based on the application of the above method, this invention provides a federated learning system for visual perception in autonomous driving. This system performs personalized aggregation and training of local models using the aforementioned graph attention network-based personalized federated learning method for medical images. The system includes:

[0040] Vehicles or roadside units in various locations are treated as clients, each equipped with a local model for lane recognition and object detection. The local model is then trained using the method described above. Each vehicle or city is treated as a node, and the server constructs a graph attention network based on these nodes. The graph attention network learns the similarity between different regions based on the model parameter features uploaded by each node. Regions with similar data distributions are assigned relatively higher aggregation weights, while regions with significant differences are assigned lower weights to prevent mis-migrating. The server performs end-to-end optimization of the aggregator using the uploaded local validation loss signal to achieve adaptive personalized aggregation.

[0041] Existing methods such as FedAvg, FedProx, and FedAMP rely on fixed aggregation rules or heuristic similarity, making it difficult to accurately reflect the true relationships between clients. When data distributions vary significantly, these methods are prone to model overfitting to local data or degenerating into averaging. The pFedGAT method provided by this invention addresses not "how to personalize the model," but rather the more fundamental issue of "how to personalize the aggregation strategy."

[0042] Beneficial Effects: This invention introduces GAT (Graph Association Graph) into PFL (Program Failure Graph) for the first time, dynamically capturing relationships between clients through graph structure to achieve fine-grained aggregation rule optimization. It proposes an end-to-end optimization paradigm, utilizing client scalar loss feedback to update GAT, with negligible additional communication overhead (only 4 bytes / client). Furthermore, this invention does not predefine association rules; the initial fully connected graph does not require manual clustering or association setting. GAT automatically uncovers potential client collaboration patterns (e.g., higher weight for clients with similar data distributions). Experiments show that the pFedGAT provided by this invention achieves optimal or near-optimal results on datasets such as Fashion-MNIST and CIFAR-10 / 100 under various Non-IID conditions and different numbers of clients. Especially in scenarios where new clients are trained for only three rounds, it can still generalize quickly and maintain high accuracy, proving that the learned "collaborative graph structure" indeed possesses transferability and robustness. Traditional methods cannot adapt to new nodes in such a short time, demonstrating the innovative breakthrough of this invention in dynamic collaboration modeling and generalization. Attached Figure Description

[0043] Figure 1 This is a flowchart of the method described in this invention;

[0044] Figure 2 This is a visualization of the CIFAR-10 data distribution; among which... Figure 2 (a) is a visualization of the data distribution under the Pathological distribution. Figure 2 (b) is a visualization of the data distribution under the IID distribution. Figure 2 (c) is a visualization of the data distribution under the Dir(0.1) distribution. Figure 2 (d) is a visualization of the data distribution under the Dir(0.5) distribution;

[0045] Figure 3 This is a visualization of the weight allocation matrix; where Figure 3 (a) and Figure 3 (b) shows the weight allocation matrices obtained by the FedAMP algorithm in the IID distribution scenario and the Pathlogical distribution scenario, respectively. Figure 3 (c) and (d) are the weight assignment matrices obtained by the CFL algorithm in the IID distribution scenario and the Pathlogical distribution scenario, respectively. Figure 3 (e) and Figure 3 (f) are the weight allocation matrices obtained by the pFedGAT algorithm in the IID distribution scenario and the Pathlogical distribution scenario, respectively.

[0046] Figure 4 This is a graph showing the generalized performance of the new client. Figure 4 (a) is a comparison chart of the generalization performance of five algorithms—Local, pFedMe, FedAMP, Per-FedAvg, and pFedGAT—on the CIFAR-10 dataset for new clients. Figure 4 (b) is a comparison chart of the generalization performance of five algorithms—Local, pFedMe, FedAMP, Per-FedAvg, and pFedGAT—on the CIFAR-100 dataset for new clients.

[0047] Figure 5 This is a graph showing the results for different numbers of clients. Detailed Implementation

[0048] Existing technologies, including FedAvg, FedProx, and FedAMP, struggle to accurately reflect the true relationships between clients. The method described in this invention requires establishing a link for cross-device loss feedback to backpropagation on the server side. Furthermore, it necessitates that the aggregator parameters converge even with heterogeneous data and a fluctuating number of clients, while avoiding increased communication overhead. This invention focuses on deploying a Graph Attention Network on the server side, treating each client as a graph node, and learning inter-node collaboration weights through a multi-head attention mechanism. Then, it utilizes the local loss uploaded by the clients to back-optimize the server-side aggregator, achieving end-to-end differentiable dynamic aggregation.

[0049] This invention captures the client graph structure using server-side GAT, dynamically calculates collaboration weights, and combines loss feedback optimization to generate a personalized model for each client. The specific steps are as follows:

[0050] (1) Local training on the client: The client uses a lightweight CNN model (3 convolutions + 3 fully connected layers) to avoid training delays caused by insufficient computing power of edge devices. During training, overfitting is suppressed by using the SGD optimizer (momentum and weight decay), and the batch size is set to 64 to balance memory usage and training efficiency. On a Raspberry Pi 4B, a single training round takes about 8 seconds, which is 33% shorter than the 12 seconds of pFedMe. After training is completed, only the compressed model parameters are uploaded to avoid the transmission of the original data, and parameter privacy is protected by AES-256 encryption.

[0051] (2) Server graph construction: After receiving the client parameters, the server first flattens the two-dimensional convolution parameters and the one-dimensional fully connected parameters into a vector using the Flatten function, and then eliminates numerical differences (such as parameter magnitude deviations caused by different client initializations) using LayerNorm normalization. Constructing an initial fully connected graph instead of predefined clustering avoids misjudgments of associations caused by manual intervention. In the CIFAR-10 ill-conditioned distribution scenario, the initial setting of the fully connected graph enables GAT to quickly identify client clusters dominated by "similar samples" without the need for additional clustering algorithms.

[0052] (3) GAT weight calculation: Dynamic weighting is achieved through three steps: "high-dimensional projection - attention score calculation - multi-head fusion". High-dimensional projection increases the node features from d dimensions to 256 dimensions, enhancing the ability to capture associations; the LeakyReLU activation function solves the gradient vanishing problem, and the negative slope of 0.2 ensures that the attention scores of clients with low similarity can still be effectively distinguished; the fusion of 8 attention heads reduces the randomness of a single attention head, and the self-weight constraint of the allocation matrix (≥0.3) ensures that the client does not rely on external knowledge and avoids model "homogenization".

[0053] (4) End-to-end optimization: The low-overhead GAT parameter update client only uploads the scalar loss (4 bytes), which reduces the communication overhead by 99.99% compared to transmitting the complete model parameters (35MB for CIFAR-10). After the server aggregates the total loss, it updates the weight matrix W and attention parameter a of GAT through backpropagation, so that the aggregation strategy dynamically adapts to changes in data distribution. In the scenario of reduced heterogeneity of Dirichlet (β=0.1→0.5), GAT can automatically adjust the attention weights, and the collaboration weight of similar clients decreases from 0.6 to 0.4, adapting to the convergence of distribution changes.

[0054] To illustrate the technical solution of the present invention in detail, it is described in conjunction with Figures 1-5 and experimental data to ensure the reproducibility and effectiveness of the technical solution.

[0055] In this embodiment, the client is set as a smartphone. The client connects to the server via WiFi 6 (100Mbps bandwidth, latency ≤10ms) through the edge node, and connects to the edge node via WiFi 4 (15Mbps bandwidth, latency ≤50ms). For the method provided by this invention, the experimental dataset and parameters are set as follows:

[0056] The dataset includes:

[0057] 1) Fashion MNIST: 10 categories of clothing images, 60k training samples (28×28 grayscale images), 10k test samples;

[0058] 2) CIFAR-10: 10 classes of natural images, 50k training samples (32×32 color images), 10k test samples;

[0059] 3) CIFAR-100: 100 classes of natural images, 50k training samples (32×32 color images), 10k test samples;

[0060] Data partitioning:

[0061] IID scenario: Each client is randomly assigned 1 / N of the training samples;

[0062] Illegible distribution: The Fashion MNIST / CIFAR-10 client contains 2 classes of samples, while the CIFAR-100 client contains 20 classes of samples;

[0063] Dirichlet distribution: β=0.1 (high heterogeneity), β=0.3 (medium heterogeneity), β=0.5 (low heterogeneity);

[0064] Hyperparameters: Communication rounds T=50, local training rounds E=5, GAT learning rate Client learning rate Attention head K=8, LayerNorm ε=1e-5, and synthetic noise variance. .

[0065] Baseline method:

[0066] Comparison of 12 mainstream PFL methods: Traditional FL methods: FedAvg, FedProx; Graph-driven methods: SFL*; Personalized FL methods: CFL, Per-FedAvg, pFedMe, FedAMP, pFedHN, FedROD, KNN-PER, FedAH; Non-collaborative methods: Local (local training only).

[0067] The implementation steps based on the above technical solution are as follows:

[0068] Taking "Medical Image Collaborative Training (simulating the CIFAR-100 dataset, 20 clients, Dirichlet (β=0.1) high heterogeneity)" as an example, the implementation steps are explained in detail:

[0069] Step 1: System Initialization (T=0 rounds) The server starts the gRPC service, initializes the GAT parameters W (256×3, 524, 500) and a (512×1), and sets T=50; the client downloads the initial CNN model from the server. Load local medical image data (e.g., client 1 contains two types of samples: "lung cancer" and "pneumonia," totaling 2500 training samples and 500 test samples); configure the SGD optimizer on the client ( (Momentum 0.9, weight decay 1e-4), batch size 64.

[0070] Step 2: Local Training on the Client (t=0 to 49 rounds) Client 1 processes the local training set in batches, updating parameters in each round:

[0071] ,

[0072] A total of 5 rounds of training will be conducted; after the training is completed, The vector is flattened to 3,524,500 dimensions, compressed using Zlib, encrypted with AES-256, and uploaded to the server via gRPC. All 20 clients perform the above operations in parallel, and the server receives and decompresses all the data. .

[0073] Step 3: Server graph construction and GAT weight calculation parameter preprocessing: Server pairs Perform LayerNorm normalization to generate node features Fully connected graph construction: Initializing the adjacency matrix The matrix consists entirely of 1s, with the client acting as a node. For node features; GAT calculation: Projection: (3,524,500 dimensions → 256 dimensions); Fraction calculation:

[0074] ,

[0075] For example, client 1 and client 5 (both containing "lung cancer" samples) With client 10 (including "fracture" samples) Normalization:

[0076] ;

[0077] Multi-head fusion: After averaging the attention of 8 heads, , ;

[0078] Personalized parameter aggregation: , Distribute to client 1.

[0079] Step 4: End-to-end optimization of client 1 loading Calculate cross-entropy loss on the local test set. After adding ±0.001 noise, the sample was uploaded to the server; the server aggregated the total loss. Update GAT parameters: ;

[0080] Step 5: Iteration and Adding New Clients. Repeat steps 2-4 until t=40 rounds. At this point, the server adds 2 new clients (including "gastritis" and "enteritis" samples). The new clients download the initial model, train for 5 rounds, and then upload their parameters. The server calculates the collaboration weights between the new clients and existing clients based on GAT (e.g., the weight with client 15 containing "digestive disease" samples reaches 0.55), and aggregates personalized parameters. Continue iterating until t=50 rounds, and client 1 saves the final model. Complete the training.

[0081] This invention achieves optimal or near-optimal results on datasets such as Fashion-MNIST and CIFAR-10 / 100 under various Non-IID conditions and different numbers of clients.

[0082] Especially in scenarios where only three rounds of training are conducted on a new client, it can still generalize quickly and maintain high accuracy, which proves that the learned "collaboration graph structure" does indeed have transferability and robustness.

[0083] Traditional methods cannot adapt to new nodes in such a short number of rounds, demonstrating the innovative breakthrough of this invention in dynamic collaborative modeling and generalization.

[0084] Based on the above implementation results, we obtain the following results:

[0085] (1) The core advantages of pFedGAT are reflected in: stable performance in highly heterogeneous scenarios: in the CIFAR-100 Dirichlet (β=0.1) scenario, the accuracy rate is 51.32%, which is 0.51 percentage points higher than FedAH, because GAT can accurately filter harmful knowledge from heterogeneous clients; strong adaptability to multiple datasets: the average accuracy rates of Fashion MNIST, CIFAR-10, and CIFAR-100 are 93.66%, 77.57%, and 39.93%, respectively, all ranking first, proving the versatility of the technical solution.

[0086] (2) Generalization performance of the new client Figure 4 shows that the accuracy of the new client in CIFAR-100 is 51.52%, which is significantly better than Local (30.30%) and pFedMe (23.00%). The reason is that GAT can quickly identify similar clients and assign high weights based on the parameter features of the new client, thus avoiding overfitting of the new client due to the small number of samples.

[0087] (3) Ablation experiments verified and compared the performance of "with GAT" and "without GAT (directly optimize the weight matrix)". The results showed that the method with GAT had higher accuracy in all scenarios, reaching 51.52% in the CIFAR-100 ill-conditioned distribution scenario, while the method without GAT only reached 50.75%. The reason is that the nonlinear transformation of GAT and multi-head attention can capture the complex relationships between clients, while direct optimization can only achieve linear weight adjustment and cannot adapt to the complex distribution of Non-IID data.

[0088] (4) pFedGAT requires 0.82GB of data to achieve CIFAR-10 35% accuracy, which is higher than FedGH (4.96MB), but the accuracy is improved by 5.76 percentage points; compared with FedKD (8.79GB), the communication overhead is reduced by 90.7%, proving the balance advantage of "performance-overhead".

[0089] The above embodiments also illustrate its application in medical imaging scenarios, namely a federated learning system for collaborative training of medical images. Furthermore, this invention can also be applied to autonomous driving scenarios, serving as a federated learning system for visual perception in autonomous driving. Through dynamic aggregation of graph attention, it achieves personalized perception models for different regions or vehicles, maintaining high accuracy and strong robustness in heterogeneous environments. Its application is described below:

[0090] In autonomous driving scenarios, road environments, climate conditions, traffic density, and camera installation angles vary significantly across different regions, resulting in image data collected by various vehicles or urban traffic systems exhibiting strong non-independent identically distributed (Non-IID) characteristics. Traditional federated learning (such as FedAvg) is prone to global model degradation or performance degradation in specific regions when aggregating perception models from different regions; for example, a detection model effective in snowy conditions in the north may perform poorly in recognizing rainy night scenes in the south. Therefore, the pFedGAT described in this invention can be applied to such heterogeneously distributed autonomous driving vision systems.

[0091] For system composition, target detection and lane recognition models can be trained locally on vehicles or roadside units (RSUs) in various locations. On the server side, each city / vehicle is treated as a node through a graph attention network (GAT).

[0092] Working principle: GAT automatically learns the similarity between different regions based on the model parameter features uploaded by each node; then it assigns higher aggregation weights to regions with similar data distribution (such as similar weather and road conditions); and assigns lower weights to regions with large differences to prevent mis-migrating; through the uploaded local validation loss signal, the server performs end-to-end optimization of the aggregator to achieve adaptive personalized aggregation.

[0093] In this system, each region obtains a perception model that is more in line with local characteristics, and the recognition accuracy is significantly improved in complex environments (such as rainy nights, tunnels, and snowy areas). Furthermore, newly connected cities or fleets can quickly obtain stable performance without the need for training from scratch. Secondly, the communication burden is extremely low, requiring only the uploading of model parameters and a single loss scalar.

Claims

1. A graph attention network-based image perception personalized federated learning method, characterized in that, The method comprises deploying a graph attention network at a server end to capture potential graph structures among clients, thereby implementing fine-grained aggregation rule optimization for local models of the clients, and implementing end-to-end optimization in combination with client loss feedback, dynamically calculating client collaboration weights, and generating personalized models for each client; the method comprises the following steps: (1) Client graph construction. In the tth communication round, each client uploads the locally trained model parameters to the server, the server performs flattening and layer normalization processing on the parameters to obtain node features of the clients, and constructs a fully connected graph topology based on the node features; (2) Attention weight distribution. The server projects the node features to a high-dimensional space through a graph attention network, calculates attention scores between the clients and performs normalization processing, fuses multi-attention head results to generate a distribution matrix, and aggregates the personalized model parameters of each client based on the distribution matrix; (3) End-to-end optimization. The server distributes the personalized model parameters to each client, the client calculates the test loss using the local test set and uploads it to the server, the server aggregates the total loss and updates the graph attention network parameters through gradient descent to complete one round of communication; (4) Repeat steps (1)-(3) until a preset number of communication rounds, and output the personalized model of each client.

2. The graph attention network-based image perception personalized federated learning method according to claim 1, characterized in that, The node feature generation process of the client is as follows: Let the model parameters of client i in the t-th round be , the flattened is flattened into a one-dimensional vector by the Flatten function, and then normalized by the LayerNorm layer to obtain the node feature , which is represented as: ， wherein, represents a matrix of the node features of d dimensions, d being the dimension of the flattened model parameters; The construction process of the fully connected graph topology is as follows: Each client is considered as a graph node, and the node features are , and all entries of the initial adjacency matrix A are set to 1, forming a fully connected graph.

3. The graph attention network-based image-aware personalized federated learning method according to claim 1, characterized in that, The specific process of step (2) comprises: High-dimensional feature projection is through a shared weight matrix The node features are projected to a high-dimensional space to obtain a high-dimensional feature matrix wherein is the output dimension after projection; The attention score calculation is to calculate the original attention score for each pair of clients, and the mathematical expression is: ； wherein, represents a is a matrix of dimension , is an activation function, , i, j represent the indices of two different clients, respectively, T represents the transpose of the matrix, and represent the high-dimensional feature matrix of the i-th client and the j-th client in the t-th round, respectively, represents vector splicing; The attention score normalization is to normalize the original attention score by a Softmax function to obtain an attention coefficient , N is the total number of clients, and k represents the kth attention head. Multi-attention head fusion and parameter aggregation is to use K independent attention heads to calculate in parallel, and each attention head outputs an attention matrix , , the distribution matrix is the average of the results of each attention head, and the mathematical expression is as follows: ， Based on the distribution matrix, the personalized model parameters of client i are aggregated: ， wherein, denotes the allocation weight between client i and client j in the t-th round, denotes the model parameters of client j in the t-th round, denotes the attention score between client i and client j on the k-th attention head in the t-th round, and denotes the k-th attention matrix in the t-th round.

4. The graph attention network-based image perception personalized federated learning method according to claim 1, characterized in that, The specific process of end-to-end optimization in step (3) comprises: Client loss computation: the client receives personalized model parameters , computes a test loss on the local test set , evaluates the personalized model parameters , computes a test loss , and uploads it to the server; Total loss aggregation: The server aggregates the test losses of all clients to get the total loss ; Graph attention network parameter update: the server updates the learnable parameters W and a of the graph attention network through gradient descent.

5. The graph attention network-based image perception personalized federated learning method according to claim 1, characterized in that, The method comprises a collaborative training scenario applied to medical image data and image perception training in an automatic driving process.

6. A federated learning system for medical image collaborative training, characterized in that, The system performs personalized aggregation and training of local models through the graph attention network-based medical image personalized federated learning method of any one of claims 1 to 4, and the system comprises: Client module: each medical institution acts as a client and is deployed with a local model for lesion detection, organ segmentation or disease auxiliary diagnosis, performs training on local medical image data, and periodically uploads model parameters and validation loss; Server module: each client is regarded as a node, a graph attention network is constructed, the similarity between institutions is learned according to the uploaded model parameters, higher aggregation weights are allocated to institutions with similar data distribution, lower weights are allocated to institutions with greater differences, and end-to-end optimization is performed on the aggregator according to the validation loss to achieve adaptive personalized aggregation.

7. The federated learning system for medical image collaborative training according to claim 6, wherein, Each client performs model training based on local medical image data, updates parameters using a stochastic gradient descent optimization algorithm, and uploads model parameters and validation loss to the server after completing local training, the server aggregates and issues updated models to realize multiple rounds of personalized collaborative training.

8. The federated learning system for medical image collaborative training according to claim 6, wherein, The server is connected with each client through a secure communication protocol, and the model parameters uploaded by the client are compressed and encrypted before transmission to ensure the privacy and security of medical image data and model parameters.

9. The federated learning system for medical image collaborative training according to claim 6, wherein, After the collaborative training is completed, each client uses the generated personalized local model to perform lesion detection, organ segmentation and disease auxiliary diagnosis on medical images to improve the diagnosis accuracy and the generalization performance of models between different institutions.

10. An automatic driving visual perception federated learning system, characterized in that, The system performs personalized aggregation and training of local models through the image perception personalized federated learning method based on the graph attention network according to any one of claims 1-4, and the system comprises: Each vehicle or roadside unit is regarded as a client, and a local model for lane recognition and target detection is deployed, and the personalized training of the local model is performed based on the method; each vehicle or city is regarded as a node, and then the server constructs a graph attention network based on the node, the graph attention network learns the similarity between different regions according to the model parameter features uploaded by each node, and regions with similar data distribution are allocated relatively higher aggregation weights, and regions with larger differences are allocated lower weights to prevent mis-migration, through the uploaded local verification loss signal, the server side optimizes the aggregator in an end-to-end manner to realize adaptive personalized aggregation.

Images