Enhanced FedACA method for prototype aggregation and classifier fusion in heterogeneous federated learning

Abstract

The invention discloses an enhanced EedACA method for prototype aggregation and classifier fusion in heterogeneous federated learning, and relates to an enhanced FedACA method. The invention aims to solve the problems of insufficient prototype expression, insufficient classifier aggregation strategy and weak individuation ability in the current heterogeneous federal learning. On the premise of ensuring the privacy of the client, the adaptive capacity of the model to the non-independent identically distributed data can be improved, the robustness and generalization ability of the global model under the conditions of data isomerism and model isomerism can be enhanced, and the method is particularly suitable for scenes such as intelligent terminal collaborative modeling, personalized recommendation and edge calculation. The invention belongs to the technical field of artificial intelligence.

Description

Technical Field

[0001] The present invention relates to an enhanced FedACA method, belonging to the technical field of artificial intelligence. Background Art

[0002] With the growing demand for data privacy, Federated Learning (FL), a decentralized distributed learning framework, has garnered widespread attention. First proposed by Google, this technology allows multiple clients to retain data locally and collaboratively train a global model by transmitting only model parameters or feature representations. This approach improves modeling capabilities while protecting user privacy. It is widely used in healthcare, finance, and other fields with stringent data privacy requirements.

[0003] However, traditional federated learning methods (such as FedAvg) primarily assume homogeneous client model structures and uniform data distribution (i.e., IID). These assumptions make them difficult to address the heterogeneous models and non-IID data distributions common in real-world environments. This severely limits their performance and generalization capabilities in real-world scenarios.

[0004] To further enhance the applicability of federated learning, heterogeneous federated learning (HtFL) has emerged. HtFL allows clients to use different model structures and develops various strategies to improve the system's learning efficiency and robustness, addressing differences in device performance and inconsistent data statistical characteristics. For example, FedPer employs a shared underlying structure combined with a personalized top-level classifier to achieve compatibility between local and global information; FedGH further proposes the use of a shared global head network to enhance server-side modeling capabilities; and FedTGP introduces globally trainable prototypes and contrastive learning mechanisms in prototype learning, enhancing both intra-class compactness and inter-class separation.

[0005] Among the many heterogeneous federated learning methods, prototype-based learning has garnered widespread attention. FedProto significantly reduces communication overhead by aggregating local class prototypes uploaded by clients to a server. However, its simple weighted averaging strategy is susceptible to factors such as data bias and inconsistent prototype scales, leading to a phenomenon known as "prototype margin shrinkage" and making it difficult to accurately capture local class distribution characteristics.

[0006] Furthermore, existing methods generally overlook the issues of classifier fusion and personalized modeling. In heterogeneous environments, each client often trains significantly different classifiers, making it difficult to cover diverse task boundaries through feature prototype sharing alone. FedReMa attempts to upload personalized classifiers to the server for matching and fusion, providing new insights into classifier personalization. However, its fusion strategies and cluster matching remain rudimentary.

[0007] In summary, heterogeneous federated learning still has the following problems that need to be solved: Insufficient prototype expression: Average aggregation ignores the complex structure within the class, resulting in insufficient prototype representation.

[0008] Blurred boundary problem: The classifier is not involved in the optimization, and the global model has difficulty in clearly dividing the boundaries between classes.

[0009] Weak personal adaptability: The model cannot adapt to diverse tasks in heterogeneous environments and lacks a fine-grained control mechanism. Summary of the Invention In order to solve the problems of insufficient prototype expression, inadequate classifier aggregation strategy and weak personalization capability in current heterogeneous federated learning, the present invention proposes an enhanced EedACA method for prototype aggregation and classifier fusion in heterogeneous federated learning.

[0010] The technical solution adopted by the present invention to solve the above problems is: the steps of the present invention include: Step 1: Adaptive K-Means clustering; In each client, the optimal number of clusters for each category is adaptively determined, the KMeans algorithm is optimized using the silhouette coefficient, multiple subclass prototypes are generated, and weighted fusion is performed to construct a local class prototype; Step 2: Cluster classifier matching; After the client uploads the local classifier, the server clusters the clients based on the similarity between the semantic outputs of the classifiers, groups similar clients, and generates a personalized global classifier. Step 3: Personalized classifier fusion; The client uses KL divergence and cross entropy to jointly optimize local and global classifiers and uses trainable fusion coefficients Control the combination ratio of the final classifier to achieve a dynamic trade-off between global generalization and local adaptation, and enhance the accuracy, stability and personalized performance of the model.

[0011] Furthermore, step 1 specifically includes: Step 101: feature extraction; The client first sends the local sample Input local feature extractor , and obtain the corresponding high-dimensional feature vector : , For each category , collect the corresponding feature set ; Step 102: Optimize the number of clusters using the silhouette coefficient; Perform multiple clustering experiments on each feature set and automatically select the optimal number of clusters using the silhouette coefficient , the silhouette coefficient is defined as: , in, Indicates samples in the same cluster The average distance to other samples, Representation sample The average distance of all samples to the nearest cluster; The final choice is The largest The optimal number of clusters : ; Step 103: Use the above optimal , perform KMeans on the feature set, with the goal of minimizing the sum of squared distances within the class: , in is the centroid of the kth subcluster; Step 104: Calculate weighted sub-prototypes; For each subcluster , calculate its center: , Calculate the sub-cluster weight according to its sample size ratio: , Aggregate all sub-archetypes to form a category The weighted prototype vector of : .

[0012] Furthermore, step 2 specifically includes: Step 201: Upload the classifier; After each round of training, the client Upload a locally trained copy of the global classifier , uploaded to the server together with the local class prototype; Step 202: Calculate the semantic representation of the separator; The server uses a set of random feature vectors , input the global classifier of each client and calculate its output probability distribution: , in, The predicted probability of client k for random input; M: temperature parameter, controlling softmax smoothness; Step 203: Calculate the similarity between classifiers; Calculate the cosine similarity between any two clients Semantic similarity between classifiers: , All clients form a similarity matrix ,in is the total number of clients; Step 204: Maximum difference grouping strategy; To cluster clients semantically, we use the "maximum difference partitioning" strategy: for each client k, we sort the similarities from high to low and find the maximum adjacent difference: , Using the maximum jump point as the threshold, select a group of clients with high similarity: ; Step 205: Determine the consensus stage; Define whether the system is currently in the "consensus stage", that is, whether it is suitable for dynamic grouping, and calculate the average value of the maximum jump difference of all clients: , if: , If CCP is set to True, it means that the information is active and the MDS grouping strategy should be implemented; otherwise, it will enter the stable period and use historical statistical dependence; Step 206: Generate a personalized global classifier; If CCP=True, the classifier is weighted averaged using the client group selected by MDS: , where |D i | is the data volume of client i; If CCP=False, weighted fusion is performed based on the historical dependency matrix G: , The matrix G records the historical aggregation frequency between clients and is used for personalized construction in the stable period.

[0013] Furthermore, step 3 specifically includes: Step 301: prepare the classifier; The client obtains two types of classifiers: Personalized global classifier from the server , generated by CHM module; Locally trained private classifier , representing the local preference feature.

[0014] Both act on local eigenvectors , output predicted logits; Step 302: Prediction output and loss construction; For the input samples, the two classifiers calculate the predicted values ​​respectively: Global output and local output; Define two loss functions: Global cross entropy loss: , Local classifier joint loss (KL + CE): , in, It is a balancing term that controls the degree of dependence of the local model on global guidance and adaptive learning; KL: KL divergence, which promotes the local model to move closer to the global model; CE: standard cross entropy loss, which maintains the ability to fit the true label; Step 303: separator fusion strategy; After completing the local training, the two classifiers are weighted and fused according to the learnable parameter μ to construct the final local model for prediction: , μ is a locally learnable fusion ratio parameter used to balance local adaptability and global generalization capability. The fused classifier serves as the output head for the next round of training and inference. If μ=1, the model is completely local; if μ=0, it is completely global.

[0015] The beneficial effects of the present invention are: 1. This invention can improve the model's adaptability to non-IID data while ensuring client privacy, and enhance the robustness and generalization ability of the global model under conditions of data and model heterogeneity. It is particularly suitable for scenarios such as intelligent terminal collaborative modeling, personalized recommendation, and edge computing. 2. FedACA's core innovation lies in its three-in-one optimization mechanism: adaptive representation (A-KMeans), personalized matching (CHM), and fusion learning (PHF) jointly improve the performance of heterogeneous federated learning. It has good versatility, scalability, and engineering deployment value, and is a key complement and improvement to existing prototype-based and personalized FL methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of the FedACA implementation steps; Figure 2 This is a model diagram of the FedACA; Figure 3 This is the t-SNE plot of the prototype separation degree of the three methods FedProto, FedTGP, and FedACA; Figure 4 This is a schematic diagram comparing the results of various methods with different local epoch numbers on the Cifar10 dataset. DETAILED DESCRIPTION

[0017] Specific implementation method 1: Figures 1 to 4 As shown in Figure 1, an enhanced EedACA method for prototype aggregation and classifier fusion in heterogeneous federated learning is presented. The specific steps include: Step 1: Adaptive K-Means clustering; In each client, the optimal number of clusters for each category is adaptively determined, the KMeans algorithm is optimized using the silhouette coefficient, multiple subclass prototypes are generated, and weighted fusion is performed to construct a local class prototype; Specifically include: Step 101: feature extraction; The client first sends the local sample Input local feature extractor , and obtain the corresponding high-dimensional feature vector : , For each category , collect the corresponding feature set ; Step 102: Optimize the number of clusters using the silhouette coefficient; Perform multiple clustering experiments on each feature set and automatically select the optimal number of clusters using the silhouette coefficient , the silhouette coefficient is defined as: , in, Indicates samples in the same cluster The average distance to other samples, Representation sample The average distance of all samples to the nearest cluster; The final choice is The largest The optimal number of clusters : ; Step 103: Use the above optimal , perform KMeans on the feature set, with the goal of minimizing the sum of squared distances within the class: , in is the centroid of the kth subcluster; Step 104: Calculate weighted sub-prototypes; For each subcluster , calculate its center: , Calculate the sub-cluster weight according to its sample size ratio: , Aggregate all sub-archetypes to form a category The weighted prototype vector of : ; Step 2: Cluster classifier matching; After the client uploads the local classifier, the server clusters the clients based on the similarity between the semantic outputs of the classifiers, groups similar clients, and generates a personalized global classifier. Specifically include: Step 201: Upload the classifier; After each round of training, the client Upload a locally trained copy of the global classifier , uploaded to the server together with the local class prototype; Step 202: Calculate the semantic representation of the separator; The server uses a set of random feature vectors , input the global classifier of each client and calculate its output probability distribution: , in, The predicted probability of client k for random input; M: temperature parameter, controlling softmax smoothness; Step 203: Calculate the similarity between classifiers; Calculate the cosine similarity between any two clients Semantic similarity between classifiers: , All clients form a similarity matrix ,in is the total number of clients; Step 204: Maximum difference grouping strategy; To cluster clients semantically, we use the "maximum difference partitioning" strategy: for each client k, we sort the similarities from high to low and find the maximum adjacent difference: , Using the maximum jump point as the threshold, select a group of clients with high similarity: ; Step 205: Determine the consensus stage; Define whether the system is currently in the "consensus stage", that is, whether it is suitable for dynamic grouping, and calculate the average value of the maximum jump difference of all clients: , if: , If CCP is set to True, it means that the information is active and the MDS grouping strategy should be implemented; otherwise, it will enter the stable period and use historical statistical dependence; Step 206: Generate a personalized global classifier; If CCP=True, the classifier is weighted averaged using the client group selected by MDS: , where |D i | is the data volume of client i; If CCP=False, weighted fusion is performed based on the historical dependency matrix G: , The matrix G records the historical aggregation frequency between clients and is used for personalized construction in the stable period; Step 3: Personalized classifier fusion; The client uses KL divergence and cross entropy to jointly optimize local and global classifiers and uses trainable fusion coefficients Control the combination ratio of the final classifier to achieve a dynamic trade-off between global generalization and local adaptation, thereby enhancing the accuracy, stability, and personalized performance of the model; Specifically include: Step 301: prepare the classifier; The client obtains two types of classifiers: Personalized global classifier from the server , generated by CHM module; Locally trained private classifier , representing the local preference feature.

[0018] Both act on local eigenvectors , output predicted logits; Step 302: Prediction output and loss construction; For the input samples, the two classifiers calculate the predicted values ​​respectively: Global output and local output; Define two loss functions: Global cross entropy loss: , Local classifier joint loss (KL + CE): , Among them, λ is a balance term that controls the degree of dependence of the local model on global guidance and adaptive learning; KL: KL divergence, which promotes the local model to move closer to the global model; CE: standard cross entropy loss, which maintains the ability to fit the true label; Step 303: separator fusion strategy; After completing the local training, the two classifiers are weighted and fused according to the learnable parameter μ to construct the final local model for prediction: , μ is a locally learnable fusion ratio parameter used to balance local adaptability and global generalization capability. The fused classifier serves as the output head for the next round of training and inference. If μ=1, the model is completely local; if μ=0, it is completely global.

[0019] The model of the present invention is mainly compared with the following methods: 1) FedProto: FedProto is a prototype-based federated learning method. It does not share model parameters. Instead, each client uploads a class prototype (i.e., feature mean) for each class of samples. The server aggregates the prototype to form a global prototype, which is then broadcast back to the client for alignment and supervision.

[0020] 2) FedGH: FedGH is a model-heterogeneous federated framework that uses a client-side personalized feature extractor (Backbone) and a unified server-side shared classification head (Global Head). After extracting features, the client sends the intermediate representation to the server, which uniformly trains the Global Head for prediction.

[0021] 3) FedTGP: Based on FedProto, FedTGP introduces trainable global prototypes and contrastive learning mechanisms, emphasizing inter-class separation and intra-class compactness to improve the expressiveness and discriminability of prototypes.

[0022] CIFAR-10 Dataset: CIFAR-10 is a classic, small-scale image classification dataset covering 10 common categories (such as airplanes, cats, and cars). Each image is 32×32 pixels in size, a low resolution that makes it suitable for basic image classification and for simulating lightweight scenarios in federated learning. The dataset contains 60,000 color images, 50,000 of which are used for training and 10,000 for testing.

[0023] Cifa100 dataset: CIFAR-100 is similar to CIFAR-10, but with 100 classes, making it more suitable for fine-grained classification tasks. Each class contains fewer examples (approximately 500 images), making it more challenging in heterogeneous data environments. The images are also 32×32 in size, making it suitable for evaluating the discriminative and generalization performance of models in environments with high class density.

[0024] Flowers102 Dataset: Flowers102 is a flower image recognition dataset released by the University of Oxford, featuring 102 flower species. Its high-resolution images, uneven sample distribution, and complex and diverse backgrounds make it more realistic for real-world applications. This dataset is widely used to evaluate the robustness and generalization of models under non-IID, class-imbalanced, and complex image conditions.

[0025] Framework: During implementation, we leveraged the computational framework HtFE (High Throughput Feature Extraction) to extract features from various sources. We also evaluated the performance of HtFE on various mobile and embedded devices. The architectures we tested included the following models: four CNN models, GoogleNet, MobileNet_v2, ResNet18, ResNet34, ResNet50, ResNet101, and ResNet152. To generate feature representations with the same feature dimension, we added an average pooling layer after each feature extractor

[18] . By default, we set K = 512.

[0026] Heterogeneous Settings: In our main experiments, we employed two widely used statistical heterogeneity settings: the pathological setting and the pragmatic setting. In the pathological setting, we assigned each client 2, 10, and 10 sampled classes from CIFAR-10, CIFAR-100, and Flowers102, respectively. Class samples were then distributed to clients with a ratio of 0.1 using a Dirichlet distribution with a default parameter of 0.1.

[0027] Comparison results of various methods on different datasets (1) Comparison results under different data distributions Table 1 lists the performance of four methods under two data distribution settings. Under the pat distribution, each client is assigned 2 / 10 / 10 categories based on different datasets. Under the dir distribution, we set the data heterogeneity parameter s = 0.1 and the total number of global epochs to 200. As can be seen, our algorithm, FedACA, achieves the best performance under all settings. In the Cifar10 real-world setting, our method outperforms the baseline method, FedTGP, by 2.98%. This improvement is primarily attributed to clearer classification boundaries and more reasonable prototype representations.

[0028] Table 1: Comparison of test accuracy (%) under pathological and practical conditions.

[0029]

[0030] 1) Comparison results on different networks Table 2 lists the performance of the four methods under conditions of model heterogeneity. Our method achieves the highest accuracy in almost all cases. The accuracy of the FedProto algorithm decreases significantly with increasing model heterogeneity, likely due to its simple weighted averaging method, which does not well represent the global prototype. In contrast, our algorithm exhibits minimal loss in accuracy, demonstrating its advantage in handling model heterogeneity.

[0031] Table 2: Test accuracy (%) of Cifar100 and Flowers102 in a real environment using heterogeneous feature extractors.

[0032]

[0033] 2) Comparison results on a large number of local clients In Table 3, we conducted large-scale training experiments using the Cifar10 and Flowers102 datasets. Specifically, we tested the FedACA algorithm with different numbers of clients (35 and 50) to evaluate its effectiveness. The results show that compared to traditional methods, the FedACA algorithm significantly improves training performance with both 35 and 50 clients. Furthermore, FedACA demonstrates the ability to effectively allocate and utilize computing resources across clients, thereby improving the overall efficiency of federated learning.

[0034] Table 3: Test accuracy (%) of Cifar100 and Flowers102 by a large number of customers.

[0035]

[0036] 3) Comparison results on different data heterogeneity Table 4 lists the performance of the four methods on the Cifar10 dataset, evaluating the impact of varying data heterogeneity. FedACA outperforms all other methods in all cases, particularly at s = 0.1, where it achieves 0.8439. FedProto experiences the largest drop in accuracy, dropping to 0.5513 at s = 0.7, highlighting the ineffectiveness of its global prototype simple weighted average method in handling model heterogeneity. FedGH and FedTGP perform well at low heterogeneity (s = 0.1), but experience a significant drop in accuracy as s increases. However, FedACA experiences the smallest drop in accuracy, maintaining 0.5956 even at high heterogeneity (s = 0.7), demonstrating its strong advantage in managing data heterogeneity.

[0037] Table 4: Test accuracy (%) under different data heterogeneity settings.

[0038]

[0039] 4) Comparison results on different feature extractors Figure 2 shows a performance comparison of different federated learning algorithms on the CIFAR-10 dataset at different feature extractor dimensions. "Dimension" represents the output size of the average pooling layer added after each feature extractor. Our method outperforms the other three algorithms across the board, consistently achieving the highest accuracy. At dimension 1024, the accuracy reaches 83.69%. Notably, at K = 64, our accuracy is 7.45% higher than the baseline algorithm, FedTGP.

[0040] 5) Ablation experiment Table 5 shows a portion of the ablation experiment. By employing the adaptive K-Means clustering method (AS-Kmeans), we successfully optimized the feature representation of local categories. Compared to the baseline model FedTGP, the addition of AS-Kmeans significantly improved the model accuracy on the CIFAR-10 dataset, increasing from 80.24% to 82.94%. Furthermore, the addition of CHM and PHF also improved accuracy, demonstrating the effectiveness of classification boundary optimization. Improvements were observed on both the CIFAR-10 and Flowers102 datasets. Finally, our FedACA achieved significant improvements on all datasets, with an accuracy increase of over 4% on the CIFAR-10 dataset, demonstrating the effectiveness of our algorithm.

[0041] Table 5: Test accuracy (%) for ablation studies using the HtFE8 model set in a real-world setting.

[0042]

[0043] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any technician familiar with the present profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical content disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement of the above embodiments made according to the technical essence of the present invention, within the spirit and principles of the present invention, without departing from the content of the technical solution of the present invention, shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. Enhanced EedACA method for prototype aggregation and classifier fusion in heterogeneous federated learning, characterized by: The specific steps include: Step 1: Adaptive K-Means clustering; In each client, the optimal number of clusters for each category is adaptively determined, the KMeans algorithm is optimized using the silhouette coefficient, multiple subclass prototypes are generated, and weighted fusion is performed to construct a local class prototype; Step 2: Cluster classifier matching; After the client uploads the local classifier, the server clusters the clients based on the similarity between the semantic outputs of the classifiers, groups similar clients, and generates a personalized global classifier. Step 3: Personalized classifier fusion; The client uses KL divergence and cross entropy to jointly optimize local and global classifiers and uses trainable fusion coefficients Control the combination ratio of the final classifier to achieve a dynamic trade-off between global generalization and local adaptation, and enhance the accuracy, stability and personalized performance of the model.

2. The enhanced EedACA method for prototype aggregation and classifier fusion in heterogeneous federated learning according to claim 1 is characterized in that: Step 1 specifically includes: Step 101: feature extraction; The client first sends the local sample Input local feature extractor , and obtain the corresponding high-dimensional feature vector : ， For each category , collect the corresponding feature set ; Step 102: Optimize the number of clusters using the silhouette coefficient; Perform multiple clustering experiments on each feature set and automatically select the optimal number of clusters using the silhouette coefficient , the silhouette coefficient is defined as: ， in, Indicates samples in the same cluster The average distance to other samples, Represents a sample The average distance of all samples to the nearest cluster; The final choice is The largest The optimal number of clusters : ； Step 103: Use the above optimal , perform KMeans on the feature set, with the goal of minimizing the sum of squared distances within the class: ， in is the centroid of the kth subcluster; Step 104: Calculate weighted sub-prototypes; For each subcluster , calculate its center: ， Calculate the sub-cluster weight according to its sample size ratio: ， Aggregate all sub-archetypes to form a category The weighted prototype vector of : 。 3. The enhanced EedACA method for prototype aggregation and classifier fusion in heterogeneous federated learning according to claim 1 is characterized in that: Step 2 specifically includes: Step 201: Upload the classifier; After each round of training, the client Upload a locally trained copy of the global classifier , uploaded to the server together with the local class prototype; Step 202: Calculate the semantic representation of the separator; The server uses a set of random feature vectors , input the global classifier of each client and calculate its output probability distribution: ， in, The predicted probability of client k for random input; M: temperature parameter, controlling softmax smoothness; Step 203: Calculate the similarity between classifiers; Calculate the cosine similarity between any two clients Semantic similarity between classifiers: ， All clients form a similarity matrix ,in is the total number of clients; Step 204: Maximum difference grouping strategy; To cluster clients semantically, we use the "maximum difference partitioning" strategy: for each client k, we sort the similarities from high to low and find the maximum adjacent difference: ， Using the maximum jump point as the threshold, select a group of clients with high similarity: ； Step 205: Determine the consensus stage; Define whether the system is currently in the "consensus stage", that is, whether it is suitable for dynamic grouping, and calculate the average value of the maximum jump difference of all clients: ， if: ， If CCP is set to True, it means that the information is active and the MDS grouping strategy should be implemented; otherwise, it will enter the stable period and use historical statistical dependence; Step 206: Generate a personalized global classifier; If CCP=True, the classifier is weighted averaged using the client group selected by MDS: ， where |D i | is the data volume of client i; If CCP=False, weighted fusion is performed based on the historical dependency matrix G: ， The matrix G records the historical aggregation frequency between clients and is used for personalized construction in the stable period.

4. The enhanced EedACA method for prototype aggregation and classifier fusion in heterogeneous federated learning according to claim 1, characterized in that: Step 3 specifically includes: Step 301: prepare the classifier; The client obtains two types of classifiers: Personalized global classifier from the server , generated by CHM module; Locally trained private classifier , representing the local preference feature. Both act on local eigenvectors , output predicted logits; Step 302: Prediction output and loss construction; For the input samples, the two classifiers calculate the predicted values ​​respectively: Global output and local output; Define two loss functions: Global cross entropy loss: ， Local classifier joint loss (KL + CE): ， in, It is a balancing term that controls the degree of dependence of the local model on global guidance and adaptive learning; KL: KL divergence, which promotes the local model to move closer to the global model; CE: standard cross entropy loss, which maintains the ability to fit the true label; Step 303: separator fusion strategy; After completing the local training, the two classifiers are weighted and fused according to the learnable parameter μ to construct the final local model for prediction: ， μ is a locally learnable fusion ratio parameter used to balance local adaptability and global generalization capability. The fused classifier serves as the output head for the next round of training and inference. If μ=1, the model is completely local; if μ=0, it is completely global.

Images