Heterogeneous remote sensing image target extraction method based on federated learning
By introducing prototype representation and privacy-preserving distributed computing into federated learning, the problems of target extraction accuracy and robustness of heterogeneous remote sensing image data are solved, achieving efficient cross-domain knowledge sharing and model optimization, and improving the accuracy and robustness of remote sensing image processing.
Patent Information
- Application Number
- CN202510977217.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-16
- Publication Date
- 2025-10-28
AI Technical Summary
Existing federated learning methods struggle to effectively improve the accuracy and robustness of target extraction when processing heterogeneous remote sensing image data, and also pose challenges to data privacy protection.
By employing prototype representation and federated learning methods, and by training feature extractors and category prototypes locally on the client side and uploading them, combined with privacy-preserving distributed computing technology, cross-domain knowledge sharing and model optimization are achieved.
While protecting data privacy, it significantly improves the accuracy and robustness of target extraction from heterogeneous remote sensing images, and enhances the model's generalization ability and cross-domain adaptability.
Smart Images
Figure CN120853005A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of remote sensing image processing technology, and in particular relates to a method for extracting targets from heterogeneous remote sensing images based on federated learning. Background Technology
[0002] In a broad sense, remote sensing image target extraction refers to the process of automatically identifying and extracting specific targets or objects from remote sensing images. It is widely used in fields such as agricultural monitoring, environmental monitoring, urban planning, and disaster assessment. With the continuous development of remote sensing technology, target extraction has become an important research direction in remote sensing image analysis. Currently, deep learning technology is widely used in remote sensing image target extraction tasks. Significant progress has been made in automatic target extraction from remote sensing images through models such as convolutional neural networks and deep residual networks. However, training deep learning models typically requires a large amount of labeled data and computational resources, especially when dealing with cross-domain or heterogeneous data, posing challenges to data sharing and privacy protection. Since remote sensing image data often involves sensitive geographic information, how to effectively train models while protecting privacy has become a significant challenge. With the widespread application of privacy-preserving distributed machine learning methods, federated learning has become one of the representative achievements, providing a distributed training solution. In federated learning, multiple clients train models locally and only share model parameters or gradients, avoiding centralized storage and transmission of data, thus ensuring data privacy protection. However, processing heterogeneous remote sensing image data remains a significant challenge in distributed privacy computing, especially when the data originates from different geographical regions, sensors, resolutions, or climatic conditions. This heterogeneity limits the generalization ability of the models. Currently, federated learning-based methods, such as FedProx and FedNova, are primarily applied to general image classification or ordinary vision tasks, with few applications specifically for remote sensing target extraction. Furthermore, directly applying existing federated learning methods to remote sensing image target extraction often fails to effectively handle these heterogeneous remote sensing data, impacting model performance and accuracy. Therefore, how to process heterogeneous remote sensing image data using federated learning methods and improve the accuracy and robustness of target extraction remains a crucial issue that urgently needs to be addressed in the fields of remote sensing image processing and federated learning. Summary of the Invention
[0003] To address the aforementioned problems, this invention discloses a target extraction method for heterogeneous remote sensing images based on federated learning. By introducing prototype representation and federated learning methods, combined with privacy-preserving distributed computing technology, this invention effectively overcomes the heterogeneity problem of remote sensing data and significantly improves the accuracy and robustness of cross-domain target extraction. Traditional federated learning methods often perform poorly when dealing with heterogeneous remote sensing data, while the innovative method proposed in this invention can achieve efficient cross-domain knowledge sharing while protecting data privacy, thereby improving the overall performance of target extraction.
[0004] This invention is achieved through the following method: a heterogeneous remote sensing image target extraction method based on federated learning, characterized by comprising the following steps:
[0005] S1: Local Data Acquisition and Secure Storage
[0006] The system comprises n clients and one central endpoint. Each client independently collects local remote sensing image data, which is strictly stored within its local system and not transmitted across nodes to ensure data privacy and regional isolation. The remote sensing image data can originate from various sensor platforms, including but not limited to satellite remote sensing, aerial photography, and UAV observation systems. Its wide geographical distribution and heterogeneous sensing mechanisms result in significant differences in the distribution characteristics and category structure of the data collected by each client.
[0007] S2: Global Model Synchronization
[0008] In the initial round, the central server synchronously distributes the initialization parameters of the deep learning model to all participating clients. If it is not the initial round, the server will first perform the S4-stage aggregation operation on the model parameters uploaded by each client, and then distribute the updated global model to each client. This distribution process ensures that each client can effectively utilize remote sensing knowledge from other regions to supplement its own training perspective and improve the model's representation ability and generalization performance without exposing its local raw data. After receiving the global model, each client continues local training based on its local remote sensing image data to further optimize the learning of feature representations and category prototypes.
[0009] The deep learning model is decoupled into a feature extractor φ t and prototype p c,k,t The system consists of two parts. The prototype is a typical feature of a category constructed by the mean of feature vectors of similar samples. Each category is represented by multiple vectors to fully reflect the diversity of the category. The feature extractor is used to extract high-dimensional semantic feature representations of ground objects in remote sensing images. The two are combined to achieve the discrimination and classification of different ground object categories.
[0010] Step S2 specifically includes the following steps:
[0011] S21: In the initial communication phase of federated learning, the central endpoint generates a global prototype vector and feature extractor during initialization and distributes them to each participating client to unify the initial state of the model and ensure the synchronization and consistency of collaborative training among the clients. The central endpoint distributes the global prototype vector and feature extractor to each client (i = 1, 2, ..., n), where t represents the aggregation round of federated learning, and t is initially set to 0. When communication is not the first time, the central endpoint will perform the aggregation operation of phase S4 on the model parameters uploaded by each client and distribute the aggregated global model to the clients.
[0012] S22: The client replaces the local prototype and feature extractor with the global prototype and feature extractor, as shown in the following formula:
[0013]
[0014]
[0015] in p represents the local prototype c,k,t Represents the global prototype. Represents the local feature extractor, φ t represents the global feature extractor. c represents the category index. What distinguishes this invention from other methods is that category c is represented by k vectors, aiming to capture the diverse features of that category more comprehensively.
[0016] S3: Local Model Training
[0017] Within the client, based on the locally preprocessed remote sensing images and their label information obtained in stage S1, the deep learning model synchronized in stage S2 is trained independently in the local environment. The constructed model can learn the discrimination features of different land cover categories in the remote sensing images, realizing the automatic identification and extraction of typical land cover targets. The entire training process is strictly limited to local execution, without involving cross-node sharing of raw data, ensuring data security and privacy protection from the source, and providing a model foundation for subsequent federated collaborative optimization and cross-domain knowledge fusion.
[0018] Step S3 specifically includes the following steps:
[0019] S31: Local Feature Extraction and Embedding Mapping. After receiving the global model, the client, based on local remote sensing image data, uses a feature extractor to extract the remote sensing image data. i Mapping to a high-dimensional embedding space, we extract deep semantic feature representations for each sample.
[0020] S32: Prototype-Distance Classification and Prediction. Within the embedding space, the client uses a classification method based on the distance between the class prototype and sample features to predict the class of local samples. This classification process can be expressed by the following formula:
[0021]
[0022] Where sim() represents the distance between the feature and the prototype.
[0023] S33: Prototype Anchoring Metric Learning and Optimization. The client computes a loss function that includes a prototype anchoring term and a classification error term. This function both constrains the tightness of the distribution of class prototypes and optimizes the classification decision boundary. The local feature extractor and class prototypes are updated through backpropagation.
[0024] The client calculates the loss using a prototype anchored metric learning method based on the prediction results and dataset labels. The loss function measures the model's predictive accuracy and guides the optimization process. This process includes the following key parts:
[0025] Cross-entropy loss (CE): In this method, cross-entropy loss ensures the accuracy of the model's classification results. It is achieved by measuring the difference between the predicted class probabilities and the true labels. During training, the model is optimized by minimizing the cross-entropy loss. The expression for cross-entropy loss is as follows:
[0026]
[0027] Where c represents the index of the target category, c′ represents all possible category indices, and y c It is the actual label associated with category c.
[0028] Prototype Contrast Alignment Loss (PAAC): In this method, Prototype Contrast Alignment Loss encourages the embedded features of samples to align with the prototype of the correct class and distinguish them from the prototype of the incorrect class. By using the prototype as an anchor, PAAC loss leverages features from client-specific regions, enhancing the global feature extractor's ability to extract features consistent with the target class, thereby improving the model's generalization ability in different environments. The expression for PAAC loss is as follows:
[0029]
[0030] Among them, s + s represents the similarity between the pixel embedding and the correct category prototype. - This represents the set of similarities to the prototype of the error category, with τ used as a temperature parameter to control the scaling of the similarity.
[0031] Prototype Center Distance Loss (PCD): In this method, prototype center distance loss primarily focuses on reducing the distance between sample embedding features and their class prototypes, enhancing the consistency of features within the same class. By minimizing the distance between sample embeddings and class prototypes, PCD loss clusters samples of the same class more tightly around the prototype. The expression for PCD loss is as follows:
[0032]
[0033] This method constructs a prototype-anchored metric learning method by weightedly combining the above three loss functions, and the specific formula is expressed as follows:
[0034]
[0035] Where λ1 and λ2 are hyperparameters controlling the loss weights.
[0036] S34 updates the local feature extractor based on the loss calculated in S33. and prototype The feature extractor is updated via backpropagation, and the prototype update process is as follows:
[0037] The client first calculates the allocation matrix L, optimizing it to ensure balanced utilization of each prototype, resulting in approximately equal pixel allocations for each prototype. Then, based on the allocation matrix L, it calculates the average embedding feature for each prototype to ensure that each prototype accurately represents the features of its category. Finally, the client updates its local prototypes using the calculated average embedding and adjusts the update step size μ to avoid overfitting. This process can be expressed by the following formula:
[0038]
[0039]
[0040] Where m represents the number of pixels and k represents the number of prototypes of each type.
[0041] S4: Multi-faceted model parameter aggregation
[0042] Based on the remote sensing feature extraction models trained locally by each client in the S3 phase, after completing a predetermined number of training rounds, each client uploads its local model parameters and corresponding category prototype vectors to the federated server. The central server, without accessing the original data, performs unified aggregation of the model information submitted by multiple parties. Specifically, weights are assigned according to the effective sample size of each client, and weighted fusion is performed on the feature extraction model parameters and category prototypes respectively to generate a globally shared model and a unified category prototype representation, thereby achieving the collaborative integration of multi-source heterogeneous remote sensing knowledge.
[0043] Step S4 specifically includes the following steps:
[0044] S41: The client sends the updated local feature extractor and prototype to the central end.
[0045] S42: The central end aggregates the local feature extractor and prototype into a global feature extractor and prototype, as shown in the following formula:
[0046]
[0047] in This represents the aggregation weight of the i-th client. The value of t on the left side of the formula becomes t+1, indicating that one aggregation has been completed.
[0048] S5: Model Convergence and Target Extraction
[0049] After multiple rounds of iterative optimization in the S2 phase, if the federated training reaches the preset convergence criterion, the model synchronization process terminates, and the final aggregated global feature extractor and category prototype vectors are exported. This global model integrates remote sensing knowledge from various clients, possessing cross-domain generalization ability and robustness. Each client can use this global model in its local environment to classify and extract targets from remote sensing images, outputting the target extraction results. Because this model integrates the distribution characteristics of multi-source heterogeneous remote sensing data, it can adapt to changes in image characteristics under different regional and sensor conditions, thereby achieving high-precision, transferable, cross-regional remote sensing target extraction.
[0050] Step S5 specifically includes the following steps:
[0051] S51: Convergence Determination. During the aggregation process, the central endpoint determines whether the convergence condition is met by monitoring the global model loss value, parameter change rate, or validation set performance changes.
[0052] S52: Final Model Export and Application. If the convergence condition is met, stop model synchronization and iteration, export the aggregated global feature extraction network and category prototype vectors, and apply them to the cross-regional remote sensing image target extraction task.
[0053] This invention provides a solution for secure collaborative learning and computation of remote sensing imagery. By uploading only the feature extractor and category prototypes instead of directly uploading the original data, it achieves secure collaborative learning and computation of remote sensing imagery while protecting data privacy. Addressing the feature shift problem in federated learning methods for heterogeneous remote sensing data, this invention employs distance-based classification and prototype anchoring metric learning. The distance-based classification method calculates the distance between the embedded features of the input sample and the category prototype, selecting the closest prototype for classification. This reduces misclassification caused by the classifier and ensures that samples are accurately classified into the most appropriate category. The prototype anchoring metric learning method enhances intra-category tightness and inter-category discriminability, improving the robustness of target extraction and classification performance in cross-domain remote sensing imagery data analysis. Attached Figure Description
[0054] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0055] Figure 1 This is a flowchart of the federated learning heterogeneous remote sensing image target extraction method according to an embodiment of the present invention;
[0056] Figure 2 This is a schematic diagram of the federated learning parameter exchange method according to an embodiment of the present invention;
[0057] Figure 3 This is a flowchart of the prototype-distance-based classification method according to an embodiment of the present invention;
[0058] Figure 4 This is a flowchart of the prototype anchoring metric learning method according to an embodiment of the present invention;
[0059] Figure 5 This is a summary of the algorithms used in the embodiments of the present invention. Detailed Implementation
[0060] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, and to make the above-mentioned objectives, features and advantages of the present invention more apparent and understandable, the technical solutions in the present invention will be further described in detail below with reference to the accompanying drawings.
[0061] This invention discloses a target extraction method for heterogeneous remote sensing images based on federated learning. By introducing prototype representation and federated learning methods, combined with privacy-preserving distributed computing technology, this invention effectively overcomes the heterogeneity problem of remote sensing data and significantly improves the accuracy and robustness of cross-domain target extraction.
[0062] This invention is achieved through the following method: a federated learning method for target extraction from remote sensing images, such as... Figure 1 As shown, it includes the following steps:
[0063] S1: Local Data Acquisition and Secure Storage
[0064] The method comprises n clients and one central terminal. Each client independently collects local remote sensing image data, which is strictly stored within its local system and not transmitted across nodes to ensure data privacy and regional isolation. Remote sensing image data can originate from various types of sensor platforms, including but not limited to satellite remote sensing, aerial photography, and UAV observation systems. Its wide geographical distribution and heterogeneous sensing mechanisms result in significant differences in the distribution characteristics and category structure of the data collected by each client.
[0065] S2: Global Model Synchronization
[0066] In the initial round, the central server synchronously distributes the model's initialization parameters to all participating clients. If it is not the initial round, the server will first perform the S4-stage aggregation operation on the model parameters uploaded by each client, and then distribute the updated global model to each client. This distribution process ensures that each client can effectively utilize remote sensing knowledge from other regions to supplement its own training perspective and improve the model's representation ability and generalization performance without exposing its local raw data. After receiving the global model, each client continues local training based on its local remote sensing image data to further optimize feature representation and class prototype learning.
[0067] S21: In the initial communication phase of federated learning, the central endpoint generates a global prototype vector and feature extractor during initialization and distributes them to each participating client to unify the initial state of the model and ensure the synchronization and consistency of collaborative training among the clients. The central endpoint distributes the global prototype vector and feature extractor to each client (i = 1, 2, ..., n), where t represents the aggregation round of federated learning, and t is initially set to 0. When communication is not the first time, the central endpoint will perform the aggregation operation of phase S4 on the model parameters uploaded by each client and distribute the aggregated global model to the clients.
[0068] S22: The client replaces the local prototype and feature extractor with the global prototype and feature extractor, as shown in the following formula:
[0069]
[0070] in p represents the local prototype c,k,t Represents the global prototype. Represents the local feature extractor, φ t represents the global feature extractor. c represents the category index. What distinguishes this invention from other methods is that category c is represented by k vectors, aiming to capture the diverse features of that category more comprehensively.
[0071] S3: Local Model Training
[0072] Within the client, based on the locally preprocessed remote sensing images and their label information obtained in stage S1, a deep learning model synchronized in stage S2 is trained independently in the local environment. The constructed model can learn the discrimination features of different land cover categories in the remote sensing images, achieving automatic identification and extraction of typical land cover targets. The entire training process is strictly limited to local execution, without involving cross-node sharing of raw data, ensuring data security and privacy protection from the source, and providing a model foundation for subsequent federated collaborative optimization and cross-domain knowledge fusion.
[0073] S34 updates the local feature extractor based on the loss calculated in S33. and prototype The feature extractor is updated via backpropagation. The prototype update process is as follows:
[0074] The client first calculates the allocation matrix L, optimizing it to ensure balanced utilization of each prototype, resulting in approximately equal pixel allocations for each prototype. Then, based on the allocation matrix L, it calculates the average embedding feature for each prototype to ensure that each prototype accurately represents the features of its category. Finally, the client updates its local prototypes using the calculated average embedding and adjusts the update step size μ to avoid overfitting. This process can be expressed by the following formula:
[0075]
[0076] Where m represents the number of pixels, and k represents the number of prototypes in each class.
[0077] S4: Multi-faceted model parameter aggregation
[0078] Based on the remote sensing feature extraction models trained locally by each client in the S3 phase, after completing a predetermined number of training rounds, each client uploads its local model parameters and corresponding category prototype vectors to the federated server. The federated server then aggregates the model information submitted by multiple parties without accessing the original data. Specifically, weights are assigned according to the effective sample size of each client, and weighted fusion is performed on the feature extraction model parameters and category prototypes to generate a globally shared model and a unified category prototype representation, thereby achieving the collaborative integration of multi-source heterogeneous remote sensing knowledge.
[0079] S41: The client sends the updated local feature extractor and prototype to the central end.
[0080] S42: The central end aggregates the local feature extractor and prototype into a global feature extractor and prototype, as shown in the following formula:
[0081]
[0082] in This represents the aggregation weight of the i-th client. The value of t on the left side of the formula becomes t+1, indicating that one aggregation has been completed.
[0083] S5: Model Convergence and Target Extraction
[0084] After multiple rounds of iterative optimization in the S2 phase, if the federated training reaches the preset convergence criterion, the model synchronization process terminates, and the final aggregated global feature extractor network and category prototype vectors are exported. This global model integrates remote sensing knowledge from various clients, possessing cross-domain generalization ability and robustness. Each client can use this global model in its local environment to classify and extract targets from remote sensing images, outputting the target extraction results. Because this model integrates the distribution characteristics of multi-source heterogeneous remote sensing data, it can adapt to changes in image characteristics under different regional and sensor conditions, thereby achieving high-precision, transferable, cross-regional remote sensing target extraction.
[0085] S51: Convergence Determination. During the aggregation process, the central endpoint determines whether the convergence condition is met by monitoring the global model loss value, parameter change rate, or validation set performance changes.
[0086] S52: Final Model Export and Application. If the convergence condition is met, stop model synchronization and iteration, export the aggregated global feature extraction network and category prototype vectors, and apply them to the cross-regional remote sensing image target extraction task.
[0087] like Figure 2 As shown, the system architecture of this embodiment includes a central server and multiple client nodes. Each client node stores only its local remote sensing image dataset, ensuring data privacy. In this method, each client node uses its local dataset to refine its local feature extractor. and prototype The process involves iterative updates. After each iteration, each client node sends its updated local model parameters and prototype vector parameters to the central server for aggregation. Once the central server has aggregated the parameters, it synchronizes the globally updated feature extraction model parameters and prototype vectors to the local models of each client. This iterative process continues until the model parameters converge to a preset threshold range.
[0088] This invention proposes a classification method based on prototype-distance, such as... Figure 3 As shown, the classification process includes the following steps:
[0089] S31: Local Feature Extraction and Embedding Mapping. After receiving the global model, the client extracts features based on local remote sensing image data using a feature extractor. Remote sensing image D i Mapping to a high-dimensional embedding space, we extract deep semantic feature representations for each sample.
[0090] S32: Prototype-Distance Classification and Prediction. Within the embedding space, the client uses a classification method based on the distance between the class prototype and sample features to predict the class of local samples. This classification process can be expressed by the following formula:
[0091]
[0092] Wherein sim is used to measure the distance between a feature and a prototype, and in this invention, cosine similarity is used as the method for measuring distance.
[0093] This invention proposes a prototype-anchored metric learning method, such as... Figure 4 As shown, it includes the following key parts:
[0094] S33: Prototype Anchoring Metric Learning and Optimization. The client computes a loss function that includes a prototype anchoring term and a classification error term. This function both constrains the tightness of the distribution of class prototypes and optimizes the classification decision boundary. The local feature extractor and class prototypes are updated through backpropagation.
[0095] The client calculates the loss using a prototype anchored metric learning method based on the prediction results and dataset labels. The loss function measures the model's predictive accuracy and guides the optimization process. This process includes the following key parts:
[0096] Cross-entropy loss (CE): In this method, cross-entropy loss ensures the accuracy of the model's classification results. It is achieved by measuring the difference between the predicted class probabilities and the true labels. During training, the model is optimized by minimizing the cross-entropy loss. The expression for cross-entropy loss is as follows:
[0097]
[0098] Where c represents the index of the target category, c′ represents all possible category indices, and y c It is the actual label associated with category c.
[0099] Prototype Contrast Alignment Loss (PAAC): In this method, Prototype Contrast Alignment Loss encourages the embedded features of samples to align with the prototype of the correct class and distinguish them from the prototype of the incorrect class. By using the prototype as an anchor, PAAC loss leverages features from client-specific regions, enhancing the global feature extractor's ability to extract features consistent with the target class, thereby improving the model's generalization ability in different environments. The expression for PAAC loss is as follows:
[0100]
[0101] Among them, s + s represents the similarity between the pixel embedding and the correct category prototype. - This represents the set of similarities to the prototype of the error category, with τ used as a temperature parameter to control the scaling of the similarity.
[0102] Prototype Center Distance Loss (PCD): In this method, prototype center distance loss primarily focuses on reducing the distance between sample embedding features and their class prototypes, enhancing the consistency of features within the same class. By minimizing the distance between sample embeddings and class prototypes, PCD loss clusters samples of the same class more tightly around the prototype. The expression for PCD loss is as follows:
[0103]
[0104] This method constructs a prototype-anchored metric learning method by weightedly combining the above three loss functions, and the specific formula is expressed as follows:
[0105]
[0106] Where λ1 and λ2 are hyperparameters that control the loss weights.
[0107] like Figure 5 As shown, the algorithm summarizes the collaboration process between the client and the central server in a prototype-based federated learning approach. The client first receives the global prototype p from the server. c,k,t and feature extractor φ t Subsequently, local remote sensing image data D was used. i High-dimensional features are extracted using a feature extractor, the loss is calculated using a prototype anchored metric learning method, and the local feature extractor is updated. and prototype After completing the local update, the client uploads the updated local prototype and feature extractor to the server. The server receives update information from all clients, updates the global prototype and global feature extractor using a weighted average method based on client contributions, and broadcasts the updated global model back to all clients. This process continues until the global model converges, thus effectively performing the target extraction task from heterogeneous remote sensing images.
[0108] In practical applications, this method was validated on BH-Pools, Inria Aerial Image Labeling (IAIL), and Global Landslide Mapping (GLM) datasets. The target extraction results show that the overall accuracy (OA) of the proposed method is 99.38%, 85.94%, and 95.76 on the three datasets, respectively, and the intersection over union (IoU) is 69.70, 40.27, and 38.97, respectively. Furthermore, this method was compared with internationally advanced federated learning methods such as FedAvg, FedProx, FedProx, DBE, and FedLGF. Experiments demonstrate that the overall accuracy and IoU of the method disclosed in this invention are superior to existing methods.
[0109] This invention proposes a target extraction method for heterogeneous remote sensing images based on federated learning, providing a novel solution to the problem of secure collaborative learning and computation of remote sensing images. Based on the innovative method proposed in this invention, secure sharing of remote sensing images is achieved, and the consistency and stability of the feature space of remote sensing data in a federated learning environment are improved, enhancing the generalization ability of the model in complex remote sensing tasks.
[0110] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.
[0111] Although the invention has been described by way of examples, those skilled in the art will recognize that the invention has many variations and modifications without departing from its spirit, and it is intended that the appended claims cover such variations and modifications without departing from the spirit of the invention.
Claims
1. A method for target extraction from heterogeneous remote sensing images based on federated learning, characterized in that, Includes the following steps: S1: Local Data Acquisition and Secure Storage It includes n clients and one central terminal. In each client environment, local remote sensing image data is collected independently. The image data is strictly kept within the local system and is not transmitted across nodes to ensure data privacy and regional isolation. S2: Global Model Synchronization In the initial round, the central server synchronously distributes the initialization parameters of the deep learning model to all participating clients. If it is not the initial round, the server will first perform the S4 stage aggregation operation on the model parameters uploaded by each client, and then distribute the updated global model to each client. After receiving the global model, each client continues to carry out local training based on local remote sensing image data to further optimize the learning of feature representation and category prototype. S3: Local Model Training Within the client, based on the locally preprocessed remote sensing images and their label information obtained in the S1 stage, the deep learning model synchronized in the S2 stage is trained independently in the local environment. The constructed model can learn the discrimination features of different land cover categories in the remote sensing images, and realize the automatic identification and extraction of typical land cover targets. The entire training process is strictly limited to local execution. S4: Multi-faceted model parameter aggregation Based on the remote sensing ground feature extraction model obtained by each client in the S3 phase through local training, after completing a predetermined number of training rounds, the client uploads the local model parameters and their corresponding category prototype vectors to the federated server. The central server aggregates model information submitted by multiple parties without accessing the original data. Specifically, it assigns weights based on the effective sample size of each client, performs weighted fusion on the feature extraction model parameters and category prototypes, and generates a globally shared model and a unified category prototype representation, thereby achieving the collaborative integration of multi-source heterogeneous remote sensing knowledge. S5: Model Convergence and Target Extraction After multiple rounds of iterative optimization in the S2 phase, if the federated training reaches the preset convergence criterion, the model synchronization process terminates, and the final aggregated global feature extractor and category prototype vector are exported. This global model integrates remote sensing knowledge from various clients and has cross-domain generalization ability and robustness. Each client can use this global model in its local environment to classify and extract targets from remote sensing images and output the target extraction results.
2. The method for target extraction from heterogeneous remote sensing images based on federated learning according to claim 1, characterized in that: The deep learning model is decoupled into a feature extractor φ t and prototype p c,k,t The system consists of two parts: the prototype is a category-specific feature constructed from the mean of feature vectors of similar samples, with each category represented by multiple vectors; the feature extractor is used to extract high-dimensional semantic feature representations of ground objects in remote sensing images; the two are combined to achieve the discrimination and classification of different ground object categories.
3. The method for target extraction from heterogeneous remote sensing images based on federated learning according to claim 2, characterized in that: Step S2 specifically includes the following steps: S21: In the initial communication phase of federated learning, the central end generates a global prototype vector and feature extractor during the initialization process and distributes them to each participating client to unify the initial state of the model and ensure the synchronization and consistency of collaborative training among the clients. The central end distributes the global prototype vector and feature extractor to each client (i = 1, 2, ..., n), where t represents the aggregation round of federated learning, and t is set to 0 at the initial distribution. When communication is not the first time, the central end will perform the aggregation operation of the S4 phase on the model parameters uploaded by each client and distribute the aggregated global model to the client. S22: The client replaces the local prototype and feature extractor with the global prototype and feature extractor, as shown in the following formula: in p represents the local prototype c,k,t Represents the global prototype. Represents the local feature extractor, φ t represents the global feature extractor; c represents the category index, and category c is represented by k vectors.
4. The method for extracting targets from heterogeneous remote sensing images based on federated learning according to claim 3, characterized in that: Step S3 specifically includes the following steps: S31: Local Feature Extraction and Embedding Mapping; After receiving the global model, the client extracts the remote sensing image data using a feature extractor based on the local remote sensing image data. i Mapping to a high-dimensional embedding space, we extract deep semantic feature representations for each sample; S32: Prototype-Distance Classification and Prediction; Within the embedding space, the client uses a classification method based on the distance between the category prototype and sample features to predict the category of local samples; this classification process is expressed by the following formula: Where sim() represents the distance between the feature and the prototype; S33: Prototype Anchoring Metric Learning and Optimization; The client calculates the loss function, which includes a prototype anchoring term and a classification error term, constraining the compactness of the distribution of class prototypes and optimizing the classification decision boundary. The local feature extractor and class prototypes are updated through backpropagation. The client calculates the loss using a prototype anchored metric learning method based on the prediction results and dataset labels. The loss function measures the model's prediction accuracy and guides the optimization process. This process includes the following key parts: Cross-entropy loss (CE): During training, the model is optimized by minimizing the cross-entropy loss; the expression for the cross-entropy loss is as follows: Where c represents the index of the target category, c′ represents all possible category indices, and y c It is the true label associated with category c; Prototype Contrast Alignment Loss (PAAC): PAAC encourages the embedded features of samples to align with the prototypes of the correct class and distinguish them from the prototypes of the incorrect class. The expression for PAAC loss is as follows: Among them, s + s represents the similarity between the pixel embedding and the correct category prototype. - This represents the set of similarities to the prototype of the error category, with τ used as a temperature parameter to control the scaling of the similarity. Prototype Center Distance Loss (PCD): By minimizing the distance between the sample embedding and the class prototype, PCD loss clusters samples of the same class more tightly around the prototype; the expression for PCD loss is as follows: By weighting and combining the above three loss functions, a prototype-anchored metric learning method is constructed, the specific formula of which is as follows: Where λ1 and λ2 are hyperparameters that control the loss weights; S34: Update the local feature extractor based on the loss calculated in S33. and prototype The feature extractor is updated via backpropagation, and the prototype update process is as follows: The client first calculates the allocation matrix L, optimizing it to ensure balanced utilization of each prototype, resulting in approximately equal pixel allocations for each prototype. Then, based on the allocation matrix L, it calculates the average embedding feature for each prototype to ensure that each prototype accurately represents the features of its category. Finally, the client updates its local prototypes using the calculated average embedding and adjusts the update step size μ to avoid overfitting. This process is expressed by the following formula: Where m represents the number of pixels and k represents the number of prototypes of each type.
5. The method for target extraction from heterogeneous remote sensing images based on federated learning according to claim 4, characterized in that: Step S4 specifically includes the following steps: S41: The client sends the updated local feature extractor and prototype to the central end; S42: The central end aggregates the local feature extractor and prototype into a global feature extractor and prototype, as shown in the following formula: in This represents the aggregation weight of the i-th client. The value of t on the left side of the formula becomes t+1, indicating that one aggregation has been completed.
6. The method for target extraction from heterogeneous remote sensing images based on federated learning according to claim 1, characterized in that: Step S5 specifically includes the following steps: S51: Convergence determination; During the aggregation process, the central end determines whether the convergence condition is met by monitoring the global model loss value, parameter change rate, or validation set performance change. S52: Final model export and application; if the convergence condition is met, stop model synchronization and iteration, export the aggregated global feature extraction network and category prototype vectors, and apply them to the cross-regional remote sensing image target extraction task.