Cooperative multi-satellite target detection method and system based on federated learning, and storage medium

Abstract

The invention discloses a collaborative multi-satellite target detection method and system based on federated learning and a storage medium, and the method comprises the steps: S1, a plurality of satellites train lightweight detection models according to the local private image data of the satellites, and the local model training comprises heterogeneous data adaptation and local model updating; s2, the ground central server receives model parameters obtained by training each satellite, and performs federal aggregation processing to generate a global model; s3, taking the ground multi-source visual large model as a teacher model, taking the global model as a student model, performing feature layer distillation and output layer distillation to obtain an optimized global model, and further obtaining global parameters; and S4, updating local models of the satellites according to the global parameters, and detecting corresponding image data of the satellites. The embodiment of the invention aims to solve the problems of low model convergence efficiency and large heterogeneous data deviation in federated learning, finally realizes efficient sharing and co-evolution of model parameters among multiple satellites, and improves the generalization ability and detection precision of in-orbit small-model cross-satellite deployment.

Description

Technical Field

[0001] This application relates to a collaborative multi-satellite target detection method, system, and storage medium based on federated learning, belonging to the field of satellite target detection and recognition. Background Technology

[0002] Current satellite-based intelligent interpretation technology primarily relies on single-satellite independent training or centralized ground training. In single-satellite mode, the satellite trains its model based on local data collected by its own sensors. However, the limited data size and insufficient target scene coverage of a single satellite result in poor model generalization performance. Centralized ground training requires transmitting data from multiple satellites back to a ground center for unified training. However, due to national security or commercial privacy concerns, cross-satellite data sharing is strictly limited, resulting in a limited actual usable data scale. Some improvement schemes attempt multi-satellite collaborative optimization using federated learning frameworks. However, traditional federated learning only updates the model through simple parameter aggregation, making it difficult for the model to adapt to the heterogeneous distribution of multi-satellite data, and frequent satellite-to-ground communication leads to resource waste. Under the current technological system, the contradiction between data privacy protection and model performance improvement has not been effectively resolved, and both the efficiency and accuracy of multi-satellite collaborative optimization face significant bottlenecks. Summary of the Invention

[0003] In view of this, this application provides a collaborative multi-satellite target detection method, system and storage medium based on federated learning. The embodiments of this application aim to solve the problems of low model convergence efficiency and large deviation of heterogeneous data in federated learning, and finally realize efficient sharing and collaborative evolution of model parameters among multiple satellites, and improve the generalization ability and detection accuracy of on-orbit small model cross-satellite deployment.

[0004] The first aspect of this application discloses a collaborative multi-satellite target detection method based on federated learning, the method comprising:

[0005] S1 satellites train lightweight detection models based on their respective local private imagery data. Local model training includes heterogeneous data adaptation and local model updates.

[0006] The S2 ground central server receives the model parameters obtained from training on each satellite and performs federated aggregation processing to generate a global model.

[0007] S3 uses a large ground-based multi-source visual model as the teacher model and a global model as the student model. It performs feature layer distillation and output layer distillation to obtain an optimized global model, and then obtains the global parameters.

[0008] S4 updates the local models of each satellite based on global parameters and performs detection on the corresponding image data of each satellite.

[0009] Furthermore, the types of image data include at least one of the following: optical images, SAR images, and infrared images;

[0010] Heterogeneous data adaptation in step S1 includes:

[0011] Based on the satellite payload type, load the corresponding pre-trained feature extractor and freeze the underlying parameters. The feature extractor includes one of the following: MobileNet series, SOCNet and ShuffleNet.

[0012] The local model update in step S1 includes:

[0013] The task output head is trained using a momentum optimizer and combined with focus loss to generate a detection model for local satellite data.

[0014] Furthermore, step S2 includes:

[0015] S21 designs a dynamic weight allocation strategy based on the amount of data from each satellite, model accuracy, and similarity in data distribution.

[0016] S22 employs differential privacy technology, adding Gaussian noise before parameter uploading to ensure that the characteristics of single-satellite data are irreversible.

[0017] S23 aggregates weighted parameters and updates the global model.

[0018] Furthermore, step S21 is as follows:

[0019]

[0020] Where, α s n represents the weight of satellite client s. s Indicates the amount of data, A s H represents the model accuracy. s λ represents the data distribution entropy, λ1, λ2 and λ3 represent the sub-weights of the corresponding parameters, and N represents the total number of satellites participating in federated learning.

[0021] Furthermore, Gaussian noise is added in step S22. Satisfying (∈,σ)-differential privacy:

[0022]

[0023] Where, θ s Let represent the local model parameters, I represent the identity matrix to ensure that noise acts independently on each dimension of the parameters, σ represent the noise standard deviation, Δ represent the sensitivity, ∈ represent the privacy budget, and δ represent the relaxation term probability.

[0024] Step S23 is as follows:

[0025]

[0026] Where t represents the training round.

[0027] Furthermore, in step S3, the feature layer distillation constrains the consistency of the feature map distributions between the student model and the teacher model through mean square error, as shown in the following equation:

[0028]

[0029] in, The loss function represents the feature layer distillation. F represents the expectation of a sample x in a data distribution D, i.e., the average of all samples. S F represents the feature map of the student model. T Feature map representing the teacher model;

[0030] The output layer distillation in step S3 uses KL divergence to align the detection box confidence with the classification probability distribution, as shown in the following formula:

[0031]

[0032] in, Denotes the KL divergence loss function. This represents the expectation calculation for a sample x in the data distribution D, i.e., the average over all samples, where C represents the total number of categories. This represents the teacher model's predicted probability for class C. This represents the predicted probability of the student model for class C.

[0033] Furthermore, a progressive update is adopted, in which the distilled global model parameters are uploaded to each satellite as initialization parameters for the next round of federated training;

[0034] Joint optimization yields the following total loss function:

[0035]

[0036] Where β represents a hyperparameter that adjusts the weights of the loss between the feature layer and the output layer;

[0037] The student parameters are updated as follows:

[0038]

[0039] Among them, the parameter θ′ after fractional distillation global Each satellite will be used for the next round of initialization, where η represents the learning rate. This represents the gradient of distillation loss.

[0040] Furthermore, the method also includes:

[0041] The detection performance of the model on multi-star heterogeneous data was verified by hardware-in-the-loop simulation test. The cross-source generalization ability was evaluated by confusion matrix and mAP index, and the federated aggregation strategy was iteratively optimized.

[0042] The second aspect of this application discloses a collaborative multi-satellite target detection system based on federated learning, the system comprising:

[0043] The first unit is used for multiple satellites to train lightweight detection models based on their respective local private image data. Local model training includes heterogeneous data adaptation and local model updating.

[0044] The second unit is used by the ground central server to receive the model parameters obtained from training on each satellite and perform federated aggregation processing to generate a global model.

[0045] The third unit is used to perform feature layer distillation and output layer distillation using a large ground-based multi-source visual model as the teacher model and a global model as the student model to obtain an optimized global model and then obtain global parameters.

[0046] The fourth unit is used to update the local models of each satellite based on global parameters and to detect the corresponding image data of each satellite.

[0047] A third aspect of this application discloses a computer-readable storage medium comprising a stored program, wherein the program, when running, controls the execution of the cooperative multi-satellite target detection method of the above embodiments in the processor of the device.

[0048] Compared with the prior art, the embodiments of this application have the following beneficial effects:

[0049] This application provides a collaborative multi-satellite target detection method, system, and storage medium based on federated learning. The method includes: S1 Multiple satellites train lightweight detection models based on their respective local private image data; local model training includes heterogeneous data adaptation and local model updating; S2 A ground-based central server receives the model parameters trained by each satellite and performs federated aggregation to generate a global model; S3 Using a large ground-based multi-source visual model as the teacher model and the global model as the student model, feature layer distillation and output layer distillation are performed to obtain an optimized global model, and thus global parameters; S4 The local models of each satellite are updated based on the global parameters, and detection is performed on the corresponding image data of each satellite. Federated learning supervised by a large visual model is introduced, utilizing the knowledge distillation loss of the large model to guide the federated aggregation process, enabling the on-satellite small models to simultaneously optimize task loss and knowledge distillation loss during local training. By fusing multi-satellite model parameters through a dynamic weight allocation strategy, effective cross-satellite knowledge transfer and synergistic improvement of model performance are achieved without transmitting the original data. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating a collaborative multi-satellite target detection method based on federated learning, provided as an embodiment of this application.

[0052] Figure 2 This is a schematic diagram of a collaborative multi-satellite target detection system based on federated learning, provided in an embodiment of this application.

[0053] Figure 3 This is a schematic diagram of a collaborative multi-satellite target detection system based on federated learning, provided as an embodiment of this application. Detailed Implementation

[0054] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0055] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0056] Example 1:

[0057] Figure 1 This is a flowchart illustrating a collaborative multi-satellite target detection method based on federated learning, provided as an embodiment of this application. Figure 1 As shown, the method may include the following steps:

[0058] S1. Satellite-based localized model training. Each satellite (client) trains a lightweight detection model based on its local private data (such as the image set of optical satellite 1 and the image set of SAR satellite N). First, heterogeneous data adaptation is performed. According to the satellite payload type (optical / SAR / infrared), a pre-trained feature extractor is dynamically loaded, and the underlying parameters are frozen to avoid overfitting. Second, local model updates are performed. A momentum optimizer (such as AdamW) is used to train the task output head, and focal loss is combined to solve the class imbalance problem, generating a detection model (satellite model 1N) specific to the satellite's data.

[0059] In this embodiment, image sets from different satellites are preprocessed, including image cropping, scaling, and normalization, to adapt to the model input requirements. Based on the satellite payload type, a feature extractor suitable for that type of data is selected or designed. For example, for optical imagery, a network pre-trained on visible light images might be chosen; for SAR imagery, a network pre-trained on synthetic aperture radar images might be selected; and for infrared imagery, a network pre-trained on infrared images might be selected. A pre-trained feature extractor matching the satellite payload type is then loaded. These pre-trained models have been trained on a large amount of similar data and are therefore able to capture the key features of that type of data.

[0060] The framework of the lightweight detection model can be the YOLO series or the SSD series, etc.; the backbone (feature extractor) can be a lightweight network such as the MobileNet series, SOCNet, ShuffleNet, etc.

[0061] S2 Federation Parameter Aggregation and Global Model Update. The ground-based central server aggregates multi-satellite model parameters to generate a global model. This stage primarily requires differentiated weighted aggregation and privacy protection. A dynamic weight allocation strategy is designed based on the amount of data from each satellite, model accuracy, and data distribution similarity. For example, satellite clients containing rare targets are assigned higher weights. Differential privacy (DP) technology is employed, adding Gaussian noise before parameter upload to ensure that single-satellite data features are irreversibly derived.

[0062] In this embodiment, differential weighted aggregation is used, with weight α. s From the amount of data n s Model accuracy A s (validation set mAP) and data distribution entropy H s Jointly determined:

[0063]

[0064] Where, α s n represents the weight of satellite client s.s Indicates the amount of data, A s H represents the model accuracy. s λ represents the data distribution entropy, λ1, λ2 and λ3 represent the sub-weights of the corresponding parameters, and N represents the total number of satellites participating in federated learning.

[0065] Upload parameters with privacy protection and add Gaussian noise. Satisfying (∈, σ)-differential privacy:

[0066]

[0067] Where, θ s Let represent the local model parameters, I represent the identity matrix, ensuring that noise acts independently on each dimension of the parameters, σ represent the noise standard deviation, Δ represent the sensitivity, ∈ represent the privacy budget, and δ represent the relaxation term probability.

[0068] Global model update, aggregating weighted parameters:

[0069]

[0070] Where t represents the training round.

[0071] Knowledge distillation under the supervision of the S3 large model. A ground-based multi-source visual large model is used as the teacher model T(x; Θ). teacher The federal global model is the student model S(x; θ). global The method employs a two-stage distillation process: feature layer distillation and output layer distillation. Feature layer distillation uses mean squared error (MSE) to constrain the consistency of feature map distributions between the student model and the teacher model, thereby enhancing the cross-source target representation capability.

[0072]

[0073] in, The loss function represents the feature layer distillation. F represents the expectation of a sample x in a data distribution D, i.e., the average of all samples. S F represents the feature map of the student model. T The feature map representing the teacher model.

[0074] Output layer distillation utilizes KL divergence to align the detection box confidence and classification probability distribution, thereby improving the ability of small models to distinguish difficult samples.

[0075]

[0076] in, Denotes the KL divergence loss function. This represents the expectation calculation for a sample x in the data distribution D, i.e., the average over all samples, where C represents the total number of categories. This represents the teacher model's predicted probability for class C. This represents the predicted probability of the student model for class C.

[0077] Furthermore, a progressive update is adopted, uploading the distilled global model parameters to each satellite as initialization parameters for the next round of federated training, forming a closed loop of "local training - federated aggregation - distillation optimization". Joint optimization, total loss function:

[0078]

[0079] Where β represents a hyperparameter that adjusts the weights of the loss between the feature layer and the output layer.

[0080] Update student parameters:

[0081]

[0082] Among them, the parameter θ′ after fractional distillation global Each satellite will be used for the next round of initialization, where η represents the learning rate.

[0083] This represents the gradient of distillation loss.

[0084] It should be noted that a ground-based multi-source visual large model refers to a large visual model that integrates multiple data sources (such as optical imagery, radar imagery, and geographic information data) for processing and analyzing ground-related visual information. Below are some typical ground-based multi-source visual large models and their applications:

[0085] 1.LandGPT

[0086] LandGPT is a multimodal parcel-level land use classification big language model trained on large-scale land use datasets (such as CN-MSLU-100K) and incorporating multi-source data including remote sensing imagery and POIs. It employs a "visual encoder (ViT) - feedforward neural network layer (MLP) - big language model (LLM)" architecture, significantly improving the accuracy of land use classification, particularly excelling in discriminating between complex scenarios such as commercial and public service facilities.

[0087] 2. RingMo

[0088] RingMo is the first generative pre-trained large-scale model for cross-modal remote sensing data, developed under the leadership of the Aerospace Information Research Institute of the Chinese Academy of Sciences. It aims to construct a general multimodal, multi-task model to provide solutions for various industry applications in the remote sensing field. It employs an unsupervised pre-training method, using either a Vision Transformer or a Swing Transformer as the encoder. It utilizes the Patch Incomplete Mask (PIMask) strategy to preserve remote sensing targets and calculates the loss by measuring the L1 distance between the reconstructed image and the original image, significantly improving the performance of tasks such as scene classification, object detection, semantic segmentation, and change detection.

[0089] 3. SkySense

[0090] SkySense is a large-scale multimodal remote sensing model with 1 billion parameters. It constructs a broad multimodal remote sensing dataset containing time series data, covering scenarios with different resolutions, spectra, and imaging mechanisms. Its architecture includes a spatial feature extraction module, a multimodal temporal fusion module, and a geocontext encoding module. It is trained using multi-granularity contrastive learning and geocontext prototype learning methods, progressively learning spatial features through pixel-level, target-level, and image-level contrastive learning, and utilizing prototype sets to represent the geocontext. This model is suitable for tasks affected by seasonality, such as crop identification, as well as remote sensing image analysis tasks requiring geocontextual information.

[0091] 4. DynamicViT

[0092] DynamicViT is a high-efficiency visual Transformer model that uses a standard visual Transformer as its backbone. Its prediction module generates probabilities for deleting / keeping tokens, and the tokens are hierarchically sparsified to reduce computational cost and inference time. During training, the prediction module and backbone network can be optimized end-to-end; during inference, the token with the most information is selected based on a predefined pruning ratio and prediction score. This model is suitable for visual tasks such as image classification, and performs particularly well in scenarios requiring efficient processing of large-scale image data.

[0093] 5. Purple East Primordial Beginning

[0094] Zidong Taichu is the world's first multimodal large-scale model with hundreds of billions of parameters, breaking through the self-supervised learning technology for cross-modal multi-task applications. It can achieve unified representation and mutual generation of multimodal data, and is suitable for various remote sensing tasks, such as land cover classification and change detection, demonstrating powerful multimodal fusion capabilities.

[0095] S4 updates the local models of each satellite based on global parameters and performs detection on the corresponding image data of each satellite.

[0096] Satellite model detection tasks mainly include target detection (identifying and locating specific targets, such as buildings, vehicles, and ships), anomaly detection (identifying areas that do not conform to the normal pattern), and target counting (counting the number of specific targets). These tasks are widely used in urban planning, environmental monitoring, disaster assessment, and other fields, providing important support for resource management and decision-making.

[0097] In some embodiments, the method further includes:

[0098] The detection performance of the model on multi-star heterogeneous data was verified by hardware-in-the-loop simulation test. The cross-source generalization ability was evaluated by confusion matrix and mAP index, and the federated aggregation strategy was iteratively optimized.

[0099] In this embodiment, the performance differences of the model across different data sources are analyzed based on the confusion matrix and mAP metric. The federated aggregation strategy is adjusted, such as adjusting the weights of each node, to optimize the model's cross-source generalization ability.

[0100] It should be noted that hardware-in-the-loop (HIL) simulation testing is a testing method that combines real physical components with a virtual simulation environment to verify the performance of a model in simulated real-world scenarios. By combining real data with virtual scenes, it can simulate various complex conditions in a safe and low-cost environment, thereby comprehensively evaluating the model's robustness, generalization ability, and performance metrics (such as confusion matrix, mAP, etc.), and supporting rapid iterative optimization.

[0101] Figure 2 This is a schematic diagram illustrating a scenario of a collaborative multi-satellite target detection system based on federated learning, provided as an embodiment of this application. Figure 2 As shown, this scenario illustrates a federated learning system for remote sensing image data processing, involving datasets collected by multiple satellites (optical satellite 1, infrared satellite 2, ..., SAR satellite N). Data collected by each satellite is used to train a corresponding local model (satellite model 1 to N). The parameters of these local models are then uploaded to a central server. The central server uses a federated algorithm to aggregate these parameters to build or update a global model. In this system, the large remote sensing visual model serves as the teacher model, while the global model acts as the student model. Knowledge distillation is used to transfer knowledge from the large remote sensing visual model to the global model. The updated global model's parameters are then distributed to each satellite, which uses its own small model and corresponding image data to perform target detection and feeds the detection results back to the ground user via a data transmission system. This global model integrates the knowledge from all satellite models, enabling more accurate analysis and interpretation of remote sensing data from different satellites while protecting data privacy because the original data does not need to be shared. The entire process aims to improve the model's generalization ability, allowing it to perform well in different remote sensing tasks.

[0102] Example 2:

[0103] Figure 3 This is a schematic diagram of a collaborative multi-satellite target detection system based on federated learning, provided as an embodiment of this application. Figure 3 As shown, the system may include the following units:

[0104] Unit 301 is used for multiple satellites to train lightweight detection models based on their respective local private image data. Local model training includes heterogeneous data adaptation and local model updating.

[0105] The second unit 302 is used by the ground central server to receive the model parameters obtained from training on each satellite and perform federated aggregation processing to generate a global model.

[0106] Unit 303 is used to perform feature layer distillation and output layer distillation using a large ground-based multi-source visual model as the teacher model and a global model as the student model to obtain an optimized global model and then obtain global parameters.

[0107] Unit 4, 304, is used to update the local models of each satellite based on global parameters and to detect the corresponding image data of each satellite.

[0108] Example 3:

[0109] Embodiments of this application also provide an electronic device, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of the present invention during runtime.

[0110] The aforementioned memory can refer to devices inside a computer used to store data and programs, including RAM, hard disks, etc. RAM can be used to temporarily store running programs and data, while hard disks can be used to store programs and data long-term. Memory enables the computer to read and write data and execute programs. The aforementioned processor is responsible for executing instructions in computer programs and performing data processing. It can also be responsible for controlling and executing various operations, including arithmetic operations, logical operations, and data transmission.

[0111] Example 4:

[0112] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of the present invention.

[0113] The aforementioned computer storage media can refer to the media used in computer memory to store certain discontinuous physical quantities. Computer storage media mainly include semiconductors, magnetic cores, magnetic drums, magnetic tapes, laser discs, etc. Computer-readable storage media include stored programs, which can be a set of instructions that a computer can recognize and execute, running on an electronic computer to meet certain information needs.

[0114] Example 5:

[0115] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0116] The aforementioned computer program products can refer to software programs that have been written, tested, and released, and can run on computers or other devices. Computer program products can include application programs, operating systems, utility software, etc., used to achieve specific functions or solve specific problems.

[0117] Example 6:

[0118] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods in various embodiments of the present invention.

[0119] The aforementioned non-volatile computer-readable storage medium can refer to a medium for storing data. Non-volatile computer-readable storage media can retain data without loss when power is off and can be used to store long-term data, such as operating systems, applications, and user files. Non-volatile storage media can include hard disk drives, solid-state drives, optical disks, and flash memory storage devices, etc.

[0120] Example 7:

[0121] Embodiments of this application also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of the present invention.

[0122] The aforementioned computer program can refer to a set of instructions used to tell the computer to perform specific tasks or operations. Computer programs can be written by programmers using specific programming languages ​​and can include algorithms, data structures, logic, and control flow. Computer programs can be used for a variety of purposes, including application software, operating systems, etc.

[0123] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0124] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0125] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0126] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0127] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0128] In summary, federated learning supervised by a large visual model utilizes the knowledge distillation loss of the large model to guide the federated aggregation process, enabling small on-satellite models to simultaneously optimize both task loss and knowledge distillation loss during local training. By fusing parameters from multiple satellite models through a dynamic weight allocation strategy, effective cross-satellite knowledge transfer and synergistic performance improvement are achieved while avoiding the transmission of original data.

[0129] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A collaborative multi-satellite target detection method based on federated learning, characterized in that, include: S1 satellites train lightweight detection models based on their respective local private imagery data. Local model training includes heterogeneous data adaptation and local model updates. The S2 ground central server receives the model parameters obtained from training on each satellite and performs federated aggregation processing to generate a global model. S3 uses a large ground-based multi-source visual model as the teacher model and a global model as the student model. It performs feature layer distillation and output layer distillation to obtain an optimized global model, and then obtains the global parameters. S4 updates the local models of each satellite based on global parameters and performs detection on the corresponding image data of each satellite.

2. The collaborative multi-satellite target detection method according to claim 1, characterized in that, Image data types include at least one of the following: optical images, SAR images, and infrared images; Heterogeneous data adaptation in step S1 includes: Based on the satellite payload type, load the corresponding pre-trained feature extractor and freeze the underlying parameters. The feature extractor includes one of the following: MobileNet series, SOCNet and ShuffleNet. The local model update in step S1 includes: The task output head is trained using a momentum optimizer and combined with focus loss to generate a detection model for local satellite data.

3. The collaborative multi-satellite target detection method according to claim 1, characterized in that, Step S2 includes: S21 designs a dynamic weight allocation strategy based on the amount of data from each satellite, model accuracy, and similarity in data distribution. S22 employs differential privacy technology, adding Gaussian noise before parameter uploading to ensure that the characteristics of single-satellite data are irreversible. S23 aggregates weighted parameters and updates the global model.

4. The collaborative multi-satellite target detection method according to claim 3, characterized in that, Step S21 is as follows: Where, α s n represents the weight of satellite client s. s Indicates the amount of data, A s H represents the model accuracy. s λ represents the data distribution entropy, λ1, λ2 and λ3 represent the sub-weights of the corresponding parameters, and N represents the total number of satellites participating in federated learning.

5. The collaborative multi-satellite target detection method according to claim 4, characterized in that, Add Gaussian noise in step S22 Satisfying (∈, σ)-differential privacy: Where, θ s Let represent the local model parameters, I represent the identity matrix to ensure that noise acts independently on each dimension of the parameters, σ represent the noise standard deviation, Δ represent the sensitivity, ∈ represent the privacy budget, and δ represent the relaxation term probability. Step S23 is as follows: Where t represents the training round.

6. The collaborative multi-satellite target detection method according to claim 1, characterized in that, In step S3, feature layer distillation constrains the consistency of feature map distributions between the student model and the teacher model through mean square error, as shown in the following equation: in, The loss function represents the feature layer distillation. F represents the expectation of a sample x in a data distribution D, i.e., the average of all samples. S F represents the feature map of the student model. T Feature map representing the teacher model; The output layer distillation in step S3 uses KL divergence to align the detection box confidence with the classification probability distribution, as shown in the following formula: in, Denotes the KL divergence loss function. This represents the expectation calculation for a sample x in the data distribution D, i.e., the average over all samples, where C represents the total number of categories. This represents the teacher model's predicted probability for class C. This represents the predicted probability of the student model for class C.

7. The collaborative multi-satellite target detection method according to claim 6, characterized in that, A progressive update method is adopted, in which the distilled global model parameters are uploaded to each satellite as initialization parameters for the next round of federated training. Joint optimization yields the following total loss function: Where β represents a hyperparameter that adjusts the weights of the loss between the feature layer and the output layer; The student parameters are updated as follows: Among them, the parameter θ′ after fractional distillation global Each satellite will be used for the next round of initialization, where η represents the learning rate. This represents the gradient of distillation loss.

8. The collaborative multi-satellite target detection method according to claim 1, characterized in that, The method also includes: The detection performance of the model on multi-star heterogeneous data was verified by hardware-in-the-loop simulation test. The cross-source generalization ability was evaluated by confusion matrix and mAP index, and the federated aggregation strategy was iteratively optimized.

9. A collaborative multi-satellite target detection system based on federated learning, characterized in that, include: The first unit is used for multiple satellites to train lightweight detection models based on their respective local private image data. Local model training includes heterogeneous data adaptation and local model updating. The second unit is used by the ground central server to receive the model parameters obtained from training on each satellite and perform federated aggregation processing to generate a global model. The third unit is used to perform feature layer distillation and output layer distillation using a large ground-based multi-source visual model as the teacher model and a global model as the student model to obtain an optimized global model and then obtain global parameters. The fourth unit is used to update the local models of each satellite based on global parameters and to detect the corresponding image data of each satellite.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the execution of the cooperative multi-satellite target detection method according to any one of claims 1 to 8 in the processor of the device.

Images