A secure distributed remote sensing data diffusion generation method based on federal multi-source information fusion

Abstract

This invention discloses a secure distributed remote sensing data diffusion generation method based on federated multi-source information fusion. The invention employs Tucker-2 reparameterization for lightweight compression of the U-Net backbone while preserving its structure. Combined with a dynamic low-rank adaptation mechanism where each client adaptively selects its rank based on its own resource budget, this allows remote sensing clients with varying computing power, memory, and energy consumption levels to effectively participate in federated diffusion training. This avoids the problem of existing methods losing multi-source observation information due to the exclusion of weaker devices. Furthermore, the low-rank adapter is decomposed into semantic factors and complementary factors. Only semantic factors representing transferable common denoising knowledge are aggregated, while complementary factors carrying specific visual styles of the sources are retained locally. This fundamentally prevents local overfitting and semantic bias from polluting the global model through aggregation. Compared to directly aggregating all factors under the same communication budget, this method achieves higher generation quality.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing data analysis technology, specifically a secure distributed remote sensing data diffusion generation method adapted to heterogeneous edge devices and based on federated multi-source information fusion. Background Technology

[0002] In the first part, high-fidelity remote sensing (RS) image generation is of great significance for Earth observation, disaster assessment, land cover monitoring, data augmentation, and large-scale scene understanding

[10] . In practical remote sensing applications, useful generative knowledge does not come from a single data source, but is distributed among satellites, UAV platforms, ground stations, sensors, and different institutions. These sources provide complementary visual observations, semantic descriptions, spatial structures, and scene-level contextual information. Therefore, remote sensing image generation is closely related to the core goal of information fusion, namely, integrating complementary knowledge from multiple sources to obtain more reliable and semantically consistent representations of complex scenes. In recent years, the stable diffusion (SD) model has shown great potential in remote sensing image generation because its potential spatial denoising process can model fine-grained spatial patterns while maintaining semantic consistency between text and images [33,45]. However, most current SD-based remote sensing image generation methods still rely on centralized training, which requires unifying and uploading multi-source remote sensing data to a central server. This assumption is often difficult to achieve due to factors such as data ownership, privacy protection, communication costs, and security restrictions

[40] . Therefore, a distributed and privacy-preserving information fusion paradigm is needed to generate multi-source remote sensing images.

[0003] However, the main drawback of existing federated diffusion generation methods is that they fail to fully consider the inconsistency in denoising updates caused by heterogeneous devices, and they mix and aggregate globally shared semantics with source-specific adaptations, lacking protection for the diffusion backbone structure, which leads to semantic drift and image distortion during multi-source semantic fusion. Summary of the Invention

[0004] The purpose of this invention is to address four pain points in existing technologies: 1) resolving privacy leaks in centralized remote sensing diffusion training data and compliance issues in cross-institutional data aggregation; 2) addressing the problems of heterogeneous memory and computing power in federated clients, resulting in weaker devices being unable to participate and the loss of multi-source observation data; 3) resolving the problem of traditional federated aggregation confusing global semantics with local sensor features, leading to semantic drift and structural distortion in generated images; 4) resolving the problem of inconsistent denoising representations caused by different low-rank adaptation ranks, resulting in unstable training and slow convergence speed; The invention provides the FedSD-LRF heterogeneous device adaptive federated diffusion framework, which enables the fusion and generation of multi-source remote sensing information without sharing the original remote sensing data.

[0005] The technical solution adopted in this invention is as follows: A secure distributed remote sensing data diffusion generation method based on federated multi-source information fusion, characterized in that: the method includes the following steps:

[0006] S1: The server performs Tucker-2 second-order tensor reparameterization on all 2D convolutional layers and attention projection layers of U-Net: compressing the number of model parameters along the channel dimension, fully preserving the original spatial convolutional kernel size, skip connections, and network input / output interfaces; initializing the tensor factor matrix using HOSVD singular value decomposition, minimizing the F-norm of the convolutional kernel to reconstruct the error, and generating a lightweight unified U-Net backbone for distribution to all clients; replacing the decomposed convolutions with a three-layer concatenated structure of a 1×1 channel compression layer, a kh×kw core convolutional layer, and a 1×1 channel expansion layer;

[0007] S2: Each client independently selects a low-rank adaptation rank r based on its own memory, computing power, and energy consumption resource thresholds, and attaches a dynamic DyLoRA low-rank adapter to the lightweight U-Net convolution and attention modules; the adapter incremental update parameters are split into semantic factor A and local complementary factor B, B is initialized as a zero matrix, and A follows a zero-mean normal distribution, and the lightweight U-Net backbone weights are frozen throughout the training process;

[0008] S3: The client-side local training constructs a composite total loss function, which integrates five constraints: standard diffusion denoising task loss, network alignment KL structure preservation loss before and after Tucker decomposition, noise prediction matching loss, cross-rank consistency loss, and low-rank factor weight decay loss. Among them, the cross-rank consistency loss ensures that the intermediate denoising representation of U-Net remains consistent under different rank configurations. For high-resolution attention feature maps with a size ≥16×16, the K / V matrix is ​​downsampled using depthwise separable convolution with a stride of 2 to reduce peak memory usage.

[0009] S4: After a single round of local training, the client only uploads the semantic factor set ΘA corresponding to all modules to the federated server, which contains the complementary factor set of sensor-specific textures and local scene biases. Permanently stored locally without being transferred;

[0010] S5: The server uses the ratio of each client's private samples to the total samples as a weighting coefficient to perform a weighted average aggregation of all uploaded semantic factors A in this round, generating a globally shared semantic factor. ;

[0011] S6: The server distributes the aggregated global semantic factors to all clients. The clients synchronize their local A factors and execute S2-S5 in a loop until the global total loss fluctuation is less than the preset threshold or the maximum number of communication rounds is reached. During the inference phase, the clients fuse the local B factors and the global A factors, input the lightweight U-Net, and generate high-fidelity multi-source remote sensing images.

[0012] In a preferred embodiment, in step S1, a federated central server and several remote sensing edge clients are deployed; the server loads a pre-trained stable diffusion model with parameter θ as the generation backbone, the model denoising network adopts the U-Net architecture, each client independently stores private remote sensing image-text pairs locally, federated communication only exchanges model parameters, the original remote sensing images and text are not transmitted throughout the process, and a privacy-preserving multi-source collaboration foundation is established.

[0013] In a preferred embodiment, during step S2, the Tucker-2 decomposition of the U-Net convolutional layers is costly for resource-constrained clients because directly fine-tuning the complete U-Net is too expensive. This application reparameterizes the convolutional kernels through Tucker-2 decomposition; this design compresses inter-channel interactions while maintaining the spatial convolutional structure.

[0014] This is crucial for maintaining RS layout and fine-grained texture during diffusion generation;

[0015] For each convolution kernel Tucker-2 decomposition provides a compact and server-aggregable parameterization method while maintaining the original U-Net topology:

[0016] (5)

[0017] in Represents the core tensor. and These are the factor matrices for the output and input channels, respectively; while keeping the spatial convolution kernel and skip connection structure unchanged, the original convolution is replaced by a... Channel compression, one The core convolution and a The channel expansion allows for a reduction in model capacity across the channel dimension, while the spatial convolution kernel and the input / output interface of U-Net remain unchanged. The specific calculation formula is as follows:

[0018] (6) For an RS image-text pair The VAE encoder maps an image to a latent representation. .

[0019] In a preferred embodiment, in step S2, the forward diffusion process is scheduled according to noise. right The disturbances include:

[0020] set up The weight tensor of the two-dimensional convolutional layer in U-Net; introduced by Tucker-2 decomposition. , and , making

[0021] (7)

[0022] During the process of compressing model capacity along the channel dimension, the input feature map The decomposed convolution is represented as:

[0023] (8)

[0024] The number of parameters from Reduce to ;

[0025] Decrease and It can reduce dominant FLOPs and active memory, allowing clients with different computing budgets to maintain the same U-Net topology while enabling lighter parameterization;

[0026] To ensure accurate approximation of the original convolutional kernel, this application minimizes the reconstruction error:

[0027] (9);

[0028] First, HOSVD initialization is performed on the channel mode of matrix $W$. and Then G is calculated; this initialization method reduces the bias between the original kernel and the decomposed kernel before federated adaptation.

[0029] In a preferred embodiment, in step S3, after obtaining the decomposed U-Net during the process of attaching a low-rank adapter to the backbone, FedSD-LRF introduces a low-rank adapter to avoid updating the complete backbone network on each client; for a fixed mapping matrix The trainable updates are parameterized as follows:

[0030] (15)

[0031] The same structural design was also applied to the Tucker-2 component. and and attention projection matrix , and ;

[0032] During initialization, B is set to zero, and A is randomly sampled. Since only the low-rank factor is optimized, the size of the trainable parameters and the main computational load are reduced from... Reduce to , and rank This can be adjusted based on the resource budget of each client; this application is initialized. and This allows the system to maintain a low-rank update to zero in the initial stage, preserving the diffusion behavior of the pre-trained system.

[0033] In a preferred embodiment, in step S3, in a multi-source remote sensing scenario, directly aggregating all adaptation parameters may entangle globally transferable semantics with source-specific differences; therefore, FedSD-LRF adopts a selective semantic factor fusion strategy: each client updates A and B simultaneously during local optimization, but only uploads semantic factor A to the server, while retaining complementary factor B locally, thereby preserving the unique visual style, sensor appearance, and image-text alignment features of each client.

[0034] In a preferred embodiment, in step S4, the trunk parameters are given during the freezing process. The client, of which the first Each client has private image-text data. The federal proliferation objective can be expressed as:

[0035] (10)

[0036] in Indicates the sample proportion.

[0037] In a preferred embodiment, in step S4, during the structure-preserving regularization process, since the decomposed U-Net may amplify the approximation error during back-diffusion, this application further constrains the decomposed denoising network to maintain its closeness to the original network; assuming and These represent the noise predictors before and after Tucker-2 reparameterization; the noise level alignment loss is...

[0038] (11)

[0039] The one-step reverse transition is also regularized in the following ways:

[0040] (12)

[0041] in and These are the inverse distributions induced by the original denoising network and the decomposed denoising network, respectively.

[0042] Together with standard diffusion loss

[0043] (13)

[0044] The structural preservation target is defined as

[0045] (14)

[0046] in and Used to balance task learning and diffusion consistency preservation;

[0047] In the process of cross-rank consistency constraints, set Represents the set of optional levels. In order to be at the level The intermediate representation below; consistency loss is defined as:

[0048] (19)

[0049] In ensuring decomposition accuracy and semantic alignment under heterogeneous ranks, the training process incorporates diffusion objectives, structure preservation regularization, cross-rank consistency, and weight decay on low-rank factors. The local training objective is as follows:

[0050] (20)

[0051] (twenty one)

[0052] in It is the regularization coefficient;

[0053] In the In each communication round, the client Choose a feasible rank based on its current memory and computing budget. Since all clients share the same decomposed U-Net interface, rank switching does not change the global synchronization protocol; furthermore, cross-rank consistency loss prevents capacity adaptation from disrupting the semantic alignment required for federated diffusion training.

[0054] For high-resolution attention layers, when the feature map size exceeds 16×16, this application further reduces activation overhead by applying depthwise separable convolutions with a stride of 2 on K and V; the attention response is computed in the downsampling space and then mapped back to the original resolution through the residual branch; this operation reduces the memory peak of the attention while preserving the main semantic response.

[0055] In a preferred embodiment, in step S5, based on the proportion of sample size for each client during the decomposition of the U-Net, the uploaded semantic factor is...

[0056] (16)

[0057] superscript This represents the client index; the corresponding local factor is not transmitted, and the specific calculation formula is as follows:

[0058] (17).

[0059] In a preferred embodiment, in step S6, during the process of weighting all uploaded A according to the sample size ratio of each client, the process is carried out in the first step. In round-robin communication, given a set of participating clients The server aggregates uploaded semantic factors in the following ways:

[0060] (18)

[0061] After aggregation In the next round, the data is redistributed to the client, thus forming a shared semantic fusion subspace among the distributed RS data sources.

[0062] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0063] 1. In this invention, the U-Net backbone is compressed in a lightweight manner while preserving its structure through Tucker-2 reparameterization. Combined with a dynamic low-rank adaptation mechanism in which each client adaptively selects the adaptation rank based on its own resource budget, remote sensing clients with different computing power, memory and energy consumption levels can effectively participate in federated diffusion training. This avoids the problem of losing multi-source observation information due to the exclusion of weak devices in existing methods. The image fidelity and structural consistency indicators on multiple remote sensing datasets are significantly better than existing heterogeneous federated learning methods.

[0064] By decomposing the low-rank adapter into semantic factors and complementary factors, only semantic factors representing transferable common denoising knowledge are aggregated, while complementary factors carrying source-specific visual styles are retained locally. This fundamentally prevents local overfitting and scene bias. By polluting the global model through aggregation, higher generation quality is achieved compared to directly aggregating all factors under the same communication budget.

[0065] 2. In this invention, the inconsistency in denoised representation caused by different clients using different adaptation ranks due to resource differences is addressed by introducing cross-rank consistency loss to force intermediate denoised semantic representations under different ranks to remain aligned. This effectively prevents the cumulative propagation of local prediction errors in the backdiffusion chain, resulting in a more stable training process and faster convergence. Furthermore, this invention transmits only semantic factors rather than all adaptation parameters during federated synchronization, significantly reducing communication bandwidth consumption. Simultaneously, the original remote sensing data of each client remains locally without needing to be shared. Semantic alignment and perceptual naturalness metrics on multiple datasets are superior to existing federated diffusion methods.

[0066] 3. In this invention, the convergence under non-convex and strongly convex objective function conditions is theoretically analyzed. It is proved that the training process converges at a standard rate to a controllable neighborhood near the optimal value. The convergence radius has a clear negative correlation with the adapter rank, cross-rank consistency coefficient, and client participation ratio, providing a theoretical basis for parameter configuration in practical deployment. It achieves consistent performance leadership on various datasets, including disaster change scenarios, multi-scene land cover classification, and fine-grained remote sensing description. The generated images outperform baseline methods in terms of scene layout preservation, object arrangement coherence, and disaster semantic reflection, demonstrating good generalization ability and practical deployment value. Attached Figure Description

[0067] Figure 1 This is a schematic diagram of the overall federated framework of FedSD-LRF in this invention;

[0068] Figure 2 This is a schematic diagram illustrating the complete technical chain of the federated diffusion generation method in this invention;

[0069] Figure 3 This is a schematic diagram of a local structure in this invention that maintains reparameterization and low-rank adaptation.

[0070] Figure 4 This is a visual comparison of the generated images of the present invention and the comparison method on a conventional remote sensing dataset;

[0071] Figure 5 This is a visual comparison of the generated images of the present invention and the comparison method in a disaster change scenario;

[0072] Figure 6 The graph shows a comparison of the accuracy, semantic alignment score, and training loss curves of the present invention and the comparative method under different communication rounds.

[0073] Figure 7 This is a hyperparameter sensitivity analysis diagram of the communication rounds and adapter rank range in this invention;

[0074] Figure 8This is a comparison chart showing the impact of different aggregation strategies on visual semantic quality in this invention;

[0075] Figure 9 This is a robustness analysis diagram of the present invention under different degrees of device heterogeneity;

[0076] Figure 10 This is a comparison chart of the communication, parameters, memory, and time overhead between the present invention and the comparative method;

[0077] Figure 11 This is a schematic diagram of the FedSD-LRF training algorithm flow of the present invention. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0079] Example:

[0080] Reference Figures 1-11 A secure distributed remote sensing data diffusion generation method based on federated multi-source information fusion is proposed. Specifically, it includes the following steps:

[0081] S1. Establish a stable diffusion model for remote sensing image generation:

[0082] Stable Diffusion (SD) provides a latent spatial generation paradigm for text-guided remote sensing image generation, enabling the joint modeling of heterogeneous visual observations and semantic descriptions through conditional denoising. This feature is particularly useful in the fusion of multi-source remote sensing information, as remote sensing scenes often contain complex spatial layouts, cross-sensor appearance differences, and fine-grained semantic relationships. Specifically, during forward diffusion, the latent representation encoded by the VAE... It will gradually be disturbed into time steps Noise potential state Given semantic conditions Denoising network The network is trained to estimate injected noise, thereby recovering a clean latent representation through iterative backdiffusion. Denoising networks typically employ a U-Net architecture, capturing multi-scale spatial features through multiple layers of convolutions and skip connections.

[0083] This potential form of denoising is crucial for remote sensing image synthesis because it preserves both the overall layout and fine-grained spatial details. The diffusion training objective is a noise regression loss:

[0084] (1)

[0085] Maintaining semantic consistency in the denoising step is crucial for the structural integrity of remote sensing images, because small prediction biases can be repeatedly propagated in the reverse process, eventually leading to semantic drift or spatial distortion in the generated image.

[0086] S2. Conduct federal stable diffusion:

[0087] In decentralized remote sensing (RS) systems, image-text data is distributed across multiple clients, such as satellite centers, unmanned aerial vehicle platforms, ground stations, and research institutions. Let... Denotes the private dataset of client i, where It is a remote sensing image. It is its semantic condition. The goal of federated semantic denoising is to learn a shared denoising model without directly accessing any local data.

[0088] Based on the framework of the latent diffusion model, each client independently optimizes the denoising network on its local data. The local diffusion target can be represented as:

[0089] (2)

[0090] in It is a time step Noisy latent variables at the location, Indicates semantic conditions. For denoising networks. The global federated diffusion objective is defined as:

[0091] (3)

[0092] In each round of communication In the middle, the server will display the current global parameters. Broadcast to a selected group of clients Each selected client performs local diffusion adaptation and receives the updated parameters. Subsequently, the server updates the global model using weighted aggregation:

[0093] (4)

[0094] This process achieves privacy-preserving remote sensing image generation by fusing distributed visual and semantic knowledge through model updates rather than raw data sharing. However, when client computational budgets, memory capacities, and data distributions differ, their local denoising updates may exhibit varying optimization quality and representation capabilities. Directly aggregating these updates can introduce semantic inconsistencies during the diffusion process, prompting FedSD-LRF's design to prioritize structure preservation and device adaptation.

[0095] S3. Maintain U-Net architecture reparameterization:

[0096] Building upon the Federal SD pipeline proposed in Step 1, this application proposes FedSD-LRF to support multi-source remote sensing diffusion generation under heterogeneous device capabilities. For example... Figure 2 As shown, the overall framework consists of distributed remote sensing clients and a central federated server. Each client holds private image and text data collected from different sensors, platforms, or institutions, and performs local diffusion model adaptation without exposing the original observation data. The shared SD model follows a standard latent denoising process, where U-Net progressively optimizes noisy latent variables under semantic guidance.

[0097] To enable federated diffusion for clients with varying resource budgets, FedSD-LRF employs a structure-preserving Tucker-2 decomposition to reparameterize SDU-Net. The original convolutional and attention-related modules are transformed into lightweight decomposition components, preserving the spatial topology and denoising interface of the backbone network. A DyLoRA adapter is introduced on top of these decomposition modules to provide local updates of tunable rank, allowing clients with different memory and computational budgets to select an appropriate fitness rank during training.

[0098] During federated synchronization, FedSD-LRF does not upload all local adaptation parameters, but only semantic factors. The data is transmitted to the server and selectively aggregated to construct a shared cross-client semantic subspace; while complementary factors... This is then retained locally on each client to preserve its visual and semantic features. The server processes data from different decomposition components (such as...). and The uploaded semantic factors are aggregated, and the updated global factors are redistributed in the next round of communication. In this way, FedSD-LRF achieves privacy-preserving multi-source semantic fusion while reducing communication and computational overhead.

[0099] Overall, FedSD-LRF constructs a unified federated generation framework by combining structure-preserving U-Net reparameterization, rank-adaptive local diffusion optimization, and selective semantic factor aggregation. This design enables heterogeneous remote sensing clients to participate in collaborative diffusion training while maintaining stable denoising semantics and source-specific adaptability.

[0100] FedSD-LRF requires a lightweight diffusion backbone network that can be shared by heterogeneous clients without disrupting the denoising topology of the original SD U-Net. Since directly fine-tuning the entire U-Net is costly for resource-constrained clients, this application reparameterizes its convolutional kernels using Tucker-2 decomposition. This design compresses inter-channel interactions while maintaining the spatial convolutional structure.

[0101] This is crucial for maintaining RS layout and fine-grained texture during diffusion generation.

[0102] For each convolution kernel Tucker-2 decomposition provides a compact and server-aggregable parameterization method while preserving the original U-Net topology:

[0103] (5)

[0104] in Represents the core tensor. and These are the factor matrices for the output and input channels, respectively. In the implementation, the original convolution is replaced by a... Channel compression, one The core convolution and a The channel expansion allows for a reduction in model capacity across the channel dimension, while the spatial convolution kernel and the U-Net's input / output interface remain unchanged.

[0105] (6)

[0106] For an RS image-text pair The VAE encoder maps an image to a latent representation. The forward diffusion process is scheduled according to noise. right Perform a disturbance:

[0107] set up This refers to the weight tensor of the two-dimensional convolutional layers in U-Net. Tucker-2 decomposition introduces this. , and , making

[0108] (7)

[0109] For the input feature map Decomposing the convolution can be represented as:

[0110] (8)

[0111] The number of parameters from Reduce to Reduce and It can reduce dominant FLOPs and active memory, allowing clients with different computing budgets to maintain the same U-Net topology while enabling lighter parameterization.

[0112] To ensure accurate approximation of the original convolutional kernel, this application minimizes the reconstruction error:

[0113] (9)

[0114] In practice, the first step is to initialize the channel mode of matrix $W$ using HOSVD. and Then calculate This initialization method reduces the bias between the original kernel and the decomposed kernel before federated adaptation. Given... The client, of which the first Each client has private image-text data. The federal proliferation objective can be expressed as:

[0115] (10)

[0116] in Indicates the sample proportion.

[0117] Since the decomposed U-Net may amplify the approximation error during back-diffusion, this application further constrains the decomposed denoising network to maintain its closeness to the original network. Let... and Let represent the noise predictor before and after Tucker-2 reparameterization, respectively. The noise level alignment loss is...

[0118] (11)

[0119] The one-step reverse transition is also regularized in the following ways:

[0120] (12)

[0121] in and These are the inverse distributions induced by the original denoising network and the decomposed denoising network, respectively.

[0122] Together with standard diffusion loss

[0123] (13)

[0124] The structural preservation target is defined as

[0125] (14)

[0126] in and Used to balance task learning and diffusion consistency preservation.

[0127] S4. Selective semantic factor fusion:

[0128] After obtaining the decomposed U-Net, FedSD-LRF introduces a low-rank adapter to avoid updating the complete backbone network at each client. For a fixed mapping matrix... Trainable updates are parameterized as

[0129] (15)

[0130] The same structural design was also applied to the Tucker-2 component. and and attention projection matrix , and Since only low-rank factors are optimized, the size of trainable parameters and the main computational cost are reduced from... Reduce to , and rank This can be adjusted based on each client's resource budget. This application initialization... and This allows the system to maintain a low-rank update to zero in the initial stage, preserving the diffusion behavior of the pre-trained system.

[0131] In multi-source remote sensing scenarios, directly aggregating all adaptation parameters may entangle globally transferable semantics with source-specific differences. Therefore, FedSD-LRF employs a selective semantic factor fusion strategy: each client updates simultaneously during local optimization. and However, only semantic factors Uploaded to the server, and complementary factors It is stored locally, thus preserving the unique visual style, sensor appearance, and text alignment features of each client.

[0132] For the decomposed U-Net, the uploaded semantic factor is

[0133] (16)

[0134] superscript This represents the client-side index. The corresponding local factor.

[0135] (17)

[0136] In the In round-robin communication, given a set of participating clients The server aggregates uploaded semantic factors in the following ways:

[0137] (18)

[0138] After aggregation In the next round, the data is redistributed to the client, thus forming a shared semantic fusion subspace among the distributed RS data sources.

[0139] Unlike standard federated averaging of full local updates, selective aggregation exposes only the migrateable portion of the local adaptation. It facilitates semantic fusion across clients while preserving This maintains local adaptability. This separation reduces the risk of introducing sensor-specific noise, local scene bias, or resource-related optimization artifacts into the global diffusion model.

[0140] S5. Perform cross-rank consistency optimization:

[0141] Because clients have varying memory and computational resource budgets, FedSD-LRF allows them to use different feasible ranks during local adaptation. However, heterogeneous rank selection can lead to representation inconsistencies between clients, especially during iterative denoising of the diffusion model. To mitigate this inconsistency, this application introduces a cross-rank consistency objective to align semantic representations generated by different rank configurations.

[0142] set up Represents the set of optional levels. In order to be at the level The intermediate representation below. Consistency loss is defined as:

[0143] (19)

[0144] This enables clients with different adaptability to maintain consistent denoising semantics, thereby reducing rank-induced drift during federated aggregation.

[0145] Combining diffusion objectives, structure preservation regularization, cross-rank consistency, and weight decay on low-rank factors, the local training objective is:

[0146] (20)

[0147] (twenty one)

[0148] in It is the regularization coefficient.

[0149] In the In each communication round, the client Choose a feasible rank based on its current memory and computing budget. Since all clients share the same decomposed U-Net interface, rank switching does not alter the global synchronization protocol. Furthermore, the cross-rank consistency loss prevents capacity adaptation from disrupting the semantic alignment required for federated diffusion training.

[0150] For high-resolution attention layers, when the feature map size exceeds 16×16, this application further reduces activation overhead by applying depthwise separable convolutions with a stride of 2 on K and V. The attention response is computed in the downsampled space and then mapped back to the original resolution via a residual branch. This operation reduces the memory peak of the attention while preserving the main semantic responses.

[0151] Cross-level consistency complements selective semantic factor fusion. Selective fusion determines which information is shared globally and which remains adaptively local, while consistency optimization makes the denoised representations at each level more stable. Together, they make federated diffusion more robust under heterogeneous computing resources and multi-source RS data distribution.

[0152] S6. Perform FedSD-LRF learning:

[0153] like Figure 11 As shown, the algorithm outlines the training process of FedSD-LRF. The server initializes the frozen SD backbone parameters. Perform structure-preserving U-Net reparameterization and maintain a global semantic adapter. In each round of federated learning, the server broadcasts the frozen basis parameters and the current global adapter to a sampled set of clients. Upon receiving the global state, each client selects a feasible rank based on its resource budget. and Synchronize and initialize local factors. To capture client-specific adaptations, the client then trains the decomposed U-Net on its private RS data samples. During local training, the client performs latent diffusion adaptation and optimizes the low-rank factor by minimizing Eq.(21). After local optimization is complete, only the updated semantic factors are uploaded. ,and The data is then retained on the client side to preserve local visual and semantic features. Finally, the server aggregates the uploaded factors using a weighted average to update the global semantic subspace for the next round. This iterative process gradually integrates semantic knowledge across clients while avoiding direct data sharing and complete model synchronization.

[0154] Experimental example:

[0155] 1. Dataset and Evaluation Metrics:

[0156] This application evaluates FedSD-LRF using three typical remote sensing image-text datasets, which cover disaster change scenarios, general land cover categories, and fine-grained remote sensing descriptions, respectively. The Remote SensingChange Captioning (RSCC) dataset contains 62,315 pairs of pre-disaster and post-disaster images, each labeled with fine-grained change descriptions covering different disaster scenarios.

[0157] This application selects four subsets: PAKISTAN-FLOODING, STVINCENT-VOLCANO, TEXAS-TORNADOES, and TONGA-VOLCANO. This application uses a standard 6:2:2 ratio to divide the training, validation, and test sets.

[0158] 2. Baseline:

[0159] This application compares FedSD-LRF with two sets of baseline methods. The first set includes representative heterogeneous or parametrically efficient federated learning methods. These methods are used to test whether general low-rank or lightweight federated optimization can handle heterogeneous RS clients. The second set includes federated generation or diffusion-oriented methods that are more directly related to the task of this application.

[0160] 3. Implementation Details: This application uses SDv1.5 as the backbone of the generative model and conducts all experiments in a simulated federated environment. Model updates employ the AdamW optimizer, combined with weight decay to achieve stable local adaptation. Consistent with cross-institutional federated learning scenarios in the RS (Research and Development) field, this federated experiment includes 10 clients. This setup reflects real-world RS collaboration, where data is typically distributed across a limited number of institutions, such as satellite centers, UAV platforms, and research institutions. To simulate device heterogeneity, this application allocates clients to three memory levels (8GB, 12GB, and 16GB), corresponding to batch sizes of 2–4, 4–8, and 8–12, respectively. The local learning rate is... The selection was based on different computational budgets. This application maintains a compression ratio of 0.6 for Tucker-2 decomposition and adjusts the adaptive rank in the range [16,128] for ablation studies. The training process consisted of 1000 rounds of communication, with each client performing 10 local iterations per round. Unless otherwise specified, all methods followed the same client partitioning and training protocol to ensure fairness in comparisons.

[0161] Table 1 compares LPIPS, PSNR, and SSIM on RSCC, RSICD, and RSITMD.

[0162]

[0163] 4. Experimental Comparison:

[0164] This application evaluates federated RS image generation in multi-source and device heterogeneous environments on RSCC, RSICD, and RSITMD. Tables 1 and 2 summarize the quantitative results compared to general heterogeneous federated learning baselines and federated generation baselines. This comparison not only examines the efficient adaptation of parameters but also analyzes whether structured decomposition and selective semantic fusion can better support diffusion generation under heterogeneous noise dynamics.

[0165] For the RSICD and RSITMD datasets, the model generates remote sensing images based on textual descriptions under semantic conditions. Full-reference fidelity metrics are calculated using PSNR, SSIM, and LPIPS, while CLIP Score, CLIP-IQA, and MUSIQ evaluate semantic alignment and perceptual naturalness from a no-reference perspective. For the RSCC dataset, the model generates post-disaster remote sensing images based on textual descriptions of disaster changes such as floods and volcanic eruptions. Given the diffuse generation characteristics, this application reports both reference-based and no-reference perceptual metrics to provide a more comprehensive evaluation.

[0166] As shown in Table 1, FedSD-LRF achieves the best performance in LPIPS, PSNR, and SSIM on RSCC, RSICD, and RSITMD. Compared to the strongest FedDifRC method in the second class, FedSD-LRF improves PSNR on RSCC from 27.51 to 29.88, SSIM from 0.693 to 0.846, and LPIPS from 0.256 to 0.239.

[0167] On RSICD, the corresponding PSNR value increased from 27.82 to 30.22, SSIM increased from 0.717 to 0.857, while LPIPS decreased from 0.239 to 0.211. On RSITD, PSNR increased from 27.66 to 29.23, SSIM increased from 0.702 to 0.811, and LPIPS decreased from 0.255 to 0.230.

[0168] These performance improvements demonstrate that, in heterogeneous federated diffusion environments, structural fidelity reparameterization and selective semantic factor fusion can jointly improve image distortion fidelity and perceptual consistency. The same trend is also reflected in Table 2, which reports the perceptual quality metrics without reference. Compared to FedDifRC, on RSCC, FedSD-LRF improves CLIP-IQA from 0.5624 to 0.6023 and MUSIQ from 53.84 to 61.16; on RSICD, CLIP-IQA to 0.6442 and MUSIQ to 60.45; and on RSITD, CLIP-IQA from 0.5976 to 0.6454 and MUSIQ from 57.31 to 67.61. Combined with the full-reference results, these improvements indicate that FedSD-LRF not only improves pixel-level and structural fidelity but also enhances visual naturalness and semantic consistency.

[0169] Figure 4 Qualitative results show that FedSD-LRF performs better in preserving scene layout and text-related semantics. On the RSIMD dataset, the generated baseball field and beach scenes have clearer structures and more realistic spatial textures. On the RSICD dataset, the semantic drift of the bridge and forest scenes is reduced, the object arrangement is more coherent, and the color distribution is more stable. Figure 5 This study further demonstrates the performance of FedSD-LRF in RSCC change scenarios. The generated post-disaster images better reflect flood-induced water accumulation, river widening, and volcanic smoke effects, while maintaining the integrity of the land cover structure. These observations indicate that the proposed method can effectively improve semantic alignment and spatial consistency in complex remote sensing image generation scenarios.

[0170] The comparison also shows that the improvements of FedSD-LRF are not solely due to parameter efficiency. General low-rank self-supervised methods reduce adaptation or communication costs but do not explicitly address the problem of mismatch accumulation in the diffusion steps. The federated generation baseline is closer to the task requirements but lacks a joint design of structure-preserving reparameterization, rank-adaptive local updates, and factor-level selective aggregation. The consistent advantage shown on both datasets highlights the benefits of coordinating low-rank optimization with the denoising dynamics in heterogeneous federated diffusion.

[0171] Table 2: Comparison of CLIP-IQA and MUSIQ on RSCC, RSICD, and RSITMD.

[0172]

[0173] Figure 6 The training performance under different communication rounds is further demonstrated. FedSD-LRF achieves higher accuracy and CLIP scores on all three datasets, while exhibiting lower training loss, indicating more stable convergence under heterogeneous client capabilities. This performance is particularly important for RS generation, as large-scale spatial consistency and fine local texture must be maintained simultaneously during the process.

[0174] 5. Hyperparameter analysis:

[0175] This application analyzes the FedSD-LRF's impact on communication rounds and LoRA rank. Sensitivity, such as Figure 7 As shown. With other settings remaining unchanged, the rank value is... Variations within the range. Increasing the number of communication rounds generally improves generation quality, indicating that repeated semantic factor fusion helps stabilize the optimization process across heterogeneous clients. As the number of rounds increases sufficiently, the performance improvement plateaus, suggesting that the federated diffusion process has converged. Rank analysis shows that a moderately low-rank configuration provides a better trade-off between model capacity and training stability. Too small a rank limits the expressiveness of local adaptation, while too large a rank offers limited benefits and may increase the optimization difficulty for weak clients.

[0176] Table 3: Ablation studies of FedSD-LRF on RSCC, RSICD and RSITMD.

[0177]

[0178] 6. Conduct ablation studies:

[0179] This application evaluates the main components of FedSD-LRF through ablation studies, including structure-preserving U-Net reparameterization, dynamic low-rank adaptation, and selective semantic factor aggregation. The results are shown in Table 3.

[0180] Low-rank parametric ablation with structure preserved.

[0181] Removing the Tucker-2-based U-Net reparameterization resulted in the greatest performance degradation across all metrics. This result demonstrates that the advantage of FedSD-LRF stems not only from reduced adaptation costs but also from a structured parameter space compatible with selective aggregation. By preserving the U-Net topology while compressing channel interactions, Tucker-2 helps maintain consistency in local updates across heterogeneous clients. Without this structure-preserving design, client updates become more fragmented, leading to poorer semantic alignment and weaker preservation of spatial structures such as terrain and buildings.

[0182] Device-adaptive low-rank optimized ablation.

[0183] Disabling dynamic LoRA and forcing a fixed low-rank configuration also degrades performance, especially in terms of PSNR and LPIPS. This indicates that uniform adaptive capabilities cannot fully match heterogeneous client resources. Rank-based adaptive low-rank optimization provides a more flexible balance between expressive power and training stability. This application compares A-only aggregation, A+B aggregation, and communication-matching A+B variants.

[0184] Table 4. Selective polymeric ablation performed on RSCC.

[0185]

[0186] Ablation experiments of selective polymerization:

[0187] Removing selective aggregation reduces semantic consistency and structural coherence. To further validate this module, this application compares it with using only... The aggregation method and simultaneous use and The aggregation method. Furthermore, this application introduces a variant of communication matching. By reducing the rank to To avoid upload costs dominating the comparison results. As shown in Table 4, under the original settings and communication matching settings, only using The aggregation still performs better. This confirms that the advantage of selective aggregation stems from separating its global semantic fusion from local adaptation, rather than simply reducing communication volume. Figure 8 This further demonstrates that only aggregation This leads to greater fluctuations in visual semantic quality across different communication rounds. These results further validate that aggregation alone... It can preserve local complementary features while incorporating globally transferable semantic knowledge, thereby improving the fidelity and semantic consistency of the generated data. Under the same communication budget, FedSD-LRF also achieves a better balance between global semantic fusion and local adaptation.

[0188] 7. Conduct an analysis of the degree of equipment heterogeneity:

[0189] This application further examines the robustness of FedSD-LRF under varying degrees of device heterogeneity by adjusting resource allocation among federated clients. In the main experiments, this application categorizes federated clients into three memory levels: 8GB, 12GB, and 16GB, as shown in Table 5. This setting reflects real-world RS collaboration scenarios, where clients may correspond to satellite centers, UAV platforms, ground stations, or computing units with different memory and processing capabilities. This application constructs three device heterogeneity settings: mild, moderate, and severe, to reveal how model behavior changes with resource imbalance as heterogeneity increases.

[0190] Table 5. Device heterogeneity settings used for robustness analysis.

[0191]

[0192] Table 6 shows the performance of FedSD-LRF under different levels of device heterogeneity.

[0193]

[0194] The total number of federated clients is fixed at 10. The light setup includes more medium and high-resource clients, while the heavy setup increases the proportion of low-resource clients. For each client, the dataset partitioning, optimizer, communication protocol, selective aggregation strategy, and evaluation metrics remain unchanged. The local batch size, low-rank decomposition, and feasible fitness level are selected based on its memory hierarchy to ensure that performance differences in the experiments are primarily due to device heterogeneity, rather than differences in data or training methods.

[0195] This application uses four metrics—LPIPS, PSNR, SSIM, and CLIP-IQA—to comprehensively evaluate perceptual distance, pixel-level fidelity, structural consistency, and no-reference visual quality. As shown in Table 5, the proposed structure-preserving Tucker-2 reparameterization method, rank-adaptive low-rank optimization, and selective semantic factor aggregation can maintain stable generation quality even when there are varying degrees of device heterogeneity and more low-resource clients participate in federated diffusion training. Figure 9 The study further demonstrates the performance robustness trends under light, medium, and heavy heterogeneous scenarios, showing whether performance remains stable as the proportion of low-resource clients increases. The smooth performance variation indicates that FedSD-LRF is not only effective in fixed heterogeneous settings but also exhibits good robustness to changes in resource imbalance.

[0196] FedSD-LRF's robustness stems from two key mechanisms. First, Tucker-2 reparameterization provides a unified U-Net decomposition interface for clients with different resource budgets, enabling heterogeneous clients to select a feasible rank without altering the global synchronization protocol. Second, FedSD-LRF only uploads and aggregates semantic factors. At the same time, local complementary factors are retained. This selective aggregation strategy effectively mitigates the impact of local perturbations caused by resource differences on the diffusion model. Therefore, it allows low-resource clients to adapt and optimize locally based on their own capabilities, while maintaining semantic fusion sharing within the global model.

[0197] 8. Communication and computing overhead:

[0198] This application analyzes the communication and computational overhead of the proposed framework FedSD-LRF. For resource-constrained clients, directly fine-tuning or synchronizing the complete diffuse U-Net incurs significant overhead. FedSD-LRF reduces communication costs by freezing the original SD backbone network, optimizing low-rank factors, and uploading semantic factors $A$ only during federated synchronization.

[0199] For a dense weight matrix Full-scale fine-tuning requires There are trainable parameters. Standard LoRA would update as... ,in , Therefore, the number of trainable parameters is In FedSD-LRF, and All optimizations are performed locally, but only uploaded. Therefore, the size of the uploaded parameters is .

[0200] For all press The cost of each upload round for the trainable layers of the index can be expressed as: during full model synchronization, During standard LoRA synchronization,

[0201] In the proposed selective semantic factor aggregation scheme,

[0202] .in, This indicates the number of bytes occupied by each parameter. Because It only includes semantic factors for uploading, therefore FedSD-LRF has lower communication overhead than the standard. Aggregation method.

[0203] The robustness of FedSD-LRF stems from the synergy between structure-preserving reparameterization and selective semantic factor aggregation. Tucker-2 reparameterization provides a unified U-Net decomposition interface for clients with different resource budgets, enabling them to select feasible ranks without altering the global synchronization protocol. Meanwhile, FedSD-LRF only uploads and aggregates semantic factors. While retaining local complementary factors This selective aggregation strategy prevents local disturbances caused by resource differences from being injected into the global diffusion model. Therefore, low-resource clients can adapt and optimize locally based on their own capabilities, while the global model maintains shared semantic fusion. Figure 10 A visual comparison of communication and computing overhead is presented.

[0204] During federated diffusion training, the selection of adapter rank r, cross-rank consistency regularization coefficient λ_cons, and the proportion of participating clients ρ in each round directly affects the final quality and training stability of the generated model.

[0205] To ensure the theoretical basis for the selection of the above parameters, this method analyzes the convergence behavior of the training process. The analysis treats the global training objective as a weighted average of the local diffusion losses of each client, and considers perturbations introduced by factors such as Tucker-2 decomposition truncation, low-rank parameterization approximation, cross-rank semantic alignment errors, and partial client participation. The analysis results show that:

[0206] (1) When the objective function is a general non-convex form, the training process converges at a standard sublinear rate to a bounded neighborhood near the stationary point. The neighborhood radius is determined by the variance of the stochastic gradient and the cumulative deviation of the above factors.

[0207] (2) When the objective function further satisfies the strong convexity condition, the training process converges at a linear rate to a controllable neighborhood near the global optimum.

[0208] The size of the convergence neighborhood mentioned above is clearly negatively correlated with the following factors: the higher the adapter rank r, the higher the accuracy of the Tucker-2 decomposition and low-rank approximation of the original model, and the smaller the convergence radius; the larger the cross-rank consistency regularization coefficient λ_cons, the tighter the denoising semantic alignment between different heterogeneous clients, and the smaller the bias introduced by rank mismatch; the higher the proportion of clients participating in each round ρ, the smaller the uncertainty disturbance brought about by client sampling.

[0209] Based on the above theoretical basis, this method suggests configuring parameters according to the distribution of client resources during actual deployment, following these principles: Prioritize higher adapter rank within resource limits; when client devices are highly heterogeneous, appropriately increase λ_cons to enhance cross-rank semantic constraints; and moderately increase the participation ratio per round if communication bandwidth allows. This approach ensures that the training process stably converges to a semantic fusion state that meets the quality requirements for remote sensing image generation in heterogeneous device environments.

[0210] In summary, this application addresses the federated multi-source information fusion problem in remote sensing diffusion generation under device heterogeneity. To alleviate the inconsistency in denoising updates caused by resource-mismatched clients, this application proposes a device-adaptive federated diffusion framework, FedSD-LRF. This method does not require redesigning the diffusion backbone network but improves the compression, adaptation, and fusion of diffusion parameters across different clients. Through Tucker-2 reparameterization, lightweighting is achieved while maintaining the U-Net denoising topology; rank-adaptive low-rank training supports clients with different resource budgets; and selective aggregation fuses only semantic factors. And complementary factors The process is kept local. This design separates global semantic fusion from source-specific adaptation, improving the stability of multi-source remote sensing generation. Experiments on the RSCC, RSICD, and RSITMD datasets demonstrate that FedSD-LRF outperforms typical federated low-rank and diffusion baseline methods on both reference-based and reference-free metrics. Future work will further extend this framework to a wider range of multi-sensor remote sensing fusion scenarios and more realistic distributed deployment environments.

[0211] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0212] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A secure distributed remote sensing data diffusion generation method based on federated multi-source information fusion, characterized in that: The method includes the following steps: S1: The server performs Tucker-2 second-order tensor reparameterization on all 2D convolutional layers and attention projection layers of U-Net: compressing the number of model parameters along the channel dimension, fully preserving the original spatial convolutional kernel size, skip connections, and network input / output interfaces; initializing the tensor factor matrix using HOSVD singular value decomposition, minimizing the F-norm of the convolutional kernel to reconstruct the error, and generating a lightweight unified U-Net backbone for distribution to all clients; replacing the decomposed convolutions with a three-layer cascaded structure of a 1×1 channel compression layer, a kh×kw core convolutional layer, and a 1×1 channel expansion layer; S2: Each client independently selects a low-rank adaptation rank r based on its own memory, computing power, and energy consumption resource thresholds, and attaches a dynamic DyLoRA low-rank adapter to the lightweight U-Net convolution and attention modules; the adapter incremental update parameters are split into semantic factor A and local complementary factor B, B is initialized as a zero matrix, and A follows a zero-mean normal distribution, and the lightweight U-Net backbone weights are frozen throughout the training process; S3: The client-side local training constructs a composite total loss function, which integrates five constraints: standard diffusion denoising task loss, network alignment KL structure preservation loss before and after Tucker decomposition, noise prediction matching loss, cross-rank consistency loss, and low-rank factor weight decay loss. Among them, the cross-rank consistency loss ensures that the intermediate denoising representation of U-Net remains consistent under different rank configurations. For high-resolution attention feature maps with a size ≥16×16, the K / V matrix is ​​downsampled using depthwise separable convolution with a stride of 2 to reduce peak memory usage. S4: After a single round of local training, the client only uploads the semantic factor set ΘA corresponding to all modules to the federated server, which contains the complementary factor set of sensor-specific textures and local scene biases. Permanently stored locally without being transferred; S5: The server uses the ratio of each client's private samples to the total samples as a weighting coefficient to perform a weighted average aggregation of all uploaded semantic factors A in this round, generating a globally shared semantic factor. ; S6: The server distributes the aggregated global semantic factors to all clients. The clients synchronize their local A factors and execute S2-S5 in a loop until the global total loss fluctuation is less than the preset threshold or the maximum number of communication rounds is reached. During the inference phase, the clients fuse the local B factors and the global A factors, input the lightweight U-Net, and generate high-fidelity multi-source remote sensing images.

2. The secure distributed remote sensing data diffusion generation method based on federated multi-source information fusion according to claim 1, characterized in that: In step S1, the server loads the pre-trained stable diffusion model parameters θ as the generation backbone, and the model denoising network... U-Net architecture is adopted; Each client holds private remote sensing image-text pairs, and under the federated communication framework, model parameter interaction replaces the sharing of raw data, establishing a foundation for multi-source collaboration with privacy protection.

3. The secure distributed remote sensing data diffusion generation method based on federated multi-source information fusion according to claim 1, characterized in that: In step S1, performing Tucker-2 decomposition on the U-Net convolutional layers is to address the problem of excessive resource overhead in fine-tuning the complete U-Net. This scheme only compresses channel interactions while fully preserving the spatial convolutional structure, thus stably maintaining the remote sensing scene layout and fine-grained texture. The convolutional kernel achieves compact reparameterization through the core tensor, input channel factor matrix, and output channel factor matrix. This is crucial for maintaining RS layout and fine-grained texture during diffusion generation; For each convolution kernel Tucker-2 decomposition provides a compact and server-aggregable parameterization method while maintaining the original U-Net topology: (5) in Represents the core tensor. and These are the factor matrices for the output and input channels, respectively; while keeping the spatial convolution kernel and skip connection structure unchanged, the original convolution is replaced by a... Channel compression, one The core convolution and a The channel expansion allows for a reduction in model capacity across the channel dimension, while the spatial convolution kernel and U-Net topology remain unchanged. The specific calculation formula is as follows: (6) For an RS image-text pair The VAE encoder maps an image to a latent representation. .

4. The secure distributed remote sensing data diffusion generation method based on federated multi-source information fusion according to claim 1, characterized in that: In step S2, the forward diffusion process is scheduled according to noise. right The disturbances include: set up The weight tensor of the two-dimensional convolutional layer in U-Net; introduced by Tucker-2 decomposition. , and , making (7) During the process of compressing model capacity along the channel dimension, the input feature map The decomposed convolution is represented as: (8) The number of parameters from Reduce to ; Decrease and It can reduce dominant FLOPs and active memory, allowing clients with different computing budgets to maintain the same U-Net topology while enabling lighter parameterization; To ensure accurate approximation of the original convolutional kernel, HOSVD is used to initialize the channel factor matrix before solving the core tensor, thus reducing the initial deviation between the lightweight network and the original network. (9)； First, HOSVD initialization is performed on the channel mode of matrix $W$. and Then G is calculated; this initialization method reduces the bias between the original kernel and the decomposed kernel before federated adaptation.

5. The secure distributed remote sensing data diffusion generation method based on federated multi-source information fusion according to claim 1, characterized in that: In step S2, during the process of attaching a low-rank adapter to the backbone, after obtaining the decomposed U-Net, FedSD-LRF introduces a low-rank adapter to avoid updating the complete backbone network on each client; for a fixed mapping matrix The trainable updates are parameterized as follows: （15） The same structural design was also applied to the Tucker-2 component. and and attention projection matrix , and ; During initialization, B is set to zero, and A is randomly sampled. Since only the low-rank factor is optimized, the size of the trainable parameters and the main computational load are reduced from... Reduce to , and rank This can be adjusted based on the resource budget of each client; this application is initialized. and This ensures that the low-rank update is zero in the initial stage, preserving the diffusion behavior of the pre-trained model. There is no additional offset initially, and the original diffusion denoising effect of the pre-trained model is fully preserved.

6. The secure distributed remote sensing data diffusion generation method based on federated multi-source information fusion according to claim 1, characterized in that: In step S2, in a multi-source remote sensing scenario, multi-source remote sensing data have differentiated sensor and local scene features. If all A and B factors are uploaded directly, it will confuse the global general semantics with the source-specific visual features. This invention updates A and B locally, uploading only semantic factor A, which has global migration capabilities, to the server, while retaining complementary factor B locally, and separately saving each client's unique visual style, sensor spectral features, and image-text matching information.

7. The secure distributed remote sensing data diffusion generation method based on federated multi-source information fusion according to claim 1, characterized in that: In step S3, the parameters are given during the process of freezing the trunk. The client, of which the first Each client has private image-text data. The federal proliferation objective can be expressed as: (10) in Indicates the sample proportion.

8. The secure distributed remote sensing data diffusion generation method based on federated multi-source information fusion according to claim 1, characterized in that: In step S4, the Tucker lightweight network is prone to amplifying approximation errors during back-diffusion. Therefore, a structure preservation regularization term is set: noise predictors before and after decomposition are defined respectively, and noise level alignment loss and back-distribution KL divergence loss are constructed, combined with the standard diffusion task loss to form a structure preservation objective; a cross-rank consistency loss is added to constrain the semantic alignment of the U-Net intermediate denoised representation under different adaptation ranks; [The text abruptly ends here, likely due to an incomplete translation or a missing section.] and These represent the noise predictors before and after Tucker-2 reparameterization; the noise level alignment loss is... (11) The one-step reverse transition is also regularized in the following ways: （12） in and These are the inverse distributions induced by the original denoising network and the decomposed denoising network, respectively. Together with standard diffusion loss （13） The structural preservation target is defined as (14) in and Used to balance task learning and diffusion consistency preservation; In the process of cross-rank consistency constraints, set Represents the set of optional levels. In order to be at the level The intermediate representation below; consistency loss is defined as: (19) To ensure decomposition accuracy and semantic alignment under heterogeneous rank constraints, the training process incorporates diffusion objectives, structure preservation regularization, cross-rank consistency, and weight decay on low-rank factors. The local training objective is as follows: (20) (21) in It is the regularization coefficient; In the In each communication round, the client Choose a feasible rank based on its current memory and computing budget. Since all clients share the same decomposed U-Net interface, rank switching does not change the global synchronization protocol; furthermore, cross-rank consistency loss prevents capacity adaptation from disrupting the semantic alignment required for federated diffusion training. For high-resolution attention layers, when the feature map size exceeds 16×16, this application further reduces activation overhead by applying depthwise separable convolutions with a stride of 2 on K and V; the attention response is computed in the downsampling space and then mapped back to the original resolution through the residual branch; this operation reduces the memory peak of attention while preserving the main semantic features to maintain the correct arrangement of image features.

9. The secure distributed remote sensing data diffusion generation method based on federated multi-source information fusion according to claim 1, characterized in that: In step S5, after a single round of training, the semantic factor set uploaded by the client includes the A matrix corresponding to all convolutions and attention layers of the U-Net. Based on the proportion of samples from each client during the decomposition of the U-Net, the uploaded semantic factors are... (16) superscript This represents the client index; the corresponding local factor is not transmitted, and the specific calculation formula is as follows: (17)。 10. The secure distributed remote sensing data diffusion generation method based on federated multi-source information fusion according to claim 1, characterized in that: In step S6, during the process of weighting all uploaded A according to the sample size ratio of each client, the process is in the first step. In round-robin communication, given a set of participating clients The server aggregates the uploaded semantic factor A using a weighted average in the following way: （18） After aggregation In the next round, the data is redistributed to clients, thereby building a unified and shared semantic fusion subspace among distributed multi-source remote sensing data.

Images