Geospatial large model low-rank adaptation and privacy protection method and system

Abstract

A method and system for low-rank adaptation and privacy protection of a large geospatial model are disclosed, involving large geospatial models and federated learning techniques. The method includes: loading a pre-trained large geospatial model and inserting a low-rank adapter, freezing the backbone and training only the adapter; extracting features based on local remote sensing ecological monitoring data and calculating local feature prototypes; performing hierarchical differential privacy and communication compression on the adapter update and prototype before uploading; a central server aggregates and generates an initial global prototype, adaptively assigning weights based on the differences between the local and global prototypes, and weighted aggregation to obtain the global model and the updated global feature prototype; distributing the global model and performing prototype alignment regularization training with the global prototype as a constraint to achieve alignment between the local and global feature spaces. The system includes modules for local training, privacy communication, global aggregation, and distribution alignment. This invention protects data privacy, reduces communication overhead, and improves cross-node generalization ability, making it suitable for distributed geospatial ecological monitoring in weak network environments.

Description

Technical Field

[0001] This invention relates to the field of geographic ecological monitoring technology, specifically to a method and system for low-rank adaptation and privacy protection of large geospatial models, which is particularly suitable for distributed geographic ecological monitoring scenarios, taking into account both data privacy protection and model generalization capabilities. Background Technology

[0002] Geographic and ecological remote sensing monitoring data comes from a wide range of sources and is widely distributed, covering multiple fields such as meteorology, water quality, and soil. Distributed monitoring has become the mainstream mode, but current technology faces significant bottlenecks. Traditional centralized model training requires the aggregation of raw monitoring data from each node, which is prone to leaking sensitive areas, pollution sources, and other private information. Furthermore, data sharing is difficult to achieve due to data privacy regulations, resulting in severe information silos. While existing federated learning schemes can avoid raw data transmission, they are not optimized for geographic and ecological monitoring scenarios, leading to high communication overhead and insufficient model generalization ability. At the same time, the unreasonable application of protection technologies such as differential privacy can easily lead to a decrease in model accuracy, and the lack of effective feature alignment mechanisms makes it impossible to adapt to the distribution differences of monitoring data from multiple nodes. Different edge monitoring nodes are located in different geographical environments, resulting in significantly different data distributions, exhibiting non-independent identically distributed (Non-IID) characteristics. This distribution difference causes client drift during the update and aggregation process in traditional federated learning, leading to slower global model convergence, reduced training stability, and difficulty in guaranteeing cross-regional monitoring accuracy, making it difficult to meet the actual needs of distributed geographic and ecological monitoring in weak network environments. Summary of the Invention

[0003] This invention provides a method for low-rank adaptation and privacy protection of a large geospatial model, comprising: loading a pre-trained large geospatial model and inserting a low-rank adapter module; freezing the backbone parameters of the large geospatial model and training only the low-rank adapter module; extracting features based on local remote sensing ecological monitoring data and calculating local feature prototypes; performing hierarchical differential privacy processing and communication compression on the updated parameters of the low-rank adapter module and the local feature prototypes, and then uploading them to a central server; aggregating each local feature prototype to generate an initial global feature prototype; adaptively allocating aggregation weights according to the differences between each local feature prototype and the initial global feature prototype, and aggregating to obtain a global model and an updated global feature prototype; distributing the global model and performing prototype alignment regularization training with the updated global feature prototype as a constraint to align each local feature with the global feature space.

[0004] According to one embodiment of the present invention, the hierarchical differential privacy processing performed on the update parameters of the low-rank adapter module and the local feature prototype specifically includes: grouping the update amount / gradient of the update parameters of the low-rank adapter module to be uploaded and / or the statistics of the local feature prototype into a spatiotemporally sensitive parameter group that interacts with spatiotemporal coding and a deep semantic parameter group; applying a smaller pruning threshold and injecting stronger noise to the spatiotemporally sensitive parameter group; and applying a larger pruning threshold and injecting weaker noise to the deep semantic parameter group.

[0005] According to one embodiment of the present invention, the communication compression of the update parameters of the low-rank adapter module and the local feature prototype specifically includes: uploading only the update amount / gradient of the update parameters of the low-rank adapter module and the local feature prototype, and performing at least one of quantization compression, sparse uploading, or group uploading on them; wherein, the quantization compression includes converting the floating-point update amount / gradient into a low-bit integer representation, and the sparse uploading includes uploading only the update amount / gradient components with an amplitude greater than a preset threshold.

[0006] According to one embodiment of the present invention, the step of generating an initial global feature prototype by aggregating each of the local feature prototypes further includes: during the first aggregation, performing an unweighted average calculation on each of the local feature prototypes to generate the initial global feature prototype; during the second and subsequent aggregations, aggregating each of the local feature prototypes based on the aggregation weights adaptively allocated in the previous aggregation process to update the initial global feature prototype.

[0007] According to one embodiment of the present invention, the adaptive allocation of aggregation weights based on the differences between each local feature prototype and the initial global feature prototype specifically includes: the aggregation weights are negatively correlated with the differences between the local feature prototypes and the initial global feature prototypes.

[0008] According to one embodiment of the present invention, the update amount / gradient of the update parameters of each of the low-rank adapter modules are weighted and aggregated based on the aggregation weight to obtain a global model; and the local feature prototypes are weighted and aggregated based on the aggregation weight to obtain an updated global feature prototype.

[0009] According to one embodiment of the present invention, the prototype alignment regularization training with the updated global feature prototype as a constraint specifically includes: during the training process of the low-rank adapter module, using the updated global feature prototype as a reference, constraining the distribution differences between each local feature prototype and the updated global feature prototype, so that the local features of each edge node remain aligned with the global feature space.

[0010] This invention also provides a low-rank adaptation and privacy protection system for a large geospatial model, comprising: a local training module for loading a pre-trained large geospatial model and inserting a low-rank adapter module, freezing the backbone parameters of the large geospatial model, and training only the low-rank adapter module; extracting features based on local remote sensing ecological monitoring data and calculating local feature prototypes; a privacy protection and communication module for performing hierarchical differential privacy processing and communication compression on the updated parameters of the low-rank adapter module and the local feature prototypes, and then uploading them to a central server; a global aggregation module for aggregating and generating an initial global feature prototype based on each local feature prototype; adaptively allocating aggregation weights based on the differences between each local feature prototype and the initial global feature prototype, and aggregating to obtain a global model and an updated global feature prototype; and a global distribution and alignment module for distributing the global model and performing prototype alignment regularization training with the updated global feature prototype as a constraint to align each local feature with the global feature space.

[0011] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the geospatial large model low-rank adaptation and privacy protection method described above.

[0012] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the geospatial large model low-rank adaptation and privacy protection method of the above embodiments.

[0013] This invention introduces low-rank adaptation technology into geographic ecological monitoring, freezing the backbone model parameters and training only the low-rank adapter, effectively reducing training and communication costs. It employs hierarchical differential privacy protection measures, combined with feature prototype alignment training, to ensure local data security while maintaining feature space consistency across monitoring nodes. Through adaptive weight allocation and aggregation strategies, it optimizes model generalization capabilities, addressing issues such as data silos, privacy leaks, and feature inconsistencies in distributed monitoring. This approach balances privacy protection, communication efficiency, and monitoring accuracy, adapting to the needs of distributed geographic ecological monitoring in weak network environments. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0015] Figure 1This is the flowchart of the low-rank adaptation and privacy protection method for large geospatial models provided by the present invention.

[0016] Figure 2 This is a schematic diagram of the injection structure of the low-rank adapter in the attention layer provided by the present invention.

[0017] Figure 3 This is a schematic diagram of the layered differential privacy processing flow provided by the present invention.

[0018] Figure 4 This is a schematic diagram of the global feature prototype weighted aggregation process under non-independent and identically distributed data provided by the present invention.

[0019] Figure 5 This is a block diagram of the geospatial large model low-rank adaptation and privacy protection system provided by the present invention.

[0020] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0021] Figure label: 100: Low-rank adaptation and privacy protection system for large geospatial models; 110: Local training module; 120: Privacy protection and communication module; 130: Global aggregation module; 140: Global distribution and alignment module; 610: Processor; 620: Communication interface; 630: Memory; 640: Communication bus. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] The following is combined with Figures 1 to 5 This invention describes a method and system for low-rank adaptation and privacy protection of large geospatial models.

[0024] Figure 1 This is the flowchart of the low-rank adaptation and privacy protection method for large geospatial models provided by this invention. For example... Figure 1 As shown, the geospatial large model low-rank adaptation and privacy protection method provided by the present invention includes: In step S100, the pre-trained geospatial large model is loaded and a low-rank adapter module is inserted. The backbone parameters of the geospatial large model are frozen, and only the low-rank adapter module is trained. In step S200, features are extracted based on local remote sensing ecological monitoring data, and local feature prototypes are calculated; In step S300, the updated parameters of the low-rank adapter module and the local feature prototype are subjected to hierarchical differential privacy processing and communication compression, and then uploaded to the central server. In step S400, an initial global feature prototype is generated by aggregating each local feature prototype. In step S500, aggregation weights are adaptively assigned based on the differences between each local feature prototype and the initial global feature prototype, and the global model and the updated global feature prototype are obtained by aggregation. In step S600, the global model is distributed, and prototype alignment regularization training is performed with the updated global feature prototype as a constraint to align each local feature with the global feature space.

[0025] Specifically, the Low-Rank Adaptive Adapter (LoRA) module described in this application is a lightweight model fine-tuning module. It connects to the attention layer of the pre-trained backbone network as a bypass branch, training only the low-rank branch parameters while keeping the backbone network parameters frozen throughout, thereby reducing model fine-tuning and communication transmission costs. In step S100, a pre-trained geospatial large model is loaded and the low-rank adapter module is inserted. The backbone parameters of the geospatial large model are frozen, and only the low-rank adapter module is trained.

[0026] Reference Figure 2 In step S100, the low-rank adapter module is connected to the attention layer of the backbone network in a bypass structure. The parameters of the backbone network are kept frozen throughout the process and do not participate in training updates.

[0027] Figure 2 In the backbone network, the attention layer contains a query matrix. Key matrix AND-value matrix The low-rank adapter module is only connected to... and The corresponding linear mapping bypass, The low-rank adapter module is not connected; LoRA is injected into the query matrix at least. Sum / or value matrix The corresponding linear mapping bypass enables federated low-rank adaptation. Each low-rank adapter module consists of matrices A and B connected sequentially. During training, only matrices A and B are updated, while the weights of the backbone network remain unchanged.

[0028] The pre-trained geospatial large model loaded in step S100 already possesses stable geographic and ecological feature extraction capabilities. After inserting the low-rank adapter module into the model, the backbone parameters of the geospatial large model remain frozen, and no further adjustments are made to the backbone parameters. The low-rank adapter module is connected to the attention layer of the backbone network as a bypass branch, and the training process is only performed on the low-rank adapter module. This module adopts a low-rank matrix bypass structure, and the model weights satisfy the following relationship.

[0029] ； 。

[0030] in, For the frozen pre-trained weight matrix, and The matrix is ​​a trainable low-rank matrix, with its rank much smaller than the smaller of the input and output dimensions of the linear layers in the backbone network. The attention layer of the backbone network contains the query matrix. Key matrix AND-value matrix Projection, low-rank adapter module at least injected to and The corresponding linear mapping bypass, and the matrix is ​​initialized with zero initialization during the early stages of training. Initialization is performed. The overall parameter scale is much smaller than that of the backbone network, which can significantly reduce the computing resources and communication data required for training while relying on the original feature extraction capabilities of the large model. This makes the entire model adaptation process more suitable for execution on resource-constrained edge monitoring nodes.

[0031] In step S200, each distributed monitoring node extracts corresponding features based on its own collected local remote sensing ecological monitoring data using a pre-initialized model. The local remote sensing ecological monitoring data covers multi-dimensional monitoring information such as meteorology, vegetation, and water quality within the geographic area corresponding to the node, and the extracted features accurately reflect the key attributes of the local data. Based on this, local feature prototypes are further calculated according to ecological element categories. The local feature prototypes are feature centroid vectors obtained by statistical analysis of categories, and the relevant calculations satisfy the following relationship.

[0032] , in, Indicates edge monitoring node In category Local feature prototypes on Represents feature extraction mapping, This indicates the frozen backbone network parameters. Indicates local low-rank adapter parameters. Indicates edge monitoring node The above belongs to the category The sample set. The local feature prototype, as the feature representation of the node's data, can stably reflect the category distribution of the current node's monitoring data, providing unified and stable basic data for subsequent global aggregation.

[0033] In step S300, hierarchical differential privacy processing and communication compression are performed simultaneously on the updated parameters of the low-rank adapter module and the local feature prototype. Hierarchical differential privacy processing groups the update amounts, gradients, and statistics of the local feature prototype of the updated parameters of the low-rank adapter module to be uploaded into two groups: a spatiotemporally sensitive parameter group (interacting with spatiotemporal coding) and a deep semantic parameter group. A smaller pruning threshold and stronger noise are injected for the spatiotemporally sensitive parameter group, while a larger pruning threshold and weaker noise are injected for the deep semantic parameter group. Through differentiated privacy protection strategies, sensitive information is not leaked while maintaining the model's feature representation accuracy as much as possible. The communication compression stage limits the upload to only the update amounts, gradients, and local feature prototypes of the low-rank adapter module, and performs at least one of quantization compression, sparse upload, or group upload. Quantization compression converts floating-point update amounts and gradients into low-bit integer representations, while sparse upload only retains update amounts and gradient components with amplitudes greater than a preset threshold for transmission. These compression methods effectively reduce the amount of data to be uploaded, adapting to communication needs in weak network environments and reducing data transmission costs.

[0034] In step S400, the central server receives the processed data uploaded by each monitoring node and performs aggregation operations based on each local feature prototype to generate an initial global feature prototype. During the first aggregation process, each local feature prototype is calculated using an unweighted average method to ensure that the initial global feature prototype objectively reflects the overall distribution of features across nodes. In the second and subsequent aggregation processes, the aggregation weights adaptively allocated in the previous aggregation round are used to aggregate each local feature prototype, achieving dynamic updates to the initial global feature prototype to better reflect the actual distribution characteristics of the data from each node.

[0035] In step S500, the central server adaptively allocates aggregation weights based on the differences between each local feature prototype and the initial global feature prototype. The magnitude of the aggregation weights is negatively correlated with these differences. Based on the allocated aggregation weights, the update amounts and gradients of the update parameters of each low-rank adapter module are weighted and aggregated to obtain the global model. Simultaneously, each local feature prototype is weighted and aggregated to obtain the updated global feature prototype. This aggregation method ensures that the global model can take into account the feature differences of each node, improves cross-node generalization ability, and provides a clear target benchmark for subsequent feature alignment.

[0036] In step S600, the central server distributes the aggregated global model to each distributed monitoring node. Upon receiving the global model, each monitoring node performs prototype alignment regularization training constrained by the updated global feature prototype. During the training of the low-rank adapter module, the updated global feature prototype is used as a benchmark to constrain the distribution differences between each local feature prototype and the updated global feature prototype. Through regularization training, the gap between local features and global features is gradually narrowed, ultimately ensuring that the local features of each edge node are consistent with the global feature space. This achieves synergistic optimization of privacy protection, communication efficiency, and monitoring accuracy in a distributed geographic ecological monitoring scenario.

[0037] Reference Figure 3 According to an embodiment of the present invention, hierarchical differential privacy processing is performed on the update parameters of the low-rank adapter module and the local feature prototype. Specifically, this includes: grouping the update amount / gradient of the update parameters of the low-rank adapter module to be uploaded and / or the statistics of the local feature prototype into a spatiotemporally sensitive parameter group that interacts with spatiotemporal coding and a deep semantic parameter group; applying a smaller pruning threshold and injecting stronger noise to the spatiotemporally sensitive parameter group; and applying a larger pruning threshold and injecting weaker noise to the deep semantic parameter group.

[0038] Specifically, the spatiotemporally sensitive parameter group carries sensitive information directly related to the geographical location and monitoring time series of the monitoring area. These parameters are directly associated with the spatial distribution characteristics and temporal variation patterns of geographic ecological monitoring data. If information leakage occurs, it will directly expose traceable sensitive content such as the location of the monitoring station, the monitoring period, and the data fluctuation patterns. Therefore, a differentiated strong privacy protection strategy is configured for this group of parameters: by setting a small pruning threshold, the magnitude of parameter updates is strictly limited to reduce the risk of leakage of sensitive information hidden in the parameters from the source and avoid leakage of spatiotemporally sensitive features during gradient updates; at the same time, high-intensity noise is injected into this group of parameters to randomly perturb the parameter values, further destroying the sensitive information carried in the parameters, achieving a high-intensity privacy protection effect, and ensuring the spatiotemporal privacy security of geographic ecological monitoring data. The deep semantic parameter set mainly represents the inherent semantic relationships and category features of ecological monitoring data, and is used to reflect the correlation patterns between different ecological elements, the distribution of data categories, etc. It does not directly contain sensitive information such as traceable geographical location and time series, and the risk of privacy leakage is relatively low. Therefore, a weak privacy protection strategy is configured for this parameter set: gradient pruning is performed using a large pruning threshold to retain the effective update information of the parameters to the greatest extent while meeting basic privacy protection requirements, and to avoid excessive constraints affecting the model's learning ability; at the same time, low-intensity noise is injected into this parameter set to only slightly perturb the parameters, so as to maintain the positive contribution of the parameters to the model performance while ensuring privacy compliance, and to balance the privacy protection effect and the model monitoring accuracy.

[0039] In practice, the low-rank adapter parameters are divided into parameter groups that interact with spatiotemporal coding. and deep semantic parameter set And record the corresponding update volume / gradient to be uploaded as follows: and Set different clipping thresholds for different parameter groups. , and different noise standard deviations , For spatiotemporal correlation gradients Injecting more noise strengthens the privacy protection of geolocation and time-series information.

[0040] The privacy protection processing employed in this embodiment of the invention specifically includes: performing differential privacy perturbation on the low-rank adapter parameter update amount / gradient and / or local feature prototype statistics to be uploaded, the perturbation process including gradient pruning and Gaussian noise injection; allocating a stricter privacy budget to parameter groups interacting with position encoding or time encoding to enhance the protection level of sensitive information; the above differential privacy perturbation process satisfies Differential privacy constraints provide a rigorous and provable guarantee of privacy and security, effectively preventing the leakage of raw geographic and ecological monitoring data.

[0041] The principle of hierarchical differential privacy processing lies in configuring differentiated privacy protection strategies for parameters with different sensitivities, thus solving the technical defects of insufficient protection for highly sensitive parameters and excessive protection for low-sensitivity parameters under the traditional unified privacy budget allocation method. By differentially processing the spatiotemporally sensitive parameter group and the deep semantic parameter group, it not only strictly guarantees the spatiotemporal privacy security of geographic ecological monitoring data and meets data compliance requirements, but also avoids excessive constraints on low-sensitivity parameters. This ensures that the model can effectively learn the semantic features of the data during the federated learning process, thereby improving the generalization ability and monitoring accuracy of the federated low-rank adaptation model for non-independent identically distributed (Non-IID) ecological monitoring data, and achieving synergistic optimization of privacy protection and model performance.

[0042] According to an embodiment of the present invention, communication compression is performed between the update parameters of the low-rank adapter module and the local feature prototype, specifically including: Upload only the update amount / gradient of the update parameters of the low-rank adapter module and the local feature prototype, and perform at least one of quantization compression, sparsity upload or group upload on them; Among them, quantization compression includes converting floating-point update amounts / gradients into low-bit integer representations, and sparse uploading includes uploading only update amount / gradient components with amplitudes greater than a preset threshold.

[0043] Specifically, the communication compression stage aims to reduce communication overhead during federated learning and adapt to transmission requirements in weak network environments. It only transmits the update amounts / gradients of the low-rank adapter module's update parameters and local feature prototypes, omitting backbone network parameters and original monitoring data, thus significantly reducing the amount of data transmitted. Building upon this, at least one of the following compression methods is employed: quantization compression, sparse uploading, or group uploading. Quantization compression converts the high-precision floating-point representation of update amounts / gradients into low-bit integer representations, reducing data width and transmission volume without losing key update information. Sparse uploading retains and uploads only update amounts / gradient components with amplitudes greater than a preset threshold, eliminating redundant components that contribute little to model updates, further reducing the amount of effective transmitted data. Group uploading can complete uploads in batches and with priority based on parameter type, importance, or data sensitivity, improving transmission stability in bandwidth-constrained scenarios. Through a combination of one or more compression methods, the communication pressure between edge nodes and the central server is significantly reduced without affecting model convergence speed and monitoring accuracy, meeting the distributed training needs of complex geographical and ecological monitoring scenarios such as weak networks and low bandwidth.

[0044] By using the aforementioned communication compression mechanism, lightweight federated training can be achieved while avoiding link blockage and training interruption caused by excessive data transmission, thereby improving the robustness and long-term operational stability of distributed training in geographic ecological monitoring scenarios.

[0045] According to an embodiment of the present invention, the method of generating an initial global feature prototype by aggregating local feature prototypes further includes: during the first aggregation, performing an unweighted average calculation on each local feature prototype to generate an initial global feature prototype; during the second and subsequent aggregations, aggregating each local feature prototype based on the aggregation weights adaptively allocated in the previous aggregation process to update the initial global feature prototype.

[0046] Specifically, in this embodiment, the initial global feature prototype refers to the first-generation global feature prototype generated during the first iteration of federated learning. After being updated in subsequent iterations, it is uniformly referred to as the global feature prototype. Both are representations of the same feature center at different iteration stages, and their meanings and calculation rules remain consistent. The generation of the global feature prototype adopts a phased aggregation strategy to adapt to the dynamic changes in data distribution during federated learning iterations, thereby improving the accuracy and stability of the global prototype.

[0047] In the first iteration aggregation phase of federated learning, the central server has not yet obtained the data distribution difference information of each edge node. Therefore, it performs unweighted average calculation on the local feature prototypes uploaded by all nodes, treating each local feature prototype as equally important, and generates the initial global feature prototype by directly averaging them. This ensures that the initial global feature prototype can objectively and evenly reflect the feature distribution of all participating nodes, providing a stable benchmark for subsequent iteration aggregation.

[0048] The central server constructs a global feature prototype that satisfies the following formula: , in, For edge monitoring nodes In category The number of samples or its approximation. For nodes Corresponding category The local feature prototype, Used to characterize categories A unified distribution center in the global feature space.

[0049] In the second and subsequent iterations of aggregation, the central server, having already completed the adaptive aggregation weight allocation in the previous iteration, possesses prior information about the differences between the data distribution of each node and the global distribution. Therefore, instead of using average aggregation, it uses the adaptive aggregation weights obtained in the previous iteration, combined with the aforementioned global feature prototype construction rules, to perform weighted aggregation of the local feature prototypes of each node, thereby dynamically updating the global feature prototype. Through this phased aggregation strategy, the first iteration obtains an initial global feature prototype to ensure fairness and universality. Subsequent iterations rely on adaptive weights to strengthen the contributions of high-quality nodes that are closer to the global distribution, while weakening the interference of abnormal nodes. This allows the updated global feature prototype to continuously conform to the non-independent and identically distributed characteristics of each edge node, effectively suppressing client drift and improving the convergence speed and generalization performance of the global model.

[0050] According to an embodiment of the present invention, the aggregation weights are adaptively allocated based on the differences between each local feature prototype and the initial global feature prototype, specifically including: The aggregate weights are negatively correlated with the difference between the local feature prototype and the initial global feature prototype.

[0051] Specifically, the aggregation weight is used to characterize the degree of matching between the local feature distribution and the global feature distribution of each edge monitoring node. Its allocation rule is negatively correlated with the distribution difference between the local feature prototype and the initial global feature prototype. That is, the smaller the difference between the local feature prototype and the initial global feature prototype of the edge monitoring node and the closer their distributions are, the larger the aggregation weight is assigned; the greater the difference between the local feature prototype and the initial global feature prototype of the edge monitoring node and the more significant the distribution deviation, the smaller the aggregation weight is assigned, thereby suppressing client drift caused by non-independent and identically distributed data.

[0052] Based on this negative correlation rule, the aggregation weights can be determined using an exponential normalization method: , in, For the first The aggregate weight of each edge monitoring node This represents the difference in distribution between the local feature prototype and the initial global feature prototype of the node. The attenuation coefficient is... This represents the total number of edge monitoring nodes.

[0053] According to an embodiment of the present invention, the update amount / gradient of the update parameters of each low-rank adapter module is weighted and aggregated based on the aggregation weight to obtain a global model; and the updated global feature prototype is obtained by weighted aggregation based on the aggregation weight.

[0054] Specifically, the central server uses the aggregate weights obtained through adaptive allocation as a basis to simultaneously execute two weighted aggregation operations, so that the global model and the global feature prototype maintain a unified optimization rhythm and distribution consistency.

[0055] On the one hand, the update amounts / gradients of the low-rank adapter module update parameters uploaded by each edge monitoring node are weighted and aggregated to obtain a global model that satisfies the following formula: ， in, This represents the parameter update amount for the global model. For the first The aggregate weights corresponding to each edge monitoring node For the first The update amount / gradient of the low-rank adapter module update parameters uploaded by each edge monitoring node. This represents the total number of edge monitoring nodes participating in the federal training.

[0056] On the other hand, the local feature prototypes of each edge monitoring node are weighted and aggregated using the same set of aggregation weights to obtain the updated global feature prototype, which satisfies the following formula: , in, For the updated category A unified distribution center in the global feature space For the first The aggregate weights corresponding to each edge monitoring node For the first Each edge monitoring node corresponds to a category The local feature prototype.

[0057] By using the aforementioned dual-path synchronous weighted aggregation, the adverse effects of high-offset nodes on global training can be effectively mitigated, the contribution of high-quality nodes can be strengthened, client drift can be further suppressed, and the convergence stability and cross-regional generalization performance of the global model under non-independent and identically distributed geographic ecological monitoring data can be improved.

[0058] According to an embodiment of the present invention, prototype alignment regularization training is performed using the updated global feature prototype as a constraint, specifically including: During the training of the low-rank adapter module, the updated global feature prototype is used as a benchmark to constrain the distribution differences between each local feature prototype and the updated global feature prototype, so that the local features of each edge node are aligned with the global feature space.

[0059] Specifically, when each edge monitoring node trains the low-rank adapter module locally, it uses the updated global feature prototype as a unified supervision benchmark. Through prototype alignment regularization constraints, it forces the locally extracted features to move closer to the global feature prototype of the corresponding category, ensuring the consistency between the local feature distribution and the global feature space.

[0060] Reference Figure 4 This illustration shows a schematic diagram of the global feature prototype weighted aggregation and alignment process under non-independent and identically distributed data provided in the embodiments of the present invention. Through adaptive weight aggregation and prototype constraint training, the unified alignment of local features and global feature space is achieved.

[0061] In this embodiment, the prototype alignment regularization loss is expressed by the following formula: , in, For prototype alignment regularization loss, For batch data used in local training, Input the samples and their corresponding labels within the batch. For the first The low-rank adapter module of each edge monitoring node processes the input sample. Extracted feature vectors, These are the fixed backbone network parameters in the model. For the first Parameters of the low-rank adapter module for each edge monitoring node. For tags The corresponding updated global feature prototype, This is the square operation for the L2 norm.

[0062] During local model training, the prototype alignment loss and task monitoring loss are weighted and fused together to guide the parameter update of the low-rank adapter module. While ensuring the accuracy of the ecological monitoring task, the local feature distribution is forced to be consistent with the global feature space, which effectively alleviates the client drift problem caused by non-independent and identically distributed data, improves cross-node feature compatibility and the overall generalization performance of the global model, and finally achieves stable alignment between the local features of each edge node and the global feature space.

[0063] like Figure 5 As shown, the present invention also provides a geospatial large model low-rank adaptation and privacy protection system 100, comprising: a local training module 110, used to load a pre-trained geospatial large model and insert a low-rank adapter module, freeze the backbone parameters of the geospatial large model, and train only the low-rank adapter module; extract features based on local remote sensing ecological monitoring data and calculate local feature prototypes; a privacy protection and communication module 120, used to perform hierarchical differential privacy processing and communication compression on the updated parameters of the low-rank adapter module and the local feature prototypes, and then upload them to a central server; a global aggregation module 130, used to aggregate and generate an initial global feature prototype based on each local feature prototype; adaptively allocate aggregation weights based on the differences between each local feature prototype and the initial global feature prototype, and aggregate to obtain a global model and an updated global feature prototype; and a global distribution and alignment module 140, used to distribute the global model and perform prototype alignment regularization training with the updated global feature prototype as a constraint, so that each local feature is aligned with the global feature space.

[0064] In this embodiment, the low-rank adaptation and privacy protection system for large geospatial models corresponds one-to-one with the corresponding steps in the aforementioned method embodiments. The specific working principles, implementation processes, and technical effects of each module can be referred to the corresponding content in the aforementioned method embodiments. To avoid repetition, they will not be repeated here.

[0065] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a geospatial large model low-rank adaptation and privacy protection method, which includes: Load the pre-trained geospatial large model and insert the low-rank adapter module, freeze the backbone parameters of the geospatial large model, and train only the low-rank adapter module; Features are extracted based on local remote sensing ecological monitoring data, and local feature prototypes are calculated; After performing hierarchical differential privacy processing and communication compression on the updated parameters of the low-rank adapter module and the local feature prototype, the data is uploaded to the central server. An initial global feature prototype is generated by aggregating the local feature prototypes described above. Based on the differences between each local feature prototype and the initial global feature prototype, aggregation weights are adaptively assigned to obtain the global model and the updated global feature prototype. The global model is then distributed, and prototype alignment regularization training is performed using the updated global feature prototype as a constraint to align each local feature with the global feature space.

[0066] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the geospatial large model low-rank adaptation and privacy protection method provided by the above methods, the method comprising: Load the pre-trained geospatial large model and insert the low-rank adapter module, freeze the backbone parameters of the geospatial large model, and train only the low-rank adapter module; Features are extracted based on local remote sensing ecological monitoring data, and local feature prototypes are calculated; After performing hierarchical differential privacy processing and communication compression on the updated parameters of the low-rank adapter module and the local feature prototype, the data is uploaded to the central server. An initial global feature prototype is generated by aggregating the local feature prototypes described above. Based on the differences between each local feature prototype and the initial global feature prototype, aggregation weights are adaptively assigned to obtain the global model and the updated global feature prototype. The global model is then distributed, and prototype alignment regularization training is performed using the updated global feature prototype as a constraint to align each local feature with the global feature space.

[0068] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0069] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for low-rank adaptation and privacy protection in large geospatial models, characterized in that, include: Load the pre-trained geospatial large model and insert the low-rank adapter module, freeze the backbone parameters of the geospatial large model, and train only the low-rank adapter module; Features are extracted based on local remote sensing ecological monitoring data, and local feature prototypes are calculated; After performing hierarchical differential privacy processing and communication compression on the updated parameters of the low-rank adapter module and the local feature prototype, the data is uploaded to the central server. An initial global feature prototype is generated by aggregating the local feature prototypes described above. Based on the differences between each local feature prototype and the initial global feature prototype, aggregation weights are adaptively assigned to obtain the global model and the updated global feature prototype. The global model is then distributed, and prototype alignment regularization training is performed using the updated global feature prototype as a constraint to align each local feature with the global feature space.

2. The method according to claim 1, characterized in that, The hierarchical differential privacy processing performed on the updated parameters of the low-rank adapter module and the local feature prototype specifically includes: The update amount / gradient of the update parameters of the low-rank adapter module to be uploaded and / or the statistics of the local feature prototype are grouped into a spatiotemporally sensitive parameter group that interacts with spatiotemporal coding and a deep semantic parameter group. A smaller pruning threshold and stronger noise are applied to the spatiotemporally sensitive parameter group; A larger pruning threshold is applied to the deep semantic parameter set, and weaker noise is injected.

3. The method according to claim 1, characterized in that, The communication compression between the updated parameters of the low-rank adapter module and the local feature prototype specifically includes: Only the update amount / gradient of the update parameters of the low-rank adapter module and the local feature prototype are uploaded, and at least one of quantization compression, sparse upload or group upload is performed on them; The quantization compression includes converting floating-point update amounts / gradients into low-bit integer representations, and the sparse uploading includes uploading only update amount / gradient components with amplitudes greater than a preset threshold.

4. The method according to claim 1, characterized in that, The step of aggregating and generating an initial global feature prototype based on each of the local feature prototypes further includes: During the initial aggregation, an unweighted average is calculated on each of the local feature prototypes to generate the initial global feature prototype; In the second and subsequent aggregations, the local feature prototypes are aggregated based on the aggregation weights adaptively allocated in the previous aggregation process, and the initial global feature prototypes are updated.

5. The method according to claim 1, characterized in that, The adaptive allocation of aggregation weights based on the differences between each local feature prototype and the initial global feature prototype specifically includes: The aggregation weights are negatively correlated with the differences between the local feature prototypes and the initial global feature prototypes.

6. The method according to claim 5, characterized in that, The update amount / gradient of the update parameters of each low-rank adapter module are weighted and aggregated based on the aggregation weight to obtain the global model; The local feature prototypes are weighted and aggregated based on the aggregation weights to obtain the updated global feature prototypes.

7. The method according to claim 1, characterized in that, The step of performing prototype alignment regularization training using the updated global feature prototype as a constraint specifically includes: During the training process of the low-rank adapter module, the updated global feature prototype is used as a reference to constrain the distribution differences between each local feature prototype and the updated global feature prototype, so that the local features of each edge node are aligned with the global feature space.

8. A low-rank adaptation and privacy protection system for large geospatial models, characterized in that, include: The local training module is used to load a pre-trained geospatial large model and insert a low-rank adapter module, freeze the backbone parameters of the geospatial large model, and train only the low-rank adapter module; it also extracts features based on local remote sensing ecological monitoring data and calculates local feature prototypes. The privacy protection and communication module is used to perform hierarchical differential privacy processing and communication compression on the updated parameters of the low-rank adapter module and the local feature prototype, and then upload them to the central server. The global aggregation module is used to aggregate and generate an initial global feature prototype based on each local feature prototype; and to adaptively allocate aggregation weights based on the differences between each local feature prototype and the initial global feature prototype, thereby aggregating to obtain a global model and an updated global feature prototype. The global distribution and alignment module is used to distribute the global model and perform prototype alignment regularization training with the updated global feature prototype as a constraint, so that each local feature is aligned with the global feature space.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the geospatial large model low-rank adaptation and privacy protection method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the geospatial large model low-rank adaptation and privacy protection method as described in any one of claims 1 to 7.

Images