Satellite-ground collaborative learning method and system and electronic equipment

Abstract

The invention relates to a satellite-ground collaborative learning method and system and electronic equipment, and relates to the technical field of satellite remote sensing, and the method comprises the steps: compressing a basic model into a small model through a ground station, and enabling the small model to serve as a satellite in-orbit model; detecting the comprehensive distribution offset of the satellite during the in-orbit reasoning period, and when the comprehensive distribution offset is greater than a preset threshold value, sending related data to a ground station; calculating a weighted difference value between a prediction result of the on-orbit model and a prediction result of the basic model based on the related data, and triggering updating of the on-orbit model when the weighted difference value exceeds a preset threshold value; loRA fine tuning is carried out on the small model through the ground station so as to realize on-orbit model updating; and the steps are repeatedly executed until the updating frequency of the on-orbit model reaches the preset frequency, and the on-orbit model is aggregated into the basic model through the ground station to achieve model updating of the basic model. According to the scheme, the overall performance of the satellite on-orbit model facing the distributed external scene can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of satellite remote sensing technology, specifically to a satellite-ground collaborative learning method, system, and electronic equipment. Background Technology

[0002] Satellite platforms equipped with intelligent models can achieve real-time analysis, processing, and autonomous decision-making of remote sensing data, significantly improving application efficiency in fields such as disaster monitoring, marine monitoring, environmental monitoring, and commercial applications. However, as remote sensing models develop towards larger scales and greater functional diversity, their computational demands are growing exponentially. Due to physical size and power supply limitations, satellite platforms face severe computational bottlenecks, forcing on-orbit models to be highly compressed versions of the original ground-based models. This results in a significant reduction in their robustness to out-of-distribution (OOD) scenarios.

[0003] Currently, all on-orbit models follow a "learn-deployment" paradigm: parameters are fixed after ground training and before launch, and updates rely entirely on occasional human intervention. This static, closed-loop training strategy is inherently unable to adapt to data drift in dynamic spatiotemporal environments, and is particularly vulnerable to sporadic and unpredictable distribution shifts such as domain OOD, fine-grained OOD, and format OOD, leading to severe performance degradation of satellite on-orbit models in actual missions. Summary of the Invention

[0004] In view of this, this disclosure provides a satellite-ground collaborative learning method, system and electronic equipment, the main purpose of which is to solve the technical problem of severe performance degradation of satellite on-orbit models when facing distributed off-site scenarios.

[0005] According to a first aspect of this disclosure, a satellite-ground collaborative learning method is provided, the method comprising: Step 1: When the satellite in orbit joins the completed satellite-ground collaborative learning system, the ground station of the satellite-ground collaborative learning system compresses the basic model into a small model and sends the small model to the satellite in orbit through the uplink, so that the satellite in orbit can use the compressed small model as its in-orbit model. Step 2: Use the OOD detector to detect the overall distribution offset during the on-orbit inference of the satellite. When the overall distribution offset is detected to be greater than the preset overall distribution offset threshold, trigger the data downlink to send the relevant data to the ground station. Step 3: Based on the relevant data received by the ground station from the on-orbit satellite, calculate the weighted difference statistics between the prediction results of the on-orbit model and the prediction results of the ground station's basic model, and trigger a selective update of the on-orbit model when the weighted difference statistics exceed a preset difference statistics threshold; Step 4: In the selective update phase of the on-orbit model, the ground station performs fine-tuning on the small model corresponding to the on-orbit satellite, generates fine-tuned update parameters, and sends the fine-tuned update parameters to the on-orbit satellite via the uplink so that the on-orbit satellite loads the update parameters and completes the update of the on-orbit model; Repeat steps 1-4 above until the cumulative number of updates to the on-orbit model reaches the preset number of updates. Then, aggregate one or more updated on-orbit models into the base model through the ground station to update the base model.

[0006] According to a second aspect of this disclosure, a satellite-ground collaborative learning system is provided, comprising: a pull module, a detection module, a first update module, a fine-tuning module, and a second update module; The pull module is used to compress the basic model into a small model through the ground station of the satellite-ground collaborative learning system when the satellite in orbit joins the completed satellite-ground collaborative learning system, and send the small model to the satellite in orbit through the uplink so that the satellite in orbit can use the compressed small model as the in-orbit model. The detection module is used to detect the comprehensive distribution offset during the on-orbit inference of the on-orbit satellite using an OOD detector. When the detected comprehensive distribution offset is greater than a preset comprehensive distribution offset threshold, the data downlink is triggered to send the relevant data to the ground station. The first update module is used to calculate the weighted difference statistics between the prediction results of the on-orbit model and the prediction results of the basic model of the ground station based on the relevant data received by the ground station from the on-orbit satellite, and to trigger a selective update of the on-orbit model when the weighted difference statistics exceed a preset difference statistics threshold. The fine-tuning module is used to perform fine-tuning on the small model corresponding to the on-orbit satellite at the ground station during the selective update phase of the on-orbit model, generate fine-tuned update parameters, and send the fine-tuned update parameters to the on-orbit satellite through the uplink so that the on-orbit satellite loads the update parameters and completes the update of the on-orbit model. The second update module is used to repeatedly execute the above steps until the cumulative number of updates of the on-orbit model reaches the preset number of updates, and then aggregate one or more updated on-orbit models into the base model through the ground station to realize the update of the base model.

[0007] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the satellite-ground collaborative learning method of the first aspect described above.

[0008] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the satellite-ground collaborative learning method of the first aspect described above.

[0009] Compared with existing technologies, the satellite-ground collaborative learning method, system, and electronic equipment disclosed herein, when an on-orbit satellite joins a completed satellite-ground collaborative learning system, compresses the basic model into a smaller model through the ground station of the satellite-ground collaborative learning system, and sends the smaller model to the on-orbit satellite via uplink, so that the on-orbit satellite can use the compressed smaller model as its on-orbit model; an OOD detector is used to detect the integrated distribution offset during the on-orbit inference of the on-orbit satellite, and when the detected integrated distribution offset is greater than a preset integrated distribution offset threshold, a data downlink is triggered to send relevant data to the ground station; based on the relevant data received by the ground station from the on-orbit satellite, calculation is performed. The system calculates the weighted statistical difference between the prediction results of the on-orbit model and the prediction results of the base model at the ground station. When the weighted statistical difference exceeds a preset statistical difference threshold, a selective update of the on-orbit model is triggered. During the selective update phase of the on-orbit model, the ground station performs fine-tuning on the small model corresponding to the on-orbit satellite, generates fine-tuned update parameters, and sends the fine-tuned update parameters to the on-orbit satellite via uplink so that the on-orbit satellite can load the update parameters and complete the update of the on-orbit model. The above steps are repeated until the cumulative number of updates of the on-orbit model reaches a preset number of updates. Then, the ground station aggregates one or more updated on-orbit models into the base model to achieve the model update of the base model.

[0010] By applying the scheme disclosed herein, a dual-mechanism OOD detector is used to achieve high-bandwidth downlink data transmission. A single update requires the transmission of no more than 7M parameters, with a data transmission volume of approximately 13.4MB (FP16 precision). This occupies only 4.2% to 8.3% of available time during typical on-orbit satellite transit times (5-10 minutes). This significantly reduces the impact of data transmission on on-orbit satellite communication resources, improves data transmission efficiency, and enables timely transmission of relevant data to the ground station under limited on-orbit satellite communication conditions, providing necessary information for model updates.

[0011] By using ground-based LoRA fine-tuning triggered by differences in output between the on-orbit satellite and the base model, efficient model updates are achieved, significantly reducing communication overhead. LoRA fine-tuning only requires adjusting a subset of model parameters to update the on-orbit model, reducing the number of parameters that need to be transmitted and updated, thereby lowering communication costs and improving the efficiency of model updates.

[0012] By adaptively distilling and aggregating multi-satellite knowledge, continuous optimization of the base model is achieved. With a seven-satellite annotation setting, the average accuracy for classification tasks improved from 59.8% to 66.0%, an improvement of 6.2%. For tasks such as domain OOD, fine-grained OOD, and format OOD, the average performance improved from 40.6% to 50.1%, an improvement of 9.5%, significantly outperforming the "learn-deployment" baseline method. In the image description task, the ROUGE-L score improved from 28.8% to 29.1%. This demonstrates that this method can effectively improve the performance of the base model, enabling it to better adapt to various out-of-distribution scenarios, thereby providing a higher-quality foundation for satellite-based models and improving the overall performance of satellite-based models when facing out-of-distribution scenarios. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0014] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating a satellite-ground collaborative learning method provided in an embodiment of this disclosure; Figure 2This is a learning flowchart of a satellite-ground collaborative learning system provided in an embodiment of the present disclosure; Figure 3 This is a schematic diagram of the structure of a satellite-ground collaborative learning system provided in an embodiment of this disclosure. Detailed Implementation

[0016] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments of this disclosure and the features described therein can be combined with each other.

[0017] The satellite-ground collaborative learning method, system, and electronic equipment of this disclosure are described below with reference to the accompanying drawings.

[0018] To address the technical problem of severe performance degradation of satellite on-orbit models when facing distributed off-site scenarios, embodiments of this disclosure provide a satellite-ground collaborative learning method, such as... Figure 1 As shown, the method includes: Step 101: When the satellite in orbit joins the completed satellite-ground collaborative learning system, the ground station of the satellite-ground collaborative learning system compresses the basic model into a small model and sends the small model to the satellite in orbit through the uplink, so that the satellite in orbit can use the compressed small model as its in-orbit model.

[0019] This disclosure allows for the initial construction of an on-orbit satellite-ground collaborative learning system (SGCL system), comprising a group of on-orbit satellites and ground stations, employing a star topology, where each on-orbit satellite is directly connected to a ground station. For example... Figure 2 As shown, in this system, the ground station is responsible for training and maintaining the base model. An important mission, and each satellite in orbit It has its own custom on-orbit model Through an asynchronous update mechanism between the ground-based basic model and the on-orbit model, the on-orbit satellite model can achieve autonomous evolution.

[0020] During the bootstrapping phase, the ground station can begin training the first base model from the initial dataset. In this embodiment, the initial dataset can be data related to the remote sensing visual language model. Simultaneously, a large amount of Earth system data, numerous datasets, and open-source base models (such as GeoChat) can be fully utilized to assist in building the initial model.

[0021] The base model can employ the CLIP-ViT-L / 14 visual encoder and the Vicuna-v1.5-7B language model. This architecture combines powerful visual feature extraction capabilities with language understanding capabilities, enabling better handling of multimodal data.

[0022] To enable the base model to quickly adapt to different tasks and scenarios, LoRA (Low-Rank Adaptation) technology can be used for efficient adaptation. LoRA technology adds a low-rank matrix to the original model parameters, achieving fine-tuning of the model with less parameter adjustment, which greatly reduces the computational and time costs of model updates.

[0023] The training process of the base model optimizes the following objective function:

[0024] In the formula, Let be the objective function. For cross-entropy loss, For visual-text alignment loss, This is the balance coefficient.

[0025] When an in-orbit satellite joins the completed SGCL system, it can request the latest model from the ground station. This request can be initiated proactively after the in-orbit satellite joins the system, or the latest model can be loaded before the in-orbit satellite is launched, depending on the actual mission requirements and system design.

[0026] After receiving a request from an in-orbit satellite, the ground station can compress its latest trained base model into a smaller model. In this embodiment, Qwen2-VL-2B can be selected as the compressed smaller model. The compression process is achieved through knowledge distillation, which is a technique for transferring knowledge from a large model (teacher model) to a smaller model (student model). During the compression process, visual-text alignment is simultaneously optimized using a mean squared error loss function, as shown below:

[0027] In the formula, For visual-text alignment loss, The embedding vector of the base model, For the embedding vector of the in-orbit satellite model, This refers to the batch size.

[0028] The ground station transmits the compressed, small model to the on-orbit satellite via uplink. Upon receiving the model, the on-orbit satellite uses it as its first on-orbit model and performs inference tasks in subsequent time periods. In this way, the on-orbit satellite can quickly obtain a high-performance model under limited computing resources and communication bandwidth, effectively improving the adaptability of the on-orbit model to different scenarios and its task execution efficiency.

[0029] Step 102: Use the OOD detector to detect the integrated distribution offset during the on-orbit inference of the satellite. When the detected integrated distribution offset is greater than the preset integrated distribution offset threshold, trigger the data downlink to send the relevant data to the ground station.

[0030] In this embodiment of the disclosure, a lightweight distributed offset monitoring module has been deployed on the satellite in orbit. This module includes three sub-mechanisms: visual anomaly detection, cue offset detection, and multimodal misalignment detection. The relevant parameters of each mechanism have been initialized based on the training set data, such as dynamic threshold, preset offset threshold, preset cosine similarity threshold, and preset comprehensive distributed offset threshold.

[0031] Specifically, such as Figure 2 As shown, the OOD detector detects the integrated distribution offset during on-orbit inference of the satellite based on feature space reconstruction error and multimodal alignment metric. When the detected integrated distribution offset exceeds a preset integrated distribution offset threshold, a data downlink is triggered to send relevant data to the ground station, which may include: Acquire the current batch of data collected by the on-orbit satellite and use the on-orbit model encoder of the on-orbit satellite to extract the visual and text features of the current batch of data; The reconstruction error of visual features is calculated by a lightweight autoencoder. When the reconstruction error is greater than a preset dynamic error threshold, the current batch of data is marked as a visual anomaly. The distribution offset of text features is calculated based on Mahalanobis distance. When the distribution offset is greater than a preset offset threshold, the current batch of data is marked as an offset anomaly. Calculate the cosine similarity between visual features and text features. When the cosine similarity is lower than the preset cosine similarity threshold, mark the current batch of data as a multimodal misalignment anomaly. Based on the detection results of visual anomalies, offset anomalies, and / or multimodal misalignment anomalies, the comprehensive distribution offset is calculated. When the comprehensive distribution offset is greater than the preset comprehensive distribution offset threshold, the data downlink is triggered to send the relevant data to the ground station.

[0032] In this embodiment, during the on-orbit operation of the satellite, observation data of batches at the current time t can be acquired in real time according to a preset data acquisition strategy. The observation data may include visual data (such as images taken by satellites in orbit) and text data (such as text descriptions and instructions related to the observation mission).

[0033] The on-orbit model encoder, carried onboard the satellite, can be used to process the current batch of data. Specifically, for visual data, the on-orbit model encoder can extract high-level semantic features of images using structures such as convolutional neural networks (CNNs); for text data, the on-orbit model encoder can extract textual features using natural language processing models (such as the Transformer architecture), ultimately obtaining the visual feature vector of the current batch of data. and text feature vectors .

[0034] The extracted visual feature vectors can be input into a lightweight autoencoder pre-deployed on an on-orbit satellite. An autoencoder can consist of an encoder and a decoder. The encoder compresses the input visual features into a low-dimensional representation, while the decoder attempts to reconstruct the original visual features from the low-dimensional representation. The reconstruction error is obtained by calculating the difference between the reconstructed visual features and the original visual features, as shown in the following formula:

[0035] In the formula, For reconstruction error, AE is a visual feature vector and a lightweight autoencoder.

[0036] The calculated reconstruction error is compared with a preset dynamic error threshold. A comparison is performed. The preset dynamic error threshold can be set based on the training set data. and According to statistics, when the reconstruction error of the current batch of data exceeds the preset dynamic error threshold, the batch of data is marked as a visual anomaly.

[0037] In this embodiment, the OOD module based on Mahalanobis distance processes the text features. First, the mean vector of the text feature vectors can be calculated based on the training set data. Covariance Matrix To improve memory efficiency, Principal Component Analysis (PCA) is used to downsample the text features of the training set, reducing computation while preserving key information. Then, for the text feature vectors of the current batch of data, the Mahalanobis distance between them and the mean of the training set features is calculated, as shown in the following formula:

[0038] In the formula, The Mahalanobis distance, For text feature vectors, The covariance matrix of the text feature vectors. Let be the covariance matrix of the text feature vectors.

[0039] The calculated Mahalanobis distance is compared with a preset offset threshold. This preset offset threshold can be obtained statistically from the training set data, such as selecting the maximum value of the Mahalanobis distance in the training set data or a value of a high percentile as the threshold. When the Mahalanobis distance of the current batch of data is greater than the preset offset threshold (e.g., ...), the threshold is set to be set. When this happens, mark the batch of data as having an offset anomaly.

[0040] In this embodiment, visual feature vectors can be calculated. and text feature vectors The cosine similarity between the pairs of pairs. The cosine similarity can be calculated using the following formula:

[0041] In the formula, Visual feature vectors and text feature vectors Cosine similarity between them For visual feature vectors, This is the text feature vector.

[0042] The calculated cosine similarity is compared with a preset cosine similarity threshold. The preset cosine similarity threshold can be set according to the actual application scenario and the requirements for multimodal consistency. When the cosine similarity of the current batch of data is lower than the preset cosine similarity threshold (e.g., ...), the similarity is calculated. When this occurs, the batch of data is marked as a multimodal misalignment anomaly.

[0043] The overall distributed offset (i.e., offset severity) can be calculated based on the detection results of visual anomalies, offset anomalies, and / or multimodal misalignment anomalies. It can be calculated using the following formula:

[0044] In the formula, This represents the overall distribution offset (i.e., the severity of the offset). For batch size, For reconstruction error, To preset the dynamic threshold of error, The Mahalanobis distance, For the preset offset threshold, Cosine similarity between visual feature vectors and text feature vectors. This is a preset cosine similarity threshold.

[0045] The calculated overall distribution offset is compared with a preset overall distribution offset threshold, which can be set based on the system's tolerance for distribution offset and actual application requirements. When the overall distribution offset exceeds the preset threshold... When this occurs, it indicates a significant distribution shift in the current batch of data, which may affect the inference performance of the on-orbit model of the on-orbit satellite. At this time, the on-orbit satellite triggers the data downlink to send relevant data (including raw visual data, text data, and information such as feature vectors, reconstruction errors, Mahalanobis distance, and cosine similarity obtained during the detection process) to the ground station to send a data offset alert to the ground station.

[0046] Step 103: Based on the relevant data received by the ground station from the on-orbit satellite, calculate the weighted difference statistics between the prediction results of the on-orbit model and the prediction results of the ground station's basic model, and trigger a selective update of the on-orbit model when the weighted difference statistics exceed the preset difference statistics threshold.

[0047] During the operation of an in-orbit satellite, such as Figure 2 As shown, due to the continuous changes in actual scenarios and tasks, the on-orbit model carried on the satellite may gradually become unable to accurately handle various situations. In order to ensure that the on-orbit model always maintains good performance, it is necessary to compare and analyze the prediction results of the on-orbit model with the prediction results of the ground station's basic model based on the relevant data received by the ground station from the on-orbit satellite, and determine whether selective updates to the on-orbit model are needed by calculating the weighted difference statistics.

[0048] Specifically, step 1: calculate the prediction probability distribution and prediction confidence of the ground station's basic model for the relevant data, and calculate the prediction probability distribution of the on-orbit model of the on-orbit satellite for the relevant data. In this embodiment, the ground station can receive relevant data transmitted by the on-orbit satellite. The relevant data can include various types of information collected by the satellite during its on-orbit operation, such as images and sensor data. This data has already undergone preliminary processing at the satellite end to extract features that can be used for model prediction. The ground station organizes and preprocesses this data to ensure that the data format meets the model input requirements.

[0049] Ground stations can use pre-trained base models to make predictions on received relevant data. During the prediction process, the base model not only outputs the predicted probability distribution of the relevant data, but also... This outputs the probability value for each possible category, along with the prediction confidence score. This confidence score reflects the reliability of the base model's predictions. For example, in an image classification task, the base model outputs the probability that an image belongs to each category, along with a confidence score that indicates the reliability of the prediction.

[0050] The on-orbit model of the in-orbit satellite also makes predictions based on the same relevant data, outputting a predicted probability distribution. The on-orbit model of an on-orbit satellite is trained on certain data before the launch of the on-orbit satellite and is used for real-time inference during the on-orbit operation of the on-orbit satellite.

[0051] Step 2: Calculate the KL divergence between the predicted probability distribution of the base model and the predicted probability distribution of the on-orbit model, and use the prediction confidence of the base model to perform a weighted calculation on the KL divergence to obtain the weighted difference value for each data point. In this embodiment, for each data point, the KL divergence between the predicted probability distribution of the ground station's basic model and the predicted probability distribution of the on-orbit model of the on-orbit satellite can be calculated. The KL divergence is an indicator used to measure the difference between two probability distributions, and its calculation formula is as follows:

[0052] In the formula, Let KL divergence be the KL divergence. To predict probability distributions based on the basic model, Predict probability distributions for in-orbit models.

[0053] The calculated KL divergence can be weighted using the prediction confidence score of the ground station's basic model. Specifically, for each data point, the KL divergence value can be multiplied by the prediction confidence score of the basic model for that data point to obtain the weighted difference value for each data point. The calculation formula is as follows:

[0054] In the formula, The weighted difference value for each data point. Let KL divergence be the KL divergence. To predict probability distributions based on the basic model, To predict the probability distribution for the in-orbit model, The prediction confidence of the basic model.

[0055] Step 3: Calculate the weighted difference statistics based on the weighted difference values ​​of all data points; In this embodiment, the median of the weighted difference values ​​of all data points can be calculated based on the weighted difference values ​​of all data points. The median is the absolute deviation from the median, where the median is the value in the middle after all weighted variances are arranged in ascending order. The median and MAD can reflect the central tendency and dispersion of the weighted variances.

[0056] The outlier threshold can be determined based on the median and the absolute deviation (MAD). The calculation formula is as follows:

[0057] In the formula, The threshold for outliers. This is the threshold coefficient, and its value can be adjusted according to the actual application scenario and the strictness of outlier detection. It is usually set to 1. MAD is the absolute deviation of the weighted difference values ​​for all data points. This is the median of the weighted differences across all data points.

[0058] Data points whose weighted variance values ​​exceed an outlier threshold can be identified as outliers. To exclude abnormally large variance values ​​caused by special circumstances or noise, and to ensure that subsequent calculations better reflect the true differences between models, these outliers can be filtered to obtain a filtered weighted variance set. .

[0059] Finally, the average of the filtered weighted difference set can be calculated. , as a weighted difference statistic.

[0060] Step 4: When the weighted difference statistic Exceeding the preset difference statistical threshold At that time, an update to the on-orbit model is triggered.

[0061] In this embodiment, the calculated weighted difference statistics can be used. Statistical threshold for differences from preset threshold Comparisons are made. The preset difference statistical threshold can be set in advance according to actual application needs and the required model update frequency. When the weighted difference statistical value... Exceeding the preset difference statistical threshold When this occurs, it indicates a significant difference between the prediction results of the on-orbit model and the prediction results of the ground station basic model. The on-orbit model may no longer be able to accurately handle the current changing scenarios and tasks. At this time, selective updates to the on-orbit model can be automatically triggered.

[0062] Step 104: In the selective update phase of the on-orbit model, the ground station performs fine-tuning on the small model corresponding to the on-orbit satellite, generates fine-tuned update parameters, and sends the fine-tuned update parameters to the on-orbit satellite via the uplink so that the on-orbit satellite loads the update parameters and completes the update of the on-orbit model.

[0063] During the operation of an in-orbit satellite, such as Figure 2 As shown, when the weighted statistical difference between the prediction results of the on-orbit model and the prediction results of the ground station's basic model exceeds a preset statistical difference threshold, it indicates that the on-orbit model may be unable to accurately handle the current changing scenarios and tasks, requiring selective updates. To efficiently and effectively update the on-orbit model, knowledge distillation techniques can be used to perform LoRA (Low-Rank Adaptation) fine-tuning. This method can significantly reduce the number of parameters transmitted during each model update while avoiding forgetting the basic context embedded in the fully fine-tuned model, thus meeting the efficiency and performance requirements of on-orbit satellite operation for model updates.

[0064] When using knowledge distillation techniques to perform LoRA fine-tuning on small models corresponding to in-orbit satellites, a composite loss function can be constructed to guide model training and achieve model updates in orbit. This composite loss function can be a weighted sum of a first loss term and a second loss term. The first loss term can be the cross-entropy loss, which measures the difference between the in-orbit model's predictions and the data labels. The second loss term can be the KL divergence term, which measures the difference between the in-orbit model's predictions and the predictions of the ground-based model. The composite loss function can be expressed as follows:

[0065] In the formula, Represents the composite loss function. Represents the balance coefficient. Represents the cross-entropy loss function. Indicates KL divergence, superscript and These represent the base model and the on-orbit model, respectively. Indicates data labels, This represents the probability distribution of the on-orbit model's predictions for the entire dataset. This represents the probability distribution of the base model's predictions for the entire dataset.

[0066] To meet the different needs of on-orbit missions, two update modes are supported: The first type is manual annotation mode: In automated mode, the base model uses the data results as pseudo-labels. However, in manual annotation mode, experts can correct the pseudo-labels. Experts review and adjust the pseudo-labels generated by the base model based on their professional knowledge and experience to ensure the accuracy and reliability of the labels.

[0067] Guided by expert-corrected pseudo-labels, LoRA fine-tuning was performed on the on-orbit model using a composite loss function. LoRA fine-tuning optimizes model performance by introducing low-rank matrices into specific layers of the model to adjust its parameters. During fine-tuning, only the parameters of the low-rank matrices are updated, while keeping other model parameters unchanged, significantly reducing the number of parameters that need to be transferred.

[0068] The second type is the automation mode: The output of the ground-based model can be directly used as a pseudo-label. The ground-based model acts as a teacher model, and its predictions provide a learning target for the on-orbit model.

[0069] The generated pseudo-labels and composite loss function can be used to fine-tune the in-orbit model using LoRA. Similarly, only the parameters of the low-rank matrix are updated, avoiding forgetting the basic context embedded in the fully fine-tuned model, while improving the model's adaptability to the current scene and task.

[0070] After the LoRA fine-tuning process described above, a new on-orbit model is generated. Guided by the composite loss function, the new on-orbit model combines prediction information from data labels and the ground-based model, resulting in improved performance.

[0071] The updated parameters of the new on-orbit model are compressed and encoded to reduce data transmission. Then, through the communication link between the ground station and the on-orbit satellite, the updated parameters are uploaded to the on-orbit satellite via uplink. After receiving the updated parameters, the on-orbit satellite updates the on-orbit model, enabling it to handle subsequent scenarios and missions with better performance.

[0072] Step 105: Repeat steps 101-104 above until the cumulative number of updates of the on-orbit model reaches the preset number of updates. Then, aggregate one or more updated on-orbit models into the base model through the ground station to achieve the update of the base model.

[0073] In a satellite-ground collaborative model update system, after completing the on-orbit model update, the ground station's base model needs to be updated. Since this update involves a many-to-one knowledge transfer from a small model (on-orbit model) to a large model (base model), and the system operates in two different modes—manual annotation and automated operation—with varying data characteristics and requirements, an adaptive knowledge distillation mechanism can be employed. This mechanism uses different update strategies depending on the system's operating mode to aggregate the updated on-orbit models from one or more on-orbit satellites into the base model via the ground station. This achieves efficient and accurate updates to the base model, improving overall system performance.

[0074] In manual annotation mode, update strategies may include: For each updated on-orbit model, collect data from its clients in the corresponding... The cross-entropy loss function value on the batch data. The cross-entropy loss function value can be used to reflect the degree of difference between the model's prediction results and the true data labels. Based on the cross-entropy loss function values ​​of multiple in-orbit models, the weight value of each in-orbit model is calculated, where the in-orbit model... weight The calculation formula is as follows:

[0075] In the formula, Indicating the on-orbit model The weight value, This represents the total number of models in orbit. Indicates client The cross-entropy loss function value on its corresponding batch of data, Represents the cross-entropy loss function. Indicates client Real data labels Indicating the on-orbit model Predicted probability distribution of the full dataset Indicates client The cross-entropy loss function value on its corresponding batch of data, The function converts the loss function values ​​into a probability distribution.

[0076] It can collect all data labels collected by N updated on-orbit models and manually annotated. Obtain the prediction probability distribution of the base model on the full dataset and the prediction probability distribution of each on-orbit model on the full dataset. Calculate the first loss function of the base model according to the following formula:

[0077] In the formula, This represents the first loss function value of the base model. and Both represent balance coefficients. Represents the cross-entropy loss function. Indicates by All data labels collected and manually annotated by the updated on-orbit model. This represents the probability distribution of the base model's predictions for the entire dataset. Indicating the on-orbit model Predicted probability distribution of the full dataset This represents the total number of models in orbit. Denotes KL divergence, Indicating the on-orbit model The weight value.

[0078] Using the calculated first loss function, optimization algorithms such as gradient descent are employed to update the parameters of the base model. By continuously adjusting the parameters of the base model, the value of the first loss function gradually decreases, thereby improving the performance of the base model.

[0079] In automated mode, update strategies may include: For each updated in-orbit model, the average similarity between its output and the outputs of all other in-orbit models is calculated. The Jensen-Shannon divergence is used to measure the similarity between two probability distributions; for the , The average similarity between the outputs of one on-orbit model and other on-orbit models. The calculation is as follows:

[0080] In the formula, Indicates the first The average similarity between the outputs of one in-orbit model and other in-orbit models. The Jensen-Shannon divergence is used to measure the similarity between two probability distributions. Indicating the on-orbit model Predicted probability distribution of the full dataset Indicating the on-orbit model Predicted probability distribution of the full dataset This indicates the total number of models in orbit.

[0081] The weight value of each on-orbit model is calculated based on the average similarity between the output of the on-orbit model and the outputs of all other on-orbit models, as shown in the following formula:

[0082] In the formula, Indicating the on-orbit model The weight value, This represents the total number of models in orbit. Indicating the on-orbit model The average similarity between the output of the model and the outputs of all other on-orbit models.

[0083] The prediction probability distribution of the base model on the full dataset and the prediction probability distribution of each in-orbit model on the full dataset can be obtained. Based on the weight values ​​and the prediction probability distributions of the base model and the in-orbit models, the second loss function of the base model is calculated. The second loss function can be expressed as follows:

[0084] In the formula, This represents the second loss function value of the base model in automated mode. Represents the balance coefficient. This represents the total number of models in orbit. Indicating the on-orbit model The weight value, Denotes KL divergence, This represents the probability distribution of the base model's predictions for the entire dataset. Indicating the on-orbit model The predicted probability distribution of the full dataset.

[0085] Using the calculated second loss function, optimization algorithms such as gradient descent are employed to update the parameters of the base model. By continuously adjusting the parameters of the base model, the value of the second loss function gradually decreases, thereby improving the performance of the base model in automated mode.

[0086] Compared to manual annotation, the cross-entropy term is omitted. Because there are no manually labeled tags available.

[0087] Model output consistency verification mechanism: To ensure the quality of pseudo-labels, the system calculates the consistency of output from multiple satellite models:

[0088] In the formula, This indicates the consistency score of the output of the on-orbit satellite model. This represents the total number of models in orbit. Indicating the on-orbit model Predicted probability distribution of the full dataset This represents the predicted probability distribution of the on-orbit model j over the entire dataset. This represents the Jensen-Shannon divergence, used to measure the similarity between two probability distributions.

[0089] when At that time, the model was considered The output is reliable, so it is incorporated into the knowledge aggregation process.

[0090] This disclosure achieves efficient knowledge transfer between satellite and ground through the collaborative work of the above six stages, effectively solving the problem of insufficient adaptability of satellite on-orbit models to distributed off-site scenarios, while keeping communication overhead at an extremely low level.

[0091] To verify the validity of this disclosure, a satellite-ground collaborative learning benchmark (SGCL-Benchmark) is constructed to simulate multi-satellite on-orbit mission execution and satellite-ground collaborative learning. This benchmark set includes seven on-orbit satellites, each performing five different missions, cycled three times (a total of 15 batches / on-orbit satellites), simulating orbital periodicity and mission diversity.

[0092] In the experiment, GeoChat was used as the ground-based model and Qwen2-VL-2B as the in-orbit model. Three scenarios were set up: configurations of 7 in-orbit satellites, 3 in-orbit satellites, and 1 in-orbit satellite, and the update modes of manual annotation and full automation were compared.

[0093] Table 1 shows a performance comparison of the seven satellites in orbit over three cycles. It can be seen that, compared to the "learn-deployment" baseline, the average accuracy of this disclosure on the classification task improved from 59.8% to 66.0%, an improvement of 6.2%. The ROUGE-L score also showed a slight improvement on the image description task, increasing from 28.8% to 29.1%.

[0094] Table 1: Performance comparison of seven satellites in orbit over three cycles (%)

[0095] Table 2 shows the average performance of this disclosure under different settings. Experimental results show that the collaborative learning effect becomes more significant with the increase in the number of participating satellites in orbit. With seven satellites in orbit and manual annotation, the average accuracy of this disclosure on the classification task reaches 66.0%, an improvement of 6.2% over the baseline. Even with a single satellite in orbit and a fully automated setting, this disclosure can still improve the classification accuracy from 65.2% to 66.8%, an improvement of 1.6%.

[0096] Table 2: Performance Comparison of SGCL under Different Settings

[0097] Table 3 details the improvement effects of this disclosure on seven different OOD tasks. Experimental results show that this disclosure provides significant improvements across all OOD tasks, with an average performance improvement of 9.5%. Specifically, in the formatted OOD task (formal offset), accuracy improved from 51.1% to 65.1%, an improvement of 14.0%; in ship type classification (fine-grained OOD), accuracy improved from 36.7% to 46.3%, an improvement of 9.6%; and in disaster scenario description (domain OOD), the ROUGE-L score improved from 24.8% to 26.5%, an improvement of 1.7%.

[0098] Table 3: The effect of SGCL on different OOD problems

[0099] To verify the reliability of the experimental results, this disclosure conducted three repeated experiments, and the results are shown in Table 4. The experimental results show that the method of this disclosure has good stability and a small standard deviation (mean standard deviation of 0.15% for classification task and 0.05% for image description task).

[0100] Table 4: Reliability verification of experimental results

[0101] Furthermore, this disclosure also includes component ablation experiments to verify the effectiveness of each module. As shown in Table 5, when the base model update function is removed, the performance of both the on-orbit satellite model and the ground base model decreases significantly; when the "adaptive" distillation mechanism is removed, the performance of both models also decreases. These results demonstrate the effectiveness of each component of this disclosure.

[0102] Table 5: Performance Comparison of SGCL Framework Components

[0103] Regarding the effectiveness of the OOD detector, this disclosure presents detailed tests in a satellite-ground co-learning benchmark. In the first two cycles, all 32 OOD tasks were successfully detected. In the third cycle, to reduce communication overhead and model oscillations, the system intentionally ignored 16 recurring OOD tasks. When the visual reconstruction error component was removed, 22 OOD tasks were detected and 10 were missed; removing the Mahalanobis distance component resulted in 20 being detected and 12 being missed; removing the multimodal alignment component resulted in 22 being detected and 10 being missed. These results demonstrate the effectiveness of the OOD detector of this disclosure.

[0104] Regarding update triggers, with a seven-satellite configuration, this disclosure performed 28 on-orbit model updates and 4 ground-based model updates in the first two cycles, accounting for 40% and 5.7% of all missions, respectively. With a three-satellite configuration, 10 on-orbit model updates and 3 ground-based model updates were performed, accounting for 33.3% and 10%, respectively. Importantly, all updates occurred when on-orbit model performance deteriorated, and in the four cases where the OOD module triggered the data downlink but the update triggers failed to activate, on-orbit model performance remained stable, confirming the accuracy of the update triggers.

[0105] Regarding communication overhead, this disclosure uses LoRA technology for on-orbit model updates, requiring the transmission of no more than 7M parameters per update, with a data transmission volume of approximately 13.4MB (FP16 precision). Considering the overhead of checksums, metadata, and other protocol elements, the actual on-orbit satellite-to-ground transmission volume is still less than 15MB / update. A typical on-orbit satellite transit lasts 5-10 minutes (300-600 seconds), and even using a low-rate downlink (5Mbps), the transmission only takes 25 seconds, accounting for only 4.2% to 8.3% of the available transit time. This on-orbit model update capability has been successfully verified on the China Orient Wisdom High Resolution 01 on-orbit satellite.

[0106] The above experimental results fully demonstrate the effectiveness of this disclosure. Compared with the "learn-deployment" baseline, this disclosure significantly improves the ability of on-orbit satellite models to handle out-of-distribution scenarios, especially excelling in domain OOD, fine-grained OOD, and format OOD tasks. Furthermore, this disclosure effectively controls communication overhead through an efficient OOD detection mechanism and update triggers, making it suitable for practical on-orbit satellite systems.

[0107] In satellite-ground collaborative learning systems, in addition to the main scheme already described, there are several alternative methods that can address the problem of insufficient adaptability of satellite on-orbit models to distributed off-site scenarios.

[0108] Satellite-Earth co-learning can be seen as a variant of edge-cloud co-learning, with a collaborative framework serving as an alternative. In this framework, a cloud-based convolutional neural network (CloudCNN) provides soft supervision for local edge convolutional neural networks (EdgeCNNs) for edge training, while EdgeCNNs handle real-time inference with visual input.

[0109] The DCCL framework can also be used as an alternative. This framework introduces an edge-cloud collaboration framework specifically designed for recommendation systems, which can be simplified to retain only its core model update mechanism.

[0110] The COLLA method can be considered an alternative. This method provides a personalized model for each device while facilitating collective learning between the cloud and end devices. The cloud model acts as a global aggregator, extracting knowledge from multiple end-device models.

[0111] The slow-fast learning mechanism can also be used as an alternative. This mechanism is used for edge-cloud collaborative recommendations and can be adapted to satellite-ground collaborative learning scenarios.

[0112] The MDPA framework is a device-cloud collaborative learning framework that enhances a user's local data by retrieving similar data from a global pool in the cloud. This framework can be simplified to use only its model update portion, without relying on satellite-side data augmentation.

[0113] The CLUE framework introduces a framework that enhances knowledge transfer in neural networks within edge computing environments using a multi-model boosting approach. This framework can serve as an alternative to improve the efficiency of knowledge transfer between satellite and ground environments.

[0114] FedRSCLIP is a distributed learning framework designed specifically for remote sensing scene classification. While the framework requires client devices to perform local training, this can be simplified to using only its model aggregation concept.

[0115] In the field of remote sensing visual language models, RemoteCLIP is the first visual-language foundational model specifically designed for remote sensing and can serve as an alternative foundational model. This model addresses the limitations of low-level features, dependence on labeled data, and the lack of language understanding in zero-shot tasks.

[0116] The lightweight Transformer architecture can also be used as an alternative. This architecture is designed for on-device deployment, improves computational efficiency, and is suitable for resource-constrained satellite platforms.

[0117] GeoChat introduces a conversational, remote sensing-specific visual language model with multi-task capabilities, supporting tasks such as image description, change detection, and object counting. This model can serve as an alternative base model.

[0118] SkySense is currently the largest multimodal remote sensing foundation model, pre-trained on time series data of 21.5 million optical and SAR data points using a novel spatiotemporal encoder. This model can serve as an alternative foundation model, providing a more comprehensive understanding of remote sensing scenes.

[0119] While these alternative solutions may not be as performant or efficient as the main solution disclosed herein, they all address, to varying degrees, the problem of insufficient adaptability of satellite on-orbit models to distributed off-orbit scenarios, providing basic functionality and effects. Through appropriate adjustments and simplifications, these solutions can adapt to the specific constraints of satellite platforms, enabling periodic model updates and performance improvements.

[0120] Based on the above Figure 1 The specific implementation of the method shown in this embodiment provides a satellite-ground collaborative learning system, such as... Figure 3 As shown, the system includes: a pull module 31, a detection module 32, a first update module 33, a fine-tuning module 34, and a second update module 35; The pull module 31 is used to compress the basic model into a small model through the ground station of the satellite-ground collaborative learning system when the on-orbit satellite joins the completed satellite-ground collaborative learning system, and send the small model to the on-orbit satellite through the uplink so that the on-orbit satellite can use the compressed small model as the on-orbit model. The detection module 32 is used to detect the comprehensive distribution offset during the on-orbit inference of the on-orbit satellite using an OOD detector. When the comprehensive distribution offset is detected to be greater than a preset comprehensive distribution offset threshold, the data downlink is triggered to send the relevant data to the ground station. The first update module 33 is used to calculate the weighted difference statistics between the prediction results of the on-orbit model and the prediction results of the basic model of the ground station based on the relevant data received by the ground station from the on-orbit satellite, and to trigger a selective update of the on-orbit model when the weighted difference statistics exceed a preset difference statistics threshold. The fine-tuning module 34 is used to perform fine-tuning on the small model corresponding to the on-orbit satellite at the ground station during the selective update phase of the on-orbit model, generate fine-tuned update parameters, and send the fine-tuned update parameters to the on-orbit satellite through the uplink so that the on-orbit satellite loads the update parameters and completes the update of the on-orbit model. The second update module 35 is used to repeatedly execute the above steps until the cumulative number of updates of the on-orbit model reaches the preset number of updates, and then aggregate one or more updated on-orbit models into the base model through the ground station to realize the update of the base model.

[0121] In specific application scenarios, the detection module 32 can be used to acquire the current batch of data collected by the on-orbit satellite and use the on-orbit model encoder of the on-orbit satellite to extract the visual features and text features of the current batch of data. The reconstruction error of the visual features is calculated by a lightweight autoencoder. When the reconstruction error is greater than a preset dynamic error threshold, the current batch of data is marked as a visual anomaly. The distribution offset of the text features is calculated based on Mahalanobis distance. When the distribution offset is greater than a preset offset threshold, the current batch of data is marked as an offset anomaly. Calculate the cosine similarity between the visual features and the text features. When the cosine similarity is lower than a preset cosine similarity threshold, mark the current batch of data as a multimodal misalignment anomaly. Based on the detection results of the visual anomaly, the offset anomaly, and / or the multimodal misalignment anomaly, the comprehensive distribution offset is calculated. When the comprehensive distribution offset is greater than the preset comprehensive distribution offset threshold, the data downlink is triggered to send the relevant data to the ground station.

[0122] In specific application scenarios, the first update module 33 can be used to calculate the prediction probability distribution and prediction confidence of the ground station's basic model for the relevant data, and to calculate the prediction probability distribution of the on-orbit model of the on-orbit satellite for the relevant data; Calculate the KL divergence between the predicted probability distribution of the base model and the predicted probability distribution of the on-orbit model, and use the prediction confidence of the base model to perform a weighted operation on the KL divergence to obtain the weighted difference value for each data point. Calculate the weighted difference statistics based on the weighted difference values ​​of all data points; When the weighted difference statistic exceeds a preset difference statistic threshold, an update to the on-orbit model is triggered.

[0123] In specific application scenarios, the first update module 33 can be used to calculate the median of the weighted difference values ​​of all data points and the absolute deviation value of the median; An outlier threshold is determined based on the median and the absolute deviation value; Data points whose weighted difference values ​​exceed the outlier threshold are identified as outliers, and the outliers are filtered to obtain a filtered weighted difference set. Calculate the average value of the filtered weighted difference set as the weighted difference statistic.

[0124] In a specific application scenario, the fine-tuning module 34 can be used at the ground station to perform LoRA fine-tuning on the small model based on a composite loss function to generate the updated parameters. The composite loss function is composed of a weighted sum of a first loss term and a second loss term. The first loss term is the cross-entropy loss of the difference between the on-orbit model prediction and the data label, and the second loss term is the KL divergence term of the difference between the on-orbit model prediction and the ground-based model prediction.

[0125] In specific application scenarios, the second update module 35 can be used to adopt an adaptive knowledge distillation mechanism and adopt different update strategies according to different system operation modes, so as to aggregate the updated knowledge of the on-orbit model of one or more on-orbit satellites into the base model through the ground station, thereby realizing the model update of the base model. The system operation modes include manual annotation mode and automated mode.

[0126] In a specific application scenario, the second update module 35 can be used in the manual annotation mode, and the update strategy includes: The weight value of each of the on-orbit models is calculated based on the cross-entropy loss of each of the multiple on-orbit models. Based on the weight values, manually annotated data labels, and the predicted probability distributions of the on-orbit model and the base model, a first loss function of the base model is calculated, and the base model is updated using the first loss function. The weight calculation formula for the on-orbit model is as follows:

[0127] In the formula, Indicating the on-orbit model The weight value, This represents the total number of models in orbit. Indicates client The cross-entropy loss function value on its corresponding batch of data, Represents the cross-entropy loss function. Indicates client Real data labels Indicating the on-orbit model Predicted probability distribution of the full dataset; The first loss function is expressed as:

[0128] In the formula, This represents the first loss function value of the base model. and Both represent balance coefficients. Represents the cross-entropy loss function. Indicates by All data labels collected and manually annotated by the updated on-orbit model. This represents the probability distribution of the base model's predictions for the entire dataset. Indicating the on-orbit model Predicted probability distribution of the full dataset This represents the total number of models in orbit. Denotes KL divergence, Indicating the on-orbit model The weight value.

[0129] In a specific application scenario, the second update module 35 can be used in the automated mode, where the update strategy includes: The weight value of each on-orbit model is calculated based on the average similarity between the output of the on-orbit model and the outputs of all other on-orbit models. Based on the weight values ​​and the predicted probability distributions of the base model and the on-orbit model, a second loss function of the base model is calculated, and the base model is updated using the second loss function. The weight calculation formula for the on-orbit model is as follows:

[0130] In the formula, Indicating the on-orbit model The weight value, This represents the total number of models in orbit. Indicating the on-orbit model The average similarity between the output of and the outputs of all other on-orbit models; The second loss function is expressed as:

[0131] In the formula, This represents the second loss function value of the base model in automated mode. Represents the balance coefficient. This represents the total number of models in orbit. Indicating the on-orbit model The weight value, Denotes KL divergence, This represents the probability distribution of the base model's predictions for the entire dataset. Indicating the on-orbit model The predicted probability distribution of the full dataset.

[0132] It should be noted that other corresponding descriptions of the functional units involved in the satellite-ground collaborative learning system provided in this embodiment can be found in [reference needed]. Figure 1 The corresponding descriptions of the Chinese methods will not be repeated here.

[0133] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0134] The above are merely specific embodiments of this disclosure, enabling those skilled in the art to understand or implement this disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A satellite-ground collaborative learning method, characterized in that, include: Step 1: When the satellite in orbit joins the completed satellite-ground collaborative learning system, the ground station of the satellite-ground collaborative learning system compresses the basic model into a small model and sends the small model to the satellite in orbit through the uplink, so that the satellite in orbit can use the compressed small model as its in-orbit model. Step 2: Use the OOD detector to detect the overall distribution offset during the on-orbit inference of the satellite. When the overall distribution offset is detected to be greater than the preset overall distribution offset threshold, trigger the data downlink to send the relevant data to the ground station. Step 3: Based on the relevant data received by the ground station from the on-orbit satellite, calculate the weighted difference statistics between the prediction results of the on-orbit model and the prediction results of the ground station's basic model, and trigger a selective update of the on-orbit model when the weighted difference statistics exceed a preset difference statistics threshold; Step 4: In the selective update phase of the on-orbit model, the ground station performs fine-tuning on the small model corresponding to the on-orbit satellite, generates fine-tuned update parameters, and sends the fine-tuned update parameters to the on-orbit satellite via the uplink so that the on-orbit satellite loads the update parameters and completes the update of the on-orbit model; Repeat steps 1-4 above until the cumulative number of updates to the on-orbit model reaches the preset number of updates. Then, aggregate one or more updated on-orbit models into the base model through the ground station to update the base model.

2. The method according to claim 1, characterized in that, The method of using an OOD detector to detect the integrated distribution offset during the on-orbit inference of the satellite, and triggering a data downlink to send relevant data to the ground station when the detected integrated distribution offset is greater than a preset integrated distribution offset threshold, includes: The current batch of data collected by the on-orbit satellite is obtained, and the visual and text features of the current batch of data are extracted using the on-orbit model encoder of the on-orbit satellite. The reconstruction error of the visual features is calculated by a lightweight autoencoder. When the reconstruction error is greater than a preset dynamic error threshold, the current batch of data is marked as a visual anomaly. The distribution offset of the text features is calculated based on Mahalanobis distance. When the distribution offset is greater than a preset offset threshold, the current batch of data is marked as an offset anomaly. Calculate the cosine similarity between the visual features and the text features. When the cosine similarity is lower than a preset cosine similarity threshold, mark the current batch of data as a multimodal misalignment anomaly. Based on the detection results of the visual anomaly, the offset anomaly, and / or the multimodal misalignment anomaly, the comprehensive distribution offset is calculated. When the comprehensive distribution offset is greater than the preset comprehensive distribution offset threshold, the data downlink is triggered to send the relevant data to the ground station.

3. The method according to claim 1, characterized in that, The step of calculating a weighted statistical difference between the prediction results of the on-orbit model and the prediction results of the ground station's basic model based on relevant data received from the on-orbit satellite by the ground station, and triggering a selective update of the on-orbit model when the weighted statistical difference exceeds a preset statistical difference threshold, includes: Calculate the prediction probability distribution and prediction confidence of the ground station's basic model for the relevant data, and calculate the prediction probability distribution of the on-orbit model of the on-orbit satellite for the relevant data; Calculate the KL divergence between the predicted probability distribution of the base model and the predicted probability distribution of the on-orbit model, and use the prediction confidence of the base model to perform a weighted operation on the KL divergence to obtain the weighted difference value for each data point. Calculate the weighted difference statistics based on the weighted difference values ​​of all data points; When the weighted difference statistic exceeds a preset difference statistic threshold, an update to the on-orbit model is triggered.

4. The method according to claim 3, characterized in that, The calculation of the weighted difference statistics based on the weighted difference values ​​of all data points includes: Calculate the median of the weighted difference values ​​for all data points and the absolute deviation of the median; An outlier threshold is determined based on the median and the absolute deviation value; Data points whose weighted difference values ​​exceed the outlier threshold are identified as outliers, and the outliers are filtered to obtain a filtered weighted difference set. Calculate the average value of the filtered weighted difference set as the weighted difference statistic.

5. The method according to claim 1, characterized in that, The step of fine-tuning the small model corresponding to the on-orbit satellite at the ground station to generate updated parameters after fine-tuning includes: At the ground station, LoRA fine-tuning is performed on the small model based on a composite loss function to generate the updated parameters. The composite loss function is composed of a weighted sum of a first loss term and a second loss term. The first loss term is the cross-entropy loss of the difference between the on-orbit model prediction and the data label, and the second loss term is the KL divergence term of the difference between the on-orbit model prediction and the ground-based model prediction.

6. The method according to claim 1, characterized in that, The step of aggregating one or more updated on-orbit models into the base model via the ground station to update the base model includes: An adaptive knowledge distillation mechanism is adopted, and different update strategies are used according to different system operation modes. The updated knowledge of the on-orbit model of one or more on-orbit satellites is aggregated into the base model through the ground station to realize the model update of the base model. The system operation modes include manual annotation mode and automated mode.

7. The method according to claim 6, characterized in that, In the manual annotation mode, the update strategy includes: The weight value of each of the on-orbit models is calculated based on the cross-entropy loss of each of the multiple on-orbit models. Based on the weight values, manually annotated data labels, and the predicted probability distributions of the on-orbit model and the base model, a first loss function of the base model is calculated, and the base model is updated using the first loss function. The weight calculation formula for the on-orbit model is as follows: In the formula, Indicating the on-orbit model The weight value, This represents the total number of models in orbit. Indicates client The cross-entropy loss function value on its corresponding batch of data, Represents the cross-entropy loss function. Indicates client Real data labels Indicating the on-orbit model Predicted probability distribution of the full dataset The function represents the transformation of the loss function values ​​into a probability distribution; The first loss function is expressed as: In the formula, This represents the first loss function value of the base model. and Both represent balance coefficients. Represents the cross-entropy loss function. Indicates by All data labels collected and manually annotated by the updated on-orbit model. This represents the probability distribution of the base model's predictions for the entire dataset. Indicating the on-orbit model Predicted probability distribution of the full dataset This represents the total number of models in orbit. Denotes KL divergence, Indicating the on-orbit model The weight value.

8. The method according to claim 6, characterized in that, In the automated mode, the update strategy includes: The weight value of each on-orbit model is calculated based on the average similarity between the output of the on-orbit model and the outputs of all other on-orbit models. Based on the weight values ​​and the predicted probability distributions of the base model and the on-orbit model, a second loss function of the base model is calculated, and the base model is updated using the second loss function. The weight calculation formula for the on-orbit model is as follows: In the formula, Indicating the on-orbit model The weight value, This represents the total number of models in orbit. Indicating the on-orbit model The average similarity between the output of and the outputs of all other on-orbit models; The second loss function is expressed as: In the formula, This represents the second loss function value of the base model in automated mode. Represents the balance coefficient. This represents the total number of models in orbit. Indicating the on-orbit model The weight value, Denotes KL divergence, This represents the probability distribution of the base model's predictions for the entire dataset. Indicating the on-orbit model The predicted probability distribution of the full dataset.

9. A satellite-ground collaborative learning system, characterized in that, include: The module includes a fetch module, a detection module, a first update module, a fine-tuning module, and a second update module. The pull module is used to compress the basic model into a small model through the ground station of the satellite-ground collaborative learning system when the satellite in orbit joins the completed satellite-ground collaborative learning system, and send the small model to the satellite in orbit through the uplink so that the satellite in orbit can use the compressed small model as the in-orbit model. The detection module is used to detect the comprehensive distribution offset during the on-orbit inference of the on-orbit satellite using an OOD detector. When the detected comprehensive distribution offset is greater than a preset comprehensive distribution offset threshold, the data downlink is triggered to send the relevant data to the ground station. The first update module is used to calculate the weighted difference statistics between the prediction results of the on-orbit model and the prediction results of the basic model of the ground station based on the relevant data received by the ground station from the on-orbit satellite, and to trigger a selective update of the on-orbit model when the weighted difference statistics exceed a preset difference statistics threshold. The fine-tuning module is used to perform fine-tuning on the small model corresponding to the on-orbit satellite at the ground station during the selective update phase of the on-orbit model, generate fine-tuned update parameters, and send the fine-tuned update parameters to the on-orbit satellite through the uplink so that the on-orbit satellite loads the update parameters and completes the update of the on-orbit model. The second update module is used to repeatedly execute the above steps until the cumulative number of updates of the on-orbit model reaches the preset number of updates, and then aggregate one or more updated on-orbit models into the base model through the ground station to realize the update of the base model.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the satellite-ground collaborative learning method as described in any one of claims 1 to 8.

Images