Satellite-ground collaborative learning method, system and electronic device

Abstract

The present disclosure relates to a satellite-ground collaborative learning method and system and electronic equipment, and relates to the technical field of satellite remote sensing. The method comprises the following steps: compressing a basic model into a small model by a ground station, and taking the small model as a satellite on-orbit model; detecting a comprehensive distribution offset during satellite on-orbit reasoning, and sending relevant data to the ground station when the comprehensive distribution offset is greater than a preset threshold; calculating a weighted difference value between a prediction result of the on-orbit model and a prediction result of the basic model based on the relevant data, and triggering on-orbit model updating when the weighted difference value exceeds a preset threshold; performing LoRA fine-tuning on the small model by the ground station to achieve on-orbit model updating; repeating the above steps until the number of on-orbit model updates reaches a preset number, and aggregating the on-orbit model into the basic model by the ground station to achieve model updating of the basic model. The present disclosure can improve the overall performance of the satellite on-orbit model when facing out-of-distribution scenarios.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of satellite remote sensing, in particular to a satellite-ground collaborative learning method, system and electronic device. BACKGROUND

[0002] The satellite platform equipment intelligent model can realize real-time analysis, processing and autonomous decision of remote sensing data, and significantly improve the application efficiency in the fields of disaster monitoring, ocean monitoring, environmental monitoring and commercial application. However, as the remote sensing model develops towards larger scale and more functional diversity, its computing demand grows exponentially, while the satellite platform is facing a serious computing power bottleneck due to the limitation of physical size and power supply, forcing the on-orbit model to be a highly compressed version of the original ground model, which significantly reduces its robustness to out-of-distribution (OOD) scenes.

[0003] Currently, all on-orbit models follow the "learning-deployment" paradigm: parameters are fixed after ground training before launch, and updates completely rely on occasional human intervention. This static, closed-loop training strategy cannot adapt to data drift in dynamic spatio-temporal environments, especially when facing accidental and unpredictable distribution shifts such as domain OOD, fine-grained OOD and format OOD, which makes the satellite on-orbit model face serious performance degradation problems in actual tasks. SUMMARY

[0004] Therefore, the present disclosure provides a satellite-ground collaborative learning method, system and electronic device, which mainly aims to solve the technical problem of serious performance degradation of satellite on-orbit model when facing out-of-distribution scenes.

[0005] According to a first aspect of the present disclosure, a satellite-ground collaborative learning method is provided, which comprises:

[0006] Step 1: when an on-orbit satellite joins the constructed satellite-ground collaborative learning system, the ground station of the satellite-ground collaborative learning system compresses the base model into a small model, and sends the small model to the on-orbit satellite through the uplink, so that the on-orbit satellite uses the compressed small model as an on-orbit model;

[0007] Step 2: using an OOD detector to detect the integrated distribution shift during on-orbit inference of the on-orbit satellite, when the integrated distribution shift is detected to be greater than a preset integrated distribution shift threshold, triggering a data downlink to send relevant data to the ground station;

[0008] Step 3: based on the relevant data received by the ground station from the on-orbit satellite, calculating a weighted difference statistic value between the prediction result of the on-orbit model and the prediction result of the base model of the ground station, and triggering selective updating of the on-orbit model when the weighted difference statistic value exceeds a preset difference statistic threshold;

[0009] Step 4: in the selective updating stage of the on-orbit model, fine-tuning the small model corresponding to the on-orbit satellite at the ground station to generate fine-tuned update parameters, and sending the fine-tuned update parameters to the on-orbit satellite through an uplink to enable the on-orbit satellite to load the update parameters and complete updating of the on-orbit model;

[0010] Steps 1-4 are repeatedly performed until the cumulative updating number of the on-orbit model reaches a preset updating number, and one or more updated on-orbit models are aggregated into the base model by the ground station to update the base model.

[0011] According to a second aspect of the present disclosure, a satellite-ground collaborative learning system is provided, which comprises a pulling module, a detection module, a first updating module, a fine-tuning module, and a second updating module.

[0012] The pulling module is configured to compress a base model into a small model when an on-orbit satellite joins the constructed satellite-ground collaborative learning system, and send the small model to the on-orbit satellite through an uplink to enable the on-orbit satellite to use the compressed small model as an on-orbit model.

[0013] The detection module is configured to detect a comprehensive distribution offset of the on-orbit satellite during on-orbit inference using an OOD detector, and trigger a data downlink to send relevant data to the ground station when the comprehensive distribution offset is detected to be greater than a preset comprehensive distribution offset threshold.

[0014] The first updating module is configured to calculate a weighted difference statistic value between the prediction result of the on-orbit model and the prediction result of the base model of the ground station based on the relevant data received by the ground station from the on-orbit satellite, and trigger selective updating of the on-orbit model when the weighted difference statistic value exceeds a preset difference statistic threshold.

[0015] The fine-tuning module is configured to fine-tune the small model corresponding to the on-orbit satellite at the ground station in the selective updating stage of the on-orbit model to generate fine-tuned update parameters, and send the fine-tuned update parameters to the on-orbit satellite through an uplink to enable the on-orbit satellite to load the update parameters and complete updating of the on-orbit model.

[0016] The second updating module is configured to repeatedly execute the above steps until the cumulative number of updates of the on-orbit model reaches a preset number of updates, and aggregate one or more updated on-orbit models into the base model through the ground station to update the base model.

[0017] According to a third aspect of the present disclosure, an electronic device is provided, comprising at least one processor, and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the satellite-ground collaborative learning method of the first aspect.

[0018] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the satellite-ground collaborative learning method of the first aspect.

[0019] Compared with the prior art, the satellite-ground collaborative learning method, system and electronic device provided by the present disclosure have the following advantages: when an on-orbit satellite is added to a constructed satellite-ground collaborative learning system, the ground station of the satellite-ground collaborative learning system compresses a base model into a small model, and sends the small model to the on-orbit satellite through an uplink, so that the on-orbit satellite uses the compressed small model as an on-orbit model; an OOD detector is used to detect a comprehensive distribution offset during on-orbit inference of the on-orbit satellite, and when the detected comprehensive distribution offset is greater than a preset comprehensive distribution offset threshold, a data downlink is triggered to send relevant data to the ground station; based on the relevant data received by the ground station from the on-orbit satellite, a weighted difference statistical value between a prediction result of the on-orbit model and a prediction result of the base model of the ground station is calculated, and when the weighted difference statistical value exceeds a preset difference statistical threshold, selective updating of the on-orbit model is triggered; during the selective updating of the on-orbit model, the ground station performs fine-tuning on the small model corresponding to the on-orbit satellite, generates fine-tuned updating parameters, and sends the fine-tuned updating parameters to the on-orbit satellite through an uplink, so that the on-orbit satellite loads the updating parameters to complete updating of the on-orbit model; the above steps are repeatedly executed until the cumulative number of updates of the on-orbit model reaches a preset number of updates, and one or more updated on-orbit models are aggregated into the base model through the ground station to update the base model.

[0020] By applying the scheme of the present disclosure, the present disclosure realizes bandwidth-efficient downlink data transmission through a double-mechanism OOD detector. Only 7M parameters need to be transmitted for a single update, and the data transmission amount is about 13.4MB (FP16 precision), which only occupies 4.2% to 8.3% of the available time in the typical on-orbit satellite transit time (5-10 minutes). This greatly reduces the occupation of on-orbit satellite communication resources by data transmission, improves the efficiency of data transmission, and enables timely transmission of relevant data to the ground station under limited on-orbit satellite communication conditions, providing necessary information for model updating.

[0021] Through ground LoRA fine-tuning triggered based on the difference between the on-orbit satellite - base model output, parameter-efficient model updating is realized, which significantly reduces the communication overhead. The LoRA fine-tuning method only needs to adjust part of the parameters of the model to realize the update of the on-orbit model, reducing the number of parameters that need to be transmitted and updated, thereby reducing the communication cost and improving the efficiency of model updating.

[0022] Through adaptive distillation and aggregation of multi-satellite knowledge, the continuous optimization of the base model is realized. In the seven-satellite annotation setting, the average accuracy of the classification task is improved from 59.8% to 66.0%, with an improvement of 6.2%; for the domain OOD, fine-grained OOD and format OOD tasks, the average performance is improved from 40.6% to 50.1%, with an improvement of 9.5%, which is significantly better than the "learning - deployment" baseline method; on the image description task, the ROUGE - L score is improved from 28.8% to 29.1%. This shows that this method can effectively improve the performance of the base model, making it better adapt to various out-of-distribution scenarios, and thus providing a better base for the on-orbit satellite model, improving the overall performance of the on-orbit satellite model when facing out-of-distribution scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0025] Figure 1 A flowchart of a satellite - ground cooperative learning method provided by the embodiments of the present disclosure is shown in the figure.

[0026] Figure 2A learning flowchart of a satellite-ground collaborative learning system provided by an embodiment of the present disclosure;

[0027] Figure 3 A structural schematic diagram of a satellite-ground collaborative learning system provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, and should be considered as merely exemplary. Thus, those skilled in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted herein. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] The satellite-ground collaborative learning method, system and electronic device of the embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0030] To solve the technical problem that the performance of a satellite on-orbit model is severely degraded when facing a distributed field scenario, the embodiments of the present disclosure provide a satellite-ground collaborative learning method, as shown in Figure 1 The method comprises the following steps.

[0031] Step 101, when an on-orbit satellite is added to a constructed satellite-ground collaborative learning system, a ground station of the satellite-ground collaborative learning system compresses a basic model into a small model, and sends the small model to the on-orbit satellite through an uplink, so that the on-orbit satellite uses the compressed small model as an on-orbit model.

[0032] The present disclosure can first construct an on-orbit satellite-ground collaborative learning system (SGCL system), which contains a group of on-orbit satellites and ground stations, and adopts a star topology, that is, each on-orbit satellite is directly connected to a ground station. As shown in Figure 2 In the system, the ground station undertakes the important task of training and maintaining a basic model , and each on-orbit satellite has its own on-orbit model , and the autonomous evolution of the on-orbit satellite model is realized through an asynchronous updating mechanism between the ground basic model and the on-orbit model.

[0033] In the boot-up stage, the ground station can train the first basic model from an initial data set. In the present embodiment, the initial data set can be selected from remote sensing visual language model related data. At the same time, a large amount of earth system data, numerous data sets and open source basic models (such as GeoChat) can be fully utilized to assist in constructing the initial model.

[0034] The base model can employ the CLIP-ViT-L / 14 visual encoder and the Vicuna-v1.5-7B language model. This architecture combines powerful visual feature extraction capabilities with language understanding capabilities, enabling better handling of multimodal data.

[0035] To enable the base model to quickly adapt to different tasks and scenarios, LoRA (Low-Rank Adaptation) technology can be used for efficient adaptation. LoRA technology adds a low-rank matrix to the original model parameters, achieving fine-tuning of the model with less parameter adjustment, which greatly reduces the computational and time costs of model updates.

[0036] The training process of the base model optimizes the following objective function:

[0037]

[0038] In the formula, Let be the objective function. For cross-entropy loss, For visual-text alignment loss, This is the balance coefficient.

[0039] When an in-orbit satellite joins the completed SGCL system, it can request the latest model from the ground station. This request can be initiated proactively after the in-orbit satellite joins the system, or the latest model can be loaded before the in-orbit satellite is launched, depending on the actual mission requirements and system design.

[0040] After receiving a request from an in-orbit satellite, the ground station can compress its latest trained base model into a smaller model. In this embodiment, Qwen2-VL-2B can be selected as the compressed smaller model. The compression process is achieved through knowledge distillation, which is a technique for transferring knowledge from a large model (teacher model) to a smaller model (student model). During the compression process, visual-text alignment is simultaneously optimized using a mean squared error loss function, as shown below:

[0041]

[0042] In the formula, For visual-text alignment loss, The embedding vectors of the base model, For the embedding vector of the in-orbit satellite model, This refers to the batch size.

[0043] The ground station transmits the compressed, small model to the on-orbit satellite via uplink. Upon receiving the model, the on-orbit satellite uses it as its first on-orbit model and performs inference tasks in subsequent time periods. In this way, the on-orbit satellite can quickly obtain a high-performance model under limited computing resources and communication bandwidth, effectively improving the adaptability of the on-orbit model to different scenarios and its task execution efficiency.

[0044] Step 102: Use the OOD detector to detect the integrated distribution offset during the on-orbit inference of the satellite. When the detected integrated distribution offset is greater than the preset integrated distribution offset threshold, trigger the data downlink to send the relevant data to the ground station.

[0045] In this embodiment of the disclosure, a lightweight distributed offset monitoring module has been deployed on the satellite in orbit. This module includes three sub-mechanisms: visual anomaly detection, cue offset detection, and multimodal misalignment detection. The relevant parameters of each mechanism have been initialized based on the training set data, such as dynamic threshold, preset offset threshold, preset cosine similarity threshold, and preset comprehensive distributed offset threshold.

[0046] Specifically, such as Figure 2 As shown, the OOD detector detects the integrated distribution offset during on-orbit inference of the satellite based on feature space reconstruction error and multimodal alignment metric. When the detected integrated distribution offset exceeds a preset integrated distribution offset threshold, a data downlink is triggered to send relevant data to the ground station, which may include:

[0047] Acquire the current batch of data collected by the on-orbit satellite and use the on-orbit model encoder of the on-orbit satellite to extract the visual and text features of the current batch of data;

[0048] The reconstruction error of visual features is calculated by a lightweight autoencoder. When the reconstruction error is greater than a preset dynamic error threshold, the current batch of data is marked as a visual anomaly.

[0049] The distribution offset of text features is calculated based on Mahalanobis distance. When the distribution offset is greater than a preset offset threshold, the current batch of data is marked as an offset anomaly.

[0050] Calculate the cosine similarity between visual features and text features. When the cosine similarity is lower than the preset cosine similarity threshold, mark the current batch of data as a multimodal misalignment anomaly.

[0051] Based on the detection results of visual anomalies, offset anomalies, and / or multimodal misalignment anomalies, the comprehensive distribution offset is calculated. When the comprehensive distribution offset is greater than the preset comprehensive distribution offset threshold, the data downlink is triggered to send the relevant data to the ground station.

[0052] In this embodiment, during the on-orbit operation of the satellite, observation data of batches at the current time t can be acquired in real time according to a preset data acquisition strategy. The observation data may include visual data (such as images taken by satellites in orbit) and text data (such as text descriptions and instructions related to the observation mission).

[0053] The on-orbit model encoder, carried onboard the satellite, can be used to process the current batch of data. Specifically, for visual data, the on-orbit model encoder can extract high-level semantic features of images using structures such as convolutional neural networks (CNNs); for text data, the on-orbit model encoder can extract textual features using natural language processing models (such as the Transformer architecture), ultimately obtaining the visual feature vector of the current batch of data. and text feature vectors .

[0054] The extracted visual feature vectors can be input into a lightweight autoencoder pre-deployed on an on-orbit satellite. An autoencoder can consist of an encoder and a decoder. The encoder compresses the input visual features into a low-dimensional representation, while the decoder attempts to reconstruct the original visual features from the low-dimensional representation. The reconstruction error is obtained by calculating the difference between the reconstructed visual features and the original visual features, as shown in the following formula:

[0055]

[0056] In the formula, For reconstruction error, AE is a visual feature vector and a lightweight autoencoder.

[0057] The calculated reconstruction error is compared with a preset dynamic error threshold. A comparison is performed. The preset dynamic error threshold can be set based on the training set data. and According to statistics, when the reconstruction error of the current batch of data exceeds the preset dynamic error threshold, the batch of data is marked as a visual anomaly.

[0058] In this embodiment, the OOD module based on Mahalanobis distance processes the text features. First, the mean vector of the text feature vectors can be calculated based on the training set data. Covariance Matrix To improve memory efficiency, Principal Component Analysis (PCA) is used to downsample the text features of the training set, reducing computation while preserving key information. Then, for the text feature vectors of the current batch of data, the Mahalanobis distance between them and the mean of the training set features is calculated, as shown in the following formula:

[0059]

[0060] In the formula, The Mahalanobis distance, For text feature vectors, The covariance matrix of the text feature vectors. Let be the covariance matrix of the text feature vectors.

[0061] The calculated Mahalanobis distance is compared with a preset offset threshold. This preset offset threshold can be obtained statistically from the training set data, such as selecting the maximum value of the Mahalanobis distance in the training set data or a value of a high percentile as the threshold. When the Mahalanobis distance of the current batch of data is greater than the preset offset threshold (e.g., ...), the threshold is set to be set. When this happens, mark the batch of data as having an offset anomaly.

[0062] In this embodiment, visual feature vectors can be calculated. and text feature vectors The cosine similarity between the pairs of pairs. The cosine similarity can be calculated using the following formula:

[0063]

[0064] In the formula, Visual feature vectors and text feature vectors Cosine similarity between them For visual feature vectors, This is the text feature vector.

[0065] The calculated cosine similarity is compared with a preset cosine similarity threshold. The preset cosine similarity threshold can be set according to the actual application scenario and the requirements for multimodal consistency. When the cosine similarity of the current batch of data is lower than the preset cosine similarity threshold (e.g., ...), the similarity is calculated. When this occurs, the batch of data is marked as a multimodal misalignment anomaly.

[0066] The overall distributed offset (i.e., offset severity) can be calculated based on the detection results of visual anomalies, offset anomalies, and / or multimodal misalignment anomalies. It can be calculated using the following formula:

[0067]

[0068] In the formula, This represents the overall distribution offset (i.e., the severity of the offset). For batch size, For reconstruction error, To preset the dynamic threshold of error, The Mahalanobis distance, For the preset offset threshold, Cosine similarity between visual feature vectors and text feature vectors. This is a preset cosine similarity threshold.

[0069] The calculated overall distribution offset is compared with a preset overall distribution offset threshold, which can be set based on the system's tolerance for distribution offset and actual application requirements. When the overall distribution offset exceeds the preset threshold... When this occurs, it indicates a significant distribution shift in the current batch of data, which may affect the inference performance of the on-orbit model of the on-orbit satellite. At this time, the on-orbit satellite triggers the data downlink to send relevant data (including raw visual data, text data, and information such as feature vectors, reconstruction errors, Mahalanobis distance, and cosine similarity obtained during the detection process) to the ground station to send a data offset alert to the ground station.

[0070] Step 103: Based on the relevant data received by the ground station from the on-orbit satellite, calculate the weighted difference statistics between the prediction results of the on-orbit model and the prediction results of the ground station's basic model, and trigger a selective update of the on-orbit model when the weighted difference statistics exceed the preset difference statistics threshold.

[0071] During the operation of an in-orbit satellite, such as Figure 2 As shown, due to the continuous changes in actual scenarios and tasks, the on-orbit model carried on the satellite may gradually become unable to accurately handle various situations. In order to ensure that the on-orbit model always maintains good performance, it is necessary to compare and analyze the prediction results of the on-orbit model with the prediction results of the ground station's basic model based on the relevant data received by the ground station from the on-orbit satellite, and determine whether selective updates to the on-orbit model are needed by calculating the weighted difference statistics.

[0072] Specifically, step 1: calculate the prediction probability distribution and prediction confidence of the ground station's basic model for the relevant data, and calculate the prediction probability distribution of the on-orbit model of the on-orbit satellite for the relevant data.

[0073] In this embodiment, the ground station can receive relevant data transmitted by the on-orbit satellite. The relevant data can include various types of information collected by the satellite during its on-orbit operation, such as images and sensor data. This data has already undergone preliminary processing at the satellite end to extract features that can be used for model prediction. The ground station organizes and preprocesses this data to ensure that the data format meets the model input requirements.

[0074] Ground stations can use pre-trained base models to make predictions on received relevant data. During the prediction process, the base model not only outputs the predicted probability distribution of the relevant data, but also... This outputs the probability value for each possible category, along with the prediction confidence score. This confidence score reflects the reliability of the base model's predictions. For example, in an image classification task, the base model outputs the probability that an image belongs to each category, along with a confidence score that indicates the reliability of the prediction.

[0075] The on-orbit model of the in-orbit satellite also makes predictions based on the same relevant data, outputting a predicted probability distribution. The on-orbit model of an on-orbit satellite is trained on certain data before the launch of the on-orbit satellite and is used for real-time inference during the on-orbit operation of the on-orbit satellite.

[0076] Step 2: Calculate the KL divergence between the predicted probability distribution of the base model and the predicted probability distribution of the on-orbit model, and use the prediction confidence of the base model to perform a weighted calculation on the KL divergence to obtain the weighted difference value for each data point.

[0077] In this embodiment, for each data point, the KL divergence between the predicted probability distribution of the ground station's basic model and the predicted probability distribution of the on-orbit model of the on-orbit satellite can be calculated. The KL divergence is an indicator used to measure the difference between two probability distributions, and its calculation formula is as follows:

[0078]

[0079] In the formula, Let KL divergence be the KL divergence. To predict probability distributions based on the basic model, Predict probability distributions for in-orbit models.

[0080] The calculated KL divergence can be weighted using the prediction confidence score of the ground station's basic model. Specifically, for each data point, the KL divergence value can be multiplied by the prediction confidence score of the basic model for that data point to obtain the weighted difference value for each data point. The calculation formula is as follows:

[0081]

[0082] In the formula, The weighted difference value for each data point. Let KL divergence be the KL divergence. To predict probability distributions based on the basic model, To predict the probability distribution for the in-orbit model, The prediction confidence of the basic model.

[0083] Step 3: Calculate the weighted difference statistics based on the weighted difference values ​​of all data points;

[0084] In this embodiment, the median of the weighted difference values ​​of all data points can be calculated based on the weighted difference values ​​of all data points. The median is the absolute deviation from the median, where the median is the value in the middle after all weighted variances are arranged in ascending order. The median and MAD can reflect the central tendency and dispersion of the weighted variances.

[0085] The outlier threshold can be determined based on the median and the absolute deviation (MAD). The calculation formula is as follows:

[0086]

[0087] In the formula, The threshold for outliers. This is the threshold coefficient, and its value can be adjusted according to the actual application scenario and the strictness of outlier detection. It is usually set to 1. MAD is the absolute deviation of the weighted difference values ​​for all data points. This is the median of the weighted differences across all data points.

[0088] Data points whose weighted variance values ​​exceed an outlier threshold can be identified as outliers. To exclude abnormally large variance values ​​caused by special circumstances or noise, and to ensure that subsequent calculations better reflect the true differences between models, these outliers can be filtered to obtain a filtered weighted variance set. .

[0089] Finally, the average of the filtered weighted difference set can be calculated. , as a weighted difference statistic.

[0090] Step 4: When the weighted difference statistic Exceeding the preset difference statistical threshold At that time, an update to the on-orbit model is triggered.

[0091] In this embodiment, the calculated weighted difference statistics can be used. Statistical threshold for differences from preset threshold Comparisons are made. The preset difference statistical threshold can be set in advance according to actual application needs and the required model update frequency. When the weighted difference statistical value... Exceeding the preset difference statistical threshold When this occurs, it indicates a significant difference between the prediction results of the on-orbit model and the prediction results of the ground station basic model. The on-orbit model may no longer be able to accurately handle the current changing scenarios and tasks. At this time, selective updates to the on-orbit model can be automatically triggered.

[0092] Step 104: In the selective update phase of the on-orbit model, the ground station performs fine-tuning on the small model corresponding to the on-orbit satellite, generates fine-tuned update parameters, and sends the fine-tuned update parameters to the on-orbit satellite through the uplink so that the on-orbit satellite can load the update parameters and complete the update of the on-orbit model.

[0093] During the operation of an in-orbit satellite, such as Figure 2 As shown, when the weighted statistical difference between the prediction results of the on-orbit model and the prediction results of the ground station's basic model exceeds a preset statistical difference threshold, it indicates that the on-orbit model may be unable to accurately handle the current changing scenarios and tasks, requiring selective updates. To efficiently and effectively update the on-orbit model, knowledge distillation techniques can be used to perform LoRA (Low-Rank Adaptation) fine-tuning. This method can significantly reduce the number of parameters transmitted during each model update while avoiding forgetting the basic context embedded in the fully fine-tuned model, thus meeting the efficiency and performance requirements of on-orbit satellite operation for model updates.

[0094] When using knowledge distillation techniques to perform LoRA fine-tuning on small models corresponding to in-orbit satellites, a composite loss function can be constructed to guide model training and achieve model updates in orbit. This composite loss function can be a weighted sum of a first loss term and a second loss term. The first loss term can be the cross-entropy loss, which measures the difference between the in-orbit model's predictions and the data labels. The second loss term can be the KL divergence term, which measures the difference between the in-orbit model's predictions and the predictions of the ground-based model. The composite loss function can be expressed as follows:

[0095]

[0096] In the formula, Represents the composite loss function. Represents the balance coefficient. Represents the cross-entropy loss function. Indicates KL divergence, superscript and These represent the base model and the on-orbit model, respectively. Indicates data labels, This represents the probability distribution of the on-orbit model's predictions for the entire dataset. This represents the probability distribution of the base model's predictions for the entire dataset.

[0097] To meet the different needs of on-orbit missions, two update modes are supported:

[0098] The first type is manual annotation mode:

[0099] In automated mode, the base model uses the data results as pseudo-labels. However, in manual annotation mode, experts can correct the pseudo-labels. Experts review and adjust the pseudo-labels generated by the base model based on their professional knowledge and experience to ensure the accuracy and reliability of the labels.

[0100] Guided by expert-corrected pseudo-labels, LoRA fine-tuning was performed on the on-orbit model using a composite loss function. LoRA fine-tuning optimizes model performance by introducing low-rank matrices into specific layers of the model to adjust its parameters. During fine-tuning, only the parameters of the low-rank matrices are updated, while keeping other model parameters unchanged, significantly reducing the number of parameters that need to be transferred.

[0101] The second type is the automation mode:

[0102] The output of the ground-based model can be directly used as a pseudo-label. The ground-based model acts as a teacher model, and its predictions provide a learning target for the on-orbit model.

[0103] The generated pseudo-labels and composite loss function can be used to fine-tune the in-orbit model using LoRA. Similarly, only the parameters of the low-rank matrix are updated, avoiding forgetting the basic context embedded in the fully fine-tuned model, while improving the model's adaptability to the current scene and task.

[0104] After the LoRA fine-tuning process described above, a new on-orbit model is generated. Guided by the composite loss function, the new on-orbit model combines prediction information from data labels and the ground-based model, resulting in improved performance.

[0105] The updated parameters of the new on-orbit model are compressed and encoded to reduce data transmission. Then, through the communication link between the ground station and the on-orbit satellite, the updated parameters are uploaded to the on-orbit satellite via uplink. After receiving the updated parameters, the on-orbit satellite updates the on-orbit model, enabling it to handle subsequent scenarios and missions with better performance.

[0106] Step 105: Repeat steps 101-104 above until the cumulative number of updates of the on-orbit model reaches the preset number of updates. Then, aggregate one or more updated on-orbit models into the base model through the ground station to achieve the update of the base model.

[0107] In a satellite-ground collaborative model update system, after completing the on-orbit model update, the ground station's base model needs to be updated. Since this update involves a many-to-one knowledge transfer from a small model (on-orbit model) to a large model (base model), and the system operates in two different modes—manual annotation and automated operation—with varying data characteristics and requirements, an adaptive knowledge distillation mechanism can be employed. This mechanism uses different update strategies depending on the system's operating mode to aggregate the updated on-orbit models from one or more on-orbit satellites into the base model via the ground station. This achieves efficient and accurate updates to the base model, improving overall system performance.

[0108] In manual annotation mode, update strategies may include:

[0109] For each updated on-orbit model, collect data from its clients in the corresponding... The cross-entropy loss function value on the batch data. The cross-entropy loss function value can be used to reflect the degree of difference between the model's prediction results and the true data labels. Based on the cross-entropy loss function values ​​of multiple in-orbit models, the weight value of each in-orbit model is calculated, where the in-orbit model... weight The calculation formula is as follows:

[0110]

[0111] In the formula, Indicating the on-orbit model The weight value, This represents the total number of models in orbit. Indicates the client The cross-entropy loss function value on its corresponding batch of data, Represents the cross-entropy loss function. Indicates the client Real data labels Indicating the on-orbit model Predicted probability distribution of the full dataset Indicates the client The cross-entropy loss function value on its corresponding batch of data, The function converts the loss function values ​​into a probability distribution.

[0112] It can collect all data labels collected by N updated on-orbit models and manually annotated. Obtain the prediction probability distribution of the base model on the full dataset and the prediction probability distribution of each on-orbit model on the full dataset. Calculate the first loss function of the base model according to the following formula:

[0113]

[0114] In the formula, This represents the first loss function value of the base model. and Both represent balance coefficients. Represents the cross-entropy loss function. Indicates by All data labels collected and manually annotated by the updated on-orbit model. This represents the probability distribution of the base model's predictions for the entire dataset. Indicating the on-orbit model Predicted probability distribution of the full dataset This represents the total number of models in orbit. Denotes KL divergence, Indicating the on-orbit model The weight value.

[0115] Using the calculated first loss function, optimization algorithms such as gradient descent are employed to update the parameters of the base model. By continuously adjusting the parameters of the base model, the value of the first loss function gradually decreases, thereby improving the performance of the base model.

[0116] In automated mode, update strategies may include:

[0117] For each updated in-orbit model, the average similarity between its output and the outputs of all other in-orbit models is calculated. The Jensen-Shannon divergence is used to measure the similarity between two probability distributions; for the , The average similarity between the outputs of one on-orbit model and other on-orbit models. The calculation is as follows:

[0118]

[0119] In the formula, Indicates the first The average similarity between the outputs of one in-orbit model and other in-orbit models. The Jensen-Shannon divergence is used to measure the similarity between two probability distributions. Indicating the on-orbit model Predicted probability distribution of the full dataset Indicating the on-orbit model Predicted probability distribution of the full dataset This indicates the total number of models in orbit.

[0120] The weight value of each on-orbit model is calculated based on the average similarity between the output of the on-orbit model and the outputs of all other on-orbit models, as shown in the following formula:

[0121]

[0122] In the formula, Indicating the on-orbit model The weight value, This represents the total number of models in orbit. Indicating the on-orbit model The average similarity between the output of the model and the outputs of all other on-orbit models.

[0123] The prediction probability distribution of the base model on the full dataset and the prediction probability distribution of each in-orbit model on the full dataset can be obtained. Based on the weight values ​​and the prediction probability distributions of the base model and the in-orbit models, the second loss function of the base model is calculated. The second loss function can be expressed as follows:

[0124]

[0125] In the formula, This represents the second loss function value of the base model in automated mode. Represents the balance coefficient. This represents the total number of models in orbit. Indicating the on-orbit model The weight value, Denotes KL divergence, This represents the probability distribution of the base model's predictions for the entire dataset. Indicating the on-orbit model The predicted probability distribution of the full dataset.

[0126] Using the calculated second loss function, optimization algorithms such as gradient descent are employed to update the parameters of the base model. By continuously adjusting the parameters of the base model, the value of the second loss function gradually decreases, thereby improving the performance of the base model in automated mode.

[0127] Compared to manual annotation, the cross-entropy term is omitted. Because there are no manually labeled tags available.

[0128] Model output consistency verification mechanism:

[0129] To ensure the quality of pseudo-labels, the system calculates the consistency of output from multiple satellite models:

[0130]

[0131] In the formula, This indicates the consistency score of the output of the on-orbit satellite model. This represents the total number of models in orbit. Indicating the on-orbit model Predicted probability distribution of the full dataset This represents the predicted probability distribution of the on-orbit model j over the entire dataset. This represents the Jensen-Shannon divergence, used to measure the similarity between two probability distributions.

[0132] when At that time, the model was considered The output is reliable, so it is incorporated into the knowledge aggregation process.

[0133] This disclosure achieves efficient knowledge transfer between satellite and ground through the collaborative work of the above six stages, effectively solving the problem of insufficient adaptability of satellite on-orbit models to distributed off-site scenarios, while keeping communication overhead at an extremely low level.

[0134] To verify the validity of this disclosure, a satellite-ground collaborative learning benchmark (SGCL-Benchmark) is constructed to simulate multi-satellite on-orbit mission execution and satellite-ground collaborative learning. This benchmark set includes seven on-orbit satellites, each performing five different missions, cycled three times (a total of 15 batches / on-orbit satellites), simulating orbital periodicity and mission diversity.

[0135] In the experiment, GeoChat was used as the ground-based model and Qwen2-VL-2B as the in-orbit model. Three scenarios were set up: configurations of 7 in-orbit satellites, 3 in-orbit satellites, and 1 in-orbit satellite, and the update modes of manual annotation and full automation were compared.

[0136] Table 1 shows a performance comparison of the seven satellites in orbit over three cycles. It can be seen that, compared to the "learn-deployment" baseline, the average accuracy of this disclosure on the classification task improved from 59.8% to 66.0%, an improvement of 6.2%. The ROUGE-L score also showed a slight improvement on the image description task, increasing from 28.8% to 29.1%.

[0137] Table 1: Performance comparison of seven satellites in orbit over three cycles (%)

[0138]

[0139] Table 2 shows the average performance of this disclosure under different settings. Experimental results show that the collaborative learning effect becomes more significant with the increase in the number of participating satellites in orbit. With seven satellites in orbit and manual annotation, the average accuracy of this disclosure on the classification task reaches 66.0%, an improvement of 6.2% over the baseline. Even with a single satellite in orbit and a fully automated setting, this disclosure can still improve the classification accuracy from 65.2% to 66.8%, an improvement of 1.6%.

[0140] Table 2: Performance Comparison of SGCL under Different Settings

[0141]

[0142] Table 3 details the improvement effects of this disclosure on seven different OOD tasks. Experimental results show that this disclosure provides significant improvements across all OOD tasks, with an average performance improvement of 9.5%. Specifically, in the formatted OOD task (formal offset), accuracy improved from 51.1% to 65.1%, an improvement of 14.0%; in ship type classification (fine-grained OOD), accuracy improved from 36.7% to 46.3%, an improvement of 9.6%; and in disaster scenario description (domain OOD), the ROUGE-L score improved from 24.8% to 26.5%, an improvement of 1.7%.

[0143] Table 3: The effect of SGCL on different OOD problems

[0144]

[0145] To verify the reliability of the experimental results, this disclosure conducted three repeated experiments, and the results are shown in Table 4. The experimental results show that the method of this disclosure has good stability and a small standard deviation (mean standard deviation of 0.15% for classification task and 0.05% for image description task).

[0146] Table 4: Reliability verification of experimental results

[0147]

[0148] Furthermore, this disclosure also includes component ablation experiments to verify the effectiveness of each module. As shown in Table 5, when the base model update function is removed, the performance of both the on-orbit satellite model and the ground base model decreases significantly; when the "adaptive" distillation mechanism is removed, the performance of both models also decreases. These results demonstrate the effectiveness of each component of this disclosure.

[0149] Table 5: Performance Comparison of SGCL Framework Components

[0150]

[0151] Regarding the effectiveness of the OOD detector, this disclosure presents detailed tests in a satellite-ground co-learning benchmark. In the first two cycles, all 32 OOD tasks were successfully detected. In the third cycle, to reduce communication overhead and model oscillations, the system intentionally ignored 16 recurring OOD tasks. When the visual reconstruction error component was removed, 22 OOD tasks were detected and 10 were missed; removing the Mahalanobis distance component resulted in 20 being detected and 12 being missed; removing the multimodal alignment component resulted in 22 being detected and 10 being missed. These results demonstrate the effectiveness of the OOD detector of this disclosure.

[0152] Regarding update triggers, with a seven-satellite configuration, this disclosure performed 28 on-orbit model updates and 4 ground-based model updates in the first two cycles, accounting for 40% and 5.7% of all missions, respectively. With a three-satellite configuration, 10 on-orbit model updates and 3 ground-based model updates were performed, accounting for 33.3% and 10%, respectively. Importantly, all updates occurred when on-orbit model performance deteriorated, and in the four cases where the OOD module triggered the data downlink but the update triggers failed to activate, on-orbit model performance remained stable, confirming the accuracy of the update triggers.

[0153] Regarding communication overhead, this disclosure uses LoRA technology for on-orbit model updates, requiring the transmission of no more than 7M parameters per update, with a data transmission volume of approximately 13.4MB (FP16 precision). Considering the overhead of checksums, metadata, and other protocol elements, the actual on-orbit satellite-to-ground transmission volume is still less than 15MB / update. A typical on-orbit satellite transit lasts 5-10 minutes (300-600 seconds), and even using a low-rate downlink (5Mbps), the transmission only takes 25 seconds, accounting for only 4.2% to 8.3% of the available transit time. This on-orbit model update capability has been successfully verified on the China Orient Wisdom High Resolution 01 on-orbit satellite.

[0154] The above experimental results fully demonstrate the effectiveness of this disclosure. Compared with the "learn-deployment" baseline, this disclosure significantly improves the ability of on-orbit satellite models to handle out-of-distribution scenarios, especially excelling in domain OOD, fine-grained OOD, and format OOD tasks. Furthermore, this disclosure effectively controls communication overhead through an efficient OOD detection mechanism and update triggers, making it suitable for practical on-orbit satellite systems.

[0155] In satellite-ground collaborative learning systems, in addition to the main scheme already described, there are several alternative methods that can address the problem of insufficient adaptability of satellite on-orbit models to distributed off-site scenarios.

[0156] Satellite-Earth co-learning can be seen as a variant of edge-cloud co-learning, with a collaborative framework serving as an alternative. In this framework, a cloud-based convolutional neural network (CloudCNN) provides soft supervision for local edge convolutional neural networks (EdgeCNNs) for edge training, while EdgeCNNs handle real-time inference with visual input.

[0157] The DCCL framework can also be used as an alternative. This framework introduces an edge-cloud collaboration framework specifically designed for recommendation systems, which can be simplified to retain only its core model update mechanism.

[0158] The COLLA method can be considered an alternative. This method provides a personalized model for each device while facilitating collective learning between the cloud and end devices. The cloud model acts as a global aggregator, extracting knowledge from multiple end-device models.

[0159] The slow-fast learning mechanism can also be used as an alternative. This mechanism is used for edge-cloud collaborative recommendations and can be adapted to satellite-ground collaborative learning scenarios.

[0160] The MDPA framework is a device-cloud collaborative learning framework that enhances a user's local data by retrieving similar data from a global pool in the cloud. This framework can be simplified to use only its model update portion, without relying on satellite-side data augmentation.

[0161] The CLUE framework introduces a framework that enhances knowledge transfer in neural networks within edge computing environments using a multi-model boosting approach. This framework can serve as an alternative to improve the efficiency of knowledge transfer between satellite and ground environments.

[0162] FedRSCLIP is a distributed learning framework designed specifically for remote sensing scene classification. While the framework requires client devices to perform local training, this can be simplified to using only its model aggregation concept.

[0163] In the field of remote sensing visual language models, RemoteCLIP is the first visual-language foundational model specifically designed for remote sensing and can serve as an alternative foundational model. This model addresses the limitations of low-level features, dependence on labeled data, and the lack of language understanding in zero-shot tasks.

[0164] The lightweight Transformer architecture can also be used as an alternative. This architecture is designed for on-device deployment, improves computational efficiency, and is suitable for resource-constrained satellite platforms.

[0165] GeoChat introduces a conversational, remote sensing-specific visual language model with multi-task capabilities, supporting tasks such as image description, change detection, and object counting. This model can serve as an alternative base model.

[0166] SkySense is currently the largest multimodal remote sensing foundation model, pre-trained on time series data of 21.5 million optical and SAR data points using a novel spatiotemporal encoder. This model can serve as an alternative foundation model, providing a more comprehensive understanding of remote sensing scenes.

[0167] While these alternative solutions may not be as performant or efficient as the main solution disclosed herein, they all address, to varying degrees, the problem of insufficient adaptability of satellite on-orbit models to distributed off-orbit scenarios, providing basic functionality and effects. Through appropriate adjustments and simplifications, these solutions can adapt to the specific constraints of satellite platforms, enabling periodic model updates and performance improvements.

[0168] Based on the above Figure 1 The specific implementation of the method shown in this embodiment provides a satellite-ground collaborative learning system, such as... Figure 3 As shown, the system includes: a pull module 31, a detection module 32, a first update module 33, a fine-tuning module 34, and a second update module 35;

[0169] The pull module 31 is used to compress the basic model into a small model through the ground station of the satellite-ground collaborative learning system when the on-orbit satellite joins the completed satellite-ground collaborative learning system, and send the small model to the on-orbit satellite through the uplink so that the on-orbit satellite can use the compressed small model as the on-orbit model.

[0170] The detection module 32 is used to detect the comprehensive distribution offset during the on-orbit inference of the on-orbit satellite using an OOD detector. When the comprehensive distribution offset is detected to be greater than a preset comprehensive distribution offset threshold, the data downlink is triggered to send the relevant data to the ground station.

[0171] The first update module 33 is used to calculate the weighted difference statistics between the prediction results of the on-orbit model and the prediction results of the basic model of the ground station based on the relevant data received by the ground station from the on-orbit satellite, and to trigger a selective update of the on-orbit model when the weighted difference statistics exceed a preset difference statistics threshold.

[0172] The fine-tuning module 34 is used to perform fine-tuning on the small model corresponding to the on-orbit satellite at the ground station during the selective update phase of the on-orbit model, generate fine-tuned update parameters, and send the fine-tuned update parameters to the on-orbit satellite through the uplink so that the on-orbit satellite loads the update parameters and completes the update of the on-orbit model.

[0173] The second update module 35 is used to repeatedly execute the above steps until the cumulative number of updates of the on-orbit model reaches the preset number of updates, and then aggregate one or more updated on-orbit models into the base model through the ground station to realize the update of the base model.

[0174] In specific application scenarios, the detection module 32 can be used to acquire the current batch of data collected by the on-orbit satellite and use the on-orbit model encoder of the on-orbit satellite to extract the visual features and text features of the current batch of data.

[0175] The reconstruction error of the visual features is calculated by a lightweight autoencoder. When the reconstruction error is greater than a preset dynamic error threshold, the current batch of data is marked as a visual anomaly.

[0176] The distribution offset of the text features is calculated based on Mahalanobis distance. When the distribution offset is greater than a preset offset threshold, the current batch of data is marked as an offset anomaly.

[0177] Calculate the cosine similarity between the visual features and the text features. When the cosine similarity is lower than a preset cosine similarity threshold, mark the current batch of data as a multimodal misalignment anomaly.

[0178] Based on the detection results of the visual anomaly, the offset anomaly, and / or the multimodal misalignment anomaly, the comprehensive distribution offset is calculated. When the comprehensive distribution offset is greater than the preset comprehensive distribution offset threshold, the data downlink is triggered to send the relevant data to the ground station.

[0179] In specific application scenarios, the first update module 33 can be used to calculate the prediction probability distribution and prediction confidence of the ground station's basic model for the relevant data, and to calculate the prediction probability distribution of the on-orbit model of the on-orbit satellite for the relevant data;

[0180] Calculate the KL divergence between the predicted probability distribution of the base model and the predicted probability distribution of the on-orbit model, and use the prediction confidence of the base model to perform a weighted operation on the KL divergence to obtain the weighted difference value for each data point.

[0181] Calculate the weighted difference statistics based on the weighted difference values ​​of all data points;

[0182] When the weighted difference statistic exceeds a preset difference statistic threshold, an update to the on-orbit model is triggered.

[0183] In specific application scenarios, the first update module 33 can be used to calculate the median of the weighted difference values ​​of all data points and the absolute deviation value of the median;

[0184] An outlier threshold is determined based on the median and the absolute deviation value;

[0185] Data points whose weighted difference values ​​exceed the outlier threshold are identified as outliers, and the outliers are filtered to obtain a filtered weighted difference set.

[0186] Calculate the average value of the filtered weighted difference set as the weighted difference statistic.

[0187] In a specific application scenario, the fine-tuning module 34 can be used at the ground station to perform LoRA fine-tuning on the small model based on a composite loss function to generate the updated parameters. The composite loss function is composed of a weighted sum of a first loss term and a second loss term. The first loss term is the cross-entropy loss of the difference between the on-orbit model prediction and the data label, and the second loss term is the KL divergence term of the difference between the on-orbit model prediction and the ground-based model prediction.

[0188] In specific application scenarios, the second update module 35 can be used to adopt an adaptive knowledge distillation mechanism and adopt different update strategies according to different system operation modes, so as to aggregate the updated knowledge of the on-orbit model of one or more on-orbit satellites into the base model through the ground station, thereby realizing the model update of the base model. The system operation modes include manual annotation mode and automated mode.

[0189] In a specific application scenario, the second update module 35 can be used in the manual annotation mode, and the update strategy includes:

[0190] The weight value of each of the on-orbit models is calculated based on the cross-entropy loss of each of the multiple on-orbit models.

[0191] Based on the weight values, manually annotated data labels, and the predicted probability distributions of the on-orbit model and the base model, a first loss function of the base model is calculated, and the base model is updated using the first loss function.

[0192] The weight calculation formula for the on-orbit model is as follows:

[0193]

[0194] In the formula, Indicating the on-orbit model The weight value, This represents the total number of models in orbit. Indicates the client The cross-entropy loss function value on its corresponding batch of data, Represents the cross-entropy loss function. Indicates the client Real data labels Indicating the on-orbit model Predicted probability distribution of the full dataset;

[0195] The first loss function is expressed as:

[0196]

[0197] In the formula, This represents the first loss function value of the base model. and Both represent balance coefficients. Represents the cross-entropy loss function. Indicates by All data labels collected and manually annotated by the updated on-orbit model. This represents the probability distribution of the base model's predictions for the entire dataset. Indicating the on-orbit model Predicted probability distribution of the full dataset This represents the total number of models in orbit. Denotes KL divergence, Indicating the on-orbit model The weight value.

[0198] In a specific application scenario, the second update module 35 can be used in the automated mode, where the update strategy includes:

[0199] The weight value of each on-orbit model is calculated based on the average similarity between the output of the on-orbit model and the outputs of all other on-orbit models.

[0200] Based on the weight values ​​and the predicted probability distributions of the base model and the on-orbit model, a second loss function of the base model is calculated, and the base model is updated using the second loss function.

[0201] The weight calculation formula for the on-orbit model is as follows:

[0202]

[0203] In the formula, Indicating the on-orbit model The weight value, This represents the total number of models in orbit. Indicating the on-orbit model The average similarity between the output of and the outputs of all other on-orbit models;

[0204] The second loss function is expressed as:

[0205]

[0206] In the formula, This represents the second loss function value of the base model in automated mode. Represents the balance coefficient. This represents the total number of models in orbit. Indicating the on-orbit model The weight value, Denotes KL divergence, This represents the probability distribution of the base model's predictions for the entire dataset. Indicating the on-orbit model The predicted probability distribution of the full dataset.

[0207] It should be noted that other corresponding descriptions of the functional units involved in the satellite-ground collaborative learning system provided in this embodiment can be found in [reference needed]. Figure 1 The corresponding descriptions of the Chinese methods will not be repeated here.

[0208] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0209] The above are merely specific embodiments of this disclosure, enabling those skilled in the art to understand or implement this disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A satellite-ground collaborative learning method, characterized in that, include: Step 1: When the satellite in orbit joins the completed satellite-ground collaborative learning system, the ground station of the satellite-ground collaborative learning system compresses the basic model into a small model and sends the small model to the satellite in orbit through the uplink, so that the satellite in orbit can use the compressed small model as its in-orbit model. Step 2: Use the OOD detector to detect the overall distribution offset during the on-orbit inference of the satellite. When the overall distribution offset is detected to be greater than the preset overall distribution offset threshold, trigger the data downlink to send the relevant data to the ground station. Step 3: Based on the relevant data received by the ground station from the on-orbit satellite, calculate the weighted difference statistics between the prediction results of the on-orbit model and the prediction results of the ground station's basic model, and trigger a selective update of the on-orbit model when the weighted difference statistics exceed a preset difference statistics threshold; Step 4: In the selective update phase of the on-orbit model, the ground station performs fine-tuning on the small model corresponding to the on-orbit satellite, generates fine-tuned update parameters, and sends the fine-tuned update parameters to the on-orbit satellite via the uplink so that the on-orbit satellite loads the update parameters and completes the update of the on-orbit model; Repeat steps 1-4 above until the cumulative number of updates of the on-orbit model reaches the preset number of updates. Then, aggregate one or more updated on-orbit models into the base model through the ground station to achieve the update of the base model. The step of calculating a weighted statistical difference between the prediction results of the on-orbit model and the prediction results of the ground station's basic model based on relevant data received from the on-orbit satellite by the ground station, and triggering a selective update of the on-orbit model when the weighted statistical difference exceeds a preset statistical difference threshold, includes: Calculate the prediction probability distribution and prediction confidence of the ground station's basic model for the relevant data, and calculate the prediction probability distribution of the on-orbit model of the on-orbit satellite for the relevant data; Calculate the KL divergence between the predicted probability distribution of the base model and the predicted probability distribution of the on-orbit model, and use the prediction confidence of the base model to perform a weighted operation on the KL divergence to obtain the weighted difference value for each data point. Calculate the weighted difference statistics based on the weighted difference values ​​of all data points; When the weighted difference statistic exceeds a preset difference statistic threshold, an update to the on-orbit model is triggered. The step of fine-tuning the small model corresponding to the on-orbit satellite at the ground station to generate updated parameters after fine-tuning includes: At the ground station, LoRA fine-tuning is performed on the small model based on a composite loss function to generate the updated parameters. The composite loss function is composed of a weighted sum of a first loss term and a second loss term. The first loss term is the cross-entropy loss of the difference between the on-orbit model prediction and the data label, and the second loss term is the KL divergence term of the difference between the on-orbit model prediction and the ground-based model prediction.

2. The method according to claim 1, characterized in that, The method of using an OOD detector to detect the integrated distribution offset during the on-orbit inference of the satellite, and triggering a data downlink to send relevant data to the ground station when the detected integrated distribution offset is greater than a preset integrated distribution offset threshold, includes: The current batch of data collected by the on-orbit satellite is obtained, and the visual and text features of the current batch of data are extracted using the on-orbit model encoder of the on-orbit satellite. The reconstruction error of the visual features is calculated by a lightweight autoencoder. When the reconstruction error is greater than a preset dynamic error threshold, the current batch of data is marked as a visual anomaly. The distribution offset of the text features is calculated based on Mahalanobis distance. When the distribution offset is greater than a preset offset threshold, the current batch of data is marked as an offset anomaly. Calculate the cosine similarity between the visual features and the text features. When the cosine similarity is lower than a preset cosine similarity threshold, mark the current batch of data as a multimodal misalignment anomaly. Based on the detection results of the visual anomaly, the offset anomaly, and / or the multimodal misalignment anomaly, the comprehensive distribution offset is calculated. When the comprehensive distribution offset is greater than the preset comprehensive distribution offset threshold, the data downlink is triggered to send the relevant data to the ground station.

3. The method according to claim 1, characterized in that, The calculation of the weighted difference statistics based on the weighted difference values ​​of all data points includes: Calculate the median of the weighted difference values ​​for all data points and the absolute deviation of the median; An outlier threshold is determined based on the median and the absolute deviation value; Data points whose weighted difference values ​​exceed the outlier threshold are identified as outliers, and the outliers are filtered to obtain a filtered weighted difference set. Calculate the average value of the filtered weighted difference set as the weighted difference statistic.

4. The method according to claim 1, characterized in that, The step of aggregating one or more updated on-orbit models into the base model via the ground station to update the base model includes: An adaptive knowledge distillation mechanism is adopted, and different update strategies are used according to different system operation modes. The updated knowledge of the on-orbit model of one or more on-orbit satellites is aggregated into the base model through the ground station to realize the model update of the base model. The system operation modes include manual annotation mode and automated mode.

5. The method according to claim 4, characterized in that, In the manual annotation mode, the update strategy includes: The weight value of each of the on-orbit models is calculated based on the cross-entropy loss of each of the multiple on-orbit models. Based on the weight values, manually annotated data labels, and the predicted probability distributions of the on-orbit model and the base model, a first loss function of the base model is calculated, and the base model is updated using the first loss function. The weight calculation formula for the on-orbit model is as follows: In the formula, Indicating the on-orbit model The weight value, This represents the total number of models in orbit. Indicates the client The cross-entropy loss function value on its corresponding batch of data, Represents the cross-entropy loss function. Indicates the client Real data labels Indicating the on-orbit model Predicted probability distribution of the full dataset The function represents the transformation of the loss function values ​​into a probability distribution; The first loss function is expressed as: In the formula, This represents the first loss function value of the base model. and Both represent balance coefficients. Represents the cross-entropy loss function. Indicates by All data labels collected and manually annotated by the updated on-orbit model. This represents the probability distribution of the base model's predictions for the entire dataset. Indicating the on-orbit model Predicted probability distribution of the full dataset This represents the total number of models in orbit. Denotes KL divergence, Indicating the on-orbit model The weight value.

6. The method according to claim 4, characterized in that, In the automated mode, the update strategy includes: The weight value of each on-orbit model is calculated based on the average similarity between the output of the on-orbit model and the outputs of all other on-orbit models. Based on the weight values ​​and the predicted probability distributions of the base model and the on-orbit model, a second loss function of the base model is calculated, and the base model is updated using the second loss function. The weight calculation formula for the on-orbit model is as follows: In the formula, Indicating the on-orbit model The weight value, This represents the total number of models in orbit. Indicating the on-orbit model The average similarity between the output of and the outputs of all other on-orbit models; The second loss function is expressed as: In the formula, This represents the second loss function value of the base model in automated mode. Represents the balance coefficient. This represents the total number of models in orbit. Indicating the on-orbit model The weight value, Denotes KL divergence, This represents the probability distribution of the base model's predictions for the entire dataset. Indicating the on-orbit model The predicted probability distribution of the full dataset.

7. A satellite-ground collaborative learning system, characterized in that, include: The module includes a fetch module, a detection module, a first update module, a fine-tuning module, and a second update module. The pull module is used to compress the basic model into a small model through the ground station of the satellite-ground collaborative learning system when the satellite in orbit joins the completed satellite-ground collaborative learning system, and send the small model to the satellite in orbit through the uplink so that the satellite in orbit can use the compressed small model as the in-orbit model. The detection module is used to detect the comprehensive distribution offset during the on-orbit inference of the on-orbit satellite using an OOD detector. When the detected comprehensive distribution offset is greater than a preset comprehensive distribution offset threshold, the data downlink is triggered to send the relevant data to the ground station. The first update module is used to calculate the weighted difference statistics between the prediction results of the on-orbit model and the prediction results of the basic model of the ground station based on the relevant data received by the ground station from the on-orbit satellite, and to trigger a selective update of the on-orbit model when the weighted difference statistics exceed a preset difference statistics threshold. The fine-tuning module is used to perform fine-tuning on the small model corresponding to the on-orbit satellite at the ground station during the selective update phase of the on-orbit model, generate fine-tuned update parameters, and send the fine-tuned update parameters to the on-orbit satellite through the uplink so that the on-orbit satellite loads the update parameters and completes the update of the on-orbit model. The second update module is used to repeatedly execute the above steps until the cumulative number of updates of the on-orbit model reaches the preset number of updates, and to aggregate one or more updated on-orbit models into the base model through the ground station to realize the update of the base model; The first update module is specifically used to calculate the prediction probability distribution and prediction confidence of the ground station's basic model for the relevant data, and to calculate the prediction probability distribution of the on-orbit model of the on-orbit satellite for the relevant data; Calculate the KL divergence between the predicted probability distribution of the base model and the predicted probability distribution of the on-orbit model, and use the prediction confidence of the base model to perform a weighted operation on the KL divergence to obtain the weighted difference value for each data point. Calculate the weighted difference statistics based on the weighted difference values ​​of all data points; When the weighted difference statistic exceeds a preset difference statistic threshold, an update to the on-orbit model is triggered. The fine-tuning module is specifically used at the ground station to perform LoRA fine-tuning on the small model based on a composite loss function to generate the updated parameters. The composite loss function is composed of a weighted sum of a first loss term and a second loss term. The first loss term is the cross-entropy loss of the difference between the on-orbit model prediction and the data label, and the second loss term is the KL divergence term of the difference between the on-orbit model prediction and the ground-based model prediction.

8. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the satellite-ground collaborative learning method as described in any one of claims 1 to 6.

Images