Satellite remote sensing image processing method and system

Abstract

This application belongs to the interdisciplinary field of satellite remote sensing and artificial intelligence technologies, and provides a satellite remote sensing image processing method and system. Based on satellite-ground collaboration, after the ground system completes the construction of the training dataset and the initial fine-tuning of the model, the satellite system optimizes and deploys the fine-tuned multimodal large model on the satellite computing platform, sequentially performing image acquisition, preprocessing, ground feature identification, and data filtering and downlink. Incremental model updates are then achieved through satellite-ground data interaction. This application improves processing efficiency, accuracy, and stability, and can be widely applied to satellite remote sensing scenarios such as disaster monitoring, agricultural yield estimation, and land surveys.

Description

Technical Field

[0001] This application relates to the fields of satellite remote sensing and artificial intelligence technology, and in particular to an on-board remote sensing image processing method and system. Background Technology

[0002] With the deep integration of remote sensing and artificial intelligence technologies, on-board remote sensing image processing has become a core development direction in the field of satellite remote sensing. By processing remote sensing images locally on-orbit, it reduces the amount of data transmitted between satellite and ground, improves mission response timeliness, and meets the high real-time requirements of scenarios such as disaster monitoring, agricultural yield estimation, and land surveys. Currently, on-board remote sensing image processing mainly relies on a technical approach combining traditional algorithms with lightweight models. Some solutions attempt to apply multimodal large models to remote sensing image analysis. While each of these technologies has its own implementation logic, they all have significant shortcomings.

[0003] Existing remote sensing image interpretation workflows typically include the following steps: acquiring high-resolution images on-board; transmitting image data to ground receiving stations; and performing ground feature identification and classification by manual personnel or deep learning models deployed on ground servers. It is evident that traditional on-board remote sensing image processing employs a separate architecture of "on-board acquisition - ground-based analysis - on-board execution." The satellite is only responsible for image acquisition and simple caching, transmitting the entire raw image to a ground processing center. The ground center processes the image using traditional machine learning algorithms or lightweight deep learning models before transmitting instructions back to the onboard system. This approach is technically mature and stable, and can adapt to the limited resources of early onboard platforms, but it suffers from poor timeliness and high pressure on satellite-to-ground transmission. In recent years, multimodal large-scale models have been gradually applied to this field due to their excellent generalization capabilities. Existing solutions often adopt the "ground pre-training - lightweight onboard deployment" approach. The ground-based system fine-tunes the model based on natural images or a small number of remote sensing images, performs lightweight processing, and then deploys it to the onboard platform for on-orbit processing. Some solutions attempt to introduce satellite-to-ground collaborative optimization of model performance. Meanwhile, in order to adapt to the limited computing power, power consumption and memory of the spaceborne platform, existing technologies optimize model deployment and operation through strategies such as lightweight inference frameworks, mixed precision inference and efficient parameter tuning, but all lack targeted customized designs.

[0004] While existing technologies have achieved initial implementation, they fall short of meeting the demands for high precision, high real-time performance, and long-term stable operation, presenting numerous key challenges. Multimodal large-scale models are primarily designed for ground platforms, with large parameter scales and high computational requirements. Even after lightweighting, they struggle to adapt to the resource constraints of spaceborne embedded platforms. Furthermore, existing optimization strategies fail to balance inference latency, memory usage, and recognition accuracy, easily leading to stuttering and crashes. Model generalization ability is insufficient; pre-training datasets primarily consist of natural images, significantly differing from the high-contrast, sparse-texture, and easily disturbed characteristics of remote sensing imagery. Existing fine-tuning schemes lack high-quality, large-scale remote sensing-specific training datasets, resulting in low accuracy and reliability in ground feature recognition. The space-ground collaborative closed-loop mechanism is incomplete; the satellite only transmits recognition results without synchronizing abnormal samples and operational logs, leading to aimless ground optimization and poor adaptability of full parameter feedback, resulting in high model update latency and easy performance degradation over long-term operation. The onboard data processing workflow is incomplete. Preprocessing can only complete basic operations and cannot effectively eliminate interference from clouds, shadows, etc. It also lacks an intelligent filtering mechanism based on the characteristics of the identification results. Invalid data is transmitted down, wasting bandwidth and failing to leverage the advantages of localized processing. In addition, the existing solution lacks a comprehensive onboard monitoring and anomaly handling mechanism, cannot monitor hardware status in real time, and lacks emergency response strategies, which can easily lead to mission interruptions and affect the stability of on-orbit operation. Summary of the Invention

[0005] In view of this, this application provides a satellite remote sensing image processing method and system to solve the problems of poor satellite-borne model adaptation, low processing time, and easy performance degradation in the prior art.

[0006] According to one aspect of this application, a satellite-based remote sensing image processing method is provided. Based on the interaction between a ground system and a satellite-based system, remote sensing image processing is implemented on the satellite-based system. The method includes: 1) Under the condition that the ground system has completed the construction of a training dataset and initial model fine-tuning, the satellite-based system deploys a multimodal large model fine-tuned by the ground system on its satellite computing platform. Through the optimization mechanism of the inference framework, the model is adapted to run in the satellite environment. 2) The satellite-based system acquires true-color images of the Earth's surface and associated metadata using remote sensing equipment onboard the satellite, and performs preprocessing operations on the acquired remote sensing images on the satellite. 3) The satellite-based system uses the deployed multimodal large model to perform ground feature identification on the preprocessed remote sensing images and outputs the results. Information to be downloaded is selected based on the features of the identification results. 4) The satellite-based system periodically downloads the identification results, abnormal samples, and operation logs to the ground system. Under the condition that the ground system completes incremental model fine-tuning based on the downloaded data and generates incremental weights, the satellite-based system receives the incremental weights returned by the ground system and updates the multimodal large model deployed on the satellite.

[0007] In one implementation, the process of the ground system constructing the training dataset is as follows: Acquire true-color satellite images covering water bodies, cultivated land, forest land, grassland, artificial surfaces, and bare land, taken by multiple satellites, as the basic training data; perform resolution unification, color space standardization, and noise suppression operations on the basic data sequentially to eliminate data differences; use a multimodal large model, such as Qwen-vl-max (a visual enhancement version of Tongyi Qianwen), to automatically predict the preprocessed basic data and generate initial labels; iterate and optimize repeatedly by adjusting the semantic expression and logical structure of the prompt words; associate the optimized labels with the corresponding preprocessed images to form a structured training dataset that meets the model training requirements. The process of the ground system completing the initial model fine-tuning is as follows: Select a multimodal model, such as Qwen3-VL-8B (a visual enhancement version of Tongyi Qianwen 3), as the model to be fine-tuned; use the LLaMA-Factory training framework to build the model fine-tuning environment and configure the hardware resources and software dependencies required for training; introduce LoRA (Low-Rank) during the fine-tuning process. The low-rank adaptation (Low-rank Adaptation) parameter optimization technique updates only the parameters of the model's visual branches and cross-modal layers, without changing the model's backbone parameters. It configures a mixed-precision training mode of FP16 (half-precision floating-point, 16-bit floating-point) or BF16 (brain half-precision floating-point, 16-bit brain floating-point) and sets a gradient accumulation strategy. The gradient accumulation steps are dynamically adjusted according to the GPU memory capacity of the training hardware to balance training efficiency and memory usage. The constructed training dataset is input into the model to be fine-tuned, and through multiple rounds of supervised optimization training, the model learns the ground feature characteristics of remote sensing images to improve its generalization ability and recognition accuracy in ground feature recognition tasks.

[0008] In one implementation, the spaceborne system deploys a multimodal large model, fine-tuned by the ground system, on a spaceborne computing platform. Through the optimization mechanism of the inference framework, the model is adapted to run in the spaceborne environment. This includes: extracting updated parameters of the fine-tuned model, where the updated parameters only include LoRA tuning parameters for the visual branch and cross-modal layers; configuring a system environment, CUDA (Unified Computing Device Architecture), and related dependency libraries adapted to model operation on the spaceborne computing platform (e.g., NVIDIA Jetson AGX Orin); loading the fine-tuned multimodal large model using the vLLM inference framework, enabling the framework's built-in efficient KV Cache (Key-Value Cache) management, block-based parallel computing, and continuous batch processing mechanisms; and optimizing the model according to the NVIDIA Jetson AGX Orin framework. The hardware performance of the Orin onboard computing platform was tested, including the number of GPU cores and memory bandwidth. The block size and batch processing scale parameters were adjusted, and FP16 mixed precision inference mode was enabled. The running status of the model on the onboard computing platform was tested to verify the running stability, task processing latency and memory usage, and to ensure that the resource constraints of real-time processing on the satellite were met.

[0009] In one implementation, the spaceborne system acquires true-color images of the Earth's surface and associated metadata using remote sensing equipment mounted on a satellite. Preprocessing operations are performed on the acquired remote sensing images on-board, including: acquiring true-color images of the Earth's surface using the satellite's onboard remote sensing camera, and simultaneously recording satellite attitude angles, image acquisition timestamps, and solar altitude angle metadata; receiving the acquired images and metadata by the onboard computing unit and temporarily caching them to ensure data integrity; performing size standardization and brightness normalization operations on the cached images to unify image input specifications; enhancing image contrast and highlighting detailed features of ground objects through histogram equalization technology to improve object recognition; using a specific algorithm to detect clouds and shadows in the images, generating corresponding mask files, and correcting the images based on the masks to eliminate interference from clouds and shadows in subsequent recognition; and normalizing the RGB channels of the images to ensure that the channel data are within a uniform numerical range, guaranteeing the consistency of image data input to the model.

[0010] In one implementation, the spaceborne system utilizes a deployed multimodal large model to perform ground feature identification on preprocessed remote sensing images and output results. This includes: inputting the preprocessed images into a multimodal large model deployed on a spaceborne computing platform and initiating the ground feature identification inference process; after model inference is completed, generating structured classification results in JSON format, the results containing identification information for various ground features such as water bodies, cultivated land, forest land, grassland, artificial surfaces, and bare land; the system extracts key information from the identification results and automatically generates concise classification summaries based on the confidence levels of various ground feature identifications; and it calls historical ground feature identification data for similar areas, compares and analyzes the current identification results with historical data, detects changes in ground feature types and extents, and generates a change detection report.

[0011] In one implementation, after filtering the information to be transmitted based on the features of the recognition results, the method further includes combining task priority and communication window to complete the transmission scheduling, including: comprehensively evaluating the recognition results from three dimensions: result stability, recognition confidence, and degree of change of ground features; if the recognition results of the image area are stable and the recognition confidence of various ground features is higher than a preset threshold, only the classification summary information is retained; if the ground features are detected to have new buildings, water area expansion, or significant changes in farmland type, the area is marked as a key area, and the complete recognition results and related data are retained; the available time period and task priority ranking of the current satellite-to-ground communication window are obtained, a transmission plan is formulated, and the relevant information of the key area is prioritized for transmission within the communication window; according to the transmission plan, the filtered information is encrypted and transmitted to the ground receiving station.

[0012] In one implementation, updating the multimodal large model deployed on the satellite includes: the onboard system periodically classifying and organizing the ground feature identification results, anomaly samples, and system operation logs stored onboard; the anomaly samples are image data with identification confidence below a preset threshold or with blurred ground feature features that cannot be clearly classified; after organization, the data is packaged; when the satellite-to-ground communication window is open, the onboard system transmits the packaged data to the ground system; after receiving the data, the ground system manually re-annotates the anomaly samples, and performs incremental fine-tuning training on the model using the existing training dataset to generate new LoRA incremental weights; the ground system transmits the newly generated LoRA incremental weights back to the onboard system via OTA, employing a data verification mechanism during transmission to ensure the integrity of the weight data; after receiving the incremental weights, the onboard system automatically loads and updates the deployed model parameters to achieve iterative optimization of model performance.

[0013] In one implementation, the method further includes a step of monitoring the onboard operating status on the spaceborne system, specifically: real-time monitoring of the GPU temperature, memory usage, power consumption, and task processing latency hardware operating parameters of the spaceborne computing platform; comparing the monitored parameters with preset normal ranges to determine if any abnormalities exist; if parameters are detected to exceed the preset normal range, automatically performing a downgrade or restart operation, wherein the downgrade operation includes reducing the image processing resolution, reducing the number of images processed simultaneously, or disabling non-core inference optimization functions; and synchronizing the operating status and abnormal information of the spaceborne system to the ground system through the FastAPI interface to support remote monitoring and intervention by ground personnel.

[0014] One implementation also includes a step of knowledge base accumulation on the ground system, specifically: collecting the operation logs of the spaceborne system, ground feature identification results, model update records, and relevant data during the incremental training process on the ground; classifying and storing the collected data into the ground knowledge base according to a preset format, establishing a data index to facilitate subsequent querying and retrieval; and conducting model performance tracking analysis, self-supervised training data expansion, and automatic adjustment of model parameters based on historical data in the knowledge base, providing data support and knowledge accumulation for model optimization and algorithm improvement in subsequent on-board AI missions.

[0015] On the other hand, this application provides an on-board remote sensing image processing system that executes any of the on-board remote sensing image processing methods described in the present application. The system includes a spaceborne system and a ground system that interact via a space-to-ground communication link. The ground system is used to construct a training dataset covering multiple typical land cover types, employing a training framework combined with efficient parameter tuning techniques, mixed-precision training, and gradient accumulation strategies. Based on the training dataset, it performs initial fine-tuning of a multimodal large model, receives recognition results, abnormal samples, and operation logs transmitted from the spaceborne system, completes incremental model fine-tuning, generates incremental weights, and transmits them back to the spaceborne system. The spaceborne system includes an on-board computing platform, a remote sensing acquisition module, a preprocessing module, an intelligent recognition module, and a data filtering and updating module. In this system, the spaceborne computing platform is used to deploy a multimodal large model fine-tuned by the ground system, and the model is adapted to run in the spaceborne environment through the optimization mechanism of the inference framework; the remote sensing acquisition module is used to acquire true-color images of the ground surface and supporting metadata; the preprocessing module is used to perform preprocessing operations on the acquired remote sensing images; the intelligent recognition module is used to use the deployed multimodal large model to identify ground features in the preprocessed remote sensing images and output the results; the data filtering and updating module is used to filter the information to be downloaded according to the characteristics of the recognition results, complete the download scheduling by combining task priority and communication window, periodically download the recognition results, abnormal samples and operation logs to the ground system, and receive the incremental weights returned by the ground system to update the model.

[0016] By employing the aforementioned technical solutions, this application provides a satellite remote sensing image processing method and system. Through the synergy of ground system preprocessing and optimized deployment of the satellite system, the finely tuned multimodal large model can run stably on the satellite computing platform. Combined with an inference framework optimization mechanism, it balances the model recognition accuracy with the resource constraints of the satellite platform's computing power, memory, and power consumption, abandoning the traditional on-board acquisition and ground-based analysis model, and achieving real-time on-orbit processing of remote sensing images. Simultaneously, the intelligent data filtering and satellite-ground collaborative update mechanism transmits only high-value information to the ground, significantly reducing the amount of data transmitted between satellite and ground. Compared to transmitting all images, the transmission efficiency is improved by more than 70%, and the timeliness of processing results in key areas is shortened from hours to minutes, efficiently adapting to scenarios with stringent time requirements such as disaster monitoring and agricultural yield estimation. Furthermore, through a satellite-ground closed-loop iteration, using a mode of transmitting abnormal samples and operation logs from the satellite system and generating incremental weights from the ground system for back transmission, continuous optimization of model performance is achieved, effectively solving the problem of accuracy decay during traditional on-orbit model operation and ensuring the stability of long-term processing results.

[0017] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This paper illustrates the architecture of an onboard remote sensing image processing system based on Qwen3-VL-8B in an embodiment of this application. Figure 2 A flowchart of an on-board remote sensing image processing method provided in an embodiment of this application is shown; Figure 3 A schematic diagram of the structure of an on-board remote sensing image processing system provided in an embodiment of this application is shown. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of them. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present application can be combined with each other.

[0020] As analyzed above, existing technologies have significant shortcomings in terms of model satellite adaptation, scene generalization, satellite-ground collaboration, data processing, and system stability, which restrict the large-scale application of satellite remote sensing image processing technology. There is an urgent need for a technical solution that is highly efficient in adaptation, closed-loop optimization, intelligent processing throughout the entire process, and stable operation to address the deficiencies of existing technologies.

[0021] This application proposes a remote sensing image processing solution centered on onboard intelligent processing and supported by satellite-ground collaborative iteration. It deploys the Qwen3-VL-8B multimodal model, with LoRA parameters efficiently fine-tuned, on the onboard Jetson AGX Orin platform. Combined with optimization mechanisms from the vLLM high-performance inference framework, including efficient KV cache management, block-based parallel computing, and dynamic batch processing, the solution achieves low-latency, stable operation of the large model in the onboard environment. This model can accurately identify typical land cover types such as water bodies, cultivated land, forest land, grassland, artificial surfaces, and bare land. Furthermore, through a multi-stage adaptive fine-tuning mechanism and a satellite-ground collaborative closed-loop design, after initial fine-tuning based on the remote sensing dataset on the ground, the onboard system periodically receives LoRA incremental weights to enable continuous model learning and self-evolution. The solution also downloads the land cover classification JSON results and keyframe summaries to the ground for retraining and validation, forming a complete closed loop of satellite-ground collaborative updates. This ensures long-term iterative optimization of model performance while adapting to the resource limitations of the onboard computing platform.

[0022] See Figure 1 This is an example of the architecture of an onboard remote sensing image processing system based on the Qwen3-VL-8B satellite. It follows a space-ground collaborative logic, divided into two main areas: a ground system and a satellite system. It presents the entire onboard remote sensing image processing workflow based on the Qwen3-VL-8B multimodal large model. The processing flow includes: the onboard system performing real-time on-orbit processing tasks, and the ground system providing support services such as data preparation, model training, and operational assurance. The two systems interact with each other via a two-way space-ground communication link, ultimately achieving the goals of real-time intelligent processing and space-ground collaborative iterative optimization.

[0023] On one hand, the ground system, as the support and guarantee end, demonstrates four core functional modules. Among them, the data collection and processing module is responsible for collecting historical true-color images from multiple satellites, standardizing preprocessing, and generating labels, providing a high-quality dataset for subsequent model training; the model fine-tuning training module, based on this dataset, uses the LLaMA-Factory framework combined with LoRA parameter efficient tuning technology to fine-tune the Qwen3-VL-8B multimodal large model for remote sensing scene adaptation, and outputs model parameters adapted for satellite operation; the ground knowledge base module stores the identification results, abnormal samples, operation logs, and model update records transmitted by the satellite system, which are used for subsequent incremental model training and performance analysis; the ground monitoring system module synchronizes the satellite system's operating status through the satellite-to-ground communication link, realizes communication window scheduling and remote intervention, and ensures stable system operation.

[0024] On the other hand, the satellite system, as the core execution end, demonstrates six functional modules and one hardware carrier. Among them, the remote sensing camera is the image acquisition device carried by the satellite, responsible for acquiring true-color images of the ground surface and supporting metadata such as attitude angle, timestamp, and solar altitude angle; Jetson AGX Orin is the core hardware of the onboard computing platform, carrying all onboard computing tasks and providing computing power support for model deployment and inference; the onboard preprocessing module performs operations such as size standardization, brightness normalization, and cloud / shadow correction on the acquired images to generate high-quality image data that meets the model input requirements; the onboard inference module loads the Qwen3-VL-8B model, which has been fine-tuned on the ground, and optimizes the recognition of ground features through the vLLM inference framework, outputting structured results and analysis reports; the task management module filters the information to be transmitted based on the characteristics of the recognition results, and schedules the data transmission timing by combining task priority and the satellite-to-ground communication window to achieve on-demand transmission; the onboard monitoring system monitors the hardware parameters of the Jetson AGX Orin (an edge artificial intelligence computing module) in real time, such as GPU temperature, memory, and power consumption, and automatically performs degradation or restart operations when abnormalities occur, and synchronizes the status to the ground.

[0025] When the ground system interacts with the satellite system, two-way communication is achieved between the two systems via two core links. The ground system transmits fine-tuned model parameters, incremental weights, and scheduling instructions to the satellite system through the model update / command transmission link, supporting on-board model deployment and updates. The satellite system, in turn, transmits ground feature recognition results, anomaly samples, and operational logs to the ground system through the recognition result / log transmission link for incremental model training and knowledge accumulation. Within the ground system, the training dataset output by the data collection and processing module is input to the model fine-tuning training module to support initial model fine-tuning. The ground monitoring system receives operational status data synchronized from the satellite, combines it with historical information from the ground knowledge base to generate scheduling instructions, and feeds them back to the satellite system. The ground knowledge base receives data transmitted from the satellite and also provides incremental training materials for the model fine-tuning training module, enabling continuous model optimization. Inside the satellite system, images and metadata collected by remote sensing cameras are input to the onboard preprocessing module for data cleaning and enhancement, and then transmitted to the onboard inference module for ground feature identification. The identification results from the onboard inference module are input to the mission management module for data filtering and downlink scheduling, and finally transmitted to the ground via the satellite-to-ground link. The onboard monitoring system monitors the operational status of the Jetson AGX Orin, the onboard inference module, and the mission management module in real time. In case of anomalies, emergency handling is triggered and the status is synchronized to the ground monitoring system. The Jetson AGX Orin, as the core hardware carrier, provides computing power support for all computing modules such as onboard preprocessing, onboard inference, and mission management.

[0026] In one example, the implementation process of a large-model remote sensing image analysis method that can run efficiently in a satellite environment is as follows.

[0027] Step 1: Collection and processing of training data for multimodal large models.

[0028] The training data for the multimodal large model uses over 2,000 true-color images taken by multiple satellites as its foundation, covering typical land cover types such as water bodies, farmland, woodland, grassland, artificial surfaces, and bare land. After preprocessing including resolution unification, color space standardization, and noise suppression, the Qwen-vl-max model is used to automatically predict and generate labels for the images. The recognition accuracy is further improved by continuously adjusting and optimizing the prompts, resulting in a high-quality, structured, and consistent training dataset.

[0029] Step 2: Fine-tuning training of the multimodal large model.

[0030] For fine-tuning of the multimodal large model, this method employs the LLaMA-Factory training framework to efficiently fine-tune the Qwen3-VL-8B multimodal model. By introducing efficient parameter tuning techniques such as LoRA (Low-Rank Adaptation) and combining mixed-precision training (FP16 / BF16) with gradient accumulation strategies, the memory usage and computational cost are significantly reduced. During training, supervised optimization is performed based on high-quality multimodal remote sensing images and accompanying text labels, enabling the model to possess stronger generalization ability and accuracy in ground feature recognition tasks.

[0031] Step 3: Loading and adapting lightweight large models to the environment.

[0032] Deploy the LoRA-tuned Qwen3-VL-8B model in the Jetson AGX Orin environment: fine-tuning is performed on the ground, updating only the visual branch and cross-modal layer parameters; inference is run on the spaceborne platform using the vLLM framework; through vLLM's efficient KV cache management, block parallel computing, and continuous batch processing mechanism, while enabling FP16 mixed precision, the memory usage is effectively reduced and the throughput is improved, thereby achieving real-time processing of multimodal tasks on the resource-constrained spaceborne computing platform.

[0033] Step 4: On-board remote sensing image acquisition and preprocessing.

[0034] True-color images of the Earth's surface are acquired via satellite-borne remote sensing cameras, covering an area of ​​500m × 500m with a resolution of 0.8m. These images are received and cached by the Jetson AGX Orin computing unit. Image acquisition also includes metadata such as attitude angles, timestamps, and solar altitude angles. Image preprocessing is performed on-board, including: size normalization and brightness normalization; histogram equalization to enhance contrast; cloud and shadow detection and mask correction; and RGB channel normalization to ensure input consistency. All steps are completed on-board and do not require transmission to the ground.

[0035] Step 5: Onboard intelligent identification, task selection and collaborative updates.

[0036] After the model runs, it intelligently identifies the input remote sensing images and automatically completes multi-category ground feature analysis and task management.

[0037] (1) Land feature identification and result output: The main land feature types identified by the model include: water, arable land, forest, grassland, artificial surface, and bare land.

[0038] The output is presented in JSON format: { "water_type": 1, "cropland_type": 0, "forest_type": 1, "grass_type": 0, "impervious_type": 0, "bareland_type": 0 The system automatically generates a classification summary and a change detection report based on the confidence level.

[0039] (2) Task screening and download decision: The on-board system automatically judges the importance and degree of change of the results: if the image area is stable and the confidence level is high, only the summary information is saved; if significant changes are detected (such as new buildings, water expansion, etc.), it is marked as a "critical area"; the download is automatically scheduled according to the task priority and communication window to reduce the amount of data. Through this mechanism, the amount of data downloaded is reduced by about 70%, and the utilization rate of transmission bandwidth is improved.

[0040] (3) Satellite-Ground Collaboration and Model Update: The satellite periodically packages and transmits the identification results, abnormal samples, and mission logs to the ground. The ground system re-annotates and incrementally fine-tunes the data, generates new LoRA weights, and transmits them back to the satellite via OTA to achieve model self-iteration and performance optimization.

[0041] (4) Anomaly monitoring and operation management: The system monitors GPU temperature, memory usage, power consumption and latency in real time. If an anomaly is detected, it will automatically perform a downgrade or restart. The system achieves satellite-to-ground operation status synchronization and remote monitoring through the FastAPI interface.

[0042] Step 4: Reverse optimization and knowledge base accumulation.

[0043] All onboard logs, inference results, and model update records are stored in a ground-based knowledge base for subsequent model performance tracking and self-supervised training. Through continuous analysis of historical samples, the system achieves long-term model optimization and automatic parameter adjustment, providing a foundation of knowledge accumulation for future onboard AI missions.

[0044] As can be seen, in the above example of this application, the Qwen3-VL-8B multimodal large model was deployed on the Jetson AGX Orin satellite platform for the first time through a combination of LoRA fine-tuning and vLLM inference framework, which solved the technical bottleneck of satellite adaptation of existing mainstream large models.

[0045] See Figure 2This application provides a satellite remote sensing image processing method based on the interaction between the ground system and the satellite system. The method includes steps S201-S204.

[0046] S201: Based on the ground system having completed the construction of the training dataset and the initial fine-tuning of the model, the spaceborne system deploys the multimodal large model fine-tuned by the ground system in the spaceborne computing platform, and realizes the adaptive operation of the model in the spaceborne environment through the optimization mechanism of the inference framework.

[0047] In this application, the multimodal large model is a visual-text multimodal model, capable of ground feature recognition based on the interaction of remote sensing imagery and text prompts. For example, The visual-text multimodal model integrates a visual encoder and a text encoder. Through a cross-modal attention mechanism, it aligns the semantics of remote sensing image features with those of text prompts. This allows it to extract visual features of ground objects in the image (such as texture, color, and spatial distribution) and parse recognition requirements from text instructions (such as ground object category and output format), thereby accurately completing ground object recognition tasks. For example, the large-scale multimodal model used in this application is Qwen3-VL-8B (Qwen3-Visual Enhanced Version, 8 billion parameters), and the auxiliary model used for generating training data labels is Qwen-vl-max (Qwen3-Visual Enhanced Version, Maximum Parameter Type). Both are large-scale visual-text multimodal models suitable for remote sensing image processing scenarios. The aforementioned model supports flexible text interaction, which can clearly identify targets, confidence thresholds, output formats, etc. through prompts, and can be adapted to different remote sensing scenarios without retraining. The visual branch is optimized for high-resolution images, which can adapt to the characteristics of high contrast and sparse texture of remote sensing images, and improve the recognition of ground features. With a moderate number of parameters (Qwen3-VL-8B), and with LoRA fine-tuning and vLLM inference framework, it can run with low latency on spaceborne embedded platforms (such as NVIDIA Jetson AGX Orin), balancing accuracy and resource consumption.

[0048] The process of constructing the training dataset for the ground system is as follows: more than 2,000 true-color satellite images from multiple satellites, covering 6 typical land features, are acquired as basic data. Resolution unification, color space standardization, and noise suppression operations are performed on the basic data in sequence to eliminate data differences. The Qwen-vl-max multimodal large model is used to automatically predict and generate initial labels for the preprocessed basic data. By adjusting the semantic expression and logical structure of the prompt words, the label accuracy is iteratively optimized, and the final label accuracy is ≥96%. The optimized labels are associated with the corresponding preprocessed images to form a structured training dataset that meets the requirements of model training. The initial fine-tuning process of the ground system model is as follows: The LLaMA-Factory training framework is adopted, and GPU server clusters and software dependencies such as Python 3.9 and PyTorch 2.0 are configured. LoRA parameter efficient tuning technology is introduced, and only the parameters of the model's visual branch and cross-modal layer are updated without changing the model's backbone parameters. FP16 mixed precision training mode is configured, and the gradient accumulation step is set to 4. The constructed training dataset is input into the model to be fine-tuned. After 50 rounds of supervised optimization training, the model learns the ground feature characteristics of remote sensing images, improving its generalization ability and recognition accuracy in ground feature recognition tasks.

[0049] In one implementation, S201 may specifically include the following steps: extracting the updated parameters of the fine-tuned model, wherein the updated parameters only include LoRA tuning parameters for the visual branch and cross-modal layers; configuring a system environment, CUDA environment, and related dependency libraries adapted to the model's operation on the NVIDIA Jetson AGX Orin onboard computing platform; loading the fine-tuned multimodal large model using the vLLM inference framework, enabling the framework's built-in efficient KV Cache management, block parallel computing, and continuous batch processing mechanism; adjusting the block size and batch processing scale parameters according to the hardware performance of the NVIDIA Jetson AGX Orin onboard computing platform, while simultaneously enabling FP16 mixed-precision inference mode; testing the model's running status on the onboard computing platform to verify its running stability, task processing latency, and memory usage, ensuring that the resource constraints for real-time onboard processing are met.

[0050] To address the core constraints of limited computing power, GPU memory, and power consumption on the spaceborne platform, a balance between model recognition accuracy and spaceborne resource consumption is achieved through efficient parameter tuning and customized inference strategies. For example, the spaceborne computing platform utilizes the NVIDIA Jetson AGX Orin embedded GPU platform, deploying the Qwen3-VL-8B multimodal large model, which has been efficiently fine-tuned with LoRA parameters. Only update parameters for the visual branch and cross-modal layers are retained. The inference framework employs the vLLM high-performance inference framework, enabling efficient KVCache management, block-based parallel computing, and FP16 mixed-precision inference mechanisms. Based on the platform's GPU core count (12 cores) and GPU memory capacity (32GB), the block size is dynamically adjusted to 64×64 pixels, and the batch processing scale is 8. In one example, after adaptation and optimization, the model's single inference latency is controlled within 200ms, GPU memory usage is stabilized below 12GB, and power consumption does not exceed 30W, meeting the resource constraints of real-time processing on the satellite and achieving low-latency and highly stable model operation in the spaceborne environment.

[0051] S202: The satellite-borne system acquires true-color images of the Earth's surface and associated metadata through remote sensing equipment carried on the satellite, and performs preprocessing operations on the acquired remote sensing images on the satellite.

[0052] In one implementation, S202 specifically includes: acquiring true-color images of the Earth's surface using a satellite-borne remote sensing camera, simultaneously recording satellite attitude angle, image acquisition timestamp, and solar altitude angle metadata; receiving the acquired images and metadata by the onboard computing unit and temporarily caching them to ensure data integrity; performing size standardization and brightness normalization operations on the cached images to unify image input specifications; enhancing image contrast and highlighting detailed features of ground objects through histogram equalization technology to improve the recognizability of ground objects; using a specific algorithm to detect clouds and shadows in the images, generating corresponding mask files, and correcting the images based on the masks to eliminate interference from clouds and shadows on subsequent recognition; and normalizing the three RGB channels of the images to ensure that the channel data are within a uniform numerical range, thus ensuring the consistency of image data input to the model.

[0053] In this application, the spaceborne system acquires true-color images of the Earth's surface and associated metadata using a high-resolution remote sensing camera mounted on a satellite. Customized preprocessing operations are then performed on the acquired remote sensing images on-board. This preprocessing involves multiple steps to eliminate interference factors in the remote sensing images and standardize data specifications, providing high-quality input for the model. Specifically, the remote sensing camera acquires true-color images with a resolution of 0.8m and a swath width of 500m × 500m. Associated metadata includes satellite attitude angle, image acquisition timestamp, and solar altitude angle. The preprocessing operations sequentially include size standardization, brightness normalization, histogram equalization enhancement, cloud and shadow detection and mask correction, and RGB channel normalization. For example, in images acquired from agricultural areas of the North China Plain, the proportion of cloud and shadow interference areas decreased from 15% to below 2% after preprocessing, and the sharpness of ground feature edges improved by 30%, effectively eliminating inherent interference in the remote sensing images and improving data consistency.

[0054] S203: The spaceborne system uses the deployed multimodal large model to identify ground features in the preprocessed remote sensing images and outputs the results. Based on the characteristics of the identification results, it selects the information to be transmitted.

[0055] In one implementation, S203 may specifically include: inputting the preprocessed image into a multimodal large model deployed on a spaceborne computing platform, and initiating the ground feature recognition inference process; after the model inference is completed, generating a structured classification result in JSON format, the result containing identification information of various ground features such as water bodies, cultivated land, forest land, grassland, artificial surfaces, and bare land; the system extracts key information from the recognition results, and automatically generates a concise classification summary based on the confidence level of various ground feature recognitions; calling ground feature recognition data of historical similar areas, comparing and analyzing the current recognition results with historical data, detecting changes in ground feature types and ranges, and generating a change detection report.

[0056] The "information to be transmitted" refers to high-value data intelligently filtered by the spaceborne system. Specifically, it can include the following four categories: The first category is core results related to land cover identification: including structured classification results in JSON format (covering identification markers and confidence levels for water bodies, cultivated land, forest land, grassland, artificial surfaces, and bare land), classification summaries generated based on confidence levels, and change detection reports comparing current identification results with historical data (such as information on new buildings, water body expansion, and changes in cultivated land type). The second category is abnormal sample data: referring to remote sensing image data with identification confidence levels below a preset threshold (70%) or with blurred land cover features that cannot be clearly classified. This type of data is used by the ground system. Subsequent manual annotation provides targeted material for incremental fine-tuning of the model; the third category is system operation logs: including hardware operation parameters of the onboard computing platform (GPU temperature, memory usage, power consumption, processing latency, etc.), model inference process records, anomaly handling records (such as degradation operations, restart logs), and task execution efficiency statistics, used for ground monitoring system operation status and problem location; the fourth category is complete data of key areas: for "key areas" where significant changes in ground features are detected, the complete identification results of the area, the corresponding pre-processed image segments, and supporting metadata (satellite attitude angle, acquisition timestamp, solar altitude angle, etc.) need to be transmitted to ensure that the ground system can fully verify the changes.

[0057] After filtering the information to be transmitted based on the characteristics of the recognition results, the process can also include scheduling the transmission by combining task priority and communication window. Specifically, the recognition results are comprehensively evaluated from three dimensions: result stability, recognition confidence, and degree of change in ground features. If the recognition results of an image area are stable and the recognition confidence of all types of ground features is higher than a preset threshold, only the classification summary information is retained. If new buildings, water area expansion, or significant changes in farmland type are detected, the area is marked as a key area, and the complete recognition results and related data are retained. The available time period and task priority ranking of the current space-to-ground communication window are obtained, and a transmission plan is formulated, prioritizing the transmission of relevant information in key areas within the communication window. According to the transmission plan, the filtered information is encrypted and transmitted to the ground receiving station to ensure data transmission security. It can be seen that the spaceborne system, through comprehensive evaluation and screening from three dimensions—result stability, recognition confidence, and degree of change in ground features—and the retention of only classification summaries in non-key areas, ultimately reduces the amount of data transmitted by approximately 70%.

[0058] In this application, on-board localized recognition reduces the downloading of all images, intelligently filters and focuses on high-value data, and reduces the bandwidth consumption between satellite and ground. The model recognizes six typical land features: water bodies, cultivated land, forest land, grassland, artificial surfaces, and bare land. It outputs structured results in JSON format, which include land feature type, coordinate range, and recognition confidence. It also generates classification summaries and change detection reports. The filtering rules are constructed from three dimensions: result stability (e.g., recognition consistency ≥95% for three consecutive frames), recognition confidence (e.g., threshold set to 90%), and degree of land feature change (e.g., area change ≥5%). The task priority is divided into disaster monitoring (highest), agricultural yield estimation, land survey, and routine monitoring. The satellite-ground communication window is 08:00-09:00 and 16:00-17:00 daily (single window bandwidth 2Mbps). For example, for remote sensing images of a certain county, after screening, only 15% of the key area data (including newly added areas of artificial land and areas where farmland type has been changed) is downloaded. The amount of satellite-to-ground data transmission is reduced by more than 70% compared to the full download. The timeliness of key area data download is improved to within 2 hours, achieving the effect of bandwidth resource optimization and priority transmission of high-value data.

[0059] S204: The onboard system periodically transmits the identification results, abnormal samples, and operation logs to the ground system. Under the condition that the ground system completes the incremental fine-tuning of the model based on the transmitted data and generates incremental weights, the system receives the incremental weights transmitted back from the ground system and updates the multimodal large model deployed on the satellite.

[0060] In one implementation, S204 may specifically include: the onboard system periodically classifying and organizing the ground feature identification results, abnormal samples, and system operation logs stored onboard. The abnormal samples are image data with identification confidence levels below a preset threshold or with blurred ground feature features that cannot be clearly classified. After organization, the data is packaged. When the space-to-ground communication window is open, the onboard system transmits the packaged data to the ground system. After receiving the data, the ground system manually re-annotates the abnormal samples and, combined with the existing training dataset, incrementally fine-tunes the model to generate new LoRA incremental weights. The ground system transmits the newly generated LoRA incremental weights back to the onboard system via OTA, employing a data verification mechanism during transmission to ensure the integrity of the weight data. After receiving the incremental weights, the onboard system automatically loads and updates the deployed model parameters to achieve iterative optimization of model performance.

[0061] This application constructs a closed-loop satellite-ground collaboration system to address the long-term performance degradation of the model through incremental optimization on the ground and iterative model iteration on the satellite. Specifically, the satellite system packages and downloads the day's data every morning at midnight. Abnormal samples are defined as image data with a recognition confidence level below 70% or unclear ground feature features that cannot be clearly classified. The ground system manually re-annotates abnormal samples and uses the LLaMA-Factory framework for incremental fine-tuning based on the original training dataset, generating LoRA incremental weights with a volume ≤500MB. These weights are then transmitted back via OTA through the satellite-ground communication window, and the satellite system automatically loads the weights to update the model parameters upon receiving them. For example, after one month of satellite-ground closed-loop iteration, the model's average recognition accuracy for six types of ground features improved from the initial 88% to 95%, with particularly significant improvements in the recognition accuracy for low-contrast bare land and small artificial surfaces. This achieves continuous iterative optimization of model performance and adapts to dynamic changes in remote sensing scenes.

[0062] In summary, the on-board remote sensing image processing method provided in this application, through the synergy of ground system preprocessing and optimized deployment of the onboard system, enables the finely tuned multimodal large model to run stably on the onboard computing platform. Combined with the inference framework optimization mechanism, it balances the model recognition accuracy with the resource constraints of the onboard platform's computing power, memory, and power consumption, abandoning the traditional onboard acquisition and ground-based analysis model, and achieving real-time on-orbit processing of remote sensing images. Simultaneously, the intelligent data filtering and space-ground collaborative update mechanism transmits only high-value information to the ground, significantly reducing the amount of space-ground data transmission. Compared to transmitting all images, the transmission efficiency is improved by more than 70%, and the timeliness of processing results in key areas is shortened from hours to minutes, efficiently adapting to scenarios with stringent time requirements such as disaster monitoring and agricultural yield estimation. Furthermore, through a closed-loop iteration between space and ground, using the onboard system to transmit abnormal samples and operation logs, and the ground system to generate incremental weights for back transmission, continuous optimization of model performance is achieved, effectively solving the problem of accuracy decay during traditional on-orbit model operation and ensuring the stability of long-term processing results.

[0063] In one implementation, the present application also includes a step of monitoring the onboard operating status on the spaceborne system, specifically: real-time monitoring of the GPU temperature, memory usage, power consumption, and task processing latency hardware operating parameters of the spaceborne computing platform; comparing the monitored parameters with preset normal ranges to determine if there are any anomalies; if the parameters are detected to exceed the preset normal range, automatically performing a downgrade or restart operation, wherein the downgrade operation includes reducing the image processing resolution, reducing the number of images processed simultaneously, or disabling non-core inference optimization functions; and synchronizing the operating status and anomaly information of the spaceborne system to the ground system through the FastAPI interface to support remote monitoring and intervention by ground personnel.

[0064] The following example illustrates the steps for monitoring the operational status of a satellite system. This step is executed in parallel with the satellite system's image acquisition, preprocessing, ground feature identification, and data filtering and downloading steps, independently consuming minimal system resources to avoid impacting core task processing. First, the satellite system uses an integrated hardware monitoring module and customized software monitoring scripts to monitor the core hardware operating parameters of the satellite computing platform in real time at a frequency of 1 time per second, quickly capturing parameter fluctuations. Specifically, GPU temperature is collected via built-in sensors with an accuracy of ±1℃, covering the GPU core and memory area; memory usage is read through the inference framework interface with an accuracy of ±512MB, simultaneously statistically analyzing the memory usage distribution at each stage; operating power consumption is collected through the power management module with an accuracy of ±0.5W, distinguishing the power consumption percentage of different modules; and task processing latency is statistically analyzed for the entire cycle of a single frame image with an accuracy of ±10ms, correlating the segmented latency of each stage to achieve anomaly tracing. Secondly, the system presets normal ranges for hardware parameters based on hardware performance limits, core mission requirements, and on-orbit environment calibration: GPU temperature 0-85℃, with a 10℃ safety redundancy; video memory usage not exceeding 20GB, with 12GB of redundant space reserved for unexpected tasks; operating power consumption 15-30W, adapting to onboard power supply constraints; single-frame task processing latency not exceeding 300ms, meeting real-time processing requirements. The monitoring module compares real-time parameters with preset ranges every second, generating a status log. When the same parameter exceeds the limit twice consecutively or a single parameter exceeds the limit threshold, it is judged as an anomaly and an emergency response mechanism is triggered to prevent the fault from escalating. Then, when parameters exceed the limits, the system automatically performs tiered degradation or restart operations based on the anomaly type, severity, and task priority, balancing fault handling with the continuity of core missions. The degradation operations include: Level 1 degradation disables non-core inference optimization functions and reduces the number of images processed in parallel at one time from 8 frames to 4 frames, balancing resource consumption and inference accuracy; Level 2 degradation, based on Level 1, reduces the image processing resolution from 512×512 pixels to 256×256 pixels, maintaining core recognition accuracy; Level 3 degradation suspends non-critical area tasks, retaining only high-level tasks. In case of severe anomalies, task progress, recognition results, and anomaly logs are saved first, then the computing platform and related modules are restarted. After restarting, it defaults to basic processing mode to avoid data loss. Finally, the spaceborne system establishes a space-to-ground monitoring link through the FastAPI interface, using HTTP / JSON protocol to ensure transmission compatibility and security, with a 100kbps transmission rate that does not consume core bandwidth. System hierarchical synchronization information: under normal conditions, synchronization occurs every 5 minutes, including average hardware parameters, task progress, and processing efficiency; under abnormal conditions, real-time synchronization is performed, covering abnormal parameters, occurrence time, emergency operations, task status, and data storage status. The ground system displays parameter curves, anomaly records, and operation logs through a visual panel. It also provides a remote intervention interface, allowing staff to send commands with a remote command transmission delay controlled within 30 seconds. This enables space-ground collaborative monitoring and intervention, improving the system's on-orbit stability and flexibility.

[0065] Therefore, by running in parallel with the core processing flow and independently consuming a small amount of resources, real-time monitoring and anomaly handling of the onboard computing platform's hardware parameters are achieved, preventing task interruptions or system crashes due to hardware parameter exceeding limits. The tiered degradation and restart strategy balances fault handling efficiency with the continuity of core tasks when dealing with anomalies. It avoids interrupting critical processing tasks due to minor anomalies and can quickly control the scope of faults through tiered responses, significantly improving the system's continuous and stable operating time and reducing the failure rate by more than 90% compared to solutions without monitoring mechanisms. Simultaneously, space-ground collaborative monitoring and remote intervention enabled by the FastAPI interface allow ground personnel to grasp the onboard system's operating status in real time, flexibly adjust processing strategies, and further enhance the system's adaptability to complex on-orbit environments, ensuring that core tasks are not affected by extreme environments or sudden failures.

[0066] In one implementation, the present application also includes a step of knowledge base accumulation on the ground system, specifically: collecting the operation logs of the spaceborne system, ground feature identification results, model update records, and relevant data during the ground incremental training process; classifying and storing the collected data into the ground knowledge base according to a preset format, establishing a data index to facilitate subsequent querying and retrieval; and conducting model performance tracking analysis, self-supervised training data expansion, and automatic adjustment of model parameters based on historical data in the knowledge base, providing data support and knowledge accumulation for model optimization and algorithm improvement of subsequent on-board AI missions.

[0067] The following example illustrates the steps involved in knowledge base accumulation on the ground system. This step is linked to the space-ground collaborative iteration and onboard monitoring steps, providing data support for continuous model optimization and algorithm iteration. First, comprehensive multi-dimensional core data is collected, covering the entire on-orbit operation process of the spaceborne system and all stages of ground system optimization, ensuring data integrity and relevance. Spaceborne system-related data includes onboard logs, full results of ground feature identification, and model parameter update records. The logs cover hardware parameter fluctuation data, anomaly handling records, and task execution efficiency statistics. Ground feature identification results include structured classification data, filtered and transmitted key area information, and change detection reports. Model update records include incremental weight versions, update times, and performance comparison data before and after updates. Ground system-related data includes hyperparameter configurations during incremental model training, training iteration curves, anomaly sample annotation results, and validation dataset evaluation reports, achieving full aggregation of onboard and ground optimization data. Second, the collected data is cleaned and organized according to a pre-defined standardized format, then categorized and stored in the ground knowledge base. Simultaneously, a multi-dimensional data index is established to improve query and retrieval efficiency. The data cleaning process removes invalid and redundant data and corrects data deviations to ensure data accuracy. Data is categorized and stored into four modules based on data type: operation log repository, recognition result repository, model archive repository, and training dataset repository. A unified data encoding rule and storage format are used to ensure data compatibility. The data index is built based on key dimensions such as data generation time, task scenario, land cover type, and model version, supporting rapid retrieval by single or combined conditions, achieving second-level access to target data. Finally, multi-dimensional in-depth applications are carried out based on historical data in the knowledge base to fully explore data value and provide comprehensive support for subsequent on-board AI missions. Performance tracking analysis is conducted by tracing model performance changes to accurately pinpoint the causes of model recognition accuracy fluctuations in different scenarios and clarify optimization directions. Self-supervised training data is expanded using historical recognition data and annotation information to supplement scarce land cover type samples and extreme scenario samples, improving the richness and coverage of the training dataset. Automatic adjustment of model parameters is achieved based on data mining results, optimizing fine-tuning strategies and inference parameter configurations to adapt to different remote sensing mission requirements. Through the above applications, a closed loop of knowledge accumulation is formed, consisting of "data collection - storage and archiving - value mining - optimization and feedback," which accumulates core data and technical experience for model optimization, algorithm improvement, and scenario expansion of on-board AI missions.

[0068] Therefore, by comprehensively collecting both onboard and ground-based optimized data, and after standardization and categorized storage, a complete knowledge base system is established. This not only provides rich material support for incremental model fine-tuning but also enables precise identification of model optimization directions through performance tracking analysis. Furthermore, it compensates for the lack of scarce samples by expanding self-supervised training data, continuously improving the model's generalization ability in complex remote sensing scenarios. Simultaneously, the accumulated knowledge base allows for model optimization and algorithm improvement for subsequent onboard AI missions to proceed without starting from scratch. It enables rapid adaptation to new scenarios and tasks based on historical data, shortening the technology development and iteration cycle, reducing development costs, and forming a complete knowledge loop of data model-application-optimization, thereby enhancing the long-term application value and technological scalability of the solution.

[0069] Corresponding to the above method, this application also provides an on-board remote sensing image processing system for executing the aforementioned on-board remote sensing image processing method. See [link to relevant documentation]. Figure 3 The system comprises a spaceborne system and a ground system that interact via a space-to-ground communication link. The ground system is used to construct a training dataset covering various typical land cover types. It employs a training framework combined with efficient parameter tuning techniques, mixed-precision training, and gradient accumulation strategies. Based on the training dataset, it performs initial fine-tuning of the multimodal large model. It receives recognition results, anomaly samples, and operation logs transmitted from the spaceborne system and performs incremental model fine-tuning, generating incremental weights which are then transmitted back to the spaceborne system. The spaceborne system includes a spaceborne computing platform, a remote sensing acquisition module, a preprocessing module, an intelligent recognition module, and a data filtering and updating module. The spaceborne computing platform is used to deploy the multimodal large model fine-tuned by the ground system, using an inference frame... The optimization mechanism of the framework enables the model to adapt and run in the spaceborne environment; the remote sensing acquisition module is used to acquire true-color images of the ground surface and supporting metadata; the preprocessing module is used to perform preprocessing operations on the acquired remote sensing images; the intelligent recognition module is used to use the deployed multimodal large model to identify ground features in the preprocessed remote sensing images and output the results; the data filtering and updating module is used to filter the information to be downloaded according to the characteristics of the recognition results, complete the download scheduling by combining task priority and communication window, periodically download the recognition results, abnormal samples and operation logs to the ground system, and receive the incremental weights returned by the ground system to update the model.

[0070] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for processing satellite remote sensing images, characterized in that, Based on the interaction between the ground system and the spaceborne system, remote sensing image processing is implemented on the spaceborne system. The method includes: Given that the ground system has completed the construction of the training dataset and the initial fine-tuning of the model, the spaceborne system deploys the multimodal large model fine-tuned by the ground system in the spaceborne computing platform. Through the optimization mechanism of the inference framework, the model is adapted to run in the spaceborne environment. The satellite-borne system acquires true-color images of the Earth's surface and associated metadata through remote sensing equipment carried on the satellite, and performs preprocessing operations on the acquired remote sensing images on the satellite. The spaceborne system uses the deployed multimodal large model to identify ground features in the preprocessed remote sensing images and outputs the results. Based on the characteristics of the identification results, it selects the information to be transmitted. The onboard system periodically transmits identification results, abnormal samples, and operation logs to the ground system. Under the condition that the ground system completes incremental fine-tuning of the model based on the transmitted data and generates incremental weights, the system receives the incremental weights transmitted back from the ground system and updates the multimodal large model deployed on the satellite.

2. The method according to claim 1, characterized in that, The process of constructing a training dataset for the ground system is as follows: acquire true-color satellite images taken by multiple satellites covering water bodies, cultivated land, forest land, grassland, artificial surfaces, and bare land as basic training data; perform resolution unification, color space standardization, and noise suppression operations on the basic data in sequence to eliminate data differences; and use a multimodal large model to automatically predict the preprocessed basic data to generate initial labels. By adjusting the semantic expression and logical structure of the prompt words, we iteratively optimize them; and associate the optimized labels with the corresponding preprocessed images to form a structured training dataset that meets the requirements of model training. The ground system completes the initial fine-tuning of the model as follows: a multimodal model is selected as the model to be fine-tuned; the LLaMA-Factory training framework is used to build the model fine-tuning environment, and the hardware resources and software dependencies required for training are configured; during the fine-tuning process, the low-rank adaptation LoRA parameter high-efficiency tuning technology is introduced, which only updates the parameters of the model's visual branch and cross-modal layer, without changing the model's backbone parameters; FP16 or BF16 mixed precision training mode is configured, and a gradient accumulation strategy is set. The number of gradient accumulation steps is dynamically adjusted according to the GPU memory capacity of the training hardware to balance training efficiency and GPU memory usage; The constructed training dataset is input into the model to be fine-tuned. Through multiple rounds of supervised optimization training, the model learns the features of land cover in remote sensing images, thereby improving its generalization ability and recognition accuracy in land cover recognition tasks.

3. The method according to claim 1, characterized in that, The spaceborne system deploys a multimodal large model, fine-tuned by the ground system, on the spaceborne computing platform. Through the optimization mechanism of the inference framework, the model is adapted to run in the spaceborne environment, including: Extract the updated parameters of the fine-tuned model, which only include the LoRA tuning parameters of the visual branch and the cross-modal layer; In the embedded computing platform and the on-board computing platform, configure the system environment, CUDA environment and related dependency libraries that are adapted to the model operation. The finely tuned multimodal large model is loaded using the vLLM inference framework, which enables the framework's built-in efficient key-value cache management, block parallel computing, and continuous batch processing mechanisms. Based on the number of GPU cores, memory bandwidth, and hardware performance of the NVIDIA Jetson AGX Orin onboard computing platform, adjust the block size and batch processing scale parameters, and enable FP16 mixed precision inference mode. The model's operation on the onboard computing platform was tested to verify its stability, task processing latency, and memory usage, ensuring that the resource constraints for real-time onboard processing were met.

4. The method according to claim 1, characterized in that, The satellite-borne system acquires true-color images of the Earth's surface and associated metadata using remote sensing equipment mounted on the satellite. Preprocessing operations are performed on the acquired remote sensing images on-board, including: The satellite acquires true-color images of the Earth's surface using its onboard remote sensing camera, and simultaneously records satellite attitude angle, image acquisition timestamp, and solar altitude angle metadata. The onboard computing unit receives the acquired images and metadata, and temporarily caches and stores them to ensure data integrity. Perform size standardization and brightness normalization operations on the cached images to unify the image input specifications; Histogram equalization technology enhances image contrast and highlights detailed features of ground objects, thereby improving their recognizability. A specific algorithm is used to detect clouds and shadows in the image, generate corresponding mask files, and then correct the image based on the mask to eliminate the interference of clouds and shadows on subsequent recognition. The three RGB channels of the image are normalized to ensure that the channel data are within a uniform value range, thus ensuring the consistency of the image data input to the model.

5. The method according to claim 1, characterized in that, The spaceborne system utilizes the deployed multimodal large model to perform ground feature identification on the preprocessed remote sensing images and outputs the results, including: The preprocessed images are input into a multimodal large model deployed on a spaceborne computing platform to initiate the ground feature recognition and inference process. After the model inference is completed, a structured classification result in JSON format is generated, which includes identification information for various land features such as water bodies, cultivated land, forest land, grassland, artificial surfaces, and bare land. The system extracts key information from the recognition results and automatically generates concise classification summaries based on the confidence level of various land features. By calling up historical feature identification data for similar areas, comparing and analyzing the current identification results with historical data, changes in feature types and extent are detected, and a change detection report is generated.

6. The method according to claim 1, characterized in that, After filtering the information to be downloaded based on the features of the recognition results, the method further includes: combining task priority and communication window to complete the download scheduling, including: The identification results are comprehensively evaluated from three dimensions: result stability, identification confidence, and degree of change in ground features. If the recognition results of an image region are stable and the confidence scores of all types of land features are higher than the preset threshold, only the classification summary information is retained; if new buildings, water body expansion, or significant changes in farmland type are detected, the region is marked as a key region and the complete recognition results and related data are retained. Obtain the available time slots and task priority ranking of the current satellite-to-ground communication window, formulate a downlink plan, and prioritize the downlink of relevant information in key areas within the communication window; According to the downlink plan, the filtered information will be encrypted and transmitted to the ground receiving station.

7. The method according to claim 1, characterized in that, The update of the multimodal large model deployed on the satellite includes: The onboard system regularly classifies and organizes the ground feature identification results, abnormal samples, and system operation logs stored on the satellite. The abnormal samples are image data with an identification confidence level lower than a preset threshold or with blurred ground feature features that cannot be clearly classified. After the data is organized, it is packaged. When the spaceborne system opens the space-to-ground communication window, it transmits the packaged data to the ground system. After receiving the data, the ground system manually re-labels the abnormal samples and, combined with the existing training dataset, incrementally fine-tunes the model to generate new LoRA incremental weights. The ground system transmits the newly generated LoRA incremental weights back to the spaceborne system via over-the-air download, and a data verification mechanism is used during the transmission process to ensure the integrity of the weight data. After receiving the incremental weights, the onboard system automatically loads and updates the deployed model parameters to achieve iterative optimization of model performance.

8. The method according to claim 1, characterized in that, It also includes the steps of performing onboard operational status monitoring on the spaceborne system, specifically: Real-time monitoring of the onboard computing platform's GPU temperature, memory usage, power consumption, and task processing latency hardware operating parameters; The monitored parameters are compared with the preset normal range to determine whether there are any abnormalities. If the parameters are detected to be outside the preset normal range, a downgrade or restart operation will be automatically performed. The downgrade operation includes reducing the image processing resolution, reducing the number of images processed at the same time, or turning off non-core inference optimization functions. The FastAPI interface is used to synchronize the operational status and anomaly information of the spaceborne system to the ground system, enabling ground personnel to conduct remote monitoring and intervention.

9. The method according to claim 1, characterized in that, This also includes the step of knowledge base accumulation in the ground system, specifically: Collect operational logs of the spaceborne system, ground feature identification results, model update records, and relevant data during the ground incremental training process; The collected data are categorized and stored in a ground knowledge base according to a preset format, and a data index is established to facilitate subsequent querying and retrieval. Based on historical data in the knowledge base, we conduct model performance tracking analysis, self-supervised training data expansion, and automatic adjustment of model parameters, providing data support and knowledge accumulation for model optimization and algorithm improvement in subsequent on-board AI missions.

10. A satellite-based remote sensing image processing system, characterized in that, The system is used to perform the on-board remote sensing image processing method according to any one of claims 1-9, the system comprising an onboard system and a ground system interacting via a satellite-to-ground communication link; The ground system is used to construct a training dataset covering a variety of typical land cover types. It adopts a training framework and combines efficient parameter tuning technology, mixed precision training and gradient accumulation strategy. Based on the training dataset, it performs initial fine-tuning of the multimodal large model, receives the identification results, abnormal samples and operation logs transmitted from the spaceborne system and completes incremental fine-tuning of the model. After generating incremental weights, it is transmitted back to the spaceborne system. The spaceborne system includes a spaceborne computing platform, a remote sensing acquisition module, a preprocessing module, an intelligent recognition module, and a data filtering and updating module. The spaceborne computing platform deploys a multimodal large model fine-tuned by the ground system, and uses an optimization mechanism in the inference framework to adapt the model for operation in the spaceborne environment. The remote sensing acquisition module acquires true-color images of the Earth's surface and associated metadata. The preprocessing module performs preprocessing operations on the acquired remote sensing images. The intelligent recognition module uses the deployed multimodal large model to identify ground features in the preprocessed remote sensing images and outputs the results. The data filtering and updating module filters information to be downloaded based on the characteristics of the recognition results, schedules downloads according to task priorities and communication windows, periodically downloads recognition results, abnormal samples, and operation logs to the ground system, and receives incremental weights from the ground system to update the model.

Images