Satellite on-orbit data processing method based on multi-modal large model

Abstract

The application discloses a satellite on-orbit data processing method based on a multi-modal large model, a large model on a satellite is deployed on the satellite, and the large model on the satellite is used for executing a data processing task; a large model on the ground is deployed in a ground data processing center, and the large model on the ground is used for executing a model training optimization task; a specific data processing procedure is as follows: S1. The large model on the satellite is used for performing on-orbit processing on received original data; S2. The original data is downloaded to the ground data processing center; S3. The large model on the ground is used for processing the original data by the ground data processing center; S4. The ground data processing center calculates a deviation between a processing result on the satellite and a processing result on the ground; S5. The processing result on the satellite is used when the deviation is less than a threshold value; S6. A third-level data processing result is used as a final result when the deviation is greater than the threshold value; S7. The large model on the ground is trained and optimized by using label data; S8. The large model on the satellite is generated; and S9. The compressed large model on the satellite is uploaded to the satellite. The method improves the data processing performance and the intelligent level of the satellite by using the multi-modal large model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of space in-orbit data processing and artificial intelligence, and in particular to a satellite in-orbit data processing method based on a multi-modal large model. BACKGROUND

[0002] With the development of space technology, the computing power on the satellite is continuously improved, and it has become possible to process the data obtained by the satellite in orbit in real time. Currently, the satellite completes the payload data processing software at the same time as the satellite is developed and is launched into orbit together with the satellite. However, the processing performance of this method is static and cannot be continuously optimized as the amount of data increases. Moreover, when it is found that the processing result has errors or deviations, the ground personnel need to modify the software again and perform sufficient verification on the ground before updating the satellite, which has problems such as long cycle, high risk, etc.

[0003] In recent years, artificial intelligence technology has developed rapidly, especially the emergence of multi-modal large models, which has greatly improved the intelligent level of data processing. It is an inevitable trend of the development of space technology to use multi-modal large model related artificial intelligence technology to improve the in-orbit data processing performance of the satellite.

[0004] The existing in-orbit data processing method has problems such as low intelligent level, long software update cycle, and inability to quickly find potential association information between different types of data. The data processing software is developed for components or tasks during the satellite development stage, and a single software cannot adapt to the diversified task data processing requirements. At the same time, if the satellite finds software problems during in-orbit operation or reconstructs the satellite hardware function through software uploading, the supporting software also needs to be updated accordingly. This process usually includes software modification scheme design, scheme confirmation, software development, software testing and verification, satellite uploading, etc., which has a long cycle and greatly limits the performance of the satellite. The existing data processing software processes different types of data independently, and it is difficult to find potential association information between different types of data obtained from different sources. SUMMARY

[0005] In view of the problems existing in the prior art, the purpose of the present application is to provide an in-orbit data processing method of a multi-modal large model, which uses a multi-modal large model to improve the diversified data processing performance and intelligent level on the satellite.

[0006] To achieve the above-mentioned purpose, the present application designs an in-orbit data processing method of a satellite based on a multi-modal large model, which deploys a large model on the satellite, the large model on the satellite is used to perform diversified data processing tasks on the satellite in orbit; a ground data processing center deploys a ground large model, the ground large model is used to perform model training and optimization tasks; the specific data processing process is as follows:

[0007] S1. Onboard processing of the received raw data using an onboard large model, and the processing result is transmitted to the ground data processing center;

[0008] S2. The ground data processing center periodically uploads commands to the satellite to download the raw data to the ground data processing center;

[0009] S3. The ground data processing center processes the raw data using a ground large model;

[0010] S4. The ground data processing center automatically calculates the deviation between the onboard processing result and the ground processing result using a pre-defined loss function;

[0011] S5. When the deviation is less than a set threshold, the onboard processing result is used, and the ground data processing center uploads a command to the satellite to suspend the download of raw data;

[0012] S6. When the deviation is greater than the set threshold, the ground data processing center uses the ground large model processing result as the final result, and the raw data and ground large model processing result are used as label data;

[0013] S7. The ground large model is trained and optimized using the label data;

[0014] S8. The trained ground large model is compressed using pruning and quantization methods to generate an onboard large model;

[0015] S9. The compressed onboard large model is uploaded to the satellite.

[0016] Further, the raw data includes satellite platform operating state data and payload collected task data.

[0017] Further, in the deployment phase, according to the onboard data processing task, a general multi-modal large model is selected, the model is fine-tuned and trained using satellite development and test phase data, and used as a ground large model; and the ground large model is compressed and used as an onboard large model.

[0018] Further, when fine-tuning and training the ground large model, a partial fine-tuning method is used, thereby maintaining the ability of the multi-modal large model bottom layer parameters while increasing the ability to adapt to specific satellite data processing tasks.

[0019] Further, the ground large model is compressed using quantization and pruning methods, thereby reducing the amount of energy and computing resources required for the onboard large model to run.

[0020] Further, the data processing tasks on the satellite are divided into two categories: single-component data processing and multi-component data correlation analysis. Single-component data processing refers to processing data of a single component of the satellite platform and the payload, and analyzing long-term accumulated data to find and predict performance degradation or failure information of the single component. Multi-component data correlation analysis refers to processing data of multiple related components of the satellite platform and the payload based on the task.

[0021] Further, a two-level satellite data processing architecture is formed: the first level of data processing is autonomous processing on the satellite, and the second level of data processing is autonomous processing on the ground. From the first level to the second level, the timeliness of data processing decreases gradually, and the accuracy of data processing increases gradually.

[0022] Further, the condition for activating the second level of data processing is set as follows: when the ground data processing center reaches the cycle time limit and the deviation between the on-board processing result and the ground large model processing result exceeds the threshold, the second level of data processing is activated.

[0023] Further, the condition for restoring to the first level of data processing is set as follows: when the deviation between the on-board processing result and the ground large model processing result is less than the threshold, the on-board transmission of raw data is suspended, and the first level of data processing is restored.

[0024] Further, in the second level of data processing, when the deviation between the on-board processing result and the ground large model processing result is less than the threshold, the ground large model processing result is used as the final result because the ground large model has higher accuracy than the on-board large model.

[0025] Further, the loss function of the single-component data processing task is defined as follows:

[0026]

[0027] where L is the loss function of the single-component data processing task, C is the on-board large model processing result vector, C i is the i-th parameter of the on-board large model processing result, K is the ground large model processing result vector, K i is the i-th parameter of the ground large model processing result.

[0028] Further, the loss function of the multi-component data correlation analysis task is defined as follows:

[0029]

[0030] where M is the loss function of the multi-component data correlation analysis task, L i is the loss function of the i-th single-component data processing task, r i is the weight of the component in the correlation analysis task, which can be flexibly adjusted according to different tasks.

[0031] Advantages:

[0032] The satellite on-orbit data processing method based on the multi-modal large model provided in the present application can process diversified data on the satellite by using the multi-modal large model, effectively reduce the workload of developing different application programs for different data processing tasks and the on-satellite software update cycle, and quickly find potential association relationships between different data.

[0033] The multi-modal large model application method provided can improve the accuracy of satellite data processing results by using the model fine-tuning method, reduce the demand of the large model on on-satellite computing resources and energy by using the compression method, and improve the utilization rate of on-satellite computing resources by combining the ground and on-satellite computing power characteristics to perform model training tasks on the ground and only perform model inference tasks on the satellite.

[0034] The data processing flow based on the multi-modal large model provided can ensure the reliability of the data processing results by comparing the on-satellite and ground data processing results, and continuously improve the accuracy of the data processing results by using the original data and the processing results to generate new label data to optimize the model.

[0035] The large model optimization suspension method provided can effectively reduce the occupation of the satellite-ground communication bandwidth resources caused by the continuous update of the on-satellite large model and the continuous transmission of the original data on the satellite. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 The figure is a flowchart of the satellite on-orbit data processing method based on the multi-modal large model of the present application. DETAILED DESCRIPTION

[0037] The technical solutions of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0038] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0039] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "linking" should be understood in a broad sense, for example, it can be fixed connection, or detachable connection, or integral connection; it can be mechanical connection, or electrical connection; it can be direct connection, or indirect connection through intermediate medium, or internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0040] The following will be described in detail in combination with Figure 1 The specific embodiments of the present application are described in detail. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0041] The present application proposes a satellite on-orbit data processing method based on a multi-modal large model, which deploys a data processing multi-modal large model on the ground data processing center and on the satellite respectively. The on-board large model only performs data processing tasks; the ground data processing center performs model training and optimization tasks; the ground data analysis personnel synchronously process the on-board raw data and compare the processing results with the on-board large model processing results, make label data to train the large model; the updated large model is uploaded to the satellite; and a management method for updating the large model is proposed.

[0042] As Figure 1 shown, according to the satellite on-orbit data processing method based on the multi-modal large model of the present application, in the satellite development stage, according to the satellite data processing task, a multi-modal large model is selected, which refers to an algorithm that can simultaneously process different data types such as text, numerical value and image. In the present application, the multi-modal large model is fine-tuned and compressed by using the data in the satellite development and test stage; the compressed and fine-tuned model (on-board large model) is deployed to the satellite and launched into orbit; the uncompressed but fine-tuned model (ground large model) is deployed in the ground data processing center.

[0043] The specific data processing process is as follows:

[0044] S1. The on-board large model is used to process the received raw data on-orbit, and the processing results are downloaded to the ground data processing center; the raw data includes satellite platform running state data and task data collected by the payload;

[0045] S2. When the ground data processing center finds that the on-board processing result is abnormal, or periodically uploads a command to the satellite to download the raw data to the ground data processing center;

[0046] S3. The ground data processing center processes the raw data by using the ground large model;

[0047] S4. The ground data processing center automatically calculates the deviation between the on-board processing result and the ground processing result using a pre-defined loss function;

[0048] S5. When the deviation is less than a set threshold, the ground large model processing result is used; the ground data processing center uploads a command to the satellite to suspend the transmission of raw data;

[0049] S6. When the deviation is greater than the set threshold, the raw data is subjected to third-level data processing, and the third-level data processing result is used as the final result; and the raw data and the third-level data processing result are made into labeled data;

[0050] S7. The ground large model is trained and optimized using the labeled data;

[0051] S8. The trained ground large model is compressed using pruning and quantization methods to generate an on-board large model;

[0052] S9. The compressed on-board large model is uploaded to the satellite.

[0053] On-board data processing tasks can be divided into two categories: single-component data processing and multi-component data correlation analysis. Single-component data processing refers to processing data of a single component of the satellite platform and the payload, and through analysis of long-term accumulated data, discovering and predicting information such as performance degradation and faults of the single component. Multi-component data correlation analysis refers to processing data of multiple related components of the satellite platform and the payload to optimize the parameter configuration of each component, thereby improving the level of satellite task execution.

[0054] The loss function of the single-component data processing task is defined as follows:

[0055]

[0056] where L is the loss function of the single-component data processing task, C is the on-board large model processing result vector, C i is the i-th parameter of the on-board large model processing result, K is the ground large model processing result vector, K i is the i-th parameter of the ground large model processing result.

[0057] For example, a large model is used to process data of a satellite orbit control component, the current orbit parameters of the satellite are T c =(a0, e0, i0, Ω0, ω0, M0), and the target orbit is T t =(a t ,e t ,i t ,Ω t ,ω t , Mt The satellite orbit control actuator settings, calculated using the on-board large model, are K=(k1,k2,k3,…,k…). n The satellite orbit control actuator settings obtained through ground-based large-scale model calculations are G=(g1,g2,g3,…,g…). n Then the loss function is:

[0058]

[0059] The loss function for multi-component data correlation analysis tasks is defined as follows:

[0060]

[0061] Where M is the loss function for multi-component data association analysis tasks, and L... i Let r be the loss function for the i-th single-component data processing task. i This is the weight of the component in the association analysis task, and the weight can be flexibly adjusted according to different tasks.

[0062] For example, a large model can be used to explore the correlation between satellite orbit control components and on-orbit target recognition data processing. When the satellite maneuvers to the target orbit, the satellite's optical payload is simultaneously activated to photograph a specific area, and the large model is used to identify targets in the images. If the target recognition performance of interest is the number of targets detected, assuming the on-board large model detects N targets... k The number of targets detected by the ground-based large model is N. g Then the loss function for the object detection task is:

[0063]

[0064] The loss function for the entire task is then:

[0065] ( )

[0066] According to the satellite on-orbit data processing method based on a multimodal large model of the present invention, a two-level satellite data processing architecture is formed: the first level of data processing is on-board autonomous processing, and the second level of data processing is ground-based autonomous processing; from the first level to the second level, the timeliness of data processing decreases progressively, while the accuracy of data processing increases progressively. Conditions for activating the second level of data processing are set: when the ground data processing center reaches its cycle limit and the deviation between the on-board processing results and the ground-based large model processing results exceeds a threshold, the second level of data processing is activated.

[0067] The conditions for resuming to Level 1 data processing are as follows: when the deviation between the on-board processing result and the ground-based large model processing result is less than a threshold, the on-board uploading and downloading of raw data is paused, and Level 1 data processing is resumed.

[0068] In the second-level data processing, when the deviation between the on-board processing result and the ground-based large model processing result is less than the threshold, the ground-based large model processing result is used as the final result because the ground-based large model has higher accuracy than the on-board large model.

[0069] The present invention provides a satellite on-orbit data processing method based on a multimodal large model, which includes deploying a multimodal large model on the satellite to process diverse data processing tasks and discover potential correlation information between multi-source data; data analysts select a general multimodal large model according to the on-board data processing task, fine-tune and train the model using data from the satellite development and testing phases, and use it as a ground large model; and compress the ground large model using a pruning method to use it as the on-board large model.

[0070] The multi-model large model application method includes fine-tuning the multi-modal large model using data from the satellite development and testing phases, compressing the large model on the satellite according to the accuracy requirements of on-board data processing, training the model using the ground-based high-computing-power environment, and only executing model inference tasks on the satellite.

[0071] In step S2, the data processing flow based on the multimodal large model includes processing satellite data simultaneously on the satellite and on the ground, and using the raw data and processing results to generate label data to continuously optimize the large models on the satellite and on the ground.

[0072] In step S4, the large model optimization pause method is used to determine the optimization pause conditions, including:

[0073] S4.1. Users define the deviation and threshold of data processing results according to the characteristics of the task and the requirements of data processing accuracy. When the requirements are met, the update of the large model on the satellite and the ground is paused.

[0074] S4.2. When the on-board data processing task changes or the processing result exceeds the threshold, restart the large model update process.

[0075] The above processing flow can continuously optimize on-board data processing performance, but the synchronous downlink of raw data from the satellite and the continuous uplinking of the large model from the ground also incur significant data transmission overhead between the satellite and ground. Therefore, when the on-board data processing task remains unchanged, ground data analysts can decide whether to suspend the downlink of raw data from the satellite and the updating of the large model from the ground based on the deviation from the on-board data processing results. The definition and threshold requirements for data processing result deviation differ for different task types and different users, and are defined by the user. When the on-board data processing task changes or the processing result exceeds the set threshold again, the process is repeated. Figure 1The data processing flow shown updates the large onboard model.

[0076] Furthermore, to improve data processing accuracy, in another preferred embodiment, the present invention can form a three-level satellite data processing architecture: the first level of data processing is on-board autonomous processing, the second level of data processing is ground-based autonomous processing, and the third level of data processing is manual processing. From the first level to the third level, the timeliness of data processing decreases progressively, while the accuracy of data processing increases progressively. The conditions for activating each level of data processing are set as follows: when the ground data processing center detects an anomaly in the on-board processing results or reaches the cycle limit, the second level of data processing is activated; when, during the second level of data processing, the deviation between the on-board processing results and the ground-based large model processing results exceeds a threshold, the third level of data processing is activated. The on-orbit data processing method proposed in this invention supports the same software to process different types of data, which can reduce the workload of on-board software development and quickly identify the correlation information between different types of data; at the same time, the on-board data processing performance can be rapidly improved through model updates.

[0077] Any process or method described in the flowcharts of this invention or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, achievable on any computer-readable medium for use by an instruction execution system, apparatus, or device. The computer-readable medium can be any medium containing a program for storage, communication, propagation, or transmission for use by an execution system, apparatus, or device, including read-only memory, magnetic disks, or optical disks.

[0078] In the description of this specification, references to terms such as "embodiment," "example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, those skilled in the art can combine or combine the different embodiments or examples described in this specification and the features therein without causing contradiction.

[0079] While embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and alterations to the above embodiments within the scope of the present invention.

Claims

1. A method for processing satellite on-orbit data based on a multimodal large model, characterized in that, A large-scale on-board model is deployed on the satellite to perform diverse on-orbit data processing tasks; a large-scale ground-based model is deployed at the ground data processing center to perform model training and optimization tasks; the specific data processing flow is as follows: S1. The satellite uses an on-board large model to process the received raw data in orbit and then transmits the processing results to the ground data processing center. S2. The ground data processing center periodically uploads commands to the satellite, which then downloads the raw data back to the ground data processing center. S3. The ground data processing center uses a large ground model to process the raw data; S4. The ground data processing center uses a pre-defined loss function to automatically calculate the deviation between the on-board processing results and the ground processing results; S5. When the deviation is less than the set threshold, the on-board processing results will continue to be used; the ground data processing center will send a command to the satellite to pause the download of raw data. S6. When the deviation exceeds the set threshold, the data processing center uses the ground model processing result as the final result; and creates label data from the original data and the ground model processing result. S7. Use labeled data to train and optimize the large-scale ground model; S8. Compress the trained ground-based large model using pruning and quantization methods to generate a satellite-based large model; S9. Upload the compressed on-board model to the satellite.

2. The satellite on-orbit data processing method based on a multimodal large model according to claim 1, characterized in that, The raw data includes satellite platform operational status data and mission data collected by the payload.

3. The satellite on-orbit data processing method based on a multimodal large model according to claim 1, characterized in that, During the deployment phase, based on the on-board data processing mission, a general multimodal large model is selected. The model is fine-tuned and trained using data from the satellite development and testing phases and used as a ground-based large model. The ground-based large model is then compressed using pruning or quantization methods and used as the on-board large model.

4. The satellite on-orbit data processing method based on a multimodal large model according to claim 1, characterized in that, Onboard data processing tasks are divided into two categories: single-component data processing and multi-component data correlation analysis. Single-component data processing refers to processing data from a single component of the satellite platform and payload, and analyzing long-term accumulated data to discover and predict performance degradation or fault information of the single component. Multi-component data correlation analysis refers to processing data from multiple related components of the satellite platform and payload, driven by the mission.

5. The satellite on-orbit data processing method based on a multimodal large model according to claim 1, characterized in that, A two-tier satellite data processing architecture has been established: the first tier of data processing is on-board autonomous processing, and the second tier of data processing is ground-based autonomous processing; from the first tier to the second tier, the timeliness of data processing decreases at each level, while the accuracy of data processing increases at each level.

6. The satellite on-orbit data processing method based on a multimodal large model according to claim 5, characterized in that, The conditions for activating the second-level data processing are set: when the ground data processing center reaches the cycle time limit and the deviation between the on-board processing results and the ground large model processing results is found to exceed the threshold, the second-level data processing is activated.

7. The satellite on-orbit data processing method based on a multimodal large model according to claim 5, characterized in that, The conditions for resuming to Level 1 data processing are as follows: when the deviation between the on-board processing result and the ground-based large model processing result is less than a threshold, the on-board uploading and downloading of raw data is paused, and Level 1 data processing is resumed.

8. The satellite on-orbit data processing method based on a multimodal large model according to claim 1, characterized in that, In the second-level data processing, when the deviation between the on-board processing result and the ground-based large model processing result is less than the threshold, the ground-based large model processing result is used as the final result because the ground-based large model has higher accuracy than the on-board large model.

9. The satellite on-orbit data processing method based on a multimodal large model according to claim 4, characterized in that, The loss function for single-component data processing tasks is defined as follows: Where L is the loss function for single-component data processing tasks, C is the vector of processing results from the on-board large model, and C i Let K be the i-th parameter of the on-board large model processing result, and K be the vector of the ground-based large model processing result. i This is the i-th parameter in the result of processing the large ground model.

10. The satellite on-orbit data processing method based on a multimodal large model according to claim 4, characterized in that, The loss function for multi-component data correlation analysis tasks is defined as follows: Where M is the loss function for multi-component data association analysis tasks, and L... i Let r be the loss function for the i-th single-component data processing task. i This is the weight of the component in the association analysis task, and the weight can be flexibly adjusted according to different tasks.

Images