A model training method, apparatus, equipment, and medium for a satellite system.

Abstract

This application relates to the field of satellite system data transmission technology, and particularly to a model training method, apparatus, device, and medium for satellite systems. The method includes: acquiring distillation datasets transmitted by various distillation satellites; performing centralized training on each distillation dataset to obtain a first average gradient; acquiring gradient data transmitted by various federated satellites, and generating a second average gradient for each federated satellite based on the gradient data; determining the global gradient of each operating satellite; updating global model parameters based on the global gradient to obtain the global model parameters for the current round, and sending the global model parameters for the current round to the operating satellites; if the global model parameters for the current round do not meet a preset convergence condition, repeating the above steps until the target global model parameters are output and meet the preset convergence condition. This application can solve the problems of excessive communication load, significant privacy and security issues, and low satellite data analysis capabilities in existing technologies.

Description

Technical Field

[0001] This application relates to the field of satellite system data transmission technology, and in particular to a model training method, apparatus, equipment and medium for a satellite system. Background Art

[0002] Traditional satellite system models primarily involve transmitting satellite-collected data to ground base stations, which then transmit it to cloud computing centers for centralized training of machine learning models. However, this approach is increasingly unsuitable for current satellite scenarios, primarily due to the following problems: First, satellites are detached in space, limiting the frequency and duration of their daily communication with ground stations, thus hindering real-time data analysis. Second, transmitting raw satellite image data consumes significant bandwidth, leading to substantial communication loads, and the raw image dataset raises privacy and security concerns.

[0003] Third, satellites launched in different eras have significant differences in energy supply and computing power. These differences may lead to inconsistencies in the time required to train machine learning models, especially when energy is limited, which may cause interruptions in the model training process.

[0004] Fourth, the accuracy of existing satellites in analyzing the collected data is not high, resulting in insufficient satellite data analysis capabilities. Consequently, the predictions made by satellites in weather forecasting, climate analysis, and other analyses based on the collected data tend to have significant errors. Summary of the Invention

[0005] This application provides a model training method, apparatus, equipment, and medium for a satellite system to solve one or more technical problems existing in the prior art, and at least provide a beneficial option or create conditions.

[0006] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0007] According to one aspect of the embodiments of this application, a model training method for a satellite system is provided, the satellite system including a ground base station and operational satellites; the operational satellites include distillation satellites and federation satellites, wherein the distillation satellites and the federation satellites are respectively satellites with low functional configuration and satellites with high functional configuration, the method is executed at the ground base station, and the method includes:

[0008] S110, acquire the distillation dataset sent by each of the distillation satellites, wherein the distillation dataset is obtained by the distillation satellites through high-order feature extraction based on the acquired original image dataset;

[0009] S120, perform centralized training on each of the distillation datasets to obtain the first average gradient of each of the distillation satellites;

[0010] S130, acquire the gradient data transmitted by each of the federal satellites, and generate a second average gradient for each of the federal satellites based on the gradient data.

[0011] S140, determine the global gradient of each of the operating satellites based on the preset first weight coefficient, the preset second weight coefficient, the first average gradient and the second average gradient;

[0012] S150, update the global model parameters according to the global gradient to obtain the global model parameters for the current round, and send the global model parameters for the current round to the operating satellite so as to train the model for each of the operating satellites according to the global model parameters for the current round;

[0013] S160, If the global model parameters of the current round do not meet the preset convergence condition, repeat steps S110-S150 until the iterative function outputs the target global model parameters to meet the preset convergence condition.

[0014] The preset convergence condition is that the model prediction accuracy of each of the operating satellites after model training based on the target global model parameters is not lower than a preset threshold during data analysis.

[0015] The distillation dataset was obtained through the following method:

[0016] The distillation satellite uses the kernel ridge regression algorithm to iteratively process the acquired raw image dataset to obtain an initial distillation set.

[0017] The initial distillation set is optimized using gradient pruning and stochastic gradient descent to obtain the distillation dataset.

[0018] In one embodiment of this application, based on the foregoing scheme, the step of performing centralized training on each of the distillation datasets to obtain the first average gradient of each of the distillation satellites includes:

[0019] Obtain a local loss function corresponding to each of the distillation datasets, and perform data gradient training on each of the distillation datasets based on each local loss function and preset model parameters to obtain gradient values ​​corresponding to each of the distillation datasets.

[0020] The first average gradient of each of the distillation satellites is calculated based on each of the gradient values.

[0021] In one embodiment of this application, based on the foregoing scheme, the step of using gradient pruning and stochastic gradient descent to optimize the features of the initial distillation set to obtain the distillation dataset includes:

[0022] Extract the sample data from the initial distillation set;

[0023] The gradients corresponding to each sample data are clipped according to preset clipping parameters, and noise data conforming to a normal distribution is added to each clipped gradient to obtain each target sample data.

[0024] The distillation dataset is obtained by using stochastic gradient descent to optimize the features of each target sample data.

[0025] In one embodiment of this application, based on the foregoing scheme, determining the global gradient of each of the operating satellites according to a preset first weighting coefficient, a preset second weighting coefficient, a first average gradient, and a second average gradient includes:

[0026] Multiply the first weight coefficient by the first average gradient to obtain the first gradient value;

[0027] Multiply the second weighting coefficient by the second average gradient to obtain the second gradient value;

[0028] The sum of the first gradient value and the second gradient value is taken as the global gradient;

[0029] Wherein, the first weighting coefficient is the quotient of the number of distillation satellites divided by the number of operational satellites, and the second weighting coefficient is the quotient of the number of federation satellites divided by the number of operational satellites.

[0030] In one embodiment of this application, based on the foregoing scheme, updating the global model parameters according to the global gradient to obtain the global model parameters for the current round includes:

[0031] Obtain the preset target learning rate, and obtain the global model parameters from the previous round from the global model parameters obtained from multiple historical model training rounds;

[0032] The global model parameters for the current round are obtained using the following formula:

[0033]

[0034] in, These are the global model parameters for the current round. These are the global model parameters from the previous round. The target learning rate, the This is the global gradient.

[0035] In one embodiment of this application, based on the foregoing scheme, the gradient data of the federal satellite is obtained by the following method:

[0036] After receiving the global model parameters from the previous round sent by the ground base station, the collected local dataset is trained based on the global model from the previous round to obtain the gradient data of the federated satellite.

[0037] Specifically, steps S110-S150 are for model training in the current round, and the gradient data of the federal satellite is used for model training in the current round.

[0038] According to one aspect of the embodiments of this application, a model training apparatus for a satellite system is provided. The satellite system includes a ground base station and operational satellites. The operational satellites include distillation satellites and federated satellites, wherein the distillation satellites and the federated satellites are satellites with lower and higher functional configurations, respectively. The ground base station stores global model parameters obtained from multiple historical rounds of model training. The apparatus is applied to the ground base station and includes:

[0039] The first acquisition unit is used to acquire the distillation dataset sent by each of the distillation satellites, wherein the distillation dataset is obtained by the distillation satellites through high-order feature extraction based on the acquired original image dataset;

[0040] The training unit is used to perform centralized training on each of the distillation datasets to obtain the first average gradient of each of the distillation satellites;

[0041] The second acquisition unit is used to acquire gradient data sent by each of the federal satellites and generate a second average gradient for each of the federal satellites based on the gradient data.

[0042] The determining unit is used to determine the global gradient of each of the operating satellites based on a preset first weighting coefficient, a preset second weighting coefficient, a first average gradient, and a second average gradient.

[0043] The output unit updates the global model parameters according to the global gradient to obtain the global model parameters for the current round, and sends the global model parameters for the current round to the operating satellite so as to train the model for each of the operating satellites according to the global model parameters for the current round.

[0044] An iterative unit is used to repeatedly update the global model parameters if the global model parameters in the current round do not meet the preset convergence condition, until the iterative function outputs the target global model parameters to meet the preset convergence condition.

[0045] The preset convergence condition is that the model prediction accuracy of each of the operating satellites after model training based on the target global model parameters is not lower than a preset threshold during data analysis.

[0046] The distillation dataset was obtained through the following method:

[0047] The distillation satellite uses the kernel ridge regression algorithm to iteratively process the acquired raw image dataset to obtain an initial distillation set.

[0048] The initial distillation set is optimized using gradient pruning and stochastic gradient descent to obtain the distillation dataset.

[0049] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, the computer program including executable instructions that, when executed by a processor, implement the model training method as described in the above embodiments.

[0050] According to one aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a memory for storing executable instructions of the processors, wherein when the executable instructions are executed by the one or more processors, the one or more processors implement the model training method as described in the above embodiments.

[0051] The beneficial effects of this application are: from the perspective of the overall logical architecture, this application classifies satellites with low functional configuration and high functional configuration, and then conducts different training methods according to different categories of satellites, thereby achieving efficient model training effect with reduced communication load.

[0052] First, for the distillation satellites with lower functional configurations, the first average gradient of each distillation satellite is obtained by centrally training the distillation dataset of the distillation satellites. Then, the gradient data sent by the federated satellites with better functional configurations is processed to obtain the second average gradient. The first and second average gradients are then weighted and fused to obtain the global gradient. The global model parameters trained in the past are then updated based on the global gradient to obtain the global model parameters for the current round.

[0053] By sending the global model parameters for the current round to the operating satellites, model training can be performed on each operating satellite based on the global model parameters for the current round. This allows the operating satellites to analyze the acquired image data themselves after model training, without having to transmit the acquired images to ground base stations for analysis. This greatly improves analysis efficiency, reduces the communication load between the satellites and ground stations, and does not involve privacy and security issues.

[0054] Furthermore, if the global model parameters of the current round do not meet the preset convergence condition, then multiple rounds of iteration are required for the global model parameters of the current round. That is, the above steps are repeated to obtain the distillation dataset and gradient data after the model training of the current round, and to further update the global model of the next round until the target global model parameters are output to meet the preset convergence condition. The preset convergence condition is that the model prediction accuracy of each of the operating satellites after model training based on the target global model parameters is not lower than a preset threshold, that is, to maximize the accuracy of the operating satellites when performing image analysis.

[0055] Therefore, this application not only solves the problem of inefficiency caused by the limitation of the number of communications between satellite and ground station, but also solves the problems of communication load and privacy security. It can also train satellites in a targeted manner according to different computing power and functional configurations. The output target global model parameters can greatly improve the accuracy of satellites when performing image analysis, and optimize the data analysis capabilities of all operating satellites in the satellite system as a whole. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly explained below. Obviously, the described drawings are only a part of the embodiments of this application, and not all of them. Those skilled in the art can obtain other design schemes and drawings based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart illustrating a model training method for a satellite system according to an embodiment of this application;

[0058] Figure 2 This is an overall logic diagram of a single round of model training according to an embodiment of this application;

[0059] Figure 3 This is a block diagram of a model training apparatus for a satellite system according to an embodiment of this application;

[0060] Figure 4 This is a schematic diagram of the system structure of an electronic device according to an embodiment of this application. DETAILED DESCRIPTION

[0061] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0062] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0063] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller node devices.

[0064] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0065] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0066] The following is a detailed description of the technical background of the embodiments of this application:

[0067] The main problems in satellite-oriented system models are: 1. Satellite bandwidth resources are limited, and transmitting a large number of raw images to ground stations will cause a huge communication burden, and the raw images involve privacy and security issues; 2. Satellites launched in different years have significant differences in energy supply and computing power. These differences may lead to inconsistencies in the time required to train machine learning models, especially when energy is limited, the model training process may be interrupted.

[0068] Therefore, in order to solve the above problems and make satellite system models (satellite federated learning models) more efficient and usable, this application proposes a model training method for satellite systems, namely a semi-federated learning framework for cross-satellite data distillation and model collaboration for satellite constellations.

[0069] Specifically, the framework is divided into two types: centralized training and federated training. This application classifies satellites with insufficient energy state and computing power (satellites with low functional configuration) as centralized training satellites. These satellites are called distillation satellites. They only need to generate distillation data locally using raw data and transmit it to the ground base station. Then, centralized model training is carried out at the ground base station. Since the distillation data is very small and has strong privacy, it can effectively solve the problems of bandwidth limitation and privacy security.

[0070] Satellites with sufficient energy and computing power (well-configured satellites) fall under the category of federated training. These satellites are called federated satellites. They can perform more complex computational tasks by transmitting model updates (i.e., gradient data) to the ground station for a single global aggregation. Global aggregation involves weighted fusion of the first average gradient obtained from centralized training and the second average gradient obtained from federated training to obtain a new global gradient. Global model parameters are then generated based on this global gradient. Finally, the ground station aggregates the centralized-trained model and the federated-trained model to generate the final global model parameters.

[0071] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0072] According to one aspect of the embodiments of this application, a model training method for a satellite system is provided. Figure 1 This is a flowchart illustrating a model training method for a satellite system according to an embodiment of this application. The satellite system includes a ground base station and operational satellites. The operational satellites include distillation satellites and federated satellites, wherein the distillation satellites and federated satellites are satellites with lower and higher functional configurations, respectively. The method is executed at the ground base station. The method includes at least steps S110 to S160, which are described in detail below:

[0073] In step S110, the distillation dataset sent by each of the distillation satellites is obtained. The distillation dataset is obtained by the distillation satellites through high-order feature extraction based on the acquired original image dataset.

[0074] Specifically, the distillation dataset is obtained by the distillation satellite through high-order feature extraction on the raw image dataset it acquires. For example, after acquiring the raw image data, the large volume of the raw image data would cause excessive communication load and consume a lot of bandwidth resources if transmitted to the ground base station. Furthermore, the raw data involves privacy and security issues. Therefore, the distillation dataset obtained by performing high-order feature extraction on the raw image dataset occupies less bandwidth, and conventional privacy parameter encryption is performed after high-order feature extraction, thus solving the privacy and security problems.

[0075] In one embodiment of this application, the distillation dataset can be obtained by the following method:

[0076] The distillation satellite uses the kernel ridge regression algorithm to iteratively process the acquired raw image dataset to obtain an initial distillation set.

[0077] The initial distillation set is optimized using gradient pruning and stochastic gradient descent to obtain the distillation dataset.

[0078] Specifically, this application proposes a data distillation method based on differential privacy kernel-induced points. This method generates differential privacy distilled samples by pruning the gradients of the distillation dataset and adding calibration noise during training. The overall goal of the data distillation process is to find a small dataset. It is derived from a large original image dataset based on the data distribution P of the learning algorithm A and the loss function l. The approximation of ξ can be expressed as follows:

[0079] ;

[0080] At the nuclear induction point, loss Classification accuracy is represented by the distance between the true and predicted labels. The learning algorithm A is Kernel Ridge Regression (KRR). In this process, the distillation dataset is first randomly initialized, and then iteratively refined by minimizing the Kernel Ridge Regression (KRR) loss. :

[0081] ;

[0082] Here, It is a regularization parameter and >0, This is the kernel matrix. During the training phase, gradient-based optimization methods (such as stochastic gradient descent) are used to update the distillation dataset until certain pre-defined convergence conditions are met. Labels are extracted from a uniform distribution of class numbers. The original KIP algorithm has an option to optimize the label using a given optimized distillation image.

[0083] In one embodiment of this application, the step of using gradient pruning and stochastic gradient descent to optimize the features of the initial distillation set to obtain the distillation dataset includes:

[0084] Extract the sample data from the initial distillation set;

[0085] The gradients corresponding to each sample data are clipped according to preset clipping parameters, and noise data conforming to a normal distribution is added to each clipped gradient to obtain each target sample data.

[0086] The distillation dataset is obtained by using stochastic gradient descent to optimize the features of each target sample data.

[0087] Specifically, in the specific distillation process, the distillation dataset is first initialized, and each sample data (input data) is... The gradient is drawn from a normal distribution. Privacy parameters (scale parameters used when adding noise) are calculated based on the set privacy level. The gradient is then clipped according to preset clipping parameters to ensure that the norm of the gradient does not exceed a threshold C. Specifically, the gradient... Cropped to That is, if the gradient Norm exceeds Then scale its size. Add noise to the clipped gradients to obtain the target sample data; the noise follows a normal distribution. Accumulate all gradients and use stochastic gradient descent (SGD) to update the distillation dataset. Finally, a privacy-preserving supporting dataset is generated. .

[0088] In step S120, centralized training is performed on each of the distillation datasets to obtain the first average gradient of each of the distillation satellites.

[0089] In one embodiment of this application, the step of performing centralized training on each of the distillation datasets to obtain the first average gradient of each of the distillation satellites includes:

[0090] Obtain a local loss function corresponding to each of the distillation datasets, and perform data gradient training on each of the distillation datasets based on each local loss function and preset model parameters to obtain gradient values ​​corresponding to each of the distillation datasets.

[0091] The first average gradient of each of the distillation satellites is calculated based on each of the gradient values.

[0092] Specifically, in intensive training, it is assumed that... Each satellite extracted its own distillation dataset, which together formed a dataset. This training method transmits distillation data to a ground station via satellite-to-ground communication for centralized training. Based on this dataset, the first average gradient is finally obtained. It can be expressed by the following formula:

[0093] ;

[0094] Where w represents the model parameters (i.e., the global model parameters from the previous round). It is the gradient value obtained from each centrally trained satellite, and this value can be expressed as: , This represents the total number of distillation samples (which can be obtained from the distillation dataset), while From the first centralized training The first satellite A sample of distillation data (obtainable from the distillation dataset). It is a local loss function (which can be calculated based on the global model parameters from the previous round).

[0095] In step S130, gradient data transmitted by each of the federal satellites is acquired, and a second average gradient of each of the federal satellites is generated based on the gradient data.

[0096] Specifically, in federated training (FL), it is assumed that there are Each of the several satellites generated its own gradient update, resulting in gradient data for each federal satellite, which together form a gradient set. These gradient data will be uploaded to ground base stations for federated averaging, ultimately generating a second average gradient. It can be expressed by the following formula:

[0097] ;

[0098] In one embodiment of this application, the gradient data of the federal satellite is obtained by the following method:

[0099] After receiving the global model parameters from the previous round sent by the ground base station, the collected local dataset is trained based on the global model from the previous round to obtain the gradient data of the federated satellite.

[0100] Specifically, steps S110-S150 involve model training for the current round, and the gradient data from the federal satellites are used for model training in the current round.

[0101] Specifically, the gradient data of the federated satellites used in the current round of model training is obtained after training the global model parameters in the previous round. Therefore, this application can continuously update the global model parameters in multiple rounds so that the final output target global model parameters can meet the preset convergence conditions.

[0102] In step S140, the global gradient of each of the operating satellites is determined according to the preset first weight coefficient, the preset second weight coefficient, the first average gradient, and the second average gradient.

[0103] The step of determining the global gradient of each of the operating satellites based on a preset first weighting coefficient, a preset second weighting coefficient, a first average gradient, and a second average gradient includes:

[0104] Multiply the first weight coefficient by the first average gradient to obtain the first gradient value;

[0105] Multiply the second weighting coefficient by the second average gradient to obtain the second gradient value;

[0106] The sum of the first gradient value and the second gradient value is taken as the global gradient;

[0107] Wherein, the first weighting coefficient is the quotient of the number of distillation satellites divided by the number of operational satellites, and the second weighting coefficient is the quotient of the number of federation satellites divided by the number of operational satellites.

[0108] Specifically, the ground base station will use the first average gradient obtained from centralized training in the current round of model training. The second average gradient obtained from federated training The weights are used to generate a global gradient G to update the global model parameters, as shown in the following equation.

[0109] ;

[0110] in, The preset first weighting coefficient, This is the preset second weighting coefficient.

[0111] In step S150, the global model parameters are updated according to the global gradient to obtain the global model parameters for the current round, and the global model parameters for the current round are sent to the operating satellites so that the models of each operating satellite can be trained according to the global model parameters for the current round.

[0112] In one embodiment of this application, updating the global model parameters according to the global gradient to obtain the global model parameters for the current round includes:

[0113] Obtain the preset target learning rate, and obtain the global model parameters from the previous round from the global model parameters obtained from multiple historical model training rounds;

[0114] The global model parameters for the current round can be obtained using the following preset iterative function formula:

[0115]

[0116] in, These are the global model parameters for the current round. These are the global model parameters from the previous round. The target learning rate, the This is the global gradient for the current round of model training.

[0117] In step S160, if the global model parameters of the current round do not meet the preset convergence condition, then steps S110-S150 are repeated until the iterative function outputs the target global model parameters to meet the preset convergence condition.

[0118] The preset convergence condition is that the model prediction accuracy of each of the operating satellites after model training based on the target global model parameters is not lower than a preset threshold during data analysis.

[0119] If the global model parameters in the current round do not meet the preset convergence condition, steps S110-S150 will be repeated, which is to perform iterative training. In the new round of training, that is, the next round of model training, when the satellite establishes a connection with the ground base station, it will first send the global model parameters of the current round to provide local model training for each satellite. After the satellite has performed local model training, the ground base station will continue to perform iterative rounds of centralized training and federated training on the distillation dataset or gradient data transmitted by the satellite until the target global model parameters are output and training stops.

[0120] Therefore, from the perspective of the overall satellite system model, we assume the satellite system is a constellation consisting of K satellites and G ground stations. Satellite k∈ Collect and store datasets of type {1,2,...,K}. Furthermore, the satellites collaboratively learn the global model w∈Rd by minimizing the global objective function, and the process can be represented by the following formula:

[0121]

[0122] in It refers to the size of the dataset. The size of the sum of all satellite datasets, and the local objective function for the satellite. Is with dataset The relevant loss function aims to minimize the global model. The loss over all client datasets, and this loss function can be expressed as:

[0123]

[0124] in Data points The training loss of the model parameters can be solved if the ground station can download the complete dataset from the satellite. The problem is that, in other words, through formula (1), the global model parameters that can achieve the minimum overall loss rate can be found in the continuous iterative training process. The overall loss rate is the sum of the loss rates of all satellites when performing data analysis, thereby improving the data analysis capabilities of all operating satellites.

[0125] In one embodiment of this application, a communication model is also proposed: in a geocentric inertial coordinate system, this application assumes that satellite k and ground station g each have a trajectory and Where t is the continuous clock time. If the link between the satellite and the ground station is connectable, then satellite k must be at its minimum elevation angle. The area visible from ground station g is referred to as the visible window in this invention. The angle of the window is calculated as follows:

[0126]

[0127] in, This represents the difference between the ground station trajectory and the satellite trajectory. This refers to the angle between the satellite and the ground station. When the satellite is within the visible angle range of the ground station, the satellite and the ground station can establish a communication connection.

[0128] Connectivity Strategy: This application utilizes the first orbital edge computing simulator and runtime services to construct a satellite constellation scenario connecting low Earth orbit satellites to ground stations. This includes 100 operational satellites and 10 ground stations. The ground segment consists of two ring arrays, each with 5 evenly distributed ground stations, located at 87°N and 87°S latitude respectively. The 100 satellites in the space segment operate in a polar orbit with an inclination of 97.3°.

[0129] The overall logic of this application can be referred to as follows: Figure 2 As shown, this application combines the advantages of federated learning and centralized learning, integrating dataset distillation technology with centralized learning to reduce communication pressure between satellite and ground stations. Differential privacy enhances data security, and the distilled data is transmitted to ground stations to fully utilize their computing power. For satellites with sufficient computing power and energy status, federated learning is used to update the local model (gradient data) and transmit it to the ground station, fully utilizing the computing resources of the federated satellites while effectively protecting the security of the original data and achieving efficient model training.

[0130] Figure 3This is a block diagram of a satellite system model training device 300 according to an embodiment of this application. The satellite system includes a ground base station and operational satellites; the operational satellites include distillation satellites and federation satellites, wherein the distillation satellites and the federation satellites are respectively satellites with lower and higher functional configurations; the device is applied to the ground base station; the device 300 includes: a first acquisition unit 301, a training unit 302, a second acquisition unit 303, a determination unit 304, and an output unit 305.

[0131] The first acquisition unit 301 is used to acquire the distillation dataset sent by each of the distillation satellites, wherein the distillation dataset is obtained by the distillation satellites through high-order feature extraction based on the acquired original image dataset;

[0132] Training unit 302 is used to perform centralized training on each of the distillation datasets to obtain the first average gradient of each of the distillation satellites;

[0133] The second acquisition unit 303 is used to acquire gradient data sent by each of the federal satellites and generate a second average gradient for each of the federal satellites based on the gradient data.

[0134] The determining unit 304 is used to determine the global gradient of each of the operating satellites based on a preset first weighting coefficient, a preset second weighting coefficient, a first average gradient, and a second average gradient.

[0135] The output unit 305 is used to update the global model parameters according to the global gradient to obtain the global model parameters of the current round, and send the global model parameters of the current round to the operating satellite so as to train the model for each of the operating satellites according to the global model parameters of the current round.

[0136] An iterative unit is used to repeat the above steps if the global model parameters of the current round do not meet the preset convergence condition, until the iterative function outputs the target global model parameters that meet the preset convergence condition.

[0137] The preset convergence condition is that the model prediction accuracy of each of the operating satellites after model training based on the target global model parameters is not lower than a preset threshold during data analysis.

[0138] In another aspect, this application also provides a computer-readable storage medium storing a program product capable of implementing the methods provided above in this specification. In some possible implementations, various aspects of this application may also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Embodiment Methods" section of this specification according to various exemplary embodiments of this application.

[0139] The program product for implementing the above-described method according to the embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of this application is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0140] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0141] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0142] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0143] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0144] In another respect, this application also provides an electronic device capable of implementing the above-described method.

[0145] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0146] The following reference Figure 4 To describe an electronic device 400 according to this embodiment of the present application. Figure 4 The electronic device 400 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0147] like Figure 4 As shown, the electronic device 400 is manifested in the form of a general-purpose computing device. The components of the electronic device 400 may include, but are not limited to: at least one processing unit 410, at least one storage unit 420, and a bus 430 connecting different system components (including storage unit 420 and processing unit 410).

[0148] The storage unit stores program code that can be executed by the processing unit 410, causing the processing unit 410 to perform the steps described in the "Embodiment Methods" section above according to various exemplary embodiments of this application.

[0149] Storage unit 420 may include readable media in the form of volatile storage units, such as random access memory (RAM) 421 and / or cache memory 422, and may further include read-only memory (ROM) 423.

[0150] Storage unit 420 may also include a program / utility 424 having a set (at least one) of program modules 425, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0151] Bus 430 can represent one or more of several types of bus structures, including a memory cell bus or memory cell control node, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0152] Electronic device 400 can also communicate with one or more external devices 1200 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 400, and / or with any device that enables electronic device 400 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 450. Furthermore, electronic device 400 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 460. As shown, network adapter 460 communicates with other modules of electronic device 400 via bus 430. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0153] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this application.

[0154] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this application, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0155] It should be understood that this application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A model training method for a satellite system, characterized in that, The satellite system includes ground base stations and operational satellites; the operational satellites include distillation satellites and federation satellites, wherein the distillation satellites and the federation satellites are satellites with low and high functional configurations, respectively. The ground base station stores global model parameters obtained from multiple historical training rounds. The method is executed at the ground base station and includes: S110, acquire the distillation dataset sent by each of the distillation satellites, wherein the distillation dataset is obtained by the distillation satellites through high-order feature extraction based on the acquired original image dataset; S120, perform centralized training on each of the distillation datasets to obtain the first average gradient of each of the distillation satellites; S130, acquire the gradient data transmitted by each of the federal satellites, and generate a second average gradient for each of the federal satellites based on the gradient data. S140, determine the global gradient of each of the operating satellites based on the preset first weight coefficient, the preset second weight coefficient, the first average gradient and the second average gradient; S150, update the global model parameters according to the global gradient to obtain the global model parameters for the current round, and send the global model parameters for the current round to the operating satellite so as to train the model for each of the operating satellites according to the global model parameters for the current round; S160, If the global model parameters of the current round do not meet the preset convergence condition, repeat steps S110-S150 until the iterative function outputs the target global model parameters to meet the preset convergence condition. The preset convergence condition is that the model prediction accuracy of each of the operating satellites after model training based on the target global model parameters is not lower than a preset threshold during data analysis. The distillation dataset was obtained through the following method: The distillation satellite uses the kernel ridge regression algorithm to iteratively process the acquired raw image dataset to obtain an initial distillation set. The initial distillation set is optimized using gradient pruning and stochastic gradient descent to obtain the distillation dataset.

2. The model training method according to claim 1, characterized in that, The step of performing centralized training on each of the distillation datasets to obtain the first average gradient of each of the distillation satellites includes: Obtain a local loss function corresponding to each of the distillation datasets, and perform data gradient training on each of the distillation datasets based on each local loss function and preset model parameters to obtain gradient values ​​corresponding to each of the distillation datasets. The first average gradient of each of the distillation satellites is calculated based on each of the gradient values.

3. The model training method according to claim 2, characterized in that, The initial distillation set is feature-optimized using gradient pruning and stochastic gradient descent to obtain the distillation dataset, which includes: Extract the sample data from the initial distillation set; The gradients corresponding to each sample data are clipped according to preset clipping parameters, and noise data conforming to a normal distribution is added to each clipped gradient to obtain each target sample data. The distillation dataset is obtained by using stochastic gradient descent to optimize the features of each target sample data.

4. The model training method according to claim 3, characterized in that, The step of determining the global gradient of each of the operating satellites based on a preset first weighting coefficient, a preset second weighting coefficient, a first average gradient, and a second average gradient includes: Multiply the first weight coefficient by the first average gradient to obtain the first gradient value; Multiply the second weighting coefficient by the second average gradient to obtain the second gradient value; The sum of the first gradient value and the second gradient value is taken as the global gradient; Wherein, the first weighting coefficient is the quotient of the number of distillation satellites divided by the number of operational satellites, and the second weighting coefficient is the quotient of the number of federation satellites divided by the number of operational satellites.

5. The model training method according to claim 4, characterized in that, The step of updating the global model parameters based on the global gradient to obtain the global model parameters for the current round includes: Obtain the preset target learning rate, and obtain the global model parameters from the previous round from the global model parameters obtained from multiple historical model training rounds; The global model parameters for the current round are obtained using the following formula: in, These are the global model parameters for the current round. These are the global model parameters from the previous round. The target learning rate, the This is the global gradient.

6. The model training method according to claim 5, characterized in that, The gradient data of the federal satellites were obtained through the following method: After receiving the global model parameters from the previous round sent by the ground base station, the collected local dataset is trained based on the global model from the previous round to obtain the gradient data of the federated satellite. Specifically, steps S110-S150 are for model training in the current round, and the gradient data of the federal satellite is used for model training in the current round.

7. A model training device for a satellite system, characterized in that, The satellite system includes a ground base station and operational satellites; the operational satellites include distillation satellites and federated satellites, the distillation satellites and federated satellites being satellites with lower and higher functional configurations, respectively; the ground base station stores global model parameters obtained from multiple historical training rounds; the device is applied to the ground base station and includes: The first acquisition unit is used to acquire the distillation dataset sent by each of the distillation satellites, wherein the distillation dataset is obtained by the distillation satellites through high-order feature extraction based on the acquired original image dataset; The training unit is used to perform centralized training on each of the distillation datasets to obtain the first average gradient of each of the distillation satellites; The second acquisition unit is used to acquire gradient data sent by each of the federal satellites and generate a second average gradient for each of the federal satellites based on the gradient data. The determining unit is used to determine the global gradient of each of the operating satellites based on a preset first weighting coefficient, a preset second weighting coefficient, a first average gradient, and a second average gradient. The output unit updates the global model parameters according to the global gradient to obtain the global model parameters for the current round, and sends the global model parameters for the current round to the operating satellite so as to train the model for each of the operating satellites according to the global model parameters for the current round. An iterative unit is used to repeatedly update the global model parameters if the global model parameters of the current round do not meet the preset convergence condition, until the iterative function outputs the target global model parameters to meet the preset convergence condition. The preset convergence condition is that the model prediction accuracy of each of the operating satellites after model training based on the target global model parameters is not lower than a preset threshold during data analysis. The distillation dataset was obtained through the following method: The distillation satellite uses the kernel ridge regression algorithm to iteratively process the acquired raw image dataset to obtain an initial distillation set. The initial distillation set is optimized using gradient pruning and stochastic gradient descent to obtain the distillation dataset.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to perform the operations performed by the method as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, The electronic device includes one or more processors and one or more memories, wherein at least one piece of program code is stored in the one or more memories, and the at least one piece of program code is loaded and executed by the one or more processors to perform the operation performed by the method as described in any one of claims 1 to 6.

Images