Space-borne heterogeneous perception federated learning method based on spiking neural network

Abstract

A spaceborne heterogeneous sensing federated learning method based on a spiking neural network comprises the following steps: firstly, initializing a global model according to a satellite-ground cooperative computing network architecture, and distributing the global model to satellite nodes of each orbit plane; thirdly, introducing a pulse neural network into each satellite node to serve as a bottom layer calculation engine, and performing low-power-consumption model training of differentiated step lengths on local remote sensing data; then, in each orbit plane, satellite nodes carry out local communication through inter-satellite links, based on the actual training step length of each satellite, a single-step normalized model update quantity and an orbit effective step length are calculated, and in-orbit model weighted aggregation for eliminating computing power bias is executed; and after a plurality of rounds of in-orbit aggregation, the ground station server collects each orbit model and executes cross-orbit global aggregation. According to the method, the problem of inconsistent federated learning targets caused by heterogeneous satellite computing power can be effectively eliminated while the satellite-borne edge computing energy consumption can be remarkably reduced, and the convergence speed and prediction precision of the model in a network environment are improved.

Description

Technical Field

[0001] This invention relates to the intersection of distributed machine learning and space-ground fusion communication. It is a federated edge learning method that combines spiking neural networks to address the heterogeneous computing power and energy consumption limitations of spaceborne equipment in dynamic space-ground collaborative computing network scenarios. Specifically, it is a spaceborne heterogeneous perception federated learning method based on spiking neural networks. Background Technology

[0002] With the rapid development of satellite manufacturing technology and the significant reduction in launch costs, low Earth orbit (LEO) mega-constellations have become a key infrastructure for the next generation of global connectivity. To process the massive amounts of multimodal Earth observation data generated by high-resolution remote sensing payloads carried by LEO satellites, the concept of Space-Ground Integrated Computing Power Networks (SG-CPN) has emerged. Traditional centralized processing requires transmitting massive amounts of raw data to ground stations via satellite-to-ground links for centralized processing. This not only introduces unacceptable communication delays but also raises serious data privacy and security risks. Therefore, introducing Federated Edge Learning (FEEL) into SG-CPNs, enabling satellites to share only model parameters rather than raw, sensitive data, has become an effective paradigm for addressing the privacy and transmission bottlenecks of massive onboard data.

[0003] However, deploying distributed deep learning in a spaceborne environment faces severe physical challenges. Traditional deep learning algorithms, such as artificial neural networks (ANNs), rely on high-precision floating-point operations and intensive matrix multiply-accumulate (MAC) operations, resulting in extremely high computational energy consumption. Since low-Earth orbit satellites primarily rely on solar panels for energy collection and limited battery power during periods of Earth's shadow, uncontrolled computational energy consumption not only rapidly depletes battery reserves but also accelerates satellite aging due to frequent charge-discharge cycles. Therefore, the energy-intensive traditional artificial neural networks are ill-suited for the "always-on" spaceborne learning tasks. To overcome this energy consumption bottleneck in spaceborne computing power, the academic community has begun to focus on third-generation neural networks—spiking neural networks (SNNs). With their brain-inspired event-driven mechanism and discrete binary pulse communication characteristics, SNNs can transform energy-intensive multiply-accumulate operations into simple accumulation (AC) operations, demonstrating energy efficiency potential several orders of magnitude higher than traditional networks, making them ideal as computing engines for resource-constrained edge nodes.

[0004] On the other hand, to alleviate the communication pressure on satellite-to-ground links, some studies in recent years have proposed hierarchical satellite federated edge learning methods based on satellite-to-ground cooperative transmission, such as adding an in-orbit aggregation mechanism in the global rounds. However, most existing satellite-to-ground hierarchical federated architectures assume that the onboard computing environment is ideally isomorphic and all use traditional federated averaging (FedAvg) or full reduction algorithms for model aggregation. In actual large-scale satellite constellations, since satellites often come from different manufacturing batches or service providers, their onboard computing hardware, such as CPU frequency and power consumption limits, exhibits severe device-level heterogeneity. This difference in hardware computing power inevitably leads to different local stochastic gradient descent (SGD) update step sizes that different satellites can perform within the same time window. At this time, if the traditional averaging algorithm in the existing architecture is still used, satellites with strong computing power and larger update step sizes will be implicitly given larger gradient weights, causing the aggregated global model to gradually deviate from the true global optimum, i.e., causing a serious "objective inconsistency" problem. Especially in scenarios where satellite observation data exhibits non-independent identical distribution (non-IID), it can lead to severe model parameter drift and a sharp decline in prediction accuracy.

[0005] In summary, existing space-ground collaborative distributed learning methods fail to effectively mitigate the energy consumption disaster caused by traditional deep learning models at the underlying level, and also fail to address the model convergence bias caused by heterogeneous satellite computing power and differences in local training step sizes at the algorithm scheduling level. How to achieve extremely low-energy local computing in dynamically changing space-ground clusters and eliminate convergence bias through heterogeneous sensing aggregation is a pressing technical challenge that needs to be solved. Summary of the Invention

[0006] To address the issues of excessive computational energy consumption in traditional federated edge learning models in current space-ground collaborative computing networks, and the failure to consider the different local training step sizes caused by the heterogeneity of satellite equipment computing power, which leads to "target inconsistency" in the global model, this invention aims to provide a spaceborne heterogeneous perception federated learning method based on spiking neural networks. This method can significantly reduce the computational energy consumption of spaceborne edge nodes, effectively eliminate model convergence deviations caused by heterogeneous equipment computing power, and improve training efficiency and model prediction accuracy in dynamic network environments.

[0007] The technical solution to achieve the purpose of this invention is a spaceborne heterogeneous sensing federated learning method based on spiking neural networks, comprising the following steps:

[0008] S1. Construction of Space-Ground Collaborative Computing Network Architecture and System Model

[0009] The system nodes participating in federated learning are divided into ground parameter servers that act as global coordinators and low-Earth orbit satellite nodes distributed across multiple orbital planes, constructing a multi-level space-ground collaborative computing network architecture; the global spiking neural network model is initialized through the ground parameter servers, and the global spiking neural network model is distributed to satellite nodes in each orbital plane using space-ground links;

[0010] S2. Introduce a spiking neural network to perform dynamic local training for heterogeneous sensing.

[0011] After acquiring local remote sensing data, the satellite nodes convert the local continuous data into discrete pulse sequences. To adapt to the extreme computational energy consumption limitations of the spaceborne environment, this method does not deploy traditional artificial neural networks on the satellite end, but instead introduces a pre-set existing spiking neural network (SNN) model as the underlying local computing engine. During local training, each satellite node evaluates its current available computing power and energy consumption status. Due to the heterogeneity of hardware capabilities of different satellite devices, each satellite node adaptively performs dynamic variable step size spiking neural network local training within the same communication round for the received global model, and obtains its own local updated model.

[0012] S3. Construction and Execution of Intra-orbit Heterogeneous Sensing Normalization Aggregation Model

[0013] Within each orbital plane, the satellite nodes communicate locally via inter-satellite links. To address the "target inconsistency" issue in federated learning synchronous training, where high-performance nodes dominate the global optimization direction due to differing actual training step sizes, this step, during model aggregation, first calculates the parameter difference between the local updated model of each satellite node and the initial model. Then, it divides the parameter difference by the actual local training step size executed by the satellite node to calculate the single-step normalized model update amount. Next, it calculates the effective orbital step size of the current orbit by combining the data weights of each satellite node. Finally, it uses the single-step normalized model update amount and the effective orbital step size to perform heterogeneous sensing-based intra-orbit model weighted aggregation, generating an orbital-level model.

[0014] S4, Execution of Global Aggregation Across Satellite and Ground Orbits

[0015] After a preset number of orbital inter-satellite link aggregation iterations, each orbital plane uploads the orbital-level model generated in step 3 to the ground parameter server of the ground station through the satellite-to-ground link within the available time window. The ground parameter server collects each orbital model and performs cross-orbit global aggregation according to the overall data volume weight of each orbit to generate a new global model. The new global model is then distributed to each satellite node to enter the next round of global iteration training.

[0016] Furthermore, the satellite-ground collaborative computing network architecture in step S1 is constructed using a hierarchical network topology, specifically including the following steps:

[0017] Divide the space segment into There are orbital planes, denoted as . ;No. Each orbital plane contains A low-orbit satellite node, represented as Satellites in the same orbit It establishes inter-satellite links only with its clockwise and counterclockwise adjacent satellites, forming a ring communication topology; the ground segment includes ground parameter servers and a collection of ground stations. The satellite is connected to the ground station within its line of sight via a satellite-to-ground link.

[0018] Furthermore, step S2 employs a leakage current integral-triggered LIF neuron model and a smoothed gradient substitution algorithm with a random mask to construct the system's underlying local computing engine. The operation and parameter updates of the computing engine specifically include the following steps:

[0019] S21. Construct the physical process of forward propagation pulse firing based on LIF neurons:

[0020] For the first spiking neural network Layer neurons, at time step The dynamic physical renewal process of the membrane potential is defined by the following equation (c):

[0021] (c)

[0022] in, This represents the membrane potential of the neuron at the current time step. This is the synaptic connection weight matrix. The binary input pulse sequence received from the previous layer. The membrane potential time leakage attenuation factor, The soft reset threshold after a pulse is delivered to a neuron;

[0023] When membrane potential Exceeding the threshold At that time, the neuron uses the Heaviside step function

[0024] Trigger and fire pulses, and their pulse firing state Defined by the following formula (d):

[0025] (d)

[0026] S22. Employ a backpropagation training process based on smooth alternative gradients with random masks:

[0027] Since the Heaviside step function in the forward propagation is not differentiable, a random noise mask following a Bernoulli distribution is introduced to ensure that the gradient is calculated during the backpropagation. It satisfies:

[0028]

[0029] in, This represents the probability that the mask value is 1;

[0030] At the same time, an alternative function is introduced to control the smoothness of the step function. ,in Represents the smoothing coefficient. The composite activation function for backpropagation is constructed as shown in equation (e):

[0031] (e)

[0032] in, This represents element-wise multiplication. To stop the gradient operator, after completing the aforementioned alternative gradient construction, each satellite node performs stochastic gradient descent training based on local data. During local model updates, the gradient of the loss function with respect to the model parameters is calculated using the alternative activation function, and the parameters are updated via backpropagation. Each satellite node executes the process independently based on its own energy budget. Update local parameters step by step.

[0033] To ensure stable convergence of the federated learning model in scenarios with highly heterogeneous devices and non-independent, identically distributed (non-IID) data, a constrained adaptive decay learning rate is used during gradient descent training. The learning rate Subject to dynamic constraints on the system's global structural constants, the total number of elements in the entire network is set to... For each satellite, the attenuation strategy is as shown in equation (j):

[0034] (j)

[0035] in, Let be the smoothing constant of the target loss function. This represents the total number of communication rounds for global cross-track aggregation. The number of iterations for intra-orbit aggregation using inter-satellite links within a single round of global communication. This is the average unweighted local update step size for all satellites in the network.

[0036] Furthermore, the orbital heterogeneous sensing normalization aggregation mathematical calculation process in step S3 includes the following steps:

[0037] definition For global cross-track communication rounds, For local iterative rounds of intra-orbit inter-satellite link communication; in the... At the start of the round, the track Satellite nodes within Received the current initial orbital-level model ;

[0038] S31. Obtain the actual local training step size of each satellite node due to differences in hardware computing power. Satellite Node Independent execution After training the local spiking neural network, the model parameters after local updates are obtained. ;

[0039] S32. Calculate the update amount of the single-step normalization model. When the models converge via inter-satellite links, the local model update difference for each satellite is calculated. The model parameter update difference is then divided by the corresponding actual training step size to obtain the normalized model update vector, as shown in equation (f).

[0040] (f);

[0041] S33. Calculate the effective orbital step length : Calculate the data volume weight within orbit based on the number of local dataset samples possessed by each satellite node. The actual training step size of each satellite node is weighted and summed using this weight, as shown in equation (g):

[0042] (g)

[0043] S34. Update and generate a new orbit-level model: Using the effective orbit step size, data volume weights of each node, and single-step normalized model update amount calculated above, calculate and generate the orbit-level model required for the next local iteration round. As shown in equation (h):

[0044] (h)

[0045] Furthermore, step S4 specifically includes:

[0046] The system defines a global optimization objective function. Local objective function for all orbits Weighted summation; continuously completed using inter-satellite links within each orbital plane. Next After local normalization and iterative aggregation, the final orbital-level model of each orbital plane is obtained. ;

[0047] Each orbital plane will connect the final orbital-level model via a satellite-to-ground link currently within its visible time window. The data is transmitted to the ground station, where the ground parameter server performs global model aggregation and update based on the overall data volume weight ratio of each orbital plane, as shown in equation (i):

[0048] (i)

[0049] in, For the first Global model parameters generated by round-robin global communication Indicates the first The total data volume weight ratio of each orbital plane; after aggregation, the ground parameter server will use the new global model. Broadcast to all satellite nodes in each orbital plane via satellite-to-ground links.

[0050] Furthermore, after step S4 is completed, an energy consumption assessment model based on a spiking neural network is introduced to evaluate the inference energy consumption performance of the global model in a spaceborne environment, and the energy consumption. Defined by the following formula:

[0051]

[0052] in, This represents the total number of synaptic operations in the network layer. This represents the firing rate of neurons in a spiking neural network. Indicates the total time step of network execution. This represents the energy consumption constant for a single 32-bit accumulation operation; the energy consumption evaluation model is used to measure the inference energy consumption characteristics of the method of the present invention in a spaceborne edge computing environment.

[0053] Beneficial effects

[0054] Compared with existing technologies, this invention introduces an event-driven SNN as the underlying computing engine, transforming the dense floating-point multiply-accumulate operations in traditional deep learning models into simple accumulation operations of discrete pulses. This not only significantly reduces the energy consumption of a single computation operation but also leverages the extreme sparsity of neuron pulse firing, effectively extending the battery life of onboard equipment and meeting the long-term on-orbit learning requirements in extremely low-power scenarios. Furthermore, addressing the issue of inconsistent local training step sizes caused by hardware differences in large-scale satellite constellations, this invention proposes a heterogeneous sensing scheduling mechanism based on normalized aggregation. By dividing the local model update amount by its actual execution step size, it rigorously eliminates the misleading influence of "computing power speed" on "gradient direction" at the mathematical aggregation level. This effectively solves the convergence bias problem in traditional federated learning where powerful nodes dominate global optimization, significantly improving model convergence speed and prediction accuracy in extremely heterogeneous environments. Furthermore, this invention adopts a decoupled architecture of "high-frequency intra-orbit aggregation + low-frequency cross-orbit global aggregation", which makes full use of the advantages of low latency and large bandwidth of inter-satellite links in the same orbital plane to perform high-frequency local parameter synchronization. This significantly reduces the communication frequency and data transmission volume of expensive satellite-to-ground links with time window limitations, and greatly alleviates the communication bottleneck problem in satellite-to-ground collaborative computing networks. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of a satellite-ground collaborative computing network architecture model according to an embodiment of the present invention;

[0057] Figure 2 This is a schematic diagram illustrating the forward and backward propagation mechanism of a spiking neural network according to an embodiment of the present invention.

[0058] Figure 3 This is a flowchart illustrating the overall workflow of a spaceborne heterogeneous sensing federated learning method based on a spiking neural network, according to an embodiment of the present invention.

[0059] Figure 4 This is a comparison chart of the test accuracy of different algorithms under the device-level heterogeneity setting in an embodiment of the present invention;

[0060] Figure 5 This is a comparison chart of the training loss of different algorithms under the device-level heterogeneity setting in an embodiment of the present invention. Detailed Implementation

[0062] The present invention will be further described and illustrated below with reference to the accompanying drawings and embodiments. It should be noted that these embodiments are preferred implementations provided under the premise of solving specific technical problems using the present invention. They detail the operation process and specific parameters of the present invention, but the scope of protection of the present invention is not limited to the following embodiments.

[0063] A spaceborne heterogeneous sensing federated learning method based on spiking neural networks includes the following steps:

[0064] S1, Workflow and Yarn Cluster Resource Modeling

[0065] S1-1, Network Topology Construction: According to the present invention, the system nodes participating in federated learning include a ground parameter server as the global coordinator, and multiple low-Earth orbit satellite nodes distributed in the space segment; the space segment is divided into The orbital plane, the first Each orbital plane contains A low-orbit satellite node.

[0066] Satellites in the same orbital plane establish inter-satellite links (ISLs) only with their clockwise and counterclockwise adjacent satellites, forming a ring communication topology. Each satellite connects to the ground station via a satellite-to-ground link (GSL) within its line of sight, enabling satellite-to-ground communication.

[0067] S1-2, Energy Consumption Assessment Model Establishment: To verify the energy consumption advantages of this invention on spaceborne equipment, a basic physical energy consumption model was established. The energy consumption of traditional artificial neural networks mainly depends on the number of 32-bit multiply-accumulate operations with dense continuous values. However, the spiking neural network introduced in this invention transforms forward propagation computation into a simple 32-bit accumulation operation through an event-driven mechanism that processes discrete binary pulses.

[0068] Its energy consumption is related to the total number of operations in the network layer, the neuron's pulse firing rate, and the total time step. Since the energy consumption of a single AC operation is much smaller than that of a MAC operation, and the pulse firing rate has extremely strong sparsity, this forms the physical theoretical basis for the low-power operation of this invention.

[0069] S2. Introduce an existing spiking neural network model to perform dynamic local training for heterogeneous sensing.

[0070] S2-1, SNN Forward Propagation Model Construction: After acquiring local remote sensing data and converting it into discrete pulse sequences, each satellite node, to adapt to extreme energy consumption limitations, does not deploy a traditional ANN at the satellite end. Instead, it introduces an existing Leakage Integral Triggered (LIF) neuron model to construct the underlying local computing engine. The physical process of dynamically updating the neuron's membrane potential includes the integration of the synaptic connection weight matrix, time leakage controlled by the decay factor, and a soft reset process after pulse firing. When the membrane potential exceeds a set threshold, the neuron triggers and fires a binary pulse through a Heaviside step function.

[0071] S2-2, Backpropagation Based on Alternate Gradients: To address the non-differentiability of the Heaviside step function in the forward propagation described above, a random noise mask following a Bernoulli distribution and a smooth alternative function are introduced to construct a composite activation function. During backpropagation gradient calculation, this smooth alternative function is used for local stochastic gradient descent training.

[0072] S2-3, Heterogeneous Sensing Dynamic Local Training: During local training, each satellite node assesses its current available computing power and energy budget. Due to the hardware heterogeneity of different batches of satellites, each node independently and adaptively performs dynamically variable step-size SNN local parameter updates within the same communication round for the received initial global model, obtaining its own locally updated model.

[0073] S3. Construction and Execution of Intra-orbit Heterogeneous Sensing Normalization Aggregation Model

[0074] Within each orbital plane, satellite nodes communicate locally via ISL. To address the issue of high-computing-power nodes dominating the global optimization direction due to varying local training step sizes—resulting in inconsistent objectives—heterogeneous sensing normalization aggregation is performed, specifically including:

[0075] S3-1. Calculate the single-step normalized model update amount: First, calculate the difference between the local updated model parameters of each satellite node and the initial model parameters. Then, strictly divide the parameter difference by the actual local SGD training step size executed by the satellite node to calculate the single-step normalized model update amount that is not limited by the node hardware execution speed.

[0076] S3-2. Calculate the effective orbit step size: Calculate the data volume weight of each satellite node in the orbit based on the number of local dataset samples it possesses. Use this weight to perform a weighted sum of the actual training step sizes of each satellite node to calculate the effective orbit step size that represents the overall update capability of the current orbit.

[0077] S3-3. Generate orbit-level model: Utilize the single-step normalized model update amount, effective orbit step size, and data volume weights of each node to perform heterogeneous sensing intra-orbit model weighted aggregation, eliminate computing power bias, and generate an orbit-level model for the next local iteration round.

[0078] S4. Execution and Convergence Boundary Constraints of Global Aggregation Between Satellites and Ground

[0079] S4-1 Cross-orbit Global Aggregation: The system's global optimization objective function is defined as a weighted sum of the local objective functions of all orbits. After performing a preset number of local normalization iterations and aggregations within each orbital plane using ISL, each orbital plane uploads its latest orbital-level model to the ground station server via GSL within the available time window. The ground parameter server performs cross-orbit global weighted aggregation based on the overall data volume weight ratio of each orbit, generating a new global model, and distributes it to each satellite node to initiate the next round of global iteration.

[0080] S4-2. Convergence Boundary Constraints: To ensure stable model convergence in scenarios with highly heterogeneous equipment and non-independent, identically distributed data, the system sets a constrained adaptive decay learning rate during SGD training. This learning rate is dynamically constrained by the smoothing constant of the target loss function, the number of global inter-orbit communication rounds, the number of intra-orbit aggregation iterations, the total number of satellites in the network, and the average unweighted local update step size. This adaptive strategy eliminates model parameter drift caused by multiple local updates, ensures the convergence of the expected upper bound of the global gradient norm, and theoretically achieves near-linear acceleration.

[0081] Example 1

[0082] See Figure 1 This embodiment discloses a spaceborne heterogeneous sensing federated learning method based on spiking neural networks. This method primarily addresses the inconsistency in federated learning objectives caused by differences in SGD training step sizes due to the heterogeneous computing hardware capabilities of satellite nodes in dynamic SG-CPN scenarios. Furthermore, to overcome the extreme energy consumption bottleneck of spaceborne equipment, this method introduces a low-power SNN as the underlying computing engine. (See reference...) Figure 3 The specific workflow of this embodiment consists of the following four main steps:

[0083] S1. Construction of Space-Ground Collaborative Computing Network Architecture and System Energy Consumption Physical Model

[0084] 11) Construction of the satellite-ground collaborative computing network architecture topology: such as Figure 1 As shown, the system nodes participating in federated learning are divided into a ground parameter server (acting as the global coordinator) and multiple low-Earth orbit satellite nodes distributed across the space segment. The space segment is defined as follows: There are orbital planes, and the set of orbital planes represents... ;No. Each orbital plane contains There are 10 low-Earth orbit satellite nodes, and the set of satellite nodes is represented as... To reduce reliance on terrestrial communication bandwidth, satellites in the same orbit Inter-satellite links (ISLs) are established only with its clockwise and counterclockwise adjacent satellites, forming a high-speed ring communication topology. The ground segment includes ground parameter servers and a collection of ground stations. The satellite establishes intermittent connections with ground stations via a satellite-to-ground link (GSL) only when it is within the visible time window, thus enabling satellite-to-ground communication.

[0085] 12) Establishment of physical evaluation model for energy consumption of spaceborne equipment: In order to verify the energy consumption advantage of the present invention on spaceborne equipment and to provide a physical basis for the subsequent introduction of low-power computing engine, this embodiment establishes a basic physical energy consumption model based on 45nm CMOS process parameters.

[0086] In tasks such as spaceborne image processing, the energy consumption of traditional artificial neural networks mainly depends on the number of 32-bit multiply-accumulate operations with dense consecutive values, and its total system energy consumption... Defined by the following formula:

[0087] (a)

[0088] To address this high energy consumption issue, this embodiment introduces a spiking neural network (SNN) as an alternative. SNN, through its event-driven mechanism that processes discrete binary pulses, transforms the dense multiplication calculations in the forward propagation into simple 32-bit accumulation operations at the hardware computation level. Its energy consumption... Defined by the following formula:

[0089] (b)

[0090] in, Represents the total number of synaptic operations in a convolutional or fully connected layer of a neural network; the energy consumption constant for a single 32-bit multiply-accumulate operation in 45nm CMOS hardware technology. pJ, while the energy consumption constant for a single 32-bit accumulation operation pJ; This represents the spiking rate and activation frequency of neurons in an SNN. This represents the total time step of network inference.

[0091] In practical remote sensing observation tasks, due to the high sparsity of the SNN event-driven mechanism, most neurons are in a resting state, making... By comparing equation (a) and equation (b), it can be seen that under the same conditions... In this embodiment, the 3.2 pJ MAC operation is completely avoided by adopting an accumulation mechanism, and only a small amount of 0.1 pJ AC operation is generated.

[0092] The physical model described above defines the theoretical basis for how this method can significantly reduce the energy consumption of satellite edge computing nodes compared to traditional deep learning frameworks.

[0093] S2. Introduce an existing spiking neural network model to perform dynamic local training for heterogeneous sensing.

[0094] See Figure 2 After acquiring local remote sensing observation data, each satellite node first converts continuous pixel or feature data into discrete binary pulse sequences. To adapt to the extreme energy consumption constraints defined in step S1 above, this embodiment does not deploy a traditional ANN at the satellite end. Instead, it introduces the existing LIF neuron model and a smooth alternative gradient algorithm with random masks to construct the system's underlying local computing engine to perform local SGD parameter updates. The specific operating mechanism of this underlying engine includes:

[0095] 21) Physical process of forward propagation pulse firing based on LIF neurons: For the first SNN deployed in a spaceborne computing unit... Layer neurons, in discrete time steps The dynamic physical update process of its membrane potential is defined by the following equation:

[0096] (c)

[0097] in, For the neuron at the current time step The membrane potential state; This represents the synaptic connection weight matrix to be optimized in this round of federated learning. The binary input pulse sequence received from the previous network layer; This is the membrane potential time leakage attenuation factor, used to control the ability to retain historical information.

[0098] It is a time constant; The soft reset threshold after a pulse is delivered to a neuron.

[0099] When membrane potential The integral accumulates over time and exceeds a set threshold. At this time, the neuron is activated via the Heaviside step function. Triggers and fires a binary pulse. Its pulse firing state...

[0100] Defined by the following formula, an output of 1 indicates the firing of a pulse, and an output of 0 indicates the remaining still:

[0101] (d)

[0102] A smooth alternative gradient backpropagation training process with random masks is employed: In federated edge learning, each node must update its local parameters by calculating the gradient of the loss function. However, due to the Heaviside step function in equation d... Since the derivative is almost zero everywhere and it is not differentiable at the threshold, the traditional backpropagation algorithm cannot be directly applied to SNN.

[0103] To enable the model to perform stochastic gradient descent (SGD) training smoothly in a spaceborne environment, this embodiment introduces a random noise mask following a Bernoulli distribution when calculating the gradient during backpropagation. ,in Hyperparameters for controlling the noise injection probability, and a smoothing substitution function for controlling the steepness of the step curve. The independent variable By combining the two, this embodiment constructs a composite activation function for backpropagation:

[0104] (e)

[0105] in, This represents element-wise multiplication. To stop the gradient operator, equation (e) is used. During forward propagation, the discrete binary properties are maintained, while during backward propagation, the gradient smoothly passes through...

[0106] It is passed to the shallow network.

[0107] During specific local model updates, the system uses this substitution function to calculate the classification loss and perform gradient descent. Due to differences in equipment batches and aging levels within a large satellite constellation, the system exhibits severe hardware heterogeneity. Each satellite node adaptively and independently performs local SGD training based on its current processor performance and energy budget. This results in different satellites performing different local SGD training cycles within the same local iteration of federated learning. Actual local training step size They are all different. This physical difference at the hardware level is the root cause of the "goal inconsistency" in the global model of federated learning.

[0108] S3. Construction and Execution of Intra-orbit Heterogeneous Sensing Normalized Aggregation Model (SpiFedHA)

[0109] In the traditional synchronous FEEL framework, it is assumed that all nodes perform the same local training step size within the same local communication round, i.e., a homogeneous environment is required. However, as described in step S2, the actual local training step size of each satellite node in SG-CPN varies. They differ. If existing federated averaging algorithms, such as FedAvg or conventional inter-satellite cascading algorithms, are directly adopted, the cumulative amount of parameter updates for powerful nodes with larger execution steps will be greater, thus implicitly dominating the optimization direction of the global model in terms of mathematical expectation. This phenomenon is called "objective inconsistency".

[0110] To completely eliminate the misleading effect of differences in computing power and speed on gradient direction, in this embodiment, satellite nodes in each orbital plane perform high-frequency local communication via ISL to execute the intra-orbit heterogeneous sensing normalization aggregation SpiFedHA algorithm. The specific logical process of its mathematical calculations and model updates includes the following steps:

[0111] definition For global cross-track communication rounds, This refers to the local iterative rounds of ISL communication within the orbit. In the... At the start of the round, the track All satellite nodes within The current initial orbital-level model is received via ISL broadcast, denoted as... .

[0112] 31) Obtaining the actual local training step size and model update for each satellite node: Due to hardware computing power constraints, different satellites can complete different numbers of SGD iterations within the same time window. The system first obtains the actual local training step size and model update for each satellite node. The actual local training step size due to differences in computing power .

[0113] satellite nodes by Starting from this point, using the alternative gradient SNN engine constructed in S2 above, execute independently... After local backpropagation training, the model parameters after local update are obtained, denoted as: .

[0114] 32) Calculate the update amount of the single-step normalized model. When the local iteration ends and the model is converged via ISL, in order to eliminate the cumulative impact of heterogeneous node computing power, this embodiment does not directly address the issue. Instead of using a weighted average, the system first calculates the total difference in local model updates for each satellite and then strictly divides it by the actual training step size of that satellite. This yields the normalized single-step update vector. The normalization operation is defined as follows:

[0115] (f)

[0116] To further clarify the technical problem solved by formula f, an example is given to illustrate its technical effect:

[0117] Assuming they are on the same orbit There are two satellites observing data of the same quality: Satellite A, equipped with a high-performance FPGA, and Satellite B, in low-power mode. In the same round... Inside, satellite A has extremely strong computing power and performed 9 steps of SGD update, namely... Its model parameters shifted significantly towards the local optimum; while Satellite B, with its weaker computing power, only performed one SGD update, i.e. Its model parameters have only shifted by a tiny distance.

[0118] If we do not use formula f for division normalization and directly add the update amounts of the two or only weight them according to the amount of data, the cumulative gradient of satellite A will be 9 times that of satellite B. This will cause the aggregated orbital model to be "severely biased" by the local optimum of satellite A, which completely violates the original intention of federated learning to optimize based on global data distribution.

[0119] By dividing the formula f of this invention by their respective step sizes, the total update amount of satellite A is divided by 9, and the update amount of satellite B is divided by 1. Therefore, regardless of how many update steps the satellite hardware performs, the update amount converging towards the center is... All values ​​are rigorously normalized to the "mathematical expectation of a single-step gradient". This mechanism completely eliminates the systematic bias caused by the computing speed of onboard hardware on model aggregation from a mathematical perspective.

[0120] 33) Calculate the effective orbital step size After eliminating heterogeneity bias, to ensure that the convergence step size of the model update is not affected, the system needs to calculate the data weight in orbit for each satellite node based on the number of local dataset samples it possesses. Subsequently, the actual training step lengths of each satellite node are weighted and summed using this data volume weight to calculate the effective orbital step length, which represents the overall computing power level of the orbit. The definition is as follows:

[0121] (g)

[0122] The system then updates and generates a new orbital-level model. Finally, the system utilizes the effective orbital step size calculated in the above steps. Data volume weights of each node and the update amount of the single-step normalized model after removing heterogeneous biases. Unbiased model updates are then performed. The orbital-level model required for the next local iteration is calculated using equation (h) below. :

[0123] (h)

[0124] Thus, this method has completed a high-frequency in-orbit federation parameter synchronization that completely overcomes hardware heterogeneity using ISL.

[0125] S4. Execution and Algorithm Theory of Global Aggregation Between Satellite and Ground Orbits: Convergence Boundary Constraints

[0126] After completing the high-frequency heterogeneous sensing normalization aggregation within orbit as described in S3, to ensure that the globally covered SG-CPN can learn the global data distribution, this embodiment performs cross-orbit global aggregation through low-frequency satellite-to-ground link communication. Simultaneously, to ensure the mathematical stability of the algorithm in extremely heterogeneous and non-IID data scenarios, the learning rate of the model must be adaptively constrained. Specifically, this includes the following steps:

[0127] 41) Global model aggregation and update for satellite-to-ground cross-orbit communication: The system defines the global optimization objective function for the entire satellite-to-ground collaborative network. Local objective function for all orbits The weighted sum. Pre-set parameters are continuously executed within each orbital plane using inter-satellite links. Next, that is After local normalization and iterative aggregation, the system obtains the final orbital-level model for each orbital plane, denoted as... .

[0128] Subsequently, each orbital plane transmits the final orbital-level model via a high-speed satellite-to-ground link currently within its visible time window. The data is then transmitted to the ground station. After collecting all orbital-level models, the ground parameter server performs a cross-orbit global model aggregation and update based on the weighted proportion of the total data volume for each orbital plane, as shown in the following formula:

[0129] (i)

[0130] in, For the first Global model parameters generated by round-robin global communication

[0131] Representing the The weighting ratio of the total data volume of each orbital plane.

[0132] After completing the weighted aggregation, the ground parameter server will use the new global model. The next round of global federated learning is initiated by broadcasting the information uplink to all satellite nodes in each orbital plane via the satellite-to-ground link.

[0133] 42) Dynamic constraints on the theoretical boundaries of algorithm convergence:

[0134] In spaceborne observation scenarios characterized by severe equipment heterogeneity and non-independent, identically distributed data, federated learning models are highly prone to divergence. This embodiment demonstrates through mathematical derivation that the method must use a constrained adaptive decay learning rate when performing local SGD training on each node. The learning rate The attenuation strategy is subject to dynamic constraints on the system's global structural constants.

[0135] Set the total number of networks included For each satellite, the adaptive attenuation strategy set by this invention is as follows:

[0136] (j)

[0137] in, This is the smoothing constant of the target loss function, i.e., the Lipschitz continuity constant. This represents the total number of communication rounds for global cross-track aggregation. The number of iterations for intra-orbit aggregation using inter-satellite links within a single round of global communication. This is the average unweighted local update step size for all satellites in the network across all rounds, which is the arithmetic mean of the actual number of hardware execution steps.

[0138] Under the dynamic constraint control of this adaptive learning rate strategy, the aggregation algorithm of this invention can effectively eliminate the systematic drift of model parameters caused by multiple local heterogeneous updates. Regarding the convergence of the global objective function, the convergence bound of the expected upper limit of the global gradient norm of this invention is strictly theoretically limited to the following order of magnitude:

[0139] (k)

[0140] The above theoretical boundary constraints prove that, under the premise of satisfying the normalized aggregation strategy and the adaptive learning rate of this invention, due to the system drift term... The decay rate is faster, and the overall convergence rate of the model is mainly determined by the dominant term. The decision was made. This conclusion, in mathematical theory, allows for the calculation of the total number of parallel nodes participating in federated learning. The method exhibits near-linear acceleration compared to the overall execution step size characteristic. Even in situations with extremely poor network connectivity across orbital planes and significant differences in satellite node computing power, such as extremely large step size variance, the method of this invention not only avoids arbitrary divergence but also increases the number of high-frequency intra-orbit aggregations. Or increase the average local execution step size Significantly reduces the need for expensive satellite-to-ground links, i.e., global rounds. Communication dependencies.

[0141] Example 2, Verification of the test scenario for the example:

[0142] To further verify the effectiveness and beneficial effects of the spaceborne heterogeneous sensing federated learning method (SpiFedHA) based on spiking neural networks described in this embodiment of the invention in practical applications, this embodiment constructs a spaceborne edge learning simulation experimental environment based on real data.

[0143] 1) Setting up a heterogeneous environment for datasets and constellations:

[0144] The experiment used the EuroSAT remote sensing image classification dataset based on Sentinel-2 satellite imagery, containing 27,000 multispectral images. To simulate the non-independent and identically distributed characteristics of the observed data features, a Dirichlet distribution and heterogeneous parameters were employed. The dataset is then split.

[0145] Regarding the SG-CPN constellation model, the Walker Delta constellation structure (80 / 4 / 1) was constructed, including... There are 1 orbital plane, and each orbital plane is evenly distributed. The satellite is set to operate at an altitude of 750 km.

[0146] 2) Simulation of heterogeneity in computing power at extreme device levels:

[0147] The core objective of this invention is to overcome target deviation caused by inconsistent local training step sizes. Therefore, to simulate real-world batch and hardware aging-induced device heterogeneity, satellites in the same orbit are divided into three categories: low-power satellites, which, due to power limitations, perform only one SGD step per round, i.e. Standard satellites perform 5-step SGD, while high-performance satellites perform 9-step SGD. By adjusting the distribution of these three types of satellites in each orbit, a step size variance index is introduced. This is used to measure the severity of heterogeneity. Specifically, a variance metric is set for extreme heterogeneous environments. Conduct a control experiment.

[0148] 3) Comparison of experimental test results and verification conclusions:

[0149] See Figure 4 and Figure 5 After multiple iterations of global communication, the test accuracy and convergence characteristics of different algorithms are compared and analyzed.

[0150] Experimental results show that in the ideal isomorphic scenario with uniform computing power, i.e., step size variance... The SpiFedHA algorithm of this invention performs exactly the same as traditional federated averaging algorithms, such as the FedMega algorithm improved for satellite scenarios, verifying the mathematically complete degeneracy characteristics of this algorithm.

[0151] However, in setting In extreme scenarios with heterogeneous computing power, traditional baseline algorithms have revealed significant shortcomings. Because the traditional FedMega algorithm cannot separate high-performance nodes during aggregation, and performs 9 steps of SGD, gradient accumulation dominates the bias, causing its test accuracy to be capped and plummet to approximately 69.5%.

[0152] In contrast, the heterogeneous sensing normalization aggregation mechanism (SpiFedHA) in this embodiment of the invention, through the correction operation in step S3 where formula f strictly divides the local update amount by the actual step size, completely eliminates the target inconsistency problem caused by hardware speed. Even under harsh conditions, the algorithm of this invention still maintains excellent robustness, with the test accuracy steadily converging and remaining at around 78.7%, achieving an absolute accuracy improvement of nearly 9.2%.

[0153] Furthermore, regarding energy consumption verification and evaluation, calculations using the physical energy consumption evaluation model in step S1 above show that the SNN architecture with alternative gradients introduced in this embodiment on the onboard computing node achieves over 30 times the single-inference energy efficiency improvement compared to the traditional ResNet-18 and ANN versions performing the same inference task, while maintaining comparable accuracy. For example, the energy consumption of some convolutional layers is reduced from... pJ decreased to pJ level.

[0154] Based on the experimental data above, this invention not only greatly alleviates the energy consumption limitations of expensive spaceborne equipment in space-ground collaborative computing networks, but also completely solves the convergence bias problem of federated learning models in scenarios with extremely heterogeneous equipment and highly dispersed observation data. It has extremely significant technological advancements and broad engineering application prospects.

[0155] The above description is merely a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A spaceborne heterogeneous sensing federated learning method based on spiking neural networks, comprising the following specific steps: S1. Construction of Space-Ground Collaborative Computing Network Architecture and System Model The system nodes participating in federated learning are divided into ground parameter servers that act as global coordinators, and low-Earth orbit satellite nodes distributed across multiple orbital planes, thus constructing a multi-level space-ground collaborative computing network architecture. The global spiking neural network model is initialized through the ground parameter server, and then distributed to satellite nodes in each orbital plane using the satellite-to-ground link. S2. Introduce a spiking neural network to perform dynamic local training for heterogeneous sensing. After acquiring local remote sensing data, the satellite node converts the local continuous data into discrete pulse sequences; and introduces a pre-defined existing spiking neural network (SNN) model as the underlying local computing engine. During local training, the satellite nodes assess their current available computing power and energy consumption status, and adaptively perform local training of the dynamically variable step-size spiking neural network within the same communication round for the received global model, thereby obtaining their respective local updated models. S3. Construction and Execution of Intra-orbit Heterogeneous Sensing Normalization Aggregation Model Within each orbital plane, the satellite nodes communicate locally via inter-satellite links. During model aggregation, the parameter difference between the local updated model of each satellite node and the initial model is first calculated. The parameter difference is then divided by the actual local training step size of the satellite node to obtain the single-step normalized model update amount. Next, the effective orbital step size of the current orbit is calculated by combining the data weights of each satellite node. The single-step normalized model update amount and the effective orbital step size are used to perform heterogeneous sensing intra-orbit model weighted aggregation to generate an orbital-level model. S4, Execution of Global Aggregation Across Satellite and Ground Orbits After a preset number of orbital inter-satellite link aggregation iterations, each orbital plane uploads the orbital-level model generated in step 3 to the ground parameter server of the ground station through the satellite-to-ground link within the available time window. The ground parameter server collects each orbital model and performs cross-orbit global aggregation according to the overall data volume weight of each orbit to generate a new global model. The new global model is then distributed to each satellite node to enter the next round of global iteration training.

2. The spaceborne heterogeneous sensing federated learning method based on spiking neural networks according to claim 1, characterized in that, The satellite-ground collaborative computing network architecture in step S1 is constructed using a hierarchical network topology, specifically including the following steps: Divide the space segment into There are orbital planes, denoted as . ;No. Each orbital plane contains A low-orbit satellite node, represented as Satellites in the same orbit It establishes inter-satellite links only with its clockwise and counterclockwise adjacent satellites, forming a ring communication topology; the ground segment includes ground parameter servers and a collection of ground stations. The satellite is connected to the ground station within its line of sight via a satellite-to-ground link.

3. The spaceborne heterogeneous sensing federated learning method based on spiking neural networks according to claim 1, characterized in that, The underlying local computing engine described in step S2 is constructed using a leakage current integral-triggered LIF neuron model and a smooth substitution gradient algorithm with a random mask. The operation and parameter update of the computing engine specifically include the following steps: S21. Construct the physical process of forward propagation pulse firing based on LIF neurons: For the first spiking neural network Layer neurons, at time step The dynamic physical update process of the membrane potential is defined by the following equation: in, This represents the membrane potential of the neuron at the current time step. This is the synaptic connection weight matrix. The binary input pulse sequence received from the previous layer. The membrane potential time leakage attenuation factor, The soft reset threshold after a pulse is delivered to a neuron; When membrane potential Exceeding the threshold At that time, the neuron uses the Heaviside step function ; Trigger and fire pulses, and determine the pulse firing status. Defined by the following formula: S22. Employ a backpropagation training process based on smooth alternative gradients with random masks: When calculating the gradient during backpropagation, a random noise mask following a Bernoulli distribution is introduced. It satisfies: ； in, This represents the probability that the mask value is 1; At the same time, an alternative function is introduced to control the smoothness of the step function. ,in Represents the smoothing coefficient. The composite activation function for backpropagation is constructed as shown in the following equation: in, This represents element-wise multiplication. To stop the gradient operator, after completing the aforementioned alternative gradient construction, each satellite node performs stochastic gradient descent training based on local data. During local model updates, the gradient of the loss function with respect to the model parameters is calculated using the alternative activation function, and the parameters are updated via backpropagation. Each satellite node executes the process independently based on its own energy budget. Update local parameters step by step.

4. The spaceborne heterogeneous sensing federated learning method based on a spiking neural network according to claim 3, characterized in that, The gradient descent training is performed using a constrained adaptive decay learning rate. ; The learning rate ; Subject to dynamic constraints on the system's global structural constants, the total number of elements in the entire network is set to... The attenuation strategy for the satellite is as follows: in, Let be the smoothing constant of the target loss function. This represents the total number of communication rounds for global cross-track aggregation. The number of iterations for intra-orbit aggregation using inter-satellite links within a single round of global communication. This is the average unweighted local update step size for all satellites in the network.

5. The spaceborne heterogeneous sensing federated learning method based on a spiking neural network according to claim 1, characterized in that, The intra-orbit heterogeneous sensing normalization aggregation mathematical calculation process in step S3 includes the following steps: definition For global cross-track communication rounds, For local iterative rounds of intra-orbit inter-satellite link communication; in the... At the start of the round, the track Satellite nodes within Received the current initial orbital-level model ; S31. Obtain the actual local training step size of each satellite node due to differences in hardware computing power. Satellite Node Independent execution After training the local spiking neural network, the model parameters after local updates are obtained. ; S32. Calculate the update amount of the single-step normalization model. When the models converge via inter-satellite links, the local model update difference for each satellite is calculated. The model parameter update difference is then divided by the corresponding actual training step size to obtain the normalized model update vector, as shown in the following formula: ； S33. Calculate the effective orbital step length : Calculate the data volume weight within orbit based on the number of local dataset samples possessed by each satellite node. The actual training step size of each satellite node is then weighted and summed using this weight, as shown in the following formula: S34. Update and generate a new orbit-level model: Using the effective orbit step size, data volume weights of each node, and single-step normalized model update amount calculated above, calculate and generate the orbit-level model required for the next local iteration round. As shown in the following formula: 。 6. The spaceborne heterogeneous sensing federated learning method based on spiking neural networks according to claim 1, characterized in that, In step S4, cross-orbit global model aggregation is performed via the satellite-to-ground link, specifically including: The system defines a global optimization objective function. Local objective function for all orbits Weighted summation; continuously completed using inter-satellite links within each orbital plane. Next After local normalization and iterative aggregation, the final orbital-level model of each orbital plane is obtained. ; Each orbital plane will connect the final orbital-level model via a satellite-to-ground link currently within its visible time window. The data is transmitted to the ground station, where the ground parameter server performs a global model aggregation and update based on the overall data volume weight ratio for each orbital plane, as shown in the following formula: in, For the first Global model parameters generated by round-robin global communication Indicates the first The total data volume weight ratio of each orbital plane; after aggregation, the ground parameter server will use the new global model. Broadcast to all satellite nodes in each orbital plane via satellite-to-ground links.

7. The spaceborne heterogeneous sensing federated learning method based on spiking neural networks according to claim 1, characterized in that, After step S4 is completed, an energy consumption assessment model based on a spiking neural network is introduced to evaluate the inference energy consumption performance of the global model in a spaceborne environment, and the energy consumption. Defined by the following formula: in, This represents the total number of synaptic operations in the network layer. This represents the firing rate of neurons in a spiking neural network. Indicates the total time step of network execution. This represents the energy consumption constant for a single 32-bit accumulation operation; the energy consumption evaluation model is used to measure the inference energy consumption characteristics of the method of the present invention in a spaceborne edge computing environment.

Images