On-orbit intelligent processing method for space-borne image based on brain-like computing

Abstract

This invention discloses an on-orbit intelligent processing method for spaceborne images based on brain-like computing, belonging to the field of on-orbit intelligent processing of spaceborne images. This invention utilizes a spiking neural network to simulate the structure of the human brain: including a feature extraction layer, a pulse coding layer, an STDP learning layer, a lateral inhibition layer, and a decision output layer; and improves model accuracy through on-orbit updates and federated learning. This invention reduces computational energy consumption and improves efficiency. Simultaneously, this invention leverages the synaptic plasticity of neurons to construct a SNN with excellent spatial and temporal resolution.

Description

Technical Field

[0001] This invention belongs to the field of on-orbit intelligent processing of spaceborne images, specifically involving an on-orbit intelligent processing method for spaceborne images based on brain-like computing, applicable to real-time on-orbit image processing of low-Earth orbit satellites. Background Technology

[0002] With the rapid development of low-Earth orbit satellite internet, SpaceX's Starlink project has deployed more than 7,000 satellites in orbit, and my country's StarNet project will also launch more than 12,000 satellites. This makes the demand for on-orbit intelligent processing of satellite images increasingly urgent. In disaster monitoring, it is necessary to identify disasters such as fires and floods in real time, and in target monitoring, it is necessary to quickly locate moving targets such as ships and aircraft. The traditional mode of relying on data to be transmitted back to the ground for processing can no longer meet the timeliness requirements.

[0003] Currently, spaceborne image processing primarily relies on data transmission back to the ground for processing. However, the bandwidth of space-to-ground communication is dynamically limited (often below 10Mbps), and full image transmission results in latency of several seconds, far exceeding the real-time requirements of scenarios such as disaster response. On-orbit processing of spaceborne images mainly depends on traditional deep neural networks, which consumes a lot of power, and the power supply capacity of satellite payloads is limited. Furthermore, the intense radiation environment of space can easily cause logic errors in traditional chips, affecting processing stability.

[0004] Neuromorphic computing, drawing inspiration from the neural mechanisms of the human brain, possesses characteristics such as high parallelism, low power consumption, and strong anti-interference capabilities. Spiking Neural Networks (SNNs), as an implementation of neuromorphic computing, offer advantages such as strong computational sparsity and low energy consumption, making them particularly suitable for spaceborne, constrained environments. However, existing SNNs are prone to performance degradation in deep networks, limiting their application in complex remote sensing tasks. Summary of the Invention

[0005] In view of this, this invention proposes an on-orbit intelligent processing method for spaceborne images based on neuromorphic computing. By introducing the Spike-Timing-Dependent Plasticity (STDP) mechanism and an adaptive network structure, the accuracy of target detection and classification is improved, while significantly reducing computational energy consumption. This solves the problems of high latency, high power consumption, and poor adaptability in existing spaceborne image processing methods.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A brain-inspired computing-based on-orbit intelligent processing method for spaceborne images includes the following steps:

[0008] Step 1: Use publicly available remote sensing datasets as training sets for model training and validation, including the EuroSAT dataset for ground object classification tasks and the satellite object detection dataset for object detection tasks.

[0009] Step 2: Construct an experimental platform simulating a spaceborne environment. This platform is based on the Intel Loihi2 neuromorphic chip and will subsequently be used to build a DSNN-based network.

[0010] Step 3, construct the feature extraction layer: use the difference of Gaussian filter to extract edge and contour features to obtain the feature map;

[0011] Step 4, Construct the pulse coding layer: Encode the feature map into a pulse sequence, converting continuous feature values ​​into discrete pulse sequences;

[0012] Step 5, Construct the STDP learning layer: Introduce an STDP-based SNN layer to achieve adaptive adjustment of synaptic weights, enabling the network to effectively learn and extract complex local feature patterns and determine the target region in the feature map;

[0013] Step 6, constructing a transverse inhibition layer: suppressing background noise through neuronal competition mechanism; suppressing background noise through inter-neuron competition mechanism: when the target region neuron fires a pulse, the membrane potential of the surrounding background neurons is suppressed: that is, the threshold is reduced by 30%, reducing false detection; adopting a "pulse counting + time window" decision mechanism: counting the total number of pulses of the target type neurons within a 100ms window, if it exceeds the threshold, it is determined to be the target, and the decision delay is controlled within 20ms. The surrounding background neurons are the neurons in the feature map other than the target region neurons;

[0014] Step 7, Construct the decision output layer: The output layer is a fully connected spiking layer that learns higher-level features and makes decisions;

[0015] Step 8: Based on the DSNN-based network obtained in Steps 3 to 7, input the training set into the constructed DSNN-based network for training. At the same time, use the publicly available pre-trained model to obtain the learned neural network. Then, optimize and tune the parameters on the validation set to obtain the optimal model and its corresponding parameters, and obtain the trained DSNN-based network.

[0016] Step 9: In practical applications, the satellite-orbiting end uses a trained DSNN-based network to process the satellite-borne images. Then, the satellite-borne end's on-orbit processing results are encrypted and uploaded to the ground. The ground updates the global model using a federated averaging algorithm and then sends the optimized difference parameters back to the satellite-borne end. The satellite-borne end uses the new parameters to fine-tune the synaptic weights and performs iterative optimization, achieving intelligent on-orbit processing of satellite-borne images based on brain-like computing.

[0017] Furthermore, step 4 is specifically implemented as follows:

[0018] A time-based encoding method is used to linearly map pixel values ​​to an initial membrane potential, which is then input to the neuron. When a certain threshold is exceeded, a pulse sequence is output at that moment.

[0019]

[0020]

[0021] in, : Membrane potential at time t; : Resting membrane potential, normalized to 0; The membrane time constant determines the rate of change of membrane potential, and its value ranges from 10 ms to 30 ms; when the membrane potential exceeds the threshold... At that time, the neuron fires a pulse and resets to X(t) is the instantaneous rate of change of the membrane potential.

[0022] Due to the adoption of the above technical solution, the beneficial effects of this invention compared with the prior art are as follows:

[0023] This invention differs from CNN-based on-orbit intelligent processing and proposes an on-orbit intelligent processing method based on DSNN suitable for spaceborne environments, which has significant advantages in low power consumption and real-time performance. It introduces STDP and lateral suppression mechanisms to improve adaptability to dynamic remote sensing scenarios. It supports on-orbit federated learning to achieve continuous model optimization. Compared with traditional GPU-based methods, this invention reduces energy consumption by about 50 times and improves computing efficiency by about 5 times. Attached Figure Description

[0024] Figure 1 This is an overall flowchart of the on-orbit intelligent processing method for spaceborne images based on neuromorphic computing in this embodiment of the invention. Detailed Implementation

[0025] The invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0026] Brain-inspired computing-based on-orbit intelligent processing methods for spaceborne images, such as Figure 1 As shown, it includes the following steps:

[0027] Step 1: Publicly available remote sensing datasets are used as training sets for model training and validation, including the EuroSAT dataset for land cover classification and a satellite object detection dataset for object detection. To evaluate the effectiveness of this invention, traditional deep learning-based methods are compared with it. The experiment first divides the dataset into three parts in an 8:1:1 ratio: training, validation, and testing. The trained model is then used to validate the effectiveness of this invention.

[0028] Step 2 involves constructing an experimental platform simulating the spaceborne environment. This platform utilizes an Intel Loihi2 neuromorphic chip-based spaceborne computing platform, integrating 1024 neuromorphic cores to support parallel computing and event-driven computing with 1.28 million virtual neurons. Power consumption is ≤15W, and radiation resistance meets LEO orbit requirements (total dose ≥100 krad). The satellite payload simulator uses the SpaceFibre interface to simulate the space-to-ground communication link, supporting dynamic bandwidth adjustment (1Mbps-100Mbps) and introducing Gaussian white noise to simulate space communication interference. A DSNN-based network will subsequently be built on this platform.

[0029] Step 3, construct the feature extraction layer: use the difference of Gaussian filter to extract edge and contour features to obtain feature maps; use the DOG filter bank and combine it with the STDP learning rule to dynamically adjust the synaptic weights.

[0030] Step 4, Construct the pulse coding layer: Encode the feature map into a pulse sequence, converting continuous feature values ​​into discrete pulse sequences;

[0031] Step 5, Construct the STDP learning layer: Introduce an STDP-based SNN layer to achieve adaptive adjustment of synaptic weights, enabling the network to effectively learn and extract complex local feature patterns and determine the target region in the feature map;

[0032] Step 6, constructing a transverse inhibition layer: suppressing background noise through neuronal competition mechanism; suppressing background noise (such as cloud reflection, sea surface clutter) through inter-neuron competition mechanism: when the target region neuron fires a pulse, the membrane potential of the surrounding background neurons is suppressed: that is, the threshold is reduced by 30%, reducing false detection; adopting a "pulse counting + time window" decision mechanism: counting the total number of pulses of the target type neurons within a 100ms window, if it exceeds the threshold, it is determined to be a target (e.g., if the number of pulses of ship type neurons is ≥50, an alarm is triggered), and the decision delay is controlled within 20ms. The surrounding background neurons are the neurons in the feature map other than the target region neurons;

[0033] Step 7, Construct the decision output layer: The output layer is a fully connected spiking layer that learns higher-level features and makes decisions;

[0034] Step 8: Based on the DSNN-based network obtained in Steps 3 to 7, input the training set into the constructed DSNN-based network for training. At the same time, use the publicly available pre-trained model to obtain the learned neural network. Then, optimize and tune the parameters on the validation set to obtain the optimal model and its corresponding parameters, and obtain the trained DSNN-based network.

[0035] Step 9: In practical applications, the satellite-orbiting end uses a trained DSNN-based network to process the satellite-borne images. Then, the satellite-borne end's on-orbit processing results are encrypted and uploaded to the ground. The ground updates the global model using a federated averaging algorithm and then sends the optimized difference parameters back to the satellite-borne end. The satellite-borne end uses the new parameters to fine-tune the synaptic weights and performs iterative optimization, achieving intelligent on-orbit processing of satellite-borne images based on brain-like computing.

[0036] Furthermore, step 4 is specifically implemented as follows:

[0037] A time-based encoding method is used to linearly map pixel values ​​to an initial membrane potential, which is then input to the neuron. When a certain threshold is exceeded, a pulse sequence is output at that moment.

[0038] The neuron model combines an integral process represented by differential equations with a firing process represented by conditional judgments, as shown in the first formula for membrane potential changes. In practical applications, discrete difference equations are used to approximate continuous differential equations, and the charging process is shown in the second formula.

[0039]

[0040]

[0041] in, : Membrane potential at time t; : Resting membrane potential, normalized to 0; The membrane time constant determines the rate of change of membrane potential, and its value ranges from 10 ms to 30 ms; when the membrane potential exceeds the threshold... At that time, the neuron fires a pulse and resets to X(t) is the instantaneous rate of change of the membrane potential.

[0042] In summary, the main advantages of this invention lie in the fact that the neuromorphic computing-based method consumes 50 times less energy and has 5 times the computing efficiency of traditional GPUs compared to traditional methods. The neuromorphic computing-based method is novel and lays the foundation for subsequent research and practical applications.

[0043] Those skilled in the art will recognize that the described embodiments are intended to help readers understand the principles of the invention and should be understood as not limiting the scope of protection of the invention to the described embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. A method for on-orbit intelligent processing of spaceborne images based on neuromorphic computing, characterized in that, Includes the following steps: Step 1: Use publicly available remote sensing datasets as training sets for model training and validation, including the EuroSAT dataset for ground object classification tasks and the satellite object detection dataset for object detection tasks. Step 2: Construct an experimental platform simulating a spaceborne environment. This platform is based on the Intel Loihi2 neuromorphic chip and will subsequently be used to build a DSNN-based network. Step 3, construct the feature extraction layer: use the difference of Gaussian filter to extract edge and contour features to obtain the feature map; Step 4, Construct the pulse coding layer: Encode the feature map into a pulse sequence, converting continuous feature values ​​into discrete pulse sequences; Step 5, Construct the STDP learning layer: Introduce an STDP-based SNN layer to achieve adaptive adjustment of synaptic weights, enabling the network to effectively learn and extract complex local feature patterns and determine the target region in the feature map; Step 6, constructing a transverse inhibition layer: suppressing background noise through neuronal competition mechanism; suppressing background noise through inter-neuron competition mechanism: when the target region neuron fires a pulse, the membrane potential of the surrounding background neurons is suppressed: that is, the threshold is reduced by 30%, reducing false detection; adopting a "pulse counting + time window" decision mechanism: counting the total number of pulses of the target type neurons within a 100ms window, if it exceeds the threshold, it is determined to be the target, and the decision delay is controlled within 20ms. The surrounding background neurons are the neurons in the feature map other than the target region neurons. Step 7, Construct the decision output layer: The output layer is a fully connected spiking layer that learns higher-level features and makes decisions; Step 8: Based on the DSNN-based network obtained in Steps 3 to 7, input the training set into the constructed DSNN-based network for training. At the same time, use the publicly available pre-trained model to obtain the learned neural network. Then, optimize and tune the parameters on the validation set to obtain the optimal model and its corresponding parameters, and obtain the trained DSNN-based network. Step 9: In practical applications, the satellite-orbiting end uses a trained DSNN-based network to process the satellite-borne images. Then, the satellite-borne end encrypts and uploads the on-orbit processing results to the ground. The ground updates the global model using the federated averaging algorithm and then sends the optimized difference parameters back to the satellite-borne end. The onboard system uses new parameters to fine-tune synaptic weights and performs iterative optimization, enabling on-orbit intelligent processing of onboard images based on brain-like computing.

2. The on-orbit intelligent processing method for spaceborne images based on neuromorphic computing according to claim 1, characterized in that, The specific method for step 4 is as follows: A time-based encoding method is used to linearly map pixel values ​​to an initial membrane potential, which is then input to the neuron. When a certain threshold is exceeded, a pulse sequence is output at that moment. in, : Membrane potential at time t; : Resting membrane potential, normalized to 0; The membrane time constant determines the rate of change of membrane potential, and its value ranges from 10 ms to 30 ms; when the membrane potential exceeds the threshold... At that time, the neuron fires a pulse and resets to X(t) is the instantaneous rate of change of the membrane potential.

Images