Remote communication integrated intelligent satellite sensing processing system

Abstract

The invention discloses a communication and remote control integrated intelligent satellite sensing processing system, and aims to solve the problems of poor timeliness, low efficiency, insufficient on-satellite computing power and the like due to mutual independence of remote sensing and communication functions and data relying on back-to-ground processing in a traditional satellite system. Through a remote communication integrated hardware system architecture, a storage and calculation integrated AI computing platform, a multi-modal intelligent processing algorithm and a satellite in-orbit remote communication collaborative algorithm, and through the synergistic effect of all the parts, the efficient remote sensing and communication functions of the micro-nano satellite under the condition of resource limitation are realized. According to the method, communication and remote sensing loads are set on a commercial satellite, deep collaboration of multi-spectrum remote sensing imaging, high-sensitivity satellite Internet of Things communication and satellite-borne AI edge calculation is achieved through a satellite-end high-computing-power AI computer, an on-satellite intelligent processing closed loop is constructed, and the satellite comprehensive sensing efficiency is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of on-orbit data processing of spacecraft, and particularly relates to an integrated communication and remote sensing intelligent satellite perception and processing system. Background Art

[0002] In recent years, the commercial space industry has developed rapidly. Due to its advantages such as short development cycle, low launch cost, and networkable deployment, low-orbit micro-nano satellites have shown great application potential in the fields of remote sensing monitoring, Internet of Things communication, space science experiments, etc. However, in the existing technology, there are multiple bottlenecks in the on-orbit data processing of low-orbit micro-nano satellites: 1. Traditional satellite systems adopt a separated design for remote sensing and communication functions, and data relies on ground backhaul processing, resulting in poor timeliness in scenarios such as disaster monitoring. Moreover, the bandwidth of the space-ground data transmission link is limited, and it cannot meet the low-latency requirements of various remote sensing applications. 2. Constrained by the volume and power consumption of the satellite platform, frequent interaction between the computing and storage units under the traditional von Neumann architecture leads to a 30%-50% waste of system power consumption. Moreover, the computing power of existing on-board computing units is generally lower than 5 TOPS, making it difficult to support the real-time fusion analysis of multi-spectral images and Internet of Things data. 3. The traditional on-board processing model relies on full-scale parameter updates, and the amount of data transmitted each time exceeds 5 MB, seriously occupying the space-ground link resources. Although some studies have proposed on-board heterogeneous computing architectures, they still adopt a separated design for storage and computing, and existing fusion algorithms are difficult to achieve sub-pixel-level spatio-temporal registration of multi-modal data (positioning deviation > 1 pixel), and cannot support the accurate discrimination of abnormal events.

[0003] 4. The application of in-memory computing technology in ground AI chips has not yet solved key challenges such as lightweight model dynamic quantization and compression, resulting in a contradiction between on-board real-time processing capabilities and resource limitations. There is an urgent need to build a new system that integrates an integrated communication and remote sensing hardware architecture, on-board in-memory computing collaboration optimization, and multi-modal intelligent analysis to break through the industrial application barriers of commercial micro-nano satellites in real-time perception, autonomous decision-making, and resource efficiency. Summary of the Invention

[0004] The purpose of the present invention is to provide an integrated communication and remote sensing intelligent satellite perception and processing system. By setting communication and remote sensing payloads on a commercial satellite, and through a high-computing-power AI computer on the satellite, the deep collaboration of multi-spectral remote sensing imaging, high-sensitivity satellite Internet of Things communication, and on-board AI edge computing is achieved to build an on-board intelligent processing closed loop, significantly improving the comprehensive perception efficiency of the satellite.

[0005] To solve the above problems, the technical solution of the present invention is as follows: A remote-sensing and communication integrated intelligent satellite sensing and processing system, comprising: a remote-sensing and communication integrated hardware system architecture, a memory-computation integrated AI computing platform, multi-modal intelligent processing algorithms, and satellite on-orbit remote-sensing and communication cooperation algorithms. Through the collaborative action of each part, efficient remote-sensing and communication functions of micro-nano satellites under resource constraints are realized; Among them, the remote-sensing and communication integrated hardware system architecture adopts an integrated design of payload and platform, sharing the power bus, star sensor, and data transmission module to improve the function density; The memory-computation integrated AI computing platform includes a hardware layer and a software layer. The hardware layer provides computing power to meet the requirements of on-board AI processing; the software layer adopts a lightweight model to reduce the model volume and improve the computing efficiency while ensuring the accuracy; The multi-modal intelligent processing algorithms are deployed on the memory-computation integrated AI computing platform to achieve spatio-temporal alignment to accurately match Internet of Things devices and remote-sensing data, as well as anomaly detection functions; The satellite on-orbit remote-sensing and communication cooperation algorithms are deployed on the memory-computation integrated AI computing platform to achieve the functions of triggering the wake-up of Internet of Things terminals by remote-sensing data and optimizing communication-assisted remote-sensing imaging parameters.

[0006] According to an embodiment of the present invention, the remote-sensing and communication integrated hardware system architecture integrates a multi-spectral imaging payload, an infrared payload, and an Internet of Things communication payload; Among them, the multi-spectral imaging payload selects a CMOSIS CMV20000 sensor, covering 8 bands of 450 - 900nm, adopting an off-axis three-mirror optical design, and achieving an 8m GSD resolution in a 500km orbit; The infrared payload covers the 8 - 12.5μm infrared spectral band, with a thermal sensitivity of 0.1K@300K, supporting the switching between staring imaging and push-broom modes; The Internet of Things communication payload adopts a software-defined radio architecture, supports LoRaWAN and NB-IoT dual-mode protocols, has a receiving sensitivity of -130dBm, is equipped with a high-gain flat antenna, and can simultaneously access 50 low-power terminals.

[0007] According to an embodiment of the present invention, the remote-sensing and communication integrated hardware system architecture adopts a carbon fiber composite bracket and a modular heat pipe cooling system to integrate the multi-spectral imaging payload, the infrared payload, and the Internet of Things communication payload in a 0.2m³ space, with a total power consumption ≤ 80W.

[0008] According to an embodiment of the present invention, the hardware layer of the memory-computation integrated AI computing platform includes: NVIDIA Jetson AGX Orin module, integrated with a 2048-core Ampere GPU and a 12-core ARM CPU, supporting 150 TOPS INT8 computing power, and connected to the payload through cameralink and LVDS buses; Storage module, using a 4TB high-performance SSD, with LDPC error correction, supporting continuous read and write of 1.5GB / s; Flexible payload interface unit, supporting multiple payload interfaces including cameralink and LVDS, and can access multispectral cameras, infrared payloads, and IoT data.

[0009] According to an embodiment of the present invention, the software layer of the memory-computation integrated AI computing platform includes: Lightweight model, using TensorRT to perform layer fusion and INT8 quantization on the LSTM network, and compressing the model volume to 18MB; Dynamic memory allocation module, building a real-time task scheduler based on ROS 2 Galactic, and preferentially allocating computing power to infrared anomaly detection tasks.

[0010] According to an embodiment of the present invention, the multimodal intelligent processing algorithm includes: Spatio-temporal alignment module, matching the GPS coordinates of IoT terminals with the coordinates of remote sensing images through the RANSAC algorithm to compensate for attitude jitter errors; and constructing a spatio-temporal correlation matrix through feature-level fusion, taking multispectral reflectance, infrared radiance, and buoy pH value as inputs, and outputting a 128-dimensional tensor; LSTM anomaly detection model, adopting a 3-layer BiLSTM + self-attention mechanism network structure, through QAT quantization-aware training; and through an incremental learning module, combining differential update algorithms and gradient compression techniques, optimizing the transmission efficiency of incremental files, and reducing the occupation of satellite communication resources by model updates.

[0011] According to an embodiment of the present invention, the spatio-temporal alignment module includes: Coordinate matching unit, matching the GPS coordinates of IoT terminals with the coordinates of remote sensing images through the RANSAC algorithm to compensate for attitude jitter errors; Preprocessing subunit for time series data, using the Kalman filter algorithm to denoise the input time series data, and reducing the impact of random noise in the data on subsequent fusion processing; Multi-feature fusion subunit, adopting a weighted fusion strategy, automatically adjusting the weight coefficients of each feature according to the importance and reliability of multispectral reflectance, infrared radiance, and buoy pH value, and realizing more accurate construction of the spatio-temporal correlation matrix.

[0012] According to an embodiment of the present invention, the LSTM anomaly detection model further adopts a probability density distribution comparison method based on KL divergence to continuously monitor the difference between the model output distribution and the normal mode distribution. When a model deviation is detected, a rollback mechanism is automatically triggered to ensure the stability of the system.

[0013] According to an embodiment of the present invention, the satellite on-orbit communication and remote sensing collaboration algorithm includes: Emergency remote sensing triggers the Internet of Things communication mode: When the infrared payload identifies an abnormal temperature, start the priority task chain: Adjust the multi-spectral camera to the high-gain mode, with the signal-to-noise ratio increased by 6 dB; Wake up the encrypted backhaul of Internet of Things terminals within a radius of 50 km, using AES-256 + OQPSK modulation; Fuse data to generate alarm information, with a compression ratio ≥ 10:1, and download it to the ground station.

[0014] According to an embodiment of the present invention, the satellite on-orbit communication and remote sensing collaboration algorithm further includes: Communication-assisted remote sensing mode: Based on the ISO 19848 standard, analyze ship position messages; for ships with illegally turned-off AIS, obtain radar reflection signals through Internet of Things terminals; drive the multi-spectral camera to switch to short-wave infrared band imaging; combine with SAR imaging results, with the resolution improved to 5 m, and generate a suspicious target report.

[0015] Due to the adoption of the above technical solutions, the present invention has the following advantages and positive effects compared with the prior art: In an embodiment of the present invention, the integrated communication and remote sensing intelligent satellite sensing and processing system aims at the problems in traditional satellite systems, such as the mutual independence of remote sensing and communication functions, data relying on ground processing for backhaul, poor timeliness, low efficiency, and insufficient on-board computing power. The present invention sets up communication and remote sensing payloads on commercial satellites, and through a high-computing-power AI computer on the satellite side, deeply collaborates multi-spectral remote sensing imaging, high-sensitivity satellite Internet of Things communication, and on-board AI edge computing to build an on-board intelligent processing closed-loop, significantly improving the comprehensive sensing efficiency of the satellite. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a block diagram of the integrated communication and remote sensing intelligent satellite sensing and processing system in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] The following further elaborates in detail on a proposed integrated communication and remote sensing intelligent satellite sensing and processing system of the present invention in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will be clearer based on the following description and the claims.

[0018] Please refer to Figure 1 , this embodiment provides an integrated communication and remote sensing intelligent satellite sensing and processing system, including: an integrated communication and remote sensing hardware system architecture, a memory-computation integrated AI computing platform, multi-modal intelligent processing algorithms, and satellite on-orbit communication and remote sensing cooperation algorithms. Through the coordinated action of each part, the high-efficiency remote sensing and communication functions of microsatellites under resource constraints are realized; Among them, the integrated communication and remote sensing hardware system architecture adopts an integrated design of payload and platform, sharing the power bus, star sensor, and data transmission module to improve the function density; The memory-computation integrated AI computing platform includes a hardware layer and a software layer. The hardware layer provides computing power to meet the requirements of on-board AI processing; the software layer adopts a lightweight model to reduce the model volume and improve the computing efficiency while ensuring the accuracy; The multi-modal intelligent processing algorithms are deployed on the memory-computation integrated AI computing platform to achieve spatio-temporal alignment to accurately match Internet of Things devices and remote sensing data, as well as anomaly detection functions; The satellite on-orbit communication and remote sensing cooperation algorithms are deployed on the memory-computation integrated AI computing platform to achieve the functions of triggering the wake-up of Internet of Things terminals by remote sensing data and optimizing the communication-assisted remote sensing imaging parameters.

[0019] Specifically, the integrated communication and remote sensing hardware system architecture is deeply integrated with a multi-spectral and infrared spectral remote sensing camera and an Internet of Things communication payload. It adopts an integrated design of payload and platform and realizes the improvement of function density by sharing the power bus, star sensor, and data transmission module. Among them, the multi-spectral imaging payload selects a CMOSIS CMV20000 sensor, covering 8 bands in the range of 450 - 900nm (including key spectral bands such as coastal blue and red edge), adopts an off-axis three-mirror optical design, and realizes an 8m GSD resolution in a 500km orbit; the infrared payload covers the 8 - 12.5μm infrared spectral band, with a thermal sensitivity of 0.1K@300K, and supports the switching between staring imaging and push-broom modes. The Internet of Things communication payload adopts a software-defined radio architecture, supports LoRaWAN and NB-IoT dual-mode protocols, has a receiving sensitivity of -130dBm, is equipped with a high-gain flat antenna, and can simultaneously access 50 low-power terminals (such as buoys, forest monitoring systems). Through a carbon fiber composite bracket and a modular heat pipe cooling system, the three payloads are integrated in a 0.2m³ space, with a total power consumption ≤80W, significantly superior to the traditional split design.

[0020] Among them, the off-axis three-mirror optical design consists of three mirrors. By reasonably setting the off-axis amount, tilt amount, spacing, and surface shape of the mirrors, an optical system without obstruction, without chromatic aberration, and with high imaging quality is achieved. Its design is usually based on the initial structure of the coaxial three-mirror system. Through field off-axis or aperture off-axis processing, the central obstruction problem is eliminated, and the aberrations are further optimized to achieve diffraction-limited imaging quality. In this embodiment, the off-axis three-mirror optical design is used for the multispectral imaging payload, and a common aperture design is adopted, that is, imaging in multiple bands is realized in the same optical path, thereby reducing the volume and weight of the system.

[0021] Through the carbon fiber composite bracket and the modular heat pipe cooling system, the three payloads are integrated in a space of 0.2 m³, which can not only effectively manage the heat load, but also ensure the light weight and high-efficiency heat dissipation of the system. Among them, the carbon fiber composite bracket is suitable for occasions with high mechanical strength and light weight requirements due to its light weight, high strength, and good heat management performance. Through the multi-level structure design, carbon fiber, polyaniline nanofibers, and silver nanowires can be integrated into a composite material to form a unique "branch-trunk" interlocked micro-nano structure, thereby significantly improving the mechanical properties and heat management performance of the material. In practical applications, the strong adhesion ability of polydopamine can be used to form the "branch-trunk" interlocked micro-nano structure of carbon fiber, enhancing the mechanical properties of the material. By the dual control strategy of synergistically controlling the thermal conductivity and infrared radiation, the heat management performance of the composite material is optimized, making it have application potential in the fields of thermal insulation and infrared stealth.

[0022] The modular heat pipe cooling system can effectively manage the heat of multiple heat sources by integrating the heat pipes and the radiator in a compact module. This design method not only improves the heat dissipation efficiency, but also reduces the overall complexity and weight of the system. In practical applications, a six-heat-pipe cooling system can be selected to provide higher heat dissipation efficiency and better heat management performance; by reasonably arranging the heat pipes, it is ensured that the heat can be evenly distributed and effectively transferred to the radiator, thereby improving the overall heat dissipation efficiency.

[0023] The in-memory computing AI computing platform designs an on-board AI processing unit based on the in-memory computing collaborative architecture. Its hardware layer uses the NVIDIA Jetson AGX Orin module, which integrates a 2048-core Ampere GPU and a 12-core ARM CPU, supports 150 TOPS INT8 computing power, and is connected to the payload through the cameralink and LVDS buses; the storage module uses a 4TB high-performance SSD (SeagateNytro® XF1230), adopts LDPC error correction, supports continuous read and write of 1.5GB / s, and has an on-orbit working life of more than 5 years through partition redundancy design; a flexible payload interface unit is adopted, which supports multiple payload interfaces such as cameralink, 2711, and LVDS, and can access multi-spectral cameras (2Gbps), infrared payloads (800Mbps), and Internet of Things data (10kbps).

[0024] The software layer of the in-memory computing AI computing platform uses a lightweight model, uses TensorRT to perform layer fusion and INT8 quantization on the LSTM network, and compresses the model volume to 18MB (accuracy loss ≤ 1.8%); supports dynamic memory allocation, constructs a real-time task scheduler based on ROS 2 Galactic, and preferentially allocates computing power to infrared anomaly detection tasks.

[0025] The multi-modal intelligent processing algorithm is deployed on the in-memory computing AI computing platform, mainly including: The spatio-temporal alignment module matches the GPS coordinates (WGS84) of the Internet of Things terminal and the remote sensing image coordinates (UTM) through the RANSAC algorithm, and compensates for the attitude jitter error (residual ≤ 0.3 pixel); constructs a spatio-temporal correlation matrix through feature-level fusion, takes multi-spectral reflectance, infrared radiance, and buoy pH value as inputs, and outputs a 128-dimensional tensor; The LSTM anomaly detection model adopts a 3-layer BiLSTM (256 nodes in the hidden layer) + self-attention mechanism network structure, and performs quantization-aware training through QAT; and through the incremental learning module, combines the differential update algorithm and gradient compression technology to optimize the transmission efficiency of incremental files and reduce the occupancy of satellite communication resources by model updates. For example, only transmit the parameter gradients triggered by ground annotation data (SGD optimizer, learning rate 1e-4) to generate a 50MB incremental file.

[0026] Furthermore, the spatio-temporal alignment module includes: The coordinate matching unit matches the GPS coordinates of the Internet of Things terminal and the remote sensing image coordinates through the RANSAC algorithm, and compensates for the attitude jitter error; The preprocessing subunit of time series data uses the Kalman filter algorithm to perform noise reduction processing on the input time series data, and reduces the impact of random noise in the data on subsequent fusion processing; The multi-feature fusion subunit adopts a weighted fusion strategy. According to the importance and reliability of multi-spectral reflectance, infrared radiation brightness, and buoy pH value, it automatically adjusts the weight coefficients of each feature to achieve a more accurate spatio-temporal correlation matrix construction.

[0027] The 3-layer BiLSTM architecture in the LSTM anomaly detection model includes: 1. Input layer: Receives sequence data with a length of 60 frames. Each frame of data contains multi-modal features such as multi-spectral reflectance, infrared radiation brightness, and buoy pH value. After preprocessing, it forms a standardized input vector to ensure the consistency of data dimensions and the rationality of numerical ranges, facilitating stable training and efficient calculation of the model.

[0028] 2. The first layer of BiLSTM: Forward LSTM: Processes the data at each moment in the input sequence in turn, calculating the hidden state from the start moment to the end moment of the sequence. The dimension of the hidden state is 256. Nonlinearity is introduced through the tanh activation function, enabling the model to learn complex nonlinear feature relationships in the sequence. The hidden state ht^forward of the forward LSTM at time t is jointly determined by the current input xt and the hidden state ht-1^forward at time t - 1, effectively capturing the forward-looking time-dependent features of the sequence. For example, in monitoring forest fires, the early signs of the fire spread trend.

[0029] Backward LSTM: Opposite to the forward LSTM, it calculates the hidden state in the reverse direction from the end moment to the start moment of the sequence, also with a dimension of 256. The hidden state ht^backward of the backward LSTM depends on the current input xt and the hidden state ht+1^backward at time t + 1, focusing on extracting the retrospective features of the sequence. For example, when analyzing ocean buoy data, considering the supplementary explanatory role of subsequent data for early abnormal fluctuations. Finally, this layer concatenates the forward and backward hidden states to form an output with a dimension of 512, providing richer bidirectional sequence information for the upper network.

[0030] 3. The second layer of BiLSTM: Input: Directly receives the 512-dimensional output sequence concatenated by the first layer of BiLSTM without additional dimensionality reduction or transformation to ensure the complete transmission of information.

[0031] Forward and backward LSTM: The structure is similar to that of the first layer, and the hidden state dimension remains 256. It extracts abstract features of the sequence at a higher level, further exploring the deep bidirectional associations and patterns in the data. For example, when processing IoT communication data, it can capture the potential connection between abnormal device data transmission and subsequent repair processes.

[0032] 4. The Third Layer BiLSTM Input: Follow the output dimension and sequence structure of the previous two layers to ensure the coherence of the model structure and the consistency of feature transmission.

[0033] Forward and backward LSTM: Continuously deepen the bidirectional feature mining of the sequence, and the hidden state dimension is stabilized at 256. At this time, the model has performed multi-layer abstraction on the input sequence and can capture more hidden and complex abnormal feature patterns. For example, when monitoring illegal maritime activities, it can associate the subtle changes in ship behavior in multiple frames of data with abnormal combinations of surrounding environmental factors.

[0034] Output: The concatenated 512-dimensional hidden state sequence, as the final output of the BiLSTM part, comprehensively contains the deep bidirectional features of the input sequence in the time dimension.

[0035] Self-attention Mechanism Module 1. Input Transformation: Project the 512-dimensional hidden state sequence output by the third layer BiLSTM through a fully connected layer into the attention space to obtain query vector (Q), key vector (K), and value vector (V). The dimensions of Q, K, and V can be set according to computing resources and model performance requirements. Generally, they can be set to 128 dimensions to reduce computational complexity while retaining sufficient expressive power.

[0036] 2. Attention Score Calculation: Adopt the scaled dot-product attention mechanism to calculate the dot product of Q and the transposed matrix of K to obtain the attention score matrix. To prevent the dot product result from being too large, causing gradient vanishing or explosion, divide the dot product result by (the dimension of Q and K) for scaling. Each element in the attention score matrix represents the correlation weight between the corresponding two positions in the sequence. For example, when analyzing multi-spectral and infrared fusion data, the degree of association between remote sensing image features at different time points and IoT monitoring data.

[0037] 3. Softmax Activation: Apply the softmax function to the attention score matrix row by row to convert the scores into a weight matrix in the form of a probability distribution. The sum of the rows of the weight matrix is 1, ensuring the numerical stability of the subsequent weighted summation operation, highlighting the key feature points in the sequence, and suppressing irrelevant information.

[0038] 4. Weighted Summation: Use the weight matrix processed by softmax and the value vector V to perform a weighted dot product operation to obtain the output sequence of the self-attention mechanism. On the basis of retaining the bidirectional features extracted by BiLSTM, this sequence redistributes weights according to the internal feature correlation of the sequence, enhances the expression of key features, makes the model more sensitive to abnormal patterns, and the output dimension remains the same as the input dimension.

[0039] Fusion and Output Layer 1. Feature Fusion: Deeply fuse the sequence processed by the self-attention mechanism with the output sequence of the 3rd layer BiLSTM. Through a fully connected layer, perform a linear transformation on the vector obtained by concatenating the two sequences in the feature dimension, integrate the sequence features of BiLSTM and the spatial features of the self-attention mechanism to enhance information, obtain the fused feature sequence, and select the ReLU activation function to introduce non-linearity and improve the model's fitting ability for complex feature relationships.

[0040] 2. Output Layer: The fused feature sequence passes through the last fully connected layer and is mapped to the output dimension required for anomaly detection. According to the specific task requirements, if it is binary classification anomaly detection (anomaly / normal), a single node is output, and the sigmoid activation function is used to compress the output value to the interval (0, 1), representing the probability that the input sequence is abnormal; if it is multi-class anomaly classification, the number of output nodes corresponds to the number of anomaly classes, and the softmax activation function is selected to obtain the probability distribution of various types of anomalies to assist the system in making accurate anomaly judgments and classifications.

[0041] Before performing QAT quantization-aware training, randomly initialize the weights of the BiLSTM and self-attention mechanism modules, and at the same time initialize the quantization parameters (scale and zero_point) to 1.0 and 0.0, laying the foundation for subsequent quantization-aware training.

[0042] Its training process includes the following steps: Forward Propagation: Input the training data sequence, and successively pass through the 3-layer BiLSTM and self-attention mechanism modules. The activation values of each module perform pseudo-quantization operations during the calculation process to simulate the quantization effect. Record the output results and loss values of the model. The loss function is selected according to the anomaly detection task type. For example, the binary cross-entropy loss function is used for binary classification tasks, and the cross-entropy loss function is used for multi-classification tasks. At the same time, an L2 regularization term can be added to prevent the model from overfitting.

[0043] Backpropagation and Parameter Update: According to the loss value, the gradients of the model parameters are calculated using the backpropagation algorithm. When calculating the gradients of the pseudo-quantization operation nodes, the STE method is adopted to directly pass the gradients to the floating-point parameters before quantization. Then, an optimization algorithm (such as the Adam optimizer) is used to update the floating-point parameters of the model. At the same time, the quantization parameters (scale and zero_point) are updated according to the histogram data of the activation values, so that the quantization parameters gradually adapt to the changes of the model parameters and optimize the quantization effect.

[0044] Iterative Optimization: Repeat the forward propagation and backpropagation processes, continuously adjust the model parameters and quantization parameters, and improve the accuracy of the quantized model while reducing the loss value. The training process continues for multiple epochs until the performance metrics (such as accuracy, recall rate, F1 value, etc.) of the model on the validation set reach a satisfactory level, and the performance gap between the quantized model and the unquantized model is controlled within a reasonable range (such as accuracy loss ≤ 1.8%).

[0045] In the incremental learning module of this embodiment, on the ground side, the newly collected labeled data is preprocessed and feature extracted to construct a training sequence. Using the QAT quantization-aware training method, the model is incrementally learned and trained, and only the part of the model parameters that change due to the introduction of new data is calculated and updated to generate a differential update package. The differential update package contains the updated parameter gradients (calculated using the SGD optimizer, learning rate 1e - 4) and quantization parameter update information, and its volume is controlled at about 50MB, reducing the occupancy of satellite communication bandwidth.

[0046] After the satellite receives the differential update package, the storage module of the on-board AI computing platform is used to temporarily store the update data. The update program is called to apply the parameter gradients in the differential update package to the current model parameters to update the model weights and quantization parameters. During the update process, a block update strategy is adopted, dividing the model parameters into multiple small blocks, updating and verifying the integrity and correctness of each small block after updating in turn, to prevent the entire model from failing due to update errors.

[0047] In this embodiment, based on the 150 TOPS INT8 computing power of the NVIDIA Jetson AGX Orin module in the in-memory computing AI computing platform, the computing power requirements of the LSTM anomaly detection model are analyzed. During the model training phase, through QAT (Quantization-Aware Training), it is ensured that the computing power requirements of the model after INT8 quantization match the computing power of the hardware platform. When the model is running, the dynamic memory allocation mechanism of the platform is used to allocate a dedicated memory area for the model, avoiding memory conflicts with other tasks and ensuring the smoothness of the model inference process. For example, when processing a 60-frame input sequence, the GPU computing power and memory bandwidth required for model inference are estimated, and the corresponding resources are reserved in advance to ensure that the model can complete the anomaly detection task within a specified time (such as within 1 second).

[0048] And the storage and cache are optimized: the model parameters are stored in the high-performance SSD storage module, and at the same time, a cache area is set in the memory of the AI computing platform. When the model performs inference, it preferentially reads frequently used parameters and intermediate results from the cache, reducing the frequent read and write operations on the SSD, extending the lifespan of the storage module, and improving the model operation efficiency. For example, the core parameters such as the weight matrix of BiLSTM and the query, key, and value transformation matrices of the self-attention mechanism are resident in the memory cache, and other auxiliary parameters are loaded from the SSD as needed to optimize the data access process.

[0049] The LSTM anomaly detection model further adopts a method of comparing probability density distributions based on KL divergence to continuously monitor the difference between the model output distribution and the normal mode distribution. When a model deviation is detected, a rollback mechanism is automatically triggered to ensure the stability of the system.

[0050] Among them, KL divergence (Kullback - Leibler Divergence) is an asymmetric metric method for measuring the difference between two probability distributions P and Q. In this solution, it is used to measure the difference between the current probability distribution Q of the model output and the probability distribution P in the normal mode. Its calculation formula is: In this embodiment, by calculating the KL divergence between the distribution Q of each model output and the normal mode distribution P, it is continuously monitored whether there are abnormal changes in the model output. The normal mode distribution P can be obtained through statistical analysis of the model outputs on a large amount of normal data, or during the model training phase, by recording the output distribution of the model on the validation set.

[0051] During the operation of the multi-modal intelligent processing software algorithm, each time the model outputs a multi-dimensional tensor, the output data is immediately collected. Since the model output may be high-dimensional and complex, it needs to be appropriately preprocessed for subsequent probability density distribution estimation. The preprocessing may include operations such as data normalization and dimensionality reduction to ensure that the data is suitable for probability density estimation methods and to improve computational efficiency.

[0052] Adopt a suitable probability density estimation method, such as Kernel Density Estimation (KDE) or the histogram method, to estimate the probability density distribution of the preprocessed model output data, and obtain the current model output distribution Q.

[0053] By statistically analyzing the KL divergence between the output distribution of the model under normal conditions and the normal mode distribution during the model training stage or system verification stage, a reasonable judgment threshold is determined. For example, the KL divergence values of multiple normal samples can be calculated, and the mean plus several times the standard deviation is taken as the threshold, so as to detect model offset events in a timely manner while ensuring a low false alarm rate.

[0054] During the operation of the system, each time the KL divergence is calculated, it is immediately compared with the preset threshold. If the KL divergence exceeds the threshold, it is judged that the model has an offset, and the corresponding rollback mechanism is triggered; otherwise, it is considered that the model output is normal, and the subsequent processing tasks are continued.

[0055] When a model offset is detected, the system immediately triggers the rollback mechanism. First, record the state information of the current model, including the input data, output results, and intermediate states inside the model at the time of offset detection, etc., for subsequent analysis and troubleshooting of the model offset cause. Then, obtain the model parameters and structure of the previous stable version from the model version management system, load them into the AI computing platform, and replace the currently offset model. During the model recovery process, ensure the compatibility of the newly loaded model with the current system environment, including checking whether the interfaces of the model with the hardware platform, software framework, and other system modules match.

[0056] In this embodiment, the fusion of ground Internet of Things data and satellite remote sensing data further includes: Data alignment: Interpolate the in-situ Internet of Things data (pH, temperature, etc.) to the remote sensing image pixel grid to generate a 10km×10km spatio-temporal cube; Feature extraction: Use a 3D convolutional network to extract the spatio-temporal correlation features of multi-source data and output a 128-dimensional embedding vector; Joint inference: Based on the Gated Recurrent Unit (GRU), realize the time series anomaly detection of multi-modal data, with a false alarm rate ≤ 2%.

[0057] The satellite on-orbit communication and remote sensing collaborative algorithm of this embodiment includes: Emergency remote sensing triggers the Internet of Things communication mode: When the infrared payload identifies an abnormal temperature (ΔT≥5K / 10 minutes), start the priority task chain: Adjust the multi-spectral camera to the high-gain mode, and the signal-to-noise ratio is increased by 6dB; Wake up the encrypted backhaul of Internet of Things terminals within a radius of 50km, and adopt AES - 256 + OQPSK modulation; Fuse data to generate alarm information, with a compression ratio ≥ 10:1, and download it to the ground station.

[0058] This satellite on-orbit communication and remote sensing collaborative algorithm further includes: Communication-assisted remote sensing mode: Based on the ISO 19848 standard, parse the ship position message; for ships with illegally turned-off AIS, obtain the radar reflection signal (RCS≥1000m²) through the Internet of Things terminal; drive the multi-spectral camera to switch to the short-wave infrared (SWIR) band for imaging; combine the SAR imaging results, and the resolution is improved to 5m to generate a suspicious target report.

[0059] Through the above communication and remote sensing integrated intelligent satellite sensing and processing system, high-performance target detection for the sparse ocean background can be achieved, and the specific implementation is as follows: Background modeling: Use historical multi-spectral data to construct a seawater reflectance reference library, and detect abnormal areas through Mahalanobis distance (threshold≥3σ); Multi-source verification: When remote sensing detects a suspected target (such as an oil film or a ship), automatically wake up the nearest Internet of Things buoy for in-situ sampling verification; Compressed transmission: Only download the ROI data of the target area (compression ratio≥10:1) for the confirmed target, and attach the Internet of Things verification label.

[0060] The communication and remote sensing integrated intelligent satellite sensing and processing system in this embodiment is applicable to the collaborative intelligent processing of remote sensing and communication on commercial micro-nano satellite platforms, especially applicable to the multi-modal data fusion and real-time intelligent analysis scenarios of micro-nano satellites (100kg class).

[0061] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, provided that these changes fall within the scope of the claims of the present invention and their equivalent technologies, they still fall within the protection scope of the present invention.

Claims

1. A remote integrated intelligent satellite sensing and processing system, characterized in that, Including: A space - borne integrated remote sensing and communication hardware system architecture, a memory - computing integrated AI computing platform, multi - modal intelligent processing algorithms, and satellite - on - orbit remote sensing and communication cooperation algorithms. Through the collaborative action of each part, the high - efficiency remote sensing and communication functions of microsatellites under resource - constrained conditions are realized; Among them, the space - borne integrated remote sensing and communication hardware system architecture adopts an integrated design of payload and platform, sharing the power bus, star sensor, and data transmission module to improve the function density; The memory - computing integrated AI computing platform includes a hardware layer and a software layer. The hardware layer provides computing power to meet the requirements of on - satellite AI processing; the software layer adopts a lightweight model to reduce the model volume and improve the computing efficiency while ensuring accuracy; The multi - modal intelligent processing algorithm is deployed on the memory - computing integrated AI computing platform to achieve spatio - temporal alignment to accurately match Internet of Things devices and remote sensing data, as well as anomaly detection functions; The satellite - on - orbit remote sensing and communication cooperation algorithm is deployed on the memory - computing integrated AI computing platform to achieve the functions of triggering the wake - up of Internet of Things terminals by remote sensing data and optimizing communication - assisted remote sensing imaging parameters.

2. The integrated intelligent satellite perception and processing system for remote control according to claim 1, wherein The space - borne integrated remote sensing and communication hardware system architecture integrates a multi - spectral imaging payload, an infrared payload, and an Internet of Things communication payload; Among them, the multi - spectral imaging payload selects a CMOSIS CMV20000 sensor, covering 8 bands in the range of 450 - 900nm, adopting an off - axis three - mirror optical design, and achieving an 8m GSD resolution in a 500km orbit; The infrared payload covers the 8 - 12.5μm infrared spectral band, with a thermal sensitivity of 0.1K@300K, and supports the switching between staring imaging and push - broom modes; The Internet of Things communication payload adopts a software - defined radio architecture, supports LoRaWAN and NB - IoT dual - mode protocols, has a receiving sensitivity of - 130dBm, is equipped with a high - gain planar antenna, and can simultaneously access 50 low - power terminals.

3. The integrated intelligent satellite perception and processing system for remote control according to claim 2, characterized in that The space - borne integrated remote sensing and communication hardware system architecture adopts a carbon fiber composite bracket and a modular heat pipe cooling system, integrating the multi - spectral imaging payload, the infrared payload, and the Internet of Things communication payload within a 0.2m³ space, with a total power consumption ≤ 80W.

4. The integrated remote intelligent satellite sensing and processing system according to claim 1, characterized in that, The hardware layer of the memory - computing integrated AI computing platform includes: NVIDIA Jetson AGX Orin module, integrating a 2048 - core Ampere GPU and a 12 - core ARM CPU, supporting 150 TOPS INT8 computing power, and connecting to the payload through cameralink and LVDS buses; A storage module, using a 4TB high - performance SSD, adopting LDPC error correction, and supporting continuous read - write of 1.5GB / s; A flexible payload interface unit, supporting multiple payload interfaces including cameralink and LVDS, and can access multi - spectral cameras, infrared payloads, and Internet of Things data.

5. The integrated intelligent satellite perception and processing system for remote control according to claim 1, characterized in that, The software layer of the memory - computing integrated AI computing platform includes: Lightweight model, which uses TensorRT to perform layer fusion and INT8 quantization on the LSTM network, compressing the model size to 18MB; Dynamic memory allocation module, which constructs a real-time task scheduler based on ROS 2 Galactic and preferentially allocates computing power to infrared anomaly detection tasks.

6. The integrated intelligent satellite perception and processing system for remote control according to claim 1, characterized in that, The multi-modal intelligent processing algorithm includes: Spatio-temporal alignment module, which matches the GPS coordinates of IoT terminals with the coordinates of remote sensing images through the RANSAC algorithm to compensate for attitude jitter errors; and constructs a spatio-temporal correlation matrix through feature-level fusion, taking multi-spectral reflectance, infrared radiance, and buoy pH value as inputs and outputting a 128-dimensional tensor; LSTM anomaly detection model, which adopts a 3-layer BiLSTM + self-attention mechanism network structure and performs quantization-aware training through QAT; and through an incremental learning module, combines the differential update algorithm and gradient compression technology to optimize the transmission efficiency of incremental files and reduce the occupancy of satellite communication resources by model updates.

7. The integrated intelligent satellite perception and processing system for remote control according to claim 6, characterized in that, The spatio-temporal alignment module includes: Coordinate matching unit, which matches the GPS coordinates of IoT terminals with the coordinates of remote sensing images through the RANSAC algorithm to compensate for attitude jitter errors; Preprocessing sub-unit for time series data, which uses the Kalman filter algorithm to denoise the input time series data and reduce the impact of random noise in the data on subsequent fusion processing; Multi-feature fusion sub-unit, which adopts a weighted fusion strategy and automatically adjusts the weight coefficients of each feature according to the importance and reliability of multi-spectral reflectance, infrared radiance, and buoy pH value to achieve a more accurate spatio-temporal correlation matrix construction.

8. The integrated intelligent satellite perception and processing system for remote control according to claim 6, characterized in that, The LSTM anomaly detection model further adopts a probability density distribution comparison method based on KL divergence to continuously monitor the difference between the model output distribution and the normal mode distribution. When a model deviation is detected, a rollback mechanism is automatically triggered to ensure the stability of the system.

9. The integrated remote intelligent satellite sensing and processing system according to claim 1, wherein The satellite on-orbit communication and remote sensing cooperation algorithm includes: Emergency remote sensing triggers the IoT communication mode: when the infrared payload identifies a temperature anomaly, start the priority task chain: Adjust the multi-spectral camera to the high-gain mode, with the signal-to-noise ratio increased by 6dB; Wake up IoT terminals within a radius of 50km to encrypt and transmit back, using AES - 256 + OQPSK modulation; Fuse data to generate alarm information, with a compression ratio ≥ 10:1, and download it to the ground station.

10. The integrated intelligent satellite perception and processing system for remote control according to claim 9, characterized in that, The satellite on-orbit communication and remote sensing cooperation algorithm further includes: Communication-assisted remote sensing mode: Based on the ISO 19848 standard, parse the ship position message; for ships that illegally turn off the AIS, obtain the radar reflection signal through the IoT terminal; drive the multi-spectral camera to switch to the short-wave infrared band for imaging; combine the SAR imaging results to improve the resolution to 5m and generate a suspicious target report.

Images