Integrated telecontrol and telemetry intelligent satellite sensing and processing system

Abstract

The application discloses a kind of through remote sensing integrated intelligent satellite sensing processing systems, for traditional satellite system, remote sensing and communication function are independent of each other, data dependence backhaul ground processing, there is poor timeliness, low efficiency, insufficient on-board computing power and other problems. Through the hardware system architecture of through remote sensing integration, the AI computing platform of storage and computing integration, multi-modal intelligent processing algorithm and satellite on-orbit through remote sensing collaborative algorithm, through the synergistic effect of each part, realize the efficient remote sensing and communication function of micro-nano satellite under the condition of limited resources. That is, communication and remote sensing load are set on commercial satellite, and through satellite end high-power AI computer, the deep cooperation of multispectral remote sensing imaging, high-sensitivity satellite internet of things communication and on-board AI edge computing, build on-board intelligent processing closed loop, significantly improve satellite comprehensive sensing efficiency.

Description

Technical Field

[0001] This invention belongs to the technical field of spacecraft on-orbit data processing, and particularly relates to an integrated intelligent satellite sensing and processing system for communication and remote sensing. Background Technology

[0002] In recent years, the commercial space industry has developed rapidly. Low-Earth orbit (LEO) micro- and nano-satellites, due to their advantages such as short development cycles, low launch costs, and network deployment capabilities, have shown great application potential in fields such as remote sensing monitoring, Internet of Things (IoT) communication, and space science experiments. However, current technologies face multiple bottlenecks in on-orbit data processing for LEO micro- and nano-satellites: 1. Traditional satellite systems adopt a design that separates remote sensing and communication functions, and data relies on ground backhaul for processing. This results in poor timeliness in scenarios such as disaster monitoring, and the bandwidth of the satellite-to-ground data transmission link is limited, which cannot meet the low latency requirements of various remote sensing applications. 2. Due to the constraints of satellite platform size and power consumption, the frequent interaction between computing and storage units under the traditional von Neumann architecture leads to a 30%-50% waste of system power consumption. Meanwhile, the computing power of existing onboard computing units is generally less than 5 TOPS, making it difficult to support the real-time fusion analysis of multispectral imagery and IoT data. 3. Traditional on-board processing models rely on full parameter updates, with single data transmission exceeding 5MB, severely consuming satellite-to-ground link resources. Although some studies have proposed on-board heterogeneous computing architectures, they still employ a storage-computing separation design, and existing fusion algorithms struggle to achieve sub-pixel-level spatiotemporal registration of multimodal data (positioning deviation > 1 pixel), failing to support accurate identification of abnormal events.

[0003] 4. The application of in-memory computing technology in ground-based AI chips has not yet solved key challenges such as dynamic quantization and compression of lightweight models, resulting in a contradiction between on-board real-time processing capabilities and limited resources. There is an urgent need to build a new system that integrates communication and remote sensing hardware architecture, on-board in-memory computing collaborative optimization and multimodal intelligent analysis, in order to break through the industrial application barriers of commercial micro-nano satellites in real-time perception, autonomous decision-making and resource efficiency. Summary of the Invention

[0004] The purpose of this invention is to provide an integrated intelligent satellite sensing and processing system. By setting up communication and remote sensing payloads on commercial satellites and using a high-performance AI computer on the satellite, multi-spectral remote sensing imaging, high-sensitivity satellite IoT communication, and onboard AI edge computing are deeply coordinated to build an onboard intelligent processing closed loop, which significantly improves the overall sensing efficiency of the satellite.

[0005] To solve the above problems, the technical solution of the present invention is as follows: A remote sensing and communication integrated intelligent satellite sensing and processing system includes: a remote sensing and communication integrated hardware system architecture, an in-memory computing AI computing platform, a multimodal intelligent processing algorithm, and a satellite on-orbit remote sensing and communication collaborative algorithm. Through the synergistic effect of each part, it realizes the efficient remote sensing and communication functions of micro-nano satellites under resource-constrained conditions. The integrated communication and remote control hardware system architecture adopts an integrated design of payload and platform, sharing a power bus, star sensor and data transmission module to improve functional density. The in-memory AI computing platform includes a hardware layer and a software layer. The hardware layer provides computing power to meet the needs of onboard AI processing. The software layer adopts a lightweight model to reduce the model size and improve computing efficiency while ensuring accuracy. The multimodal intelligent processing algorithm is deployed on the in-memory AI computing platform to achieve spatiotemporal alignment for accurate matching of IoT devices and remote sensing data, as well as anomaly detection function; The satellite on-orbit communication and remote sensing collaborative algorithm is deployed on the in-memory AI computing platform to realize the functions of remote sensing data triggering IoT terminal wake-up and communication-assisted remote sensing imaging parameter optimization.

[0006] According to one embodiment of the present invention, the integrated communication and remote sensing hardware system architecture integrates a multispectral imaging payload, an infrared payload, and an Internet of Things communication payload into one unit; The multispectral imaging payload uses a CMOSIS CMV20000 sensor, covering eight bands from 450 to 900 nm, and adopts an off-axis three-mirror optical design to achieve 8 m GSD resolution at a 500 km orbit. The infrared payload covers the 8–12.5 μm infrared spectrum, has a thermal sensitivity of 0.1 K @ 300 K, and supports switching between staring imaging and push-broom modes. The IoT communication payload adopts a software-defined radio architecture, supports LoRaWAN and NB-IoT dual-mode protocols, has a receiving sensitivity of -130dBm, is equipped with a high-gain flat panel antenna, and can simultaneously connect 50 low-power terminals.

[0007] According to one embodiment of the present invention, the integrated communication and remote sensing hardware system architecture adopts a carbon fiber composite bracket and a modular heat pipe heat dissipation system, integrating the multispectral imaging payload, infrared payload and IoT communication payload in a 0.2m³ space, with a total power consumption of ≤ 80W.

[0008] According to an embodiment of the present invention, the hardware layer of the in-memory AI computing platform includes: The NVIDIA Jetson AGX Orin module integrates a 2048-core Ampere GPU and a 12-core ARM CPU, supports 150 TOPS INT8 computing power, and connects to the payload via CameraLink and LVDS buses; The storage module uses a 4TB high-performance SSD with LDPC error correction and supports continuous read / write speeds of 1.5GB / s. The flexible payload interface unit supports multiple payload interfaces, including CameraLink and LVDS, and can connect to multispectral cameras, infrared payloads, and IoT data.

[0009] According to an embodiment of the present invention, the software layer of the in-memory AI computing platform includes: A lightweight model is created by using TensorRT to perform layer fusion and INT8 quantization on the LSTM network, compressing the model size to 18MB. The dynamic memory allocation module, based on ROS 2 Galactic, builds a real-time task scheduler that prioritizes allocating computing power to infrared anomaly detection tasks.

[0010] According to an embodiment of the present invention, the multimodal intelligent processing algorithm includes: The spatiotemporal alignment module matches the GPS coordinates of the IoT terminal with the coordinates of the remote sensing image using the RANSAC algorithm to compensate for attitude jitter error; and constructs a spatiotemporal correlation matrix through feature-level fusion, taking multispectral reflectance, infrared radiance and buoy pH value as inputs, and outputting a 128-dimensional tensor. The LSTM anomaly detection model employs a 3-layer BiLSTM network structure with a self-attention mechanism and is trained using QAT quantization. Furthermore, an incremental learning module, combined with differential update algorithms and gradient compression techniques, optimizes the transmission efficiency of incremental files and reduces the impact of model updates on satellite communication resources.

[0011] According to an embodiment of the present invention, the spatiotemporal alignment module includes: The coordinate matching unit uses the RANSAC algorithm to match the GPS coordinates of the IoT terminal with the coordinates of the remote sensing image to compensate for attitude jitter error. The preprocessing subunit for time series data uses the Kalman filter algorithm to denoise the input time series data, reducing the impact of random noise in the data on subsequent fusion processing. The multi-feature fusion subunit adopts a weighted fusion strategy, which automatically adjusts the weight coefficients of each feature according to the importance and reliability of multispectral reflectance, infrared radiance and buoy pH value, so as to achieve a more accurate spatiotemporal correlation matrix construction.

[0012] According to one embodiment of the present invention, the LSTM anomaly detection model further employs a probability density distribution comparison method based on KL divergence to monitor the difference between the model output distribution and the normal mode distribution in real time. When a model shift is detected, a rollback mechanism is automatically triggered to ensure the stability of the system.

[0013] According to an embodiment of the present invention, the satellite on-orbit communication and remote sensing collaborative algorithm includes: Emergency remote sensing triggers IoT communication mode: When the infrared payload detects an abnormal temperature, a priority task chain is initiated. Adjusting the multispectral camera to its highest gain mode improved the signal-to-noise ratio by 6dB. Encrypted data transmission from IoT terminals within a 50km wake-up radius is achieved using AES-256 + OQPSK modulation. Data is merged to generate alarm information with a compression ratio of ≥ 10:1, and then transmitted to the ground station.

[0014] According to an embodiment of the present invention, the satellite on-orbit communication and remote sensing coordination algorithm further includes: Communication-assisted remote sensing mode: Based on the ISO 19848 standard, it parses ship position messages; for ships that have illegally turned off AIS, it acquires radar reflection signals through IoT terminals; it drives the multispectral camera to switch to shortwave infrared imaging; combined with SAR imaging results, the resolution is improved to 5m, and a suspicious target report is generated.

[0015] Because the present invention adopts the above technical solution, it has the following advantages and positive effects compared with the prior art: The integrated intelligent satellite sensing and processing system in one embodiment of this invention addresses the problems of poor timeliness, low efficiency, and insufficient onboard computing power in traditional satellite systems, where remote sensing and communication functions are independent and data relies on ground-based data transmission for processing. This invention addresses these issues by deploying communication and remote sensing payloads on commercial satellites and utilizing a high-performance AI computer on the satellite to deeply coordinate multi-spectral remote sensing imaging, high-sensitivity satellite IoT communication, and onboard AI edge computing, thus constructing an onboard intelligent processing closed loop and significantly improving the overall satellite sensing efficiency. Attached Figure Description

[0016] Figure 1 This is a block diagram of an integrated intelligent satellite sensing and processing system for communication and remote sensing in one embodiment of the present invention. Detailed Implementation

[0017] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a further detailed explanation of the intelligent satellite sensing and processing system integrating communication and remote sensing proposed in this invention. The advantages and features of this invention will become clearer from the following description and claims.

[0018] Please refer to Figure 1 This embodiment provides an integrated intelligent satellite sensing and processing system, including: an integrated communication and remote sensing hardware system architecture, an in-memory computing AI computing platform, a multimodal intelligent processing algorithm, and a satellite on-orbit communication and remote sensing collaborative algorithm. Through the synergistic effect of each part, the system enables efficient remote sensing and communication functions of micro-nano satellites under resource-constrained conditions. Among them, the integrated hardware system architecture of telemetry and remote sensing adopts an integrated design of payload and platform, sharing a power bus, star sensor and data transmission module to improve functional density; The in-memory AI computing platform comprises a hardware layer and a software layer. The hardware layer provides computing power to meet the needs of onboard AI processing. The software layer employs a lightweight model to reduce model size and improve computing efficiency while ensuring accuracy. Multimodal intelligent processing algorithms are deployed on an in-memory AI computing platform to achieve spatiotemporal alignment for accurate matching of IoT devices and remote sensing data, as well as anomaly detection functionality; The satellite-based on-orbit communication and remote sensing collaborative algorithm is deployed on an in-memory AI computing platform to enable remote sensing data to trigger the wake-up of IoT terminals and to optimize remote sensing imaging parameters with communication assistance.

[0019] Specifically, this integrated remote sensing and communication hardware system architecture deeply integrates multispectral and infrared remote sensing cameras with an IoT communication payload. It adopts an integrated payload and platform design, achieving enhanced functional density through a shared power bus, star sensor, and data transmission module. The multispectral imaging payload uses a CMOSIS CMV20000 sensor, covering eight bands from 450-900nm (including key spectral bands such as coastal blue and red edges), employing an off-axis three-mirror optical design to achieve 8m GSD resolution at a 500km orbit. The infrared payload covers the 8–12.5μm infrared spectral band, with a thermal sensitivity of 0.1K@300K, supporting staring imaging and pushbroom mode switching. The IoT communication payload uses a software-defined radio architecture, supporting LoRaWAN and NB-IoT dual-mode protocols, with a receiving sensitivity of -130dBm, and is equipped with a high-gain flat panel antenna, capable of simultaneously connecting 50 low-power terminals (such as buoys and forest monitoring systems). By integrating three loads into a 0.2m³ space using a carbon fiber composite bracket and a modular heat pipe cooling system, the total power consumption is ≤80W, which is significantly better than the traditional split design.

[0020] The off-axis three-mirror optical design consists of three mirrors. By rationally setting the off-axis amount, tilt amount, spacing, and surface shape of the mirrors, an optical system with unobstructed, chromatic aberration-free, and high-quality imaging can be achieved. Its design is typically based on the initial structure of a coaxial three-mirror system, eliminating central obstruction problems through off-axis field of view or aperture processing, and further optimizing aberrations to achieve diffraction-limited imaging quality. This embodiment uses the off-axis three-mirror optical design for a multispectral imaging payload, employing a common aperture design, i.e., achieving imaging of multiple wavelengths in the same optical path, thereby reducing system size and weight.

[0021] By integrating three loads within a 0.2 m³ space using a carbon fiber composite support and a modular heat pipe cooling system, the system effectively manages heat load while ensuring lightweight construction and efficient heat dissipation. The carbon fiber composite support, due to its lightweight, high strength, and excellent thermal management performance, is suitable for applications requiring high mechanical strength and lightweight design. Through multi-level structural design, carbon fiber, polyaniline nanofibers, and silver nanowires can be integrated into a single composite material, forming a unique "branch-trunk" interlocking micro / nanostructure, significantly improving the material's mechanical and thermal management properties. In practical applications, the strong adhesion of polydopamine can be used to form a branch-trunk interlocking micro / nanostructure of carbon fiber, enhancing the material's mechanical properties. By employing a dual-control strategy that synergistically controls thermal conductivity and infrared radiation, the thermal management performance of the composite material is optimized, making it potentially applicable in thermal insulation and infrared stealth fields.

[0022] Modular heat pipe cooling systems effectively manage heat from multiple heat sources by integrating heat pipes and radiators into a compact module. This design not only improves heat dissipation efficiency but also reduces the overall system complexity and weight. In practical applications, a six-heat pipe cooling system can be selected to provide higher heat dissipation efficiency and better thermal management performance; by rationally arranging the heat pipes, heat can be evenly distributed and effectively transferred to the radiator, thereby improving overall heat dissipation efficiency.

[0023] The in-memory computing AI computing platform is based on an in-memory computing collaborative architecture design for its onboard AI processing unit. Its hardware layer adopts the NVIDIA Jetson AGX Orin module, which integrates a 2048-core Ampere GPU and a 12-core ARM CPU, supporting 150 TOPSINT8 computing power. It connects to the payload via cameralink and LVDS buses. The storage module uses a 4TB high-performance SSD (Seagate Nytro® XF1230) with LDPC error correction, supports continuous read and write speeds of 1.5GB / s, and has an on-orbit lifespan of more than 5 years through partitioned redundancy design. It adopts a flexible payload interface unit that supports multiple payload interfaces such as cameralink, 2711, and LVDS, and can connect to multispectral cameras (2Gbps), infrared payloads (800Mbps), and IoT data (10kbps).

[0024] The software layer of the in-memory AI computing platform adopts a lightweight model, using TensorRT to perform layer fusion and INT8 quantization on the LSTM network, compressing the model size to 18MB (accuracy loss ≤1.8%); it supports dynamic memory allocation, and builds a real-time task scheduler based on ROS 2 Galactic to prioritize the allocation of computing power to infrared anomaly detection tasks.

[0025] Multimodal intelligent processing algorithms are deployed on an in-memory AI computing platform, mainly including: The spatiotemporal alignment module matches the GPS coordinates (WGS84) of the IoT terminal with the coordinates (UTM) of the remote sensing image using the RANSAC algorithm to compensate for attitude jitter error (residual ≤ 0.3 pixels); the feature-level fusion constructs a spatiotemporal correlation matrix, taking multispectral reflectance, infrared radiance and buoy pH value as inputs, and outputting a 128-dimensional tensor; The LSTM anomaly detection model employs a 3-layer BiLSTM network structure (256 hidden nodes) with a self-attention mechanism, trained using QAT quantization. An incremental learning module, combined with differential update algorithms and gradient compression techniques, optimizes the transmission efficiency of incremental files, reducing the resource consumption of satellite communication during model updates. For example, transmitting only the parameter gradients triggered by ground-labeled data (SGD optimizer, learning rate 1e-4) generates a 50MB incremental file.

[0026] Furthermore, the spatiotemporal alignment module includes: The coordinate matching unit uses the RANSAC algorithm to match the GPS coordinates of the IoT terminal with the coordinates of the remote sensing image to compensate for attitude jitter error. The preprocessing subunit for time series data uses the Kalman filter algorithm to denoise the input time series data, reducing the impact of random noise in the data on subsequent fusion processing. The multi-feature fusion subunit adopts a weighted fusion strategy, which automatically adjusts the weight coefficients of each feature according to the importance and reliability of multispectral reflectance, infrared radiance and buoy pH value, so as to achieve a more accurate spatiotemporal correlation matrix construction.

[0027] The 3-layer BiLSTM architecture in the LSTM anomaly detection model includes: 1. Input layer: Receives sequence data of length 60 frames. Each frame contains multimodal features such as multispectral reflectance, infrared radiance and buoy pH value. After preprocessing, it forms a standardized input vector to ensure data dimensionality consistency and reasonable numerical range, which facilitates stable model training and efficient computation.

[0028] 2. First layer BiLSTM: Forward LSTM: This method processes data from each time step of the input sequence sequentially, calculating the hidden state from the start to the end of the sequence. The hidden state dimension is 256, and a non-linearity is introduced through the tanh activation function, enabling the model to learn complex non-linear feature relationships within the sequence. The hidden state ht^forward at time t is jointly determined by the current input xt and the hidden state ht-1^forward at time t-1, effectively capturing forward-looking time-dependent features of the sequence, such as early signs of fire spread trends when monitoring forest fires.

[0029] Backward LSTM: The reverse of the forward LSTM, it computes the hidden state backward from the end of the sequence to the beginning, also with a dimension of 256. The hidden state ht^backward of the backward LSTM depends on the current input xt and the hidden state ht+1^backward at time t+1, focusing on extracting retrospective features of the sequence, such as considering the supplementary explanation of previous anomalous fluctuations by subsequent data when analyzing ocean buoy data. Finally, this layer concatenates the forward and backward hidden states to form a 512-dimensional output, providing richer bidirectional sequence information for upper-layer networks.

[0030] 3. Second layer BiLSTM: Input: Directly receive the 512-dimensional output sequence after the first layer of BiLSTM is spliced, without the need for additional dimensionality reduction or transformation, ensuring complete information transmission.

[0031] Forward and Backward LSTM: The structure is similar to the first layer, and the hidden state dimension remains at 256. It extracts abstract features from the sequence at a higher level, further uncovering deeper bidirectional correlations and patterns in the data. For example, when processing IoT communication data, it can capture the potential connection between abnormal device data transmission and subsequent repair processes.

[0032] 4. Third layer BiLSTM Input: The output dimensions and sequence structure of the first two layers are used to ensure the coherence of the model structure and the consistency of feature transfer.

[0033] Forward and Backward LSTM: Continuously deepens the bidirectional feature mining of the sequence, with the hidden state dimension stabilized at 256. At this point, the model has performed multi-layer abstraction of the input sequence, enabling it to capture more hidden and complex abnormal feature patterns, such as when monitoring illegal maritime activities, it can correlate subtle changes in ship behavior in multiple frames of data with abnormal combinations of surrounding environmental factors.

[0034] Output: The concatenated 512-dimensional hidden state sequence serves as the final output of the BiLSTM part, fully containing the bidirectional deep features of the input sequence in the time dimension.

[0035] Self-attention mechanism module 1. Input Transformation: The 512-dimensional hidden state sequence output by the third layer BiLSTM is passed through a fully connected layer and projected onto the attention space to obtain the query vector (Q), key vector (K), and value vector (V). The dimensions of Q, K, and V can be set according to the computational resources and model performance requirements. Generally, they can be set to 128 dimensions to reduce computational complexity while retaining sufficient expressive power.

[0036] 2. Attention Score Calculation: A scaled dot product attention mechanism is used to calculate the dot product of the transposes of Q and K, yielding the attention score matrix. To prevent the dot product from becoming too large and causing gradient vanishing or exploding, the dot product is scaled by dividing by (the dimensions of Q and K). Each element in the attention score matrix represents the correlation weight between two corresponding positions in the sequence, such as the degree of correlation between remote sensing image features at different time points and IoT monitoring data when analyzing multispectral and infrared fusion data.

[0037] 3. Softmax Activation: The softmax function is applied row-wise to the attention score matrix, transforming the scores into a weight matrix in probability distribution form. The row sum of the weight matrix is ​​1, ensuring the numerical stability of subsequent weighted summation operations, highlighting key feature points in the sequence, and suppressing irrelevant information.

[0038] 4. Weighted Summation: A weighted dot product is performed between the weight matrix processed by softmax and the value vector V to obtain the output sequence of the self-attention mechanism. This sequence retains the bidirectional features extracted by BiLSTM, and the weights are redistributed according to the correlation of features within the sequence to enhance the expression of key features, making the model more sensitive to abnormal patterns, and maintaining the consistency between the output dimension and the input dimension.

[0039] Fusion and Output Layer 1. Feature Fusion: The sequence processed by the self-attention mechanism is fused with the output sequence of the third-layer BiLSTM through deep feature fusion. A fully connected layer is used to perform a linear transformation on the vector concatenated from the two sequences along their feature dimensions, integrating the sequence features of the BiLSTM with the spatial feature enhancement information of the self-attention mechanism to obtain a fused feature sequence. The ReLU activation function is chosen to introduce non-linearity and improve the model's ability to fit complex feature relationships.

[0040] 2. Output Layer: The fused feature sequence passes through the last fully connected layer and is mapped to the output dimension required for anomaly detection. Depending on the specific task requirements, for binary anomaly detection (abnormal / normal), a single node is output, using the sigmoid activation function to compress the output value to the (0,1) interval, representing the probability that the input sequence is an anomaly; for multi-class anomaly classification, the number of output nodes corresponds to the number of anomaly categories, and the softmax activation function is used to obtain the probability distribution of each anomaly category, assisting the system in accurate anomaly judgment and classification.

[0041] Before performing QAT quantization-aware training, the weights of the BiLSTM and self-attention mechanism modules are randomly initialized, and the quantization parameters (scale and zero point) are initialized to 1.0 and 0.0, respectively, laying the foundation for subsequent quantization-aware training.

[0042] Its training process includes the following steps: Forward propagation: The input training data sequence passes through three layers of BiLSTM and a self-attention mechanism module. During the calculation of the activation values ​​in each module, pseudo-quantization is performed to simulate the effects of quantization. The model's output and loss value are recorded. The loss function is selected based on the anomaly detection task type; for example, binary cross-entropy loss is used for binary classification tasks, and cross-entropy loss is used for multi-class classification tasks. An L2 regularization term can also be added to prevent overfitting.

[0043] Backpropagation and parameter update: Based on the loss value, the gradient of the model parameters is calculated using the backpropagation algorithm. When calculating the gradient of the pseudo-quantization operation node, the STE method is used to directly pass the gradient to the floating-point parameters before quantization. Then, an optimization algorithm (such as the Adam optimizer) is used to update the floating-point parameters of the model, and the quantization parameters (scale and zero point) are updated according to the histogram data of the activation values, so that the quantization parameters gradually adapt to the changes in the model parameters and optimize the quantization effect.

[0044] Iterative optimization: Repeat the forward and backward propagation processes, continuously adjusting model parameters and quantization parameters to reduce the loss value while improving the accuracy of the quantized model. The training process continues for multiple epochs until the model's performance metrics (such as accuracy, recall, F1 score, etc.) on the validation set reach a satisfactory level, and the performance gap between the quantized and unquantized models is controlled within a reasonable range (e.g., accuracy loss ≤ 1.8%).

[0045] In this embodiment, the incremental learning module preprocesses and extracts features from newly acquired labeled data on the ground to construct a training sequence. Using the QAT quantization-aware training method, the model is incrementally trained, calculating and updating only the model parameters that have changed due to the introduction of new data, generating a differential update package. This differential update package contains the updated parameter gradients (calculated using the SGD optimizer with a learning rate of 1e-4) and quantized parameter update information, with a size controlled to approximately 50MB to reduce satellite communication bandwidth usage.

[0046] After receiving the differential update packet, the satellite temporarily stores the updated data using the storage module of the onboard AI computing platform. It then calls the update program to apply the parameter gradients from the differential update packet to the current model parameters, updating the model weights and quantization parameters. During the update process, a block update strategy is employed, dividing the model parameters into multiple small blocks. Each block is updated sequentially, and the completeness and correctness of the updated block are verified to prevent the entire model from failing due to update errors.

[0047] This embodiment analyzes the computational requirements of an LSTM anomaly detection model based on the 150 TOPS INT8 computing power of the NVIDIA Jetson AGX Orin module in the in-memory AI computing platform. During model training, QAT quantization-based training is used to ensure that the model's computational requirements after INT8 quantization match the hardware platform's capabilities. During model runtime, the platform's dynamic memory allocation mechanism is utilized to allocate a dedicated memory area for the model, avoiding memory conflicts with other tasks and ensuring the smoothness of the model's inference process. For example, when processing a 60-frame input sequence, the GPU computing power and memory bandwidth required for model inference are estimated, and corresponding resources are reserved in advance to ensure that the model can complete the anomaly detection task within a specified time (e.g., within 1 second).

[0048] Storage and caching are optimized: model parameters are stored in high-performance SSD storage modules, while a cache area is set up in the memory of the AI ​​computing platform. When the model performs inference, frequently used parameters and intermediate results are read from the cache first, reducing frequent read and write operations on the SSD, extending the lifespan of the storage module, and improving model running efficiency. For example, the core parameters such as the BiLSTM weight matrix and the query, key, and value transformation matrices of the self-attention mechanism are cached in memory, while other auxiliary parameters are loaded from the SSD as needed, optimizing the data access process.

[0049] The LSTM anomaly detection model further employs a probability density distribution comparison method based on KL divergence to monitor the difference between the model output distribution and the normal mode distribution in real time. When a model shift is detected, a rollback mechanism is automatically triggered to ensure the stability of the system.

[0050] The KL divergence is an asymmetric measure of the difference between two probability distributions P and Q. In this scheme, it is used to measure the difference between the current probability distribution Q output by the model and the probability distribution P under normal conditions. Its calculation formula is as follows: This embodiment monitors whether there are abnormal changes in the model output in real time by calculating the KL divergence between the distribution Q of the model's output each time and the normal pattern distribution P. The normal pattern distribution P can be obtained by statistical analysis of the model's output on a large amount of normal data, or by recording the model's output distribution on the validation set during the model training phase.

[0051] During the execution of multimodal intelligent processing software algorithms, the output data is collected immediately after each model outputs a multidimensional tensor. Due to the potentially high dimensionality and complexity of the model output, appropriate preprocessing is required for subsequent probability density distribution estimation. Preprocessing may include data normalization, dimensionality reduction, and other operations to ensure the data is suitable for probability density estimation methods and to improve computational efficiency.

[0052] By employing appropriate probability density estimation methods, such as kernel density estimation (KDE) or histogram methods, the probability density distribution of the preprocessed model output data is estimated to obtain the current model output distribution Q.

[0053] By statistically analyzing the KL divergence between the model's output distribution under normal conditions and the normal pattern distribution during the model training or system validation phases, a reasonable judgment threshold can be determined. For example, the KL divergence values ​​of multiple normal samples can be calculated, and the mean plus a certain number of standard deviations can be used as the threshold, thereby detecting model shift events in a timely manner while ensuring a low false alarm rate.

[0054] During system operation, after each calculation of the KL divergence, it is immediately compared with a pre-set threshold. If the KL divergence exceeds the threshold, the model is considered to have shifted, triggering the corresponding rollback mechanism; otherwise, the model output is considered normal, and subsequent processing tasks continue.

[0055] When a model offset is detected, the system immediately triggers a rollback mechanism. First, it records the current model's state information, including the input data, output results, and intermediate states at the time of offset detection, to facilitate subsequent analysis and investigation of the cause of the offset. Then, it retrieves the parameters and structure of the previous stable version of the model from the model version management system and loads it into the AI ​​computing platform, replacing the currently offset model. During model recovery, it ensures the compatibility of the newly loaded model with the current system environment, including checking whether the model matches the hardware platform, software framework, and interfaces with other system modules.

[0056] In this embodiment, the fusion of terrestrial IoT data and satellite remote sensing data further includes: Data alignment: Interpolate IoT in-situ data (pH, temperature, etc.) to the remote sensing image pixel grid to generate a 10km×10km spatiotemporal cube; Feature extraction: A 3D convolutional network is used to extract the spatiotemporal correlation features of multi-source data and output a 128-dimensional embedding vector. Joint inference: Time series anomaly detection based on gated recurrent units (GRU) for multimodal data, with a false alarm rate of ≤2%.

[0057] The satellite on-orbit communication and remote sensing coordination algorithm in this embodiment includes: Emergency remote sensing triggers IoT communication mode: When the infrared payload detects an abnormal temperature (ΔT≥5K / 10 minutes), the priority task chain is activated. Adjusting the multispectral camera to its highest gain mode improved the signal-to-noise ratio by 6dB. Encrypted data transmission from IoT terminals within a 50km wake-up radius is achieved using AES-256 + OQPSK modulation. Data is merged to generate alarm information with a compression ratio of ≥ 10:1, and then transmitted to the ground station.

[0058] The satellite's on-orbit communication and remote sensing collaborative algorithm further includes: Communication-assisted remote sensing mode: Based on the ISO 19848 standard, the system parses ship position messages; for ships that have illegally turned off AIS, it acquires radar reflection signals (RCS≥1000m²) through IoT terminals; it drives the multispectral camera to switch to shortwave infrared (SWIR) band imaging; combined with SAR imaging results, the resolution is improved to 5m, and a suspicious target report is generated.

[0059] The aforementioned integrated intelligent satellite sensing and processing system can achieve high-efficiency target detection against sparse ocean backgrounds, specifically as follows: Background modeling: A seawater reflectance benchmark library was constructed using historical multispectral data, and anomalous regions were detected by Mahalanobis distance (threshold ≥ 3σ). Multi-source verification: When remote sensing detects a suspected target (such as an oil slick or a ship), it automatically wakes up the nearest IoT buoy to perform in-situ sampling verification; Compressed transmission: For confirmed targets, only the ROI data of the target area is transmitted (compression ratio ≥10:1), and an IoT verification tag is attached.

[0060] The integrated remote sensing and communication intelligent satellite sensing and processing system in this embodiment is suitable for remote sensing and communication collaborative intelligent processing of commercial micro-nano satellite platforms, and is especially suitable for multimodal data fusion and real-time intelligent analysis scenarios of micro-nano satellites (100kg class).

[0061] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they shall still fall within the protection scope of the present invention.

Claims

1. A remote sensing and intelligent satellite sensing and processing system integrating communication and remote control, characterized in that, include: The integrated hardware system architecture for remote sensing and communication and the AI ​​computing platform are connected through a payload interface. Through the synergistic effect of the two, the micro-nano satellite can achieve efficient remote sensing and communication functions under resource-constrained conditions. The integrated hardware system architecture of the communication and remote sensing system adopts an integrated design of payload and platform, integrating multispectral imaging payload, infrared payload and Internet of Things communication payload into one, sharing a power bus, star sensor and data transmission module to improve functional density. The AI ​​computing platform comprises a hardware layer and a software layer. The hardware layer uses an NVIDIA Jetson AGX Orin module, integrating a 2048-core Ampere GPU and a 12-core ARM CPU, supporting 150 TOPS INT8 computing power, and is connected to the payload via CameraLink and LVDS bus. The software layer uses a lightweight LSTM model based on TensorRT quantization to reduce model size and improve computational efficiency while maintaining accuracy. The AI ​​computing platform deploys a multimodal intelligent processing algorithm and a satellite on-orbit communication and remote sensing coordination algorithm. The multimodal intelligent processing algorithm includes: a spatiotemporal alignment module, which matches the GPS coordinates of the IoT terminal with the coordinates of the remote sensing image through the RANSAC algorithm to compensate for attitude jitter error and achieve sub-pixel-level spatiotemporal registration; and an LSTM anomaly detection model, which adopts a 3-layer BiLSTM + self-attention mechanism network structure and achieves anomaly detection function through QAT quantization perception training. The satellite on-orbit communication and remote sensing collaborative algorithm enables remote sensing data to trigger the wake-up of IoT terminals and facilitates communication-assisted optimization of remote sensing imaging parameters.

2. The integrated intelligent satellite sensing and processing system for communication and remote sensing as described in claim 1, characterized in that, The multispectral imaging payload uses a CMOSIS CMV20000 sensor, covering eight bands from 450 to 900 nm, and adopts an off-axis three-mirror optical design to achieve 8 m GSD resolution at a 500 km orbit. The infrared payload covers the 8–12.5 μm infrared spectrum, has a thermal sensitivity of 0.1 K @ 300 K, and supports switching between staring imaging and push-broom modes. The IoT communication payload adopts a software-defined radio architecture, supports LoRaWAN and NB-IoT dual-mode protocols, has a receiving sensitivity of -130dBm, is equipped with a high-gain flat panel antenna, and can simultaneously connect 50 low-power terminals.

3. The integrated intelligent satellite sensing and processing system for communication and remote sensing as described in claim 2, characterized in that, The integrated hardware system architecture of the communication and remote sensing system adopts a carbon fiber composite bracket and a modular heat pipe cooling system, integrating the multispectral imaging payload, infrared payload and IoT communication payload in a 0.2m³ space, with a total power consumption of ≤ 80W.

4. The integrated intelligent satellite sensing and processing system for communication and remote sensing as described in claim 1, characterized in that, The hardware layer of the AI ​​computing platform also includes: The storage module uses a 4TB high-performance SSD with LDPC error correction and supports continuous read / write speeds of 1.5GB / s. The flexible payload interface unit supports multiple payload interfaces, including CameraLink and LVDS, and can connect to multispectral cameras, infrared payloads, and IoT data.

5. The integrated intelligent satellite sensing and processing system for communication and remote sensing as described in claim 1, characterized in that, The software layer of the AI ​​computing platform also includes: The dynamic memory allocation module, based on ROS 2 Galactic, builds a real-time task scheduler that prioritizes allocating computing power to infrared anomaly detection tasks.

6. The integrated intelligent satellite sensing and processing system for communication and remote sensing as described in claim 1, characterized in that, The spatiotemporal alignment module also constructs a spatiotemporal correlation matrix through feature-level fusion, taking multispectral reflectance, infrared radiance and buoy pH value as inputs, and outputting a 128-dimensional tensor. The LSTM anomaly detection model also optimizes the transmission efficiency of incremental files by combining incremental learning modules with differential update algorithms and gradient compression techniques, thereby reducing the impact of model updates on satellite communication resources.

7. The integrated intelligent satellite sensing and processing system for communication and remote sensing as described in claim 6, characterized in that, The spatiotemporal alignment module includes: The coordinate matching unit uses the RANSAC algorithm to match the GPS coordinates of the IoT terminal with the coordinates of the remote sensing image to compensate for attitude jitter error. The preprocessing subunit for time series data uses the Kalman filter algorithm to denoise the input time series data, reducing the impact of random noise in the data on subsequent fusion processing. The multi-feature fusion subunit adopts a weighted fusion strategy, which automatically adjusts the weight coefficients of each feature according to the importance and reliability of multispectral reflectance, infrared radiance and buoy pH value, so as to achieve a more accurate spatiotemporal correlation matrix construction.

8. The integrated intelligent satellite sensing and processing system for communication and remote sensing as described in claim 6, characterized in that, The LSTM anomaly detection model further employs a probability density distribution comparison method based on KL divergence to monitor the difference between the model output distribution and the normal mode distribution in real time. When a model shift is detected, a rollback mechanism is automatically triggered to ensure the stability of the system.

9. The integrated intelligent satellite sensing and processing system for communication and remote sensing as described in claim 1, characterized in that, The satellite on-orbit communication and remote sensing coordination algorithm includes: Emergency remote sensing triggers IoT communication mode: When the infrared payload detects an abnormal temperature, a priority task chain is initiated. Adjusting the multispectral camera to its highest gain mode improved the signal-to-noise ratio by 6dB. Encrypted data transmission from IoT terminals within a 50km wake-up radius is achieved using AES-256 + OQPSK modulation. Data is merged to generate alarm information with a compression ratio of ≥ 10:1, and then transmitted to the ground station.

10. The integrated intelligent satellite sensing and processing system for communication and remote sensing as described in claim 9, characterized in that, The satellite on-orbit communication and remote sensing collaborative algorithm further includes: Communication-assisted remote sensing mode: Based on the ISO 19848 standard, it parses ship position messages; for ships that have illegally turned off AIS, it acquires radar reflection signals through IoT terminals; it drives the multispectral camera to switch to shortwave infrared imaging; combined with SAR imaging results, the resolution is improved to 5m, and a suspicious target report is generated.

Images