A lightweight cloud detection method, system and device for a satellite-borne FPGA and a storage medium

Abstract

This invention discloses a lightweight cloud detection method, system, device, and storage medium for spaceborne FPGAs. Addressing the challenges of deploying large remote sensing models on constrained hardware and low detection accuracy in complex scenes, this method dynamically segments and quantizes multispectral remote sensing images using INT8. The images are then input into a lightweight network for hardware-aware reconstruction. The original activation function is replaced with a pure combinational logic ReLU6 to eliminate fixed-point overflow and latency. A Ghost Module is used to decouple feature extraction, overcoming off-chip memory access bottlenecks. A cascaded CBAM attention mechanism effectively suppresses high-albedo background interference, inferring the cloud mask. Finally, based on cloud cover statistics, invalid data is autonomously physically erased or compressed and downloaded. This invention achieves cross-layer coupling between the algorithm and the underlying hardware, significantly reducing computational resource consumption while maintaining detection accuracy for complex backgrounds such as thin clouds and snowfields. This releases orders of magnitude more space-to-ground communication bandwidth, meeting the real-time on-orbit processing needs of micro- and nano-satellites.

Description

Technical Field

[0001] This invention relates to the fields of remote sensing image processing, deep learning, and on-board edge computing technology for micro-nano satellites. Specifically, it relates to a lightweight instance segmentation cloud detection method, system, device, and storage medium for platforms with extremely limited computing resources, such as onboard FPGAs. Background Technology

[0002] With the rapid evolution of high-resolution Earth observation systems, the amount of spaceborne remote sensing data has surged exponentially, placing enormous bandwidth pressure on space-to-ground communication links. Statistics show that approximately two-thirds of the Earth's surface is covered by clouds year-round. In the traditional "store first, then download, then process on the ground" model, a large number of invalid images obscured by clouds not only consume limited spaceborne storage capacity but also waste expensive downlink bandwidth, resulting in significant delays in acquiring high-value remote sensing data. Therefore, deploying edge computing nodes on satellites to achieve "on-board cloud detection and real-time removal of invalid data" has become a core requirement for next-generation intelligent micro / nano satellites.

[0003] However, porting existing cloud detection algorithms to real spaceborne platforms still faces a significant technological hurdle. Mainstream technologies suffer from the following major drawbacks:

[0004] 1. Traditional physical thresholding methods have poor generalization ability and low accuracy:

[0005] Traditional methods for detecting clouds on satellites often employ threshold segmentation algorithms based on spectral differences (such as ACCA and Fmask). These methods are extremely inflexible and are prone to large-scale false detections when dealing with high-altitude snow, deserts, or man-made objects with high albedo in cities due to overlapping spectral features. Furthermore, for thin clouds with semi-transparent edges, fixed thresholds often fail to extract complete cloud boundaries, leading to serious missed detections.

[0006] 2. Conventional deep learning networks are extremely incompatible with spaceborne FPGA hardware:

[0007] In recent years, methods based on convolutional neural networks (such as U-Net and the standard YOLO series) have achieved extremely high accuracy in remote sensing cloud detection, demonstrating excellent robustness in complex scenarios. However, their deployment on spaceborne platforms has encountered fatal physical constraints. Real-world microsatellites in orbit are limited by extremely stringent size, weight, and power consumption (SWaP) constraints, and their onboard computing cores mostly use field-programmable gate arrays (FPGAs) with extremely low power consumption (typically below 10W). Their on-chip static random access memory (BRAM) capacity is extremely small, and the number of underlying computing units is extremely limited.

[0008] Taking the mainstream YOLOv8 architecture as an example, its widely used SiLU activation function involves complex nonlinear exponential operations ( FPGA underlying logic is extremely unfriendly to floating-point exponentiation. If this activation function is forcibly synthesized, it requires a massive amount of lookup tables (LUTs) for approximate fitting. This will not only exhaust logic resources instantly, but also disrupt the pipeline throughput of data flow and introduce fatal inference latency.

[0009] Meanwhile, standard convolution operations in conventional networks generate a large number of redundant feature maps, and dense matrix multiplications produce a huge amount of floating-point operations (FLOPs). With the limited memory bandwidth of FPGAs, this can easily lead to frequent off-chip reads and writes, causing the system inference frame rate to fall far short of the requirements for real-time processing.

[0010] In summary, existing technologies struggle to strike a balance between high-precision detection and the extreme resource constraints of spaceborne hardware. There is an urgent need in this field for a novel, lightweight cloud detection solution that co-optimizes algorithms and hardware to eliminate complex computations that are incompatible with the underlying hardware, significantly reduce the number of parameters and computational overhead, and achieve integrated real-time processing of sensing and computing on micro- and nano-satellites. Summary of the Invention

[0011] To address the challenges of deploying existing high-precision deep learning models on spaceborne FPGA hardware with extremely limited computing resources, and the low detection accuracy of traditional physical thresholding methods in complex scenarios, this invention provides a lightweight cloud detection method, system, device, and storage medium for spaceborne FPGAs. By performing deep hardware-aware reconstruction of existing instance segmentation networks from an algorithm-hardware collaborative perspective, it eliminates complex nonlinear operations that are extremely unfriendly to the underlying hardware. While ensuring cloud detection accuracy in complex backgrounds (such as bright snowfields and thin clouds), it significantly reduces the number of model parameters and computational complexity, thereby meeting the stringent requirements of real-time sensing and computing integration in orbit for micro-nano satellites.

[0012] According to one aspect of the present invention, a lightweight cloud detection method for spaceborne FPGAs is provided. The method operates on a spaceborne heterogeneous computing system comprising a processing system and a programmable logic unit, and includes the following steps: The processing system acquires multispectral remote sensing images, performs dynamic slicing and 8-bit integer fixed-point quantization preprocessing to generate image blocks, and pushes the image blocks into the programmable logic terminal via the AXI4-Stream bus. The lightweight instance segmentation network deployed in the programmable logic terminal performs parallel feature extraction and mask inference on the image patch and outputs a prediction tensor; The processing system performs mask decoding on the prediction tensor to generate a binarized cloud mask, calculates the cloud coverage rate of the current image patch based on the binarized cloud mask, and compares the cloud coverage rate with a preset threshold. If the cloud coverage rate is greater than or equal to the preset threshold, the processing system directly erases the physical address cache of the image block in the external memory through the file system; if the cloud coverage rate is less than the preset threshold, the processing system schedules the hardware compression engine to encode the image block and push it into the cache queue of the satellite-to-ground downlink transmitter.

[0013] As a further technical solution, the lightweight instance segmentation network uses YOLOv8n-seg as the baseline architecture for hardware-aware reconstruction and includes the following hardware acceleration circuits deployed in the programmable logic terminal: The activation function after all convolutional layers is configured as a ReLU6 activation circuit based on pure combinational logic, which is instantiated in the underlying hardware mapping as a cascaded digital comparator and multiplexer with zero clock cycle delay. The backbone network is constructed using GhostModule to decouple feature extraction into two independent stages: intrinsic feature generation and phantom feature generation. The multiply-accumulator array inside the programmable logic terminal performs standard convolution operations to generate intrinsic feature maps, while the depthwise separable convolution accelerated pipeline built based on shift registers performs linear transformation operations to generate phantom feature maps. The intrinsic feature maps and phantom feature maps are concatenated along the channel dimension at the data write-out end of the on-chip static random access memory. Between the spatial pyramid pooling layer and the instance segmentation head of the backbone network, a convolutional block attention module is cascaded and embedded to focus features along the channel dimension and the spatial dimension. The channel attention inference uses a parallel accumulating tree circuit to synchronously perform global max pooling and average pooling, while the spatial attention inference generates a two-dimensional spatial weight mask through a hardware convolutional window.

[0014] As a further technical solution, the dynamic block cutting and 8-bit integer fixed-point quantization preprocessing includes: The original remote sensing image is cropped into image patches of a preset spatial dimension using a dynamic sliding window mechanism, and a preset overlap step size is set between adjacent image patches. Perform symmetric or asymmetric 8-bit integer quantization on the image block to be detected, and push the quantized image data stream into the programmable logic terminal.

[0015] As a further technical solution, the mask decoding includes: generating the binarized cloud mask matrix through nonmaximum suppression and inverse quantization decoding; the cloud coverage rate is calculated by traversing the mask matrix using a hardware accumulator to calculate the total number of cloud pixels.

[0016] As a further technical solution, the lightweight instance segmentation network undergoes training followed by quantization before deployment. The 8-bit integer fixed-point quantization is matched with the upper limit hard truncation of the ReLU6 activation circuit to eliminate the risk of data overflow during fixed-point inference.

[0017] According to one aspect of this specification, a lightweight cloud detection system for spaceborne FPGAs is provided, deployed on a micro / nano satellite on-orbit platform, the system comprising: Image acquisition and preprocessing module: configured to receive high-speed image bitstreams from onboard optical sensors and perform image slicing, 8-bit integer fixed-point quantization, and data stream format conversion operations; the image acquisition and preprocessing module is deployed in the processing system area of ​​the onboard heterogeneous computing chip; Onboard hardware inference module: Deployed in the programmable logic area of ​​the same onboard heterogeneous computing chip, it has embedded a hardware acceleration IP core of lightweight instance segmentation network and a quantized model parameter bit stream, which is used to perform feature extraction, mask inference and mask decoding, and output a binary cloud mask. Onboard adaptive decision-making module: Deployed in the processing system area, configured to calculate cloud coverage based on the binarized cloud mask, and autonomously schedule the data physical erasure or compression transmission process of the satellite-to-ground communication link; wherein, the data physical erasure includes directly erasing the physical address cache of invalid data in the external memory through the file system, and the compression transmission process includes scheduling the hardware compression engine to encode and push it into the cache queue of the satellite-to-ground downlink transmitter.

[0018] As a further technical solution, the system is deployed based on a heterogeneous computing architecture of system-on-chip; the image acquisition and preprocessing module and the on-board adaptive decision-making module are deployed in the processing system area; and the on-board hardware inference module is deployed in the programmable logic area.

[0019] As a further technical solution, the hardware acceleration IP core in the onboard hardware inference module includes: a cascaded digital comparator and multiplexer for executing the ReLU6 activation circuit, a multiply-accumulator array and a depth-separable convolution acceleration pipeline based on shift registers for executing the GhostModule, and a parallel accumulating tree circuit and a hardware convolution window for executing the convolution block attention module.

[0020] According to one aspect of the present invention, a computer device is provided, comprising: a computing unit and a field-programmable gate array (FPGA); and a memory storing computer program instructions and a bitstream file for configuring hardware circuits; when the program instructions and the bitstream file are loaded and executed by the computing unit and the FPGA, the lightweight cloud detection method for spaceborne FPGAs is implemented.

[0021] According to one aspect of the present invention, a computer-readable storage medium is provided, which stores computer program instructions and a bitstream file for configuring FPGA hardware circuitry. When the program instructions and bitstream file are loaded and executed by the computing unit and field-programmable gate array of an onboard processing system, all steps of the lightweight cloud detection method for onboard FPGA are implemented.

[0022] Compared with existing technologies, this invention is deeply compatible with the physical characteristics of the underlying heterogeneous computing hardware of micro-nano satellites, realizing cross-layer coupling between algorithms and hardware, and achieving the following significant progress and unexpected technical effects:

[0023] 1. Completely eliminate the hardware pinpointing problem and achieve ultra-low latency inference.

[0024] Technical relevance: ReLU6, a pure combinational logic implementation, replaces the complex exponential operations. The SiLU activation function.

[0025] Substantial Benefits: When porting existing large-scale remote sensing models to FPGAs, fitting nonlinear exponential operations instantly exhausts extremely valuable DSP slices and lookup tables (LUTs) resources, and incurs computational delays over multiple clock cycles. The replacement strategy of this invention reduces the physical circuitry to a simple two-stage digital comparator logic, achieving nonlinear mapping with zero clock cycle delay. Simultaneously, the upper limit hard truncation characteristic of ReLU6 perfectly matches the 8-bit (INT8) quantization range, fundamentally eliminating the activation value overflow problem that easily occurs in continuous multiplication and addition operations of the underlying data stream, significantly improving the single-event upset stability of spaceborne equipment in strong radiation environments.

[0026] 2. Breaking through the bottleneck of spaceborne storage, achieving a leapfrog improvement in energy efficiency.

[0027] Technical relevance: The introduction of the Ghost Module decouples global dense convolution into "few-channel intrinsic extraction + shift register linear mapping".

[0028] Practical benefits: Dense matrix multiplication of high-resolution remote sensing images generates massive off-chip memory accesses, easily hitting the FPGA's "memory wall." The feature decoupling mechanism of this invention significantly reduces the number of model parameters by 26.1% and floating-point operations (FLOPs) by 16.7% without sacrificing the effective receptive field. In real-world hardware mapping, this algorithm-level reduction dramatically decreases the frequency of read / write operations to onboard DDR memory, greatly freeing up memory bandwidth. This allows the system to maintain a high frame rate (FPS) throughput for real-time on-orbit processing even with a stringent power consumption budget of less than 10W.

[0029] 3. Zero computing power burden resolves "snow cloud obfuscation," significantly enhancing robustness in complex scenarios.

[0030] Technical relevance: A spatial and channel-based dual-dimensional CBAM attention mechanism is cascaded after the SPPF layer.

[0031] Practical Benefits: Addressing the industry pain points of traditional physical thresholding methods, which are prone to false detections in high-altitude snow cover, brightly lit man-made objects in cities, and missed detections of thin clouds, this invention utilizes the massive computing power saved by a lightweight backbone network to perfectly compensate for the minimal storage overhead of the CBAM module (only an increase of 0.01M parameters). During physical inference, this mechanism is like applying a "dynamic voltage gain" to the cloud texture region on a digital circuit, effectively suppressing the activation response of high-albedo background features. Experimental verification shows that this model significantly reduces the false detection rate in heavily disturbed snow scenes, and the extracted semi-transparent thin cloud boundaries are sharper and more closely resemble the real shape.

[0032] 4. Construct an adaptive decision-making closed loop, releasing satellite-to-ground transmission bandwidth by orders of magnitude.

[0033] Technology Relevance: A heterogeneous processing pipeline based on SoC architecture design, featuring "dynamic scheduling and autonomous erasure on the PS side + high-speed inference on the PL side".

[0034] Substantial Benefits: This invention breaks away from the inefficient open-loop operation mode of traditional microsatellites that involves "full storage first, then blind downloading." The integrated sensing and computing architecture of this invention places cloud cover assessment (Rcloud) and cleansing decisions directly at the very end of sensor acquisition. By erasing low-value image blocks with excessive cloud cover in real time at the physical memory level, over 60% of invalid remote sensing payloads are precisely filtered from the data source. This not only significantly extends the lifespan of onboard solid-state storage but also allows limited space-to-ground communication bandwidth to be fully allocated to high-value clear-sky images, improving the overall response speed and economic efficiency of the Earth observation system by an order of magnitude. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a schematic diagram of the overall process of the lightweight cloud detection method provided in the embodiments of the present invention.

[0037] Figure 2 This is a hardware deployment architecture diagram of a lightweight cloud detection system for spaceborne FPGA provided in an embodiment of the present invention.

[0038] Figure 3 This is a diagram showing the internal topology and hardware mapping of a lightweight network (Ghost-RC-YOLO) provided in an embodiment of the present invention.

[0039] Figure 4 This is a comparison chart of the convergence curves of the model provided in the embodiments of the present invention during the ground training phase.

[0040] Figure 5 This is a comparison chart of the cloud detection performance of different models provided in the embodiments of the present invention in complex remote sensing scenarios. Detailed Implementation

[0041] The terms “comprising” and “having”, and any variations thereof, in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, product, or apparatus that includes a series of steps or units, not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form new technical solutions. Such combinations are not bound by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0043] This invention provides a lightweight cloud detection method for spaceborne FPGAs. The method relies on a spaceborne heterogeneous computing system comprising a processing system (PS-side) and programmable logic (PL-side). See reference [link to relevant documentation]. Figure 1 As shown, the specific steps include:

[0044] Step S1: On-board acquisition, dynamic tiling, and hardware-level quantization preprocessing of multispectral remote sensing images

[0045] Specifically, step S1 includes:

[0046] S1.1 Dynamic Sliding Window Segmentation: The PS-side controls the onboard optical sensor to capture the raw high-resolution remote sensing image bitstream in real time, and temporarily stores it in the onboard external double data rate synchronous dynamic random access memory (DDR) via the direct memory access (DMA) controller. Considering the capacity limit of the FPGA's on-chip static random access memory (BRAM), a dynamic sliding window mechanism is used to crop the raw image into segments with spatial dimensions of [missing information]. (For example The set of image patches to be detected. To prevent the cloud edge features from being truncated due to patching, a preset overlap step size (e.g., 10%) is set between adjacent image patches;

[0047] S1.2 Hardware-level Fixed-Point Quantization: To adapt to the FPGA's underlying fixed-point arithmetic pipeline, the PS performs symmetric or asymmetric 8-bit integer (INT8) quantization operations on the image block to be detected. Taking asymmetric quantization as an example, its physical mapping formula is:

[0048]

[0049] in, These are the original floating-point pixel values. This is the quantization scaling factor. This is the zero-point offset. This is the truncation function. The quantized image data stream is pushed into the hardware acceleration unit at the PL end via the AXI4-Stream high-speed video bus.

[0050] Step S2: Hardware-Software Co-operation Feature Extraction and Mask Inference Based on Ghost-RC-YOLO Architecture

[0051] The PL-side hardware acceleration unit receives the quantized image blocks and calls a lightweight instance segmentation network (Ghost-RC-YOLO) pre-synthesized and programmed into the gate array for parallel feature extraction. See also... Figure 3As shown, the network uses YOLOv8n-seg as the baseline architecture for hardware-aware reconstruction, and its core data path and hardware mapping mechanism include:

[0052] S2.1 Nonlinear activation mapping based on pure combinational logic: Replace the SiLU activation function after all convolutional layers in the baseline network with the hardware-friendly ReLU6 activation function. For the input quantized feature map Its forward propagation calculation formula is:

[0053]

[0054] Low-level hardware mapping: In the register-transfer level (RTL) circuitry of the FPGA, the ReLU6 function is instantiated as a pure combinational logic circuit with zero clock cycle delay, completely eliminating the need to call lookup tables (LUTs) to fit the exponential function in traditional activation functions. This circuit avoids the enormous resource overhead of traditional methods. It consists of two cascaded digital comparators and multiplexers: the first comparator determines the sign bit of the input value; if it is negative, it outputs 0. The second comparator checks if the value exceeds the quantization limit of the corresponding integer 6; if it does, it forces hardware truncation. This mechanism perfectly eliminates the risk of data overflow during INT8 fixed-point inference.

[0055] S2.2 Low-rank redundant feature generation based on Ghost Module: Ghost Modules are used to replace the standard dense convolutions in the backbone network. For input channels with... Feature tensor The number of output channels generated is Feature tensor The process is decoupled into the following physical execution steps:

[0056] (1) Intrinsic Feature Extraction: The DSP48E multiply-accumulator array inside the FPGA is scheduled to perform a standard convolution operation, generating a number of features. ( ,generally intrinsic feature map : ,in Standard convolutional kernel weights;

[0057] (2) Phantom Feature Mapping: Mapping the intrinsic feature map The input is fed into a depthwise separable convolution acceleration pipeline built on shift registers, and a low-hardware-overhead linear transformation operation is applied to each intrinsic feature channel. Generate a corresponding number of phantom feature maps : ;

[0058] (3) Physical memory reorganization: At the data write end of the on-chip BRAM, the physical memory is reorganized. and Direct concatenation along the channel dimension outputs a complete feature representation. .

[0059] S2.3 Multidimensional Attention Decision to Resist Background Interference: Convolutional Block Attention (CBAM) modules are cascaded between the Spatial Pyramid Pooling (SPPF) layer and the instance segmentation head of the backbone network. The entire backbone network is composed of multiple GhostModules stacked together, each module generating a Y. After the final Spatial Pyramid Pooling (SPPF) processing, the resulting feature map is F.

[0060] (1) Channel attention inference: Using a parallel accumulator tree circuit on an FPGA, the input features are... Global max pooling and average pooling are performed simultaneously, and then the results are fed into a shared multilayer perceptron (MLP) quantized into fully connected matrix multiplication to compute and generate a one-dimensional channel weight vector. ,implement ( (representing element-wise multiplication) to extract feature channels that are highly responsive to cloud textures;

[0061] (2) Spatial attention inference: for After pooling and splicing along the channel dimension, through Hardware convolutional windows infer two-dimensional spatial weight masks ,implement This mechanism dynamically amplifies the activation voltage signal of the target area (cloud boundary) in the physical circuit, effectively suppressing false detection interference caused by high albedo background (such as high-altitude snow).

[0062] Step S3: On-board mask decoding, cloud coverage calculation, and adaptive data decision-making.

[0063] Specifically, step S3 includes:

[0064] S3.1 Mask Decoding and Binarization: The instance segmentation head at the PL end outputs a prediction tensor, which is then subjected to Fast Non-Maximum Suppression (NMS) to remove redundant bounding boxes. After inverse quantization decoding, a binarized cloud mask matrix with the same spatial size as the original image patch is generated based on a set confidence threshold. (Where cloud pixels are marked as 1 and background pixels as 0); the predicted tensor is a direct prediction result generated by the instance segmentation head performing convolution inference on F″;

[0065] S3.2 Cloud Cover Statistics and Erasure / Download Scheduling: The hardware accumulator iterates through the mask matrix to calculate the total number of pixels with a value of 1, thus determining the cloud coverage of the current image patch. i represents the row index (vertical coordinate of pixels) of the mask matrix in the spatial dimension; j represents the column index (horizontal coordinate of pixels) of the mask matrix in the spatial dimension; traversing i and j means that the hardware accumulator checks all pixels of the entire image block one by one and counts the number of cloud pixels.

[0066] The PS terminal reads the cloud coverage rate and compares it with the reconfigurable cloud cleaning threshold. Comparative decision-making:

[0067] like If the image block is determined to be invalid occlusion data, the PS end directly erases the physical address cache of the data in DDR through the file system, preventing it from entering the delivery queue;

[0068] like Once determined to be a valid clear-sky image, the PS terminal schedules the hardware compression engine to encode the image and push it into the buffer queue of the satellite-to-ground downlink transmitter.

[0069] As one implementation method, this embodiment details how to obtain the Ghost-RC-YOLO network model described in this invention on a ground computing platform and compile it into a bitstream adaptable to spaceborne hardware.

[0070] 1. Hardware and software environment and hyperparameter configuration:

[0071] The offline training phase of the model in this invention was conducted on a high-performance computing platform equipped with an NVIDIA A800 GPU, and the underlying deep learning framework used was PyTorch. To ensure the model has generalization ability, the training dataset used was the standard 95-Cloud public dataset (containing 95 Landsat-8 remote sensing images).

[0072] To adapt to the input specifications of the spaceborne FPGA, all remote sensing images were cropped to an input resolution of 384×384 pixels. The model optimizer was configured as stochastic gradient descent (SGD), the batch size was set to 16, and the total number of training epochs was set to 50 epochs.

[0073] like Figure 4As shown in the training convergence curve, after replacing the traditional activation function with ReLU6, a pure combinational logic, the model's loss value can still decrease rapidly and stably and converge within the set 50 epochs, without gradient explosion or vanishing, proving the effectiveness of the lightweight reconstruction scheme of this invention at the mathematical and algorithmic levels.

[0074] 2. Post-training quantization (PTQ) and instruction set compilation:

[0075] After obtaining the model weights with FP32 precision, typical scene images extracted from the 95-Cloud dataset were used as a calibration set to statistically analyze the dynamic range of activation values ​​for each network layer. A quantization scaling factor was calculated to safely truncate the floating-point data path into 8-bit integer (INT8) fixed-point numbers. Finally, using a deep learning compiler tailored to the target deployment platform (onboard FPGA), the quantized computation graph was mapped into a hardware configuration bitstream file and an executable instruction set, which were then burned into the onboard non-volatile memory.

[0076] As one implementation, this embodiment discloses the hardware entity topology on which the present invention relies. The system of the present invention is deployed on a heterogeneous computing chip (including a processing system PS and a programmable logic PL) based on a system-on-a-chip (SoC) architecture.

[0077] Processing system side (PS side):

[0078] It is responsible for the overall satellite system-level scheduling. It receives the real-time bitstream from the multispectral camera and writes it to the DDR cache via direct memory access (DMA). Then, it uses a sliding window mechanism with a 10% overlap rate to crop the large-scale image into 384×384 pixel image blocks to be detected, and pushes them to the PL terminal via the AXI4-Stream high-speed bus.

[0079] Programmable logic module (PL) side:

[0080] Internally, the Ghost-RC-YOLO hardware acceleration IP core is instantiated.

[0081] In the nonlinear activation mapping region, the ReLU6 module is synthesized as a cascaded digital comparator and multiplexer circuit. When the sign bit of the input INT8 data is negative, the output is 0; when it exceeds the upper limit (corresponding to the integer 6), it is forcibly truncated, achieving nonlinear calculation with zero clock cycle delay and no overflow.

[0082] In the feature extraction decoupling region, the Ghost Module offloads the computation. The DSP48E multiply-accumulator array performs a small number of standard convolutions to generate "intrinsic features"; the depthwise separable convolution array composed of shift registers performs linear transformations to generate "phantom features", and finally, channel concatenation is performed at the on-chip BRAM write-out end.

[0083] In the attention inference region, the parallel accumulating tree circuit synchronously performs max / average pooling, and infers the channel and spatial weight masks by combining the quantized matrix multiplication. The activation gain is applied to specific pixel coordinates (cloud boundaries) through a hardware multiplier.

[0084] As one implementation method, this embodiment illustrates the service execution logic of a microsatellite during its on-orbit operation.

[0085] After the PL outputs the prediction tensor, it undergoes nonmaximum suppression (NMS) and dequantization decoding to generate a binarized cloud mask with a spatial size of 384×384.

[0086] The hardware accumulator iterates through the mask, calculating the percentage of pixels with a value of 1, to determine the cloud coverage of the current image patch. The PS-side processor reads this cloud coverage percentage.

[0087] When cloud coverage When 60% (which can be dynamically reconstructed from the ground) is reached, the PS end directly erases the physical address of the image in DDR through the file system, realizing "real-time discarding of invalid images".

[0088] When cloud coverage When the image reaches 60%, it is determined to be a valid clear-sky image. The PS terminal schedules the onboard hardware compression IP core to encode and compress it, and pushes it into the downlink transmitter's buffer queue to wait for cross-border transmission.

[0089] As one implementation method, in order to verify the inventiveness and significant progress of the technical solution of the present invention, ablation experiments and visual comparisons were conducted under the same software and hardware testing benchmarks.

[0090] 1. Ablation experiment verification:

[0091] Table 1 Performance Comparison of Different Module Combinations

[0092] YOLOv8n-seg SiLU - 3.26 4.36 61.89 38.49 42.53 23.77 YOLOv8-ReLU6 ReLU6 - 3.26 4.36 60.96 37.3 41.38 22.60 YOLOv8-CBAM ReLU6 CBAM 3.27 4.36 61.31 37.64 41.39 23.05 Ghost-YOLO ReLU6 CBAM 2.41 3.63 59.56 36.1 39.44 21.66

[0093] As shown in Table 1, using YOLOv8n-seg as the baseline model, the core reconstruction modules of this invention are introduced sequentially:

[0094] After introducing Ghost Module and ReLU6, the number of model parameters and floating-point operations (FLOPs) decreased significantly, effectively alleviating the computation and storage bottleneck of FPGA;

[0095] By further embedding the CBAM module, although the number of parameters increased by a very small amount (approximately 0.01M), it effectively compensated for the accuracy loss caused by the lightweight design under the stringent mAP50-95 metric, achieving an optimal balance between the system's precision and recall. The final form of this invention, Ghost-RC-YOLO, compared to the baseline, maintains a high level of detection capability while reducing the number of parameters by 26.1% and the computational load by 16.7%.

[0096] 2. Visual contrast in complex scenes:

[0097] As attached Figure 5 As shown, when faced with high-albedo backgrounds such as high-altitude snow cover, traditional methods and benchmark models are prone to large-area false detections due to similar spectral features. This invention, benefiting from the dynamic feature suppression of the CBAM mechanism in both spatial and channel dimensions, successfully distinguishes differences in cloud and snow textures, significantly reducing the false detection rate. Furthermore, in thin cloud scenarios, the mask boundaries generated by this invention are sharper and more complete, effectively solving the industry-wide technical challenge of edge-missed detections.

[0098] Based on the same inventive concept as the foregoing method embodiments, this invention also provides a lightweight cloud detection system for spaceborne FPGAs. The system is deployed on an in-orbit micro / nano satellite platform. (See attached...) Figure 2 As shown, it includes:

[0099] Image acquisition and preprocessing module: configured to receive high-speed image bitstreams from onboard optical sensors and perform image slicing, 8-bit integer fixed-point quantization, and data stream format conversion operations; the image acquisition and preprocessing module is deployed in the processing system area of ​​the onboard heterogeneous computing chip;

[0100] Onboard hardware inference module: Deployed in the programmable logic area of ​​the same onboard heterogeneous computing chip, it has embedded a hardware acceleration IP core of lightweight instance segmentation network and a quantized model parameter bit stream, which is used to perform feature extraction, mask inference and mask decoding, and output a binary cloud mask.

[0101] Onboard adaptive decision-making module: Deployed in the processing system area, configured to calculate cloud coverage based on the binarized cloud mask, and autonomously schedule the data physical erasure or compression transmission process of the satellite-to-ground communication link; wherein, the data physical erasure includes directly erasing the physical address cache of invalid data in the external memory through the file system, and the compression transmission process includes scheduling the hardware compression engine to encode and push it into the cache queue of the satellite-to-ground downlink transmitter.

[0102] Preferably, the system is deployed based on a heterogeneous computing architecture of system-on-a-chip; the image acquisition and preprocessing module and the on-board adaptive decision-making module are deployed in the processing system area; and the on-board hardware inference module is deployed in the programmable logic area.

[0103] Based on the same inventive concept as the foregoing embodiments, this embodiment of the invention also provides a computer device configured to operate in a spaceborne environment for executing the lightweight cloud detection method for spaceborne FPGAs described in this invention.

[0104] The computer device includes: at least one computing unit, a field-programmable gate array (FPGA), and a memory. Wherein:

[0105] Computing unit: can be an ARM architecture processor, RISC-V processor, or other embedded processor core suitable for spaceborne applications, used to run software program instructions and implement the logic control functions of the processing system side (PS side), including image slicing, fixed-point quantization, cloud coverage calculation, data erasure and downlink scheduling, etc.

[0106] Field-Programmable Gate Array (FPGA): Integrated with the computing unit on the same chip (e.g., in a heterogeneous system-on-a-chip architecture) or interconnected via a high-speed bus. The FPGA is used to configure hardware circuitry and deploy hardware acceleration IP cores for lightweight instance partitioning networks.

[0107] Memory: The memory is a non-volatile memory (such as Flash, EEPROM, or solid-state storage disk), which stores computer program instructions and bitstream files used to configure FPGA hardware circuitry. The bitstream file is a hardware configuration file generated after ground training, quantization, and compilation, containing model parameters quantized by 8-bit integers and circuit connection information for programmable logic terminals.

[0108] When the computer device is powered on, the computing unit loads and executes computer program instructions from memory, while the FPGA reads a bitstream file from memory for configuration. This instantiates a lightweight instance segmentation network hardware acceleration IP core at the programmable logic level, comprising a ReLU6 pure combinational logic activation circuit, a GhostModule feature decoupling circuit, and a convolutional block attention module. The computing unit and the FPGA work together to implement the steps of the lightweight cloud detection method for spaceborne FPGAs.

[0109] The computer device in this embodiment can be the core processing unit of a micro-nano satellite-borne integrated electronic system, or it can be a specially designed onboard intelligent computing module.

[0110] Based on the same inventive concept as the foregoing embodiments, this embodiment of the invention also provides a computer-readable storage medium, which is a non-transitory computer-readable storage medium, such as ROM, RAM, disk, solid-state drive, flash memory, programmable read-only memory (PROM), or EPROM, etc.

[0111] The computer-readable storage medium stores:

[0112] 1. Computer program instructions: When loaded and executed by the computing unit (such as the processor at the PS end) of the onboard processing system, these instructions are used to implement the steps related to the processing system end in the method, including: dynamic slicing, fixed-point quantization, cloud coverage calculation, cloud amount threshold comparison, physical address erasure, hardware compression engine scheduling, and encoding downlink decision-making.

[0113] 2. Bitstream file for configuring FPGA hardware circuitry: The bitstream file contains hardware configuration data generated after ground training, post-trained quantization (PTQ), and mapping by the deep learning compiler. When the bitstream file is loaded by the Field Programmable Gate Array (FPGA) of the onboard processing system, the programmable logic area of ​​the FPGA is configured as an inference engine containing the following hardware acceleration circuitry:

[0114] A ReLU6 activation circuit based on pure combinational logic (consisting of a cascaded digital comparator and a multiplexer).

[0115] The feature extraction circuit built on GhostModule includes a multiply-accumulator array for intrinsic feature extraction, a shift register for phantom feature generation, and a depthwise separable convolution pipeline.

[0116] The cascaded embedded convolutional block attention module circuit includes a parallel accumulating tree circuit, a shared multilayer perceptron circuit, and a hardware convolutional window circuit.

[0117] When the computer program instructions and bitstream file are loaded and executed by the computing unit and field-programmable gate array of the onboard processing system, all steps of the lightweight cloud detection method for onboard FPGA are implemented in a coordinated manner.

[0118] The computer-readable storage medium of this embodiment can be used as firmware or software distribution package for spaceborne equipment. After being written on the ground, it can be launched into orbit with the satellite. It can also be updated in orbit through on-board grease injection or uplink commands.

[0119] In summary, this invention discloses a lightweight cloud detection method, system, and medium for spaceborne FPGAs. Addressing the challenges of deploying large remote sensing models on constrained hardware and the low detection accuracy in complex scenes, this method dynamically segments and quantizes multispectral remote sensing images using INT8. The images are then input into a lightweight network for hardware-aware reconstruction. The original activation function is replaced with a pure combinational logic ReLU6 to eliminate fixed-point overflow and latency. A Ghost Module is used to decouple feature extraction, overcoming off-chip memory access bottlenecks. A cascaded CBAM attention mechanism effectively suppresses high-albedo background interference, inferring the cloud mask. Finally, based on cloud cover statistics, invalid data is autonomously physically erased or compressed and downloaded. This invention achieves cross-layer coupling between the algorithm and the underlying hardware, significantly reducing computational resource consumption while maintaining detection accuracy for complex backgrounds such as thin clouds and snowfields. This releases orders of magnitude more space-to-ground communication bandwidth, meeting the real-time on-orbit processing needs of micro- and nano-satellites.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight cloud detection method for a spaceborne FPGA, characterized in that, The method relies on a spaceborne heterogeneous computing system that includes a processing system and a programmable logic system, and includes the following steps: The processing system acquires multispectral remote sensing images, performs dynamic slicing and 8-bit integer fixed-point quantization preprocessing to generate image blocks, and pushes the image blocks into the programmable logic terminal via the AXI4-Stream bus. The lightweight instance segmentation network deployed in the programmable logic terminal performs parallel feature extraction and mask inference on the image patch and outputs a prediction tensor; The processing system performs mask decoding on the prediction tensor to generate a binarized cloud mask, calculates the cloud coverage rate of the current image patch based on the binarized cloud mask, and compares the cloud coverage rate with a preset threshold. If the cloud coverage rate is greater than or equal to the preset threshold, the processing system will directly erase the physical address cache of the image block in the external storage through the file system. If the cloud coverage rate is less than the preset threshold, the processing system will schedule the hardware compression engine to encode the image block and push it into the buffer queue of the satellite-to-ground downlink transmitter.

2. The lightweight cloud detection method for spaceborne FPGAs according to claim 1, characterized in that, The lightweight instance segmentation network uses YOLOv8n-seg as the baseline architecture for hardware-aware reconstruction and includes the following hardware acceleration circuits deployed in the programmable logic terminal: The activation function after all convolutional layers is configured as a ReLU6 activation circuit based on pure combinational logic, which is instantiated in the underlying hardware mapping as a cascaded digital comparator and multiplexer with zero clock cycle delay. The backbone network is constructed using GhostModule to decouple feature extraction into two independent stages: intrinsic feature generation and phantom feature generation. The multiply-accumulator array inside the programmable logic terminal performs standard convolution operations to generate intrinsic feature maps, while the depthwise separable convolution accelerated pipeline built based on shift registers performs linear transformation operations to generate phantom feature maps. The intrinsic feature maps and phantom feature maps are concatenated along the channel dimension at the data write-out end of the on-chip static random access memory. Between the spatial pyramid pooling layer and the instance segmentation head of the backbone network, a convolutional block attention module is cascaded and embedded to focus features along the channel dimension and the spatial dimension. The channel attention inference uses a parallel accumulating tree circuit to synchronously perform global max pooling and average pooling, while the spatial attention inference generates a two-dimensional spatial weight mask through a hardware convolutional window.

3. The lightweight cloud detection method for spaceborne FPGAs according to claim 1, characterized in that, The dynamic block segmentation and 8-bit integer fixed-point quantization preprocessing includes: The original remote sensing image is cropped into image patches of a preset spatial dimension using a dynamic sliding window mechanism, and a preset overlap step size is set between adjacent image patches. Perform symmetric or asymmetric 8-bit integer quantization on the image block to be detected, and push the quantized image data stream into the programmable logic terminal.

4. The lightweight cloud detection method for spaceborne FPGAs according to claim 1, characterized in that, The mask decoding includes: generating the binarized cloud mask matrix through nonmaximum suppression and inverse quantization decoding; the cloud coverage rate is calculated by traversing the mask matrix using a hardware accumulator to calculate the total number of cloud pixels.

5. The lightweight cloud detection method for spaceborne FPGAs according to claim 1, characterized in that, The lightweight instance segmentation network is trained and then quantized before deployment. The 8-bit integer fixed-point quantization is matched with the upper limit hard truncation of the ReLU6 activation circuit to eliminate the risk of data overflow during fixed-point inference.

6. A lightweight cloud detection system for spaceborne FPGAs, characterized in that, Deployed on a micro-nano satellite in-orbit platform, the system includes: Image acquisition and preprocessing module: configured to receive high-speed image bitstreams from onboard optical sensors and perform image slicing, 8-bit integer fixed-point quantization, and data stream format conversion operations; the image acquisition and preprocessing module is deployed in the processing system area of ​​the onboard heterogeneous computing chip; Onboard hardware inference module: Deployed in the programmable logic area of ​​the same onboard heterogeneous computing chip, it has embedded a hardware acceleration IP core of lightweight instance segmentation network and a quantized model parameter bit stream, which is used to perform feature extraction, mask inference and mask decoding, and output a binary cloud mask. Onboard adaptive decision-making module: Deployed in the processing system area, configured to calculate cloud coverage based on the binarized cloud mask, and autonomously schedule the data physical erasure or compression transmission process of the satellite-to-ground communication link; wherein, the data physical erasure includes directly erasing the physical address cache of invalid data in the external memory through the file system, and the compression transmission process includes scheduling the hardware compression engine to encode and push it into the cache queue of the satellite-to-ground downlink transmitter.

7. The lightweight cloud detection system for spaceborne FPGAs according to claim 6, characterized in that, The system is deployed based on a heterogeneous computing architecture of system-on-chip; the image acquisition and preprocessing module and the on-board adaptive decision module are deployed in the processing system area; the on-board hardware inference module is deployed in the programmable logic area.

8. The lightweight cloud detection system for spaceborne FPGAs according to claim 6, characterized in that, The hardware acceleration IP cores in the onboard hardware inference module include: a cascaded digital comparator and multiplexer for executing ReLU6 activation circuitry; a multiply-accumulator array and a shift register-based depth-separable convolution acceleration pipeline for executing GhostModule; and a parallel accumulator tree circuit and a hardware convolution window for executing convolution block attention modules.

9. A computer device, characterized in that, include: Computing unit and field-programmable gate array; And a memory, on which computer program instructions and bitstream files for configuring hardware circuits are stored; When the program instructions and bitstream file are loaded and executed by the computing unit and the field-programmable gate array, the lightweight cloud detection method for spaceborne FPGA as described in any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions and a bitstream file for configuring FPGA hardware circuits. When the program instructions and bitstream file are loaded and executed by the computing unit and field-programmable gate array of the onboard processing system, all steps of the lightweight cloud detection method for onboard FPGA as described in any one of claims 1 to 5 are implemented.

Images