An on-orbit target detection method for remote sensing satellite visible light original image

Abstract

The application discloses a kind of on-orbit target detection methods for remote sensing satellite visible light original image, comprising: obtaining the original remote sensing image without radiation and geometric correction preprocessing;Subsequently, the image is subjected to edge perception enhancement processing to strengthen its structure and edge features;The feature after enhancement is subjected to deformable attention enhancement processing, to improve the feature perception ability of the model to geometric distortion target;Through the lightweight feature pyramid structure, multi-scale feature extraction and fusion are carried out, while reducing the computational complexity, the feature expression ability is maintained;Finally, based on the fusion feature, target detection is completed, and class, position and confidence information are output, to realize end-to-end on-orbit original image target recognition.The on-orbit target detection method of the application skips the traditional preprocessing link, significantly saves on-board computing power resources, and improves the real-time performance of interpretation and the timeliness of information acquisition.

Description

Technical Field

[0001] This invention relates to on-orbit target detection technology for remote sensing satellites, and in particular to an on-orbit target detection method for raw visible light images from remote sensing satellites. Background Technology

[0002] In recent years, with the rapid development of remote sensing satellites and their payloads, the spatial resolution of remote sensing imagery has been continuously improved. In particular, visible light remote sensing imagery, with its intuitive information expression and high-resolution characteristics, provides a solid foundation for accurate automatic target identification, enabling its widespread application in fields such as agricultural monitoring, disaster early warning, and smart cities.

[0003] The rise of deep learning has further promoted the deep integration of target detection technology and remote sensing, significantly improving the intelligent analysis capabilities of large-scale surface targets. Meanwhile, with the improvement of satellite intelligence, research focus has gradually shifted from ground-based processing to on-orbit processing. On-orbit target detection not only supports satellites in autonomously planning observation tasks based on recognition results, but also enables rapid information transmission after detecting targets of interest, improving observation timeliness and mission efficiency. Furthermore, it alleviates storage and communication pressures by filtering and compressing raw data, thus becoming an important direction for the development of intelligent remote sensing.

[0004] On-orbit processing technology is constantly evolving, and on-orbit target detection is a crucial component and an important means of remote sensing image interpretation. Its main task is to automatically identify and locate specific target objects, such as buildings, vehicles, ships, and vegetation, from large-scale remote sensing images. This task typically includes target classification, detection, and precise localization. By processing and analyzing features in remote sensing images, the required target information is extracted, which is key to the application of remote sensing images containing rich surface information.

[0005] The current processing flow of remote sensing satellite on-orbit target detection technology, from data acquisition to detection and interpretation, is as follows: Level 0 remote sensing signals are transmitted to the ground station via a data link, and after data analysis, scene segmentation and cataloging, CCD stitching, radiometric correction, etc., Level 1 remote sensing images are formed; Level 1 remote sensing images undergo atmospheric correction and system geometric correction (coarse geometric correction) to form Level 2 remote sensing images; Level 2 remote sensing images undergo fine geometric correction to form Level 3 remote sensing images; Level 3 remote sensing images can be processed according to specific needs, such as image fusion, image stitching, orthorectification, to generate higher-level image products; finally, target detection is performed on the highly processed advanced products. It is evident that target detection in remote sensing images is a relatively late step in the traditional processing flow.

[0006] Remote sensing image target detection aims to automatically identify and locate specific features or targets in aerial or satellite imagery. Its basic principle is to distinguish targets from complex backgrounds by utilizing differences in spectral characteristics, texture structure, geometric morphology, and spatial context. Traditional methods often rely on manually designed features and classifiers, while modern methods are primarily based on deep learning, using models such as convolutional neural networks or Transformers to perform end-to-end feature extraction and semantic modeling of images. Under the supervision of large-scale labeled samples, the model learns multi-scale, multi-directional discriminative features, achieving joint prediction of target category and location, thereby improving detection accuracy and robustness under complex scenes and scale variations.

[0007] Current on-orbit target detection processes for remote sensing typically rely on multi-stage data preprocessing, including radiometric correction, atmospheric correction, geometric correction, orthorectification, and image fusion. These preprocessing steps aim to improve image quality and restore ground reflectivity and geometric accuracy, but they also significantly increase the complexity and time cost of data processing. In real-time onboard missions, this traditional "preprocessing-detection" paradigm is difficult to apply directly due to limitations in computing power, energy consumption, and data transmission bandwidth. In contrast, if target detection could be performed directly on uncorrected raw remote sensing images, it would not only significantly shorten the processing chain from data acquisition to interpretation but also improve the real-time performance and overall efficiency of onboard processing. This need has driven research into intelligent detection architectures for raw image processing.

[0008] Existing on-orbit target detection technologies still have the following drawbacks in practical applications:

[0009] I. Although current on-orbit target detection has achieved a leap from ground to satellite, it still follows the ground-based process. The core of this process is that target detection is performed after numerous preprocessing steps. Target detection is based on highly processed optical remote sensing images. If this process is continued, these preprocessing steps before detection will impose a heavy computational burden. Since computational power in the satellite environment is already very precious, this traditional detection paradigm is not suitable for satellite application. A better on-orbit detection paradigm needs to be found.

[0010] Second, current on-orbit target detection is performed after numerous preprocessing stages, which introduces a significant delay to target detection interpretation in time-sensitive missions. Skipping these preprocessing stages would significantly improve the real-time performance of target interpretation, enhance the timeliness of information acquisition, and support satellite intelligence and real-time decision-making.

[0011] Third, to bypass preprocessing and perform detection directly on the raw imagery, it is essential to address the interference introduced by the raw features. The raw image data acquired by optical remote sensing payloads differs significantly from mature ground-based remote sensing data products. Raw images acquired by spaceborne sensors typically lack radiometric and geometric correction, and commonly suffer from detector noise, color distortion, uneven exposure, and geometric aberrations. These degradation factors significantly increase the difficulty of detection, and existing algorithms are mostly trained on high-quality ground-based remote sensing products, lacking the ability to adapt to raw payload data, making it difficult to guarantee their performance in on-orbit scenarios.

[0012] It should be noted that the information disclosed in the background section above is only for understanding the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0013] The main objective of this invention is to overcome the deficiencies in the aforementioned background technology and provide an on-orbit target detection method for raw visible light images from remote sensing satellites.

[0014] To achieve the above objectives, the present invention adopts the following technical solution:

[0015] An on-orbit target detection method for raw visible light imagery from remote sensing satellites includes the following steps:

[0016] S1. Acquire raw visible light remote sensing images from satellite payloads, the raw remote sensing images being unprocessed by radiometric and geometric correction preprocessing;

[0017] S2. Perform edge-aware enhancement processing on the original remote sensing image to strengthen the structural and edge features in the image;

[0018] S3. Perform deformable attention enhancement on the edge-enhanced features to improve the model's feature perception and spatial adaptability to geometrically distorted targets.

[0019] S4. Multi-scale feature extraction and fusion are performed through a feature pyramid structure to maintain feature expressiveness while reducing computational complexity.

[0020] S5. Target detection is performed based on the fused features, and the target category, location and confidence information are output to achieve on-orbit end-to-end raw image target recognition.

[0021] Further, in step S2, the edge-aware enhancement processing specifically includes:

[0022] High-frequency feature information is extracted from the input image, and the high-frequency feature information is calculated by at least one of the following methods: the difference between the input image and the image after average pooling; or the gradient intensity of the input image is calculated based on the gradient direction convolution kernel;

[0023] The high-frequency feature information is mapped to an edge feature map through a convolutional layer and an activation function.

[0024] The edge feature map is channel-stitched with the original input image, and the intensity of the edge feature map in the fusion is controlled by a learnable factor to form an enhanced fused input feature, so as to provide an explicit structural prior in the input stage.

[0025] Furthermore, in step S2, in the edge perception enhancement process, high-frequency feature information is mapped to an edge feature map through convolution and the Sigmoid activation function, and the edge feature map is fused with the original input image by channel stitching.

[0026] Furthermore, in step S3, the deformable attention enhancement processing specifically includes:

[0027] A deformable attention mechanism is introduced at the junction of the backbone network and the neck network.

[0028] A reference point grid is generated based on the input feature map, and the query features generated by linearly projecting the input feature map through a light quantum network are used to calculate the spatial offset of each reference point.

[0029] The input feature map is resampled based on the offset to obtain the feature representation after deformation sensing.

[0030] Based on the resampled features, key features and value features are calculated and input together with the query features into the multi-head self-attention module for calculation to obtain an enhanced feature representation.

[0031] The enhanced feature representation is directly passed to the large target detection head to enhance the model's ability to detect large targets with geometric distortions.

[0032] Furthermore, step S3 also includes:

[0033] Deformable attention computation is performed before the output features of the backbone network are fused into the neck features;

[0034] The direct connection between the large detection head and the previous module in the original structure was removed, and the output of the deformable attention module was directly connected to the large detection head instead.

[0035] Further, in step S4, the lightweight feature pyramid structure specifically includes:

[0036] In the back-end and neck modules of the backbone network, Ghost convolutions are used to replace some standard convolutions;

[0037] Ghost convolution reduces the overall computational cost by first generating a main feature map using a small number of standard convolutions, then deriving redundant feature maps from the main feature map using a set of inexpensive channel-wise convolutions, and finally stitching all the feature maps together to form a complete output.

[0038] Standard convolutional modules are retained at the front of the backbone network to maintain low-level semantic integrity, while lightweight bottleneck modules containing Ghost convolutions are used in the rear and neck layers of the backbone network for feature reconstruction and fusion.

[0039] Furthermore, in step S4, the lightweight feature pyramid structure is constructed by replacing the standard convolution in the standard bottleneck structure with Ghost convolution, and additional depthwise separable convolutional paths are enabled during the downsampling stage to maintain feature representation capability.

[0040] Furthermore, the method also includes a model training and on-orbit deployment process, specifically including:

[0041] At the ground end, the detection model is trained and optimized based on the original remote sensing dataset containing radiation distortion and geometric distortion;

[0042] The trained model is uploaded to the satellite's on-orbit computing platform for deployment;

[0043] During operation in orbit, the original remote sensing images are directly input into the model for target detection, and the detection results are output.

[0044] Based on the detection results, the target area of ​​interest is sliced ​​and extracted, and then downloaded preferentially.

[0045] The model is iteratively updated and optimized based on the downlink data, forming a closed-loop evolution mechanism.

[0046] Furthermore, in the model training and on-orbit deployment process, model updates specifically include:

[0047] The model can be retrained or incrementally optimized using new samples acquired in orbit, complex scene data, and false detection samples.

[0048] The updated model parameters are periodically generated, verified, and then re-uploaded to the satellite to achieve continuous evolution of the on-orbit model.

[0049] A computer program product includes a computer program that, when executed by a processor, implements the on-orbit target detection method for raw visible light imagery from remote sensing satellites.

[0050] The present invention has the following beneficial effects:

[0051] This invention proposes an on-orbit target detection method for raw visible light imagery from remote sensing satellites. This method enables direct target detection on-board raw images without any radiometric or geometric correction preprocessing, effectively bypassing the cumbersome data preprocessing steps in traditional workflows. This not only significantly shortens the overall processing chain from data acquisition to information interpretation and reduces the consumption of valuable onboard computing resources, but also greatly improves the real-time performance of target detection and the timeliness of information acquisition, providing key technical support for achieving intelligent autonomous perception and rapid response from satellites.

[0052] Specifically, the RAWDet target detection architecture designed for on-orbit applications in this invention explicitly enhances structural and edge features in images by introducing an Edge-Aware Input Module (EIM). This effectively overcomes the low contrast and color degradation problems commonly found in original images due to the lack of atmospheric correction, improving the model's feature extraction stability and target recognition robustness under radiometric distortion conditions. Addressing shape distortion caused by the lack of geometric correction in original images, this invention utilizes a Large Target Deformable Attention Enhancement Module (LDAE). This module leverages a deformable attention mechanism to enable the model to adaptively focus on key target regions, thereby enhancing feature perception and spatial adaptability for geometrically deformed targets, significantly improving detection accuracy and robustness. Furthermore, by constructing a Ghost Lightweight Feature Pyramid (GLFP) and replacing some standard convolutions with Ghost convolutions in key network layers, a structural reduction in model parameter size and computational complexity is achieved while maintaining basic feature extraction capabilities. This makes the model more suitable for the stringent constraints of computing power, power consumption, and storage on spaceborne embedded platforms, improving the engineering feasibility of on-orbit deployment.

[0053] In summary, this invention does not pursue detection accuracy in isolation or simply compress the model. Instead, through the aforementioned targeted modular design, it synergistically optimizes accuracy, efficiency, and robustness to degradation features of the original imagery. Furthermore, this invention constructs a complete application process encompassing ground training, on-orbit deployment, intelligent detection, result downloading, and model iterative updates, forming a continuously optimized closed-loop system. This system can significantly reduce the burden of satellite-to-ground communication by prioritizing the downloading of target region slices and supports continuous model evolution using new on-orbit data, thereby comprehensively improving the practicality, adaptability, and long-term performance of the on-orbit intelligent sensing system.

[0054] Other beneficial effects of the embodiments of the present invention will be further described below. Attached Figure Description

[0055] Figure 1 This is a diagram illustrating the overall architecture of the RAWDet target detection architecture according to an embodiment of the present invention.

[0056] Figure 2 This is a schematic diagram of the edge-aware input module (EIM) according to an embodiment of the present invention;

[0057] Figure 3 This is a schematic diagram of the structure of the Large Target Deformable Attention Enhancement Module (LDAE) according to an embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram illustrating the generation of features from standard convolution and Ghost convolution in this embodiment of the invention.

[0059] Figure 5 This is a comparative diagram of the standard bottleneck structure and the Ghost bottleneck structure of this invention embodiment;

[0060] Figure 6 This is a flowchart illustrating the overall process of the on-orbit target detection method for raw visible light images from remote sensing satellites, as described in this invention. Detailed Implementation

[0061] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0062] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0063] This invention aims to bypass the traditional remote sensing image preprocessing stage and directly perform real-time target detection on uncorrected raw visible light images on satellite, shortening the processing chain and saving onboard resources. To this end, an on-orbit target detection method for raw visible light images from remote sensing satellites is proposed, establishing an end-to-end detection architecture (named RAWDet) to enhance robustness to radiation degradation features, improve adaptability to geometric distortion, and achieve structural lightweighting of the model. While ensuring detection accuracy, it significantly improves the feasibility of on-orbit deployment and the timeliness of information acquisition.

[0064] See Figures 1 to 6 This invention provides an on-orbit target detection method for raw visible light images from remote sensing satellites, comprising the following steps:

[0065] Step S1: Acquire raw visible light remote sensing images collected by the satellite payload. The raw remote sensing images have not undergone radiometric and geometric correction preprocessing.

[0066] In some embodiments, the original remote sensing images directly retain the original data characteristics acquired by the satellite payload without correction processing. They generally exhibit degradation features such as detector noise, color distortion, uneven exposure, and geometric distortion, and can be directly used for further detection in orbit.

[0067] Step S2: Perform edge-aware enhancement processing on the original remote sensing image to enhance the structural and edge features in the image.

[0068] In some embodiments, see Figure 2 In step S2, the edge-aware enhancement processing specifically includes: extracting high-frequency feature information from the input image, wherein the high-frequency feature information is calculated in at least one of the following ways: the difference between the input image and the image after average pooling; or calculating the gradient intensity of the input image based on the gradient direction convolution kernel; mapping the high-frequency feature information to an edge feature map through a convolutional layer and an activation function; concatenating the edge feature map with the original input image through channels, and controlling the intensity of the edge feature map in the fusion process through a learnable factor to form an enhanced fused input feature, so as to provide an explicit structural prior in the input stage.

[0069] In some embodiments, in step S2, the high-frequency feature information is mapped to an edge feature map through convolution and the Sigmoid activation function, and the edge feature map is fused with the original input image by channel stitching.

[0070] Step S3: Perform deformable attention enhancement processing on the edge-enhanced features to improve the model's feature perception and spatial adaptability to geometrically distorted targets.

[0071] In some embodiments, see Figure 3 The structural design of the Large Target Deformable Attention Enhancement Module (LDAE) shown includes the following specific steps: introducing a deformable attention mechanism at the connection between the backbone network and the neck network; generating a reference point grid based on the input feature map, and calculating the query features generated by linearly projecting the input feature map through a light quantum network to predict the spatial offset of each reference point; resampling the input feature map according to the offset to obtain a deformable-aware feature representation; calculating key features and value features based on the resampled features, and inputting them together with the query features into a multi-head self-attention module for calculation to obtain an enhanced feature representation; and directly transmitting the enhanced feature representation to the large target detection head to enhance the model's ability to detect large targets with geometric distortions.

[0072] In some embodiments, step S3 further includes: performing deformable attention calculation before the backbone network output features enter the neck feature fusion; canceling the direct connection between the large detection head and the previous module in the original structure, and replacing it with the output of the deformable attention module being directly connected to the large detection head.

[0073] Step S4: Multi-scale feature extraction and fusion are performed through a lightweight feature pyramid structure, which reduces computational complexity while maintaining feature expressive power.

[0074] See Figure 4 In some embodiments, step S4 specifically includes the lightweight feature pyramid structure:

[0075] In the back-end and neck modules of the backbone network, Ghost convolutions are used to replace some standard convolutions;

[0076] Ghost convolution reduces the overall computational cost by first generating a main feature map using a small number of standard convolutions, then deriving redundant feature maps from the main feature map using a set of inexpensive channel-wise convolutions, and finally stitching all the feature maps together to form a complete output.

[0077] Standard convolutional modules are retained at the front of the backbone network to maintain low-level semantic integrity, while lightweight bottleneck modules containing Ghost convolutions are used in the rear and neck layers of the backbone network for feature reconstruction and fusion.

[0078] See Figure 5 In some embodiments, in step S4, the lightweight feature pyramid structure is constructed by replacing the standard convolution in the standard bottleneck structure with Ghost convolution, and additional depthwise separable convolutional paths are enabled during the downsampling phase to maintain feature expressiveness.

[0079] Step S5: Target detection is performed based on the fused features, and the target category, location and confidence information are output to achieve on-orbit end-to-end original image target recognition.

[0080] In some embodiments, the target detection is implemented based on a detection architecture improved from YOLO11. The fused multi-scale features are input into the corresponding detection heads to classify and locate different types of targets such as buildings, vehicles, ships, and vegetation. The output confidence information is used to filter valid detection results, providing a basis for subsequent slice extraction and priority download of target areas of interest, ensuring the accuracy and practicality of on-orbit detection.

[0081] In some embodiments, the method further includes a model training and on-orbit deployment process, specifically including: training and optimizing the detection model on the ground based on the original remote sensing dataset containing radiation distortion and geometric distortion; uploading the trained model to the satellite on-orbit computing platform for deployment; during on-orbit operation, directly inputting the original remote sensing image into the model for target detection and outputting the detection results; extracting slices of the target area of ​​interest based on the detection results and prioritizing their download; iteratively updating and optimizing the model based on the downloaded data to form a closed-loop evolution mechanism.

[0082] In some embodiments, the model training and on-orbit deployment process includes, specifically, the following: retraining or incrementally optimizing the model using new samples, complex scenario data and false detection samples acquired on-orbit; periodically generating updated model parameters, verifying them and then re-uploading them to the satellite to achieve continuous evolution of the on-orbit model.

[0083] This invention proposes an on-orbit target detection method for raw visible light remote sensing satellite imagery. By directly performing end-to-end on-orbit target detection on raw, unprocessed visible light remote sensing images, it effectively skips the computationally intensive radiometric and geometric correction steps in traditional processes, thus significantly saving valuable onboard computing resources and greatly improving the timeliness from data acquisition to information interpretation. The RAWDet architecture established by this invention specifically introduces an edge-aware input module to enhance the robustness of feature extraction for radiometrically degraded images, utilizes a large-target deformable attention mechanism to adapt to geometrically distorted targets, and further reduces model complexity while maintaining accuracy through a lightweight Ghost design. Finally, it constructs a complete closed-loop system including training, deployment, detection, and model iterative updates, achieving synergistic optimization in terms of raw image adaptability, onboard resource utilization efficiency, and engineering feasibility.

[0084] The following further describes specific embodiments of the present invention and examples of its algorithm implementation.

[0085] This invention presents an on-orbit target detection method for raw visible light imagery from remote sensing satellites. The method establishes a target detection architecture, RAWDet (Raw Remote Sensing Imagery Oriented Detection), which improves upon the YOLO11 detection architecture to adapt to the characteristics of uncorrected raw remote sensing images. Addressing the issues of uneven illumination and color degradation in raw images due to lack of atmospheric correction, which typically leads to weakened image edges and loss of detail, thus reducing feature separability, this invention introduces an Edge-aware Input Module (EIM) at the front end of the backbone network. By strengthening structural and edge information, it guides the network to maintain stable and discriminative feature representation even under radiometric distortion conditions. To address the geometric distortion problem caused by the lack of geometric correction in raw remote sensing images, this invention introduces a Large-object Deformable Attention Enhancement (LDAE) module. Geometric distortion causes significant shape differences in targets of the same category under different imaging conditions, leading to a decrease in intra-class feature clustering. To alleviate this issue, LDAE is based on a deformable attention (DA) mechanism. By deformably enhancing deep features, the model can better focus on key regions of geometrically distorted targets, thus obtaining more robust shape representations. Furthermore, LDAE directly transmits the enhanced high-level semantic features to the large target detection head, helping to improve the model's detection accuracy on large-scale, complexly deformed targets. Further, this invention designs the Ghost Lightweight Feature Pyramid (GLFP), introducing Ghost convolutions in the backbone and neck structures to replace some standard convolutions, achieving structural lightweighting of the network. This design significantly reduces the number of parameters and computational overhead while maintaining feature extraction capabilities, providing greater resource adaptability for on-orbit deployment.

[0086] RAWDet's overall architecture is as follows Figure 1As shown in the diagram. Here, EIM represents the edge-aware input module, AP represents average pooling, Conv represents convolution, Concat represents concatenation, GhostBottleneck represents the Ghost bottleneck, GhostConv represents Ghost convolution, DWConv represents depthwise convolution, GC3k2 represents the GhostC3k2 module, C3k2 represents the C3k2 module, DAttention represents deformable attention (its parameters are further explained below), Multi-head Attention represents multi-head attention, Upsample represents upsampling, C2PSA represents the C2PSA module, SPPF represents fast spatial pyramid pooling, and Head represents the detection head. RAWDet mainly consists of three parts: a backbone for feature extraction, a neck for feature fusion, and a head for target prediction. This embodiment introduces EIM at the front of the backbone network to highlight target edge features and improve the model's feature extraction capability in the absence of color guidance. LDAE is introduced at the connection between the backbone and the neck, utilizing the deformable attention mechanism to achieve adaptive feature focusing in the high-level semantic space, enhancing the model's ability to perceive deformable targets. Furthermore, Ghost bottlenecks are integrated into multiple C3k2 modules in the rear section and neck of the backbone, forming GC3k2 modules that significantly reduce the number of parameters and computational load. Overall, RAWDet's design fully considers the unique characteristics of raw remote sensing imagery in terms of radiometric degradation, geometric distortion, and computationally limited on-orbit environments, significantly improving computational efficiency and deployment friendliness while ensuring detection accuracy.

[0087] (1) Edge-sensing input

[0088] Raw remote sensing images, without atmospheric correction, often exhibit low contrast and color degradation, leading to a weakened spectral difference between the target and the background. This makes it difficult for convolutional networks to effectively extract edge and structural information from the raw pixels. To enhance the structural sensitivity of the input stage, this invention introduces an edge-aware input module (EIM) at the front end of the backbone network to explicitly provide prior edge features, thereby improving the network's feature representation ability under degradation conditions.

[0089] Figure 2 The structure of the edge-aware input module (EIM) according to an embodiment of the present invention is shown: (a) Conceptual diagram: edge features are extracted and fused with the original image to provide edge priors. (b) Implementation in average pooling mode: edge features are calculated by the difference between the input image and the pooled image, modulated by a convolutional layer, and then fused as enhanced input.

[0090] The core idea of ​​the Edge-Aware Input Module (EIM) is to construct a learnable edge enhancement operator to extract and nonlinearly modulate high-frequency components from the input image, and then fuse the high-frequency components with the original image. Given an input image:

[0091] (1)

[0092] EIM first from Extracting high-frequency information Then calculate the edge map. :

[0093] (2)

[0094] in This indicates that a 3×3 convolution and a sigmoid activation form a mapping function, thereby enabling learnable high-frequency information processing. Different... Corresponding to different high-frequency modeling methods:

[0095] Average pooling mode (Avg):

[0096] (3)

[0097] Sobel pattern (Sobel):

[0098] (4)

[0099] in , It uses a classic Sobel convolution kernel to extract the gradient direction strength;

[0100] During the fusion phase, EIM provides two feature integration methods: additive fusion and channel stitching. This invention uses the latter to avoid interference from radiometric distribution while preserving the independent expressive power of edge features. The fused input is represented as follows:

[0101] (5)

[0102] It is a learnable factor that controls the intensity of edge information.

[0103] EIM enables the model to obtain explicit structural priors at the input stage, allowing the feature extraction process to not only rely on spectral information but also directly utilize geometric and contour cues. The EIM module significantly improves the model's response strength to edge targets under conditions of weak color and low contrast in the original image, thereby enhancing the robustness of the detection network in radiometrically degraded scenes.

[0104] (2) Large target deformability enhances attention

[0105] Raw remote sensing images are typically uncorrected geometrically and are prone to geometric distortion due to factors such as pose changes, terrain undulations, and differences in imaging perspective during the imaging process. This distortion can cause similar targets to exhibit significantly different shapes and spatial distributions in different regions or under different imaging conditions, thereby exacerbating intra-class feature differences, weakening the geometric consistency of targets, and posing challenges to deep feature extraction and spatial localization. To improve the robustness of the model under geometrically diverse conditions, this invention introduces a deformable attention (DA) mechanism to adaptively focus on deep features, achieving deformation-insensitive feature modeling.

[0106] Traditional self-attention employs a fixed global sampling method in the spatial dimension, making it difficult to model long-range dependencies under complex geometric deformations. For example... Figure 1 and Figure 3 As shown, the deformable attention mechanism introduces learnable offsets, enabling each query point to dynamically aggregate key information from different spatial locations. Given an input feature map... Generating points A uniform grid is used as a reference. To obtain the offset of each reference point, the feature map is linearly projected to obtain the query token. x is the feature data to be processed, which is then input into a light quantum network. Generate offset Use offset For the input feature map The offset is then performed, and resampling is carried out using bilinear interpolation. Then calculate the deformed key-value embedding: Combined with the previously calculated query Multi-head self-attention computation is performed. Features from each head are concatenated and projected. To obtain the final output .in, W q , W k , W v These are the linear transformation weight matrices for query (q), key (k), and value (v), respectively.

[0107] In natural scenes, large-scale objects typically possess complex contour structures and long spatial extensions, making their local deformation under geometric distortion more pronounced. In contrast, the deformation of small-scale objects is primarily manifested at the texture level. Therefore, geometric distortion has a significantly greater impact on large-scale objects than on small-scale objects. Based on this, the LDAE proposed in this invention specifically introduces a deformable attention mechanism at the connection between the trunk and neck to enhance the model's spatial adaptability to large-scale objects.

[0108] Specifically, the DAttention module is placed after the SPPF module and before the C2PSA module at the end of the backbone, and the branch connection between C2PSA and the large detector head in the original structure is removed, replaced by DAttention directly connecting to the large detector head. This design allows the high-level features at the end of the backbone to be deformably enhanced before entering the neck region, while ensuring that the large detector head directly receives feature representations containing deformation-aware information. By introducing a deformable attention mechanism at the stage with the richest semantics in the feature flow, the model can significantly enhance its ability to perceive the geometric deformation of the target while preserving high-level semantic information.

[0109] (3) Ghost Lightweight Feature Pyramid

[0110] In-orbit applications impose strict constraints on the computational overhead and storage resources of the model. Therefore, RAWDet is designed with a focus on reducing the number of parameters and computational load. To address this, this invention proposes the Ghost Lightweight Feature Pyramid (GLFP) to significantly reduce model complexity while maintaining detection accuracy.

[0111] Figure 4 The comparison between standard convolution and Ghost convolution in feature generation is shown: (a) Standard convolution generates all feature maps through time-consuming convolution operations. (b) Ghost convolution generates some inherent features and derives additional features through lightweight linear transformations, thereby reducing computational overhead.

[0112] Ghost convolutions are widely used in the design of lightweight networks, achieving or exceeding the feature learning capabilities of standard convolutions (SC) at a lower cost. The methods by which standard convolutions and Ghost convolutions generate feature maps are as follows: Figure 4 As shown. Standard convolution in generating When outputting a feature map, it is necessary to perform... Multiply-accumulate (MAC) operations have a parameter count and computational complexity that are proportional to the number of input channels. Number of output channels and kernel size Proportional. In contrast, Ghost convolution breaks this process down into two steps: first, it generates the main feature maps (called explicit features) through a small number of standard convolutions, the number of which is... Then, the remaining features are generated from these explicit features using a set of inexpensive channel-wise convolutions. There are one redundant feature. Therefore, the total computational cost is approximately:

[0113] (6)

[0114] in, This is the feature compression ratio (usually set to 2). The size of the channel-wise convolution kernel (e.g., 1×1 or 3×3). Ghost convolution can theoretically reduce the computational cost to approximately [a fraction of the original]. .

[0115] Inspired by Ghost convolutions, this invention designs a Ghost Bottleneck as a lightweight basic unit. It replaces the standard convolutional layers in a typical bottleneck module with Ghost convolutions, thereby significantly reducing network complexity without sacrificing feature representation capabilities. Figure 5 As shown. Figure 5 The comparison between the standard bottleneck structure and the Ghost bottleneck structure of this invention is shown: (a) The standard bottleneck uses two convolutional blocks and a residual shortcut for feature transformation. (b) The Ghost bottleneck replaces the standard convolution with ghost convolution to reduce computational cost; the dashed part is activated only during the downsampling stage.

[0116] Unlike a global replacement of the entire feature pyramid, GLFP employs a selective lightweight strategy: the front of the backbone network handles basic feature extraction, which is crucial for the model's recognition capabilities; therefore, the first two C3k2 modules are retained to maintain the integrity of low-level semantic features. The rear and neck layers of the backbone are primarily responsible for high-level feature reconstruction and fusion. In these layers, Ghost bottlenecks replace the standard bottlenecks in the original C3k2, forming the improved module GC3k2, achieving a balance between performance and complexity. Through this structural improvement, RAWDet achieves significant parameter compression and FLOP reduction with minimal loss of detection accuracy.

[0117] Application process

[0118] The application process of this invention can be summarized as a closed-loop system of "ground training - on-orbit deployment - intelligent detection - result download - model update", and the specific process is as follows.

[0119] Ground Training: Offline pre-training and optimization of the model are completed on the ground. Based on historical remote sensing imagery and a constructed degraded remote sensing dataset, the proposed on-orbit target detection model is thoroughly trained. During training, factors such as radiometric distortion, geometric distortion, and imaging noise commonly found in original remote sensing images are comprehensively considered. Through targeted network structure design and loss optimization, the model acquires robust feature extraction and target discrimination capabilities for uncorrected original images. After training, the model is validated, compressed, and its inference performance is evaluated on a ground-based high-performance computing platform, generating a final deployment model that meets the constraints of spaceborne computing resources.

[0120] The method for constructing the original remote sensing dataset containing radiometric and geometric distortions is as follows:

[0121] Based on the actual mission requirements, suitable publicly available remote sensing imagery datasets for target detection, such as DOTA, DIOR, and FGSC-23, are acquired. Then, simulation methods are used to degrade these mature remote sensing datasets, forming original remote sensing datasets that include both radiometric and geometric distortions. The degradation methods include three parts: geometric degradation, radiometric degradation, and additive degradation. Geometric degradation uses quadratic polynomials to adjust pixel positions, introducing geometric distortion. Radiometric degradation uses the 6S radiative transfer model to adjust pixel values, achieving radiometric distortion. Additive degradation introduces noise to reduce the signal-to-noise ratio, simulating the original imagery before denoising. This technique can produce Level 1 remote sensing imagery simulation data for in-orbit target detection model development without the need for data collection, cleaning, and annotation processes, significantly reducing data production cycle and cost, and providing reliable data support and knowledge transfer pathways for data-driven in-orbit detection model development. Taking the DOTA dataset as an example, we first performed a degradation simulation on the DOTA dataset, and then used the slicing and annotation tools provided by the official DOTA team to crop the images into sub-images of size 1024×1024. After slicing, the training set contains 15,749 images, and the validation set contains 6,297 images.

[0122] Due to the rich detail in remote sensing images, a high-resolution input size of 1024*1024 was used for training. To ensure sufficient model convergence, the training lasted for 300 epochs using the SGD optimizer. The initial learning rate was set to 0.01, the final learning rate ratio was set to 0.01, the momentum coefficient was set to 0.937, the weight decay was set to 0.0005, and the batch size was 16. The loss in object detection consisted of three parts: classification loss (BCE Loss), confidence loss (Obj Loss), and bounding box regression loss (CIoU Loss). Furthermore, data augmentation techniques such as HSV color enhancement, Mosaic enhancement, and left / right flipping were utilized during training. The specific training steps were as follows: 1. Input the dataset and labeled data into the network in batches for forward propagation. 2. The network outputs the prediction results and calculates the loss value with the labels. 3. The network performs backpropagation based on the loss value, updating the weight parameters through the SGD optimizer. 4. Repeat the above process continuously, with the loss value decreasing until the network's weight parameters stabilize.

[0123] In-orbit deployment: The trained model is uploaded and deployed to the satellite's in-orbit computing platform. Through the satellite-to-ground communication link, the validated model parameters and inference program are uploaded to the embedded computing unit within the satellite payload, and model initialization and runtime environment configuration are completed. This process can be completed during normal satellite operation without affecting the imaging tasks of existing remote sensing payloads, enabling rapid in-orbit deployment of the model.

[0124] Intelligent Detection: This function performs on-orbit target detection during satellite operation. After the remote sensing payload acquires raw image data, the images do not require complete ground-level radiometric and geometric correction processing; they are directly input into the on-orbit detection model for real-time or near-real-time inference. The model automatically identifies and locates targets in the images, outputting information such as target category, location, and confidence level, thereby achieving autonomous on-board perception and preliminary information filtering.

[0125] Results Download: After completing on-orbit detection, slice download and data filtering are performed based on the detection results. For detected regions of interest, the system automatically crops corresponding local images or target slices from the original imagery and prioritizes the download of relevant detection results and target image data. Compared to full image download, this method significantly reduces the amount of communication data, improves the utilization efficiency of the satellite-to-ground link, and enhances the timeliness of acquiring key information.

[0126] Model Updates: Continuous model updates and optimizations are achieved based on downlinked data. The ground-based system can utilize newly acquired samples, complex scene data, and false detection samples to retrain or incrementally optimize the model, and periodically generate updated model parameters. After validation, the new model version can be uploaded back to the satellite, enabling on-orbit iterative updates and gradually improving the system's detection performance under different regions and imaging conditions, forming a continuous evolution mechanism of "on-orbit application—ground optimization—redeployment."

[0127] In summary, this invention proposes an on-orbit target detection method for raw visible light imagery from remote sensing satellites. Through an end-to-end detection architecture, it directly processes uncorrected raw images on-board, thus skipping traditional preprocessing steps, significantly shortening the processing chain, saving onboard computing power, and improving the real-time performance of information interpretation. The important innovative contributions and outstanding features of this invention include:

[0128] (1) A target detection architecture RAWDet for original remote sensing images is proposed. An end-to-end detection framework is constructed, which can be directly applied to the original images, shorten the processing link and improve the model's adaptability to degradation features.

[0129] (2) An edge-aware input module (EIM) is proposed to enhance the stability of feature extraction and target recognition under degradation conditions by using explicit edge features.

[0130] (3) Design large target deformable attention enhancement (LDAE) to improve the detection accuracy and robustness of the model for geometrically distorted targets by deformable attention.

[0131] (4) Construct Ghost Lightweight Feature Pyramid (GLFP), which achieves structural lightweighting by introducing Ghost convolution in key layers, reducing the number of parameters and computational burden, making it suitable for on-orbit deployment.

[0132] Compared with the prior art, the significant technical advantages of the present invention are reflected in the following aspects:

[0133] Existing spaceborne target detection methods typically rely on complex radiometric correction, geometric correction, and data preprocessing processes performed on the ground. These methods are difficult to directly adapt to uncorrected raw remote sensing images and are limited in application due to constraints in spaceborne computing power, power consumption, and bandwidth. The on-orbit target detection scheme proposed in this invention is designed holistically for raw remote sensing images. By introducing a robust feature modeling mechanism targeting radiometric and geometric distortions, the model can directly and effectively detect targets in the raw images, significantly shortening the processing chain from data acquisition to target interpretation and improving the real-time performance and autonomy of on-orbit processing.

[0134] Furthermore, this invention fully considers the resource constraints of spaceborne embedded platforms at the model structure level. Through lightweight feature modeling and an efficient attention mechanism, it effectively reduces the model parameter scale and computational complexity while ensuring detection accuracy, thus improving the engineering feasibility of on-orbit deployment. Compared with existing methods that only pursue detection accuracy or simply rely on model compression, this invention achieves synergistic optimization of accuracy, efficiency, and robustness.

[0135] Furthermore, this invention constructs an application process that combines ground training with on-orbit inference, and supports the sliced ​​downlink of detection results and iterative model updates, effectively reducing the burden of satellite-to-ground communication, improving the efficiency of acquiring key target information, and forming a continuously evolving on-orbit intelligent sensing closed-loop system. In summary, this invention surpasses the closest existing technology in terms of adaptability to original remote sensing images, efficiency of spaceborne resource utilization, and completeness of engineering applications, demonstrating significant technological advancement and practical value.

[0136] This invention also provides a storage medium for storing a computer program, which, when executed, performs at least the methods described above.

[0137] This invention also provides a control device, including a processor and a storage medium for storing a computer program; wherein the processor executes the computer program by performing at least the method described above.

[0138] This invention also provides a processor that executes a computer program, at least performing the methods described above.

[0139] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc or CD-ROM; magnetic surface memory can be disk storage or magnetic tape storage. The storage media described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0140] In the several embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0141] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0142] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0143] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0144] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0145] The methods disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0146] The features disclosed in the several product embodiments provided by this invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0147] The features disclosed in the several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0148] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various equivalent substitutions or obvious modifications can be made without departing from the concept of the present invention, and all such modifications, achieving the same performance or application, should be considered within the scope of protection of the present invention.

Claims

1. An on-orbit target detection method for raw visible light imagery from remote sensing satellites, characterized in that, Includes the following steps: S1. Acquire raw visible light remote sensing images from satellite payloads, the raw remote sensing images being unprocessed by radiometric and geometric correction preprocessing; S2. Perform edge-aware enhancement processing on the original remote sensing image to strengthen the structural and edge features in the image; S3. Perform deformable attention enhancement on the edge-enhanced features to improve the model's feature perception and spatial adaptability to geometrically distorted targets. In step S3, the deformable attention enhancement processing specifically includes: generating a reference point grid based on the input feature map, and performing calculations on the query features generated by linearly projecting the input feature map through a light quantum network to predict the spatial offset of each reference point; resampling the input feature map according to the offset to obtain the deformable-aware feature representation; calculating key features and value features based on the resampled features, and inputting them together with the query features into a multi-head self-attention module for calculation to obtain the enhanced feature representation; S4. Multi-scale feature extraction and fusion are performed through a feature pyramid structure to maintain feature expressiveness while reducing computational complexity. S5. Target detection is performed based on the fused features, and the target category, location and confidence information are output to achieve on-orbit end-to-end raw image target recognition.

2. The on-orbit target detection method for raw visible light images from remote sensing satellites as described in claim 1, characterized in that, In step S2, the edge-aware enhancement processing specifically includes: High-frequency feature information is extracted from the input image, and the high-frequency feature information is calculated by at least one of the following methods: the difference between the input image and the image after average pooling; or the gradient intensity of the input image is calculated based on the gradient direction convolution kernel; The high-frequency feature information is mapped to an edge feature map through a convolutional layer and an activation function. The edge feature map is channel-stitched with the original input image, and the intensity of the edge feature map in the fusion is controlled by a learnable factor to form an enhanced fused input feature, so as to provide an explicit structural prior in the input stage.

3. The on-orbit target detection method for raw visible light imagery from remote sensing satellites as described in claim 1, characterized in that, In step S2, during the edge-aware enhancement process, high-frequency feature information is mapped to an edge feature map through convolution and the Sigmoid activation function. The edge feature map is then fused with the original input image via channel stitching.

4. The on-orbit target detection method for raw visible light imagery from remote sensing satellites as described in claim 1 or 2, characterized in that, In step S3, the deformable attention enhancement processing specifically further includes: A deformable attention mechanism is introduced at the junction of the backbone network and the neck network. The enhanced feature representation is directly passed to the large target detection head to enhance the model's ability to detect large targets with geometric distortions.

5. The on-orbit target detection method for raw visible light imagery from remote sensing satellites as described in claim 4, characterized in that, Step S3 also includes: Deformable attention computation is performed before the output features of the backbone network are fused into the neck features; The direct connection between the large detection head and the previous module in the original structure was removed, and the output of the deformable attention module was directly connected to the large detection head instead.

6. The on-orbit target detection method for raw visible light imagery from remote sensing satellites as described in claim 1, characterized in that, In step S4, the lightweight feature pyramid structure specifically includes: In the back-end and neck modules of the backbone network, Ghost convolutions are used to replace some standard convolutions; Ghost convolution reduces the overall computational cost by first generating a main feature map using a small number of standard convolutions, then deriving redundant feature maps from the main feature map using a set of inexpensive channel-wise convolutions, and finally stitching all the feature maps together to form a complete output. Standard convolutional modules are retained at the front of the backbone network to maintain low-level semantic integrity, while lightweight bottleneck modules containing Ghost convolutions are used in the rear and neck layers of the backbone network for feature reconstruction and fusion.

7. The on-orbit target detection method for raw visible light imagery from remote sensing satellites as described in claim 6, characterized in that, In step S4, the lightweight feature pyramid structure is constructed by replacing the standard convolution in the standard bottleneck structure with Ghost convolution, and additional depthwise separable convolutional paths are enabled during the downsampling stage to maintain feature representation capability.

8. The on-orbit target detection method for raw visible light imagery from remote sensing satellites as described in claim 1, characterized in that, It also includes the model training and on-orbit deployment process, specifically including: At the ground level, the detection model is trained and optimized based on the original remote sensing dataset containing radiation and geometric distortions; The trained model is uploaded to the satellite's on-orbit computing platform for deployment; During operation in orbit, the original remote sensing images are directly input into the model for target detection, and the detection results are output. Based on the detection results, the target area of ​​interest is sliced ​​and extracted, and then downloaded preferentially. The model is iteratively updated and optimized based on the downlink data, forming a closed-loop evolution mechanism.

9. The on-orbit target detection method for raw visible light imagery from remote sensing satellites as described in claim 8, characterized in that, In the model training and on-orbit deployment process, model updates specifically include: The model can be retrained or incrementally optimized using new samples acquired in orbit, complex scene data, and false detection samples. The updated model parameters are periodically generated, verified, and then re-uploaded to the satellite to achieve continuous evolution of the on-orbit model.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the on-orbit target detection method for raw visible light images from remote sensing satellites as described in any one of claims 1 to 9.

Images