Satellite on-orbit target detection and identification method and system and storage medium

Abstract

The application discloses a satellite in-orbit ground target detection and identification method and system and a storage medium, and belongs to the technical fields of remote sensing image intelligent interpretation, satellite-borne intelligent processing, lightweight neural networks and visual state space models. The method comprises the following steps: pre-processing remote sensing images, wherein the pre-processing comprises normalization, cropping and size adjustment; and inputting the pre-processed remote sensing images into a trained multi-stage network model to obtain target identification results or target detection results. The embodiment of the application aims at solving problems such as limited satellite in-orbit computing resources, small remote sensing target size and large attitude change, limited receptive field of traditional visual models, insufficient precision of lightweight models and the like, and constructs a remote sensing image target detection and identification model which has low computing overhead, high identification precision and fast reasoning capability, thereby providing algorithm support for on-board real-time intelligent interpretation, fast screening and low-latency information downlink.

Description

Technical Field

[0001] This invention relates to a method, system, and storage medium for on-orbit ground target detection and identification via satellite, belonging to the technical fields of intelligent interpretation of remote sensing images, on-board intelligent processing, lightweight neural networks, and visual state space models. Background Technology

[0002] With the continuous improvement of satellite payload resolution and observation frequency, the amount of remote sensing image data is growing exponentially. Traditional processing methods typically involve downloading raw images to ground servers before target detection, identification, or scene interpretation, which is insufficient to meet the stringent real-time requirements of tasks such as emergency response, maritime target surveillance, and airport / port monitoring. To reduce data download pressure and improve mission response speed, on-orbit satellite computing has become an important development direction for intelligent remote sensing processing.

[0003] Existing intelligent interpretation models for remote sensing images mainly include Convolutional Neural Networks (CNNs), Transformer visual models, and the emerging visual Mamba model based on the Selective State Space (SSM) architecture. CNNs extract local features such as texture and edges using local convolutional kernels, achieving relatively low complexity, but are limited by their receptive field, making it difficult to efficiently model long-range dependencies. Transformers rely on self-attention mechanisms and possess excellent global context modeling capabilities, but their computational complexity and memory overhead increase quadratically with image resolution. The visual Mamba model, based on the selective state space mechanism, combines linear complexity with long-range dependency modeling capabilities, thus demonstrating great potential for on-board computing.

[0004] Although some Mamba-like methods have been attempted for tasks such as remote sensing image classification, change detection, semantic segmentation, and road extraction, and some lightweight Mamba models have been adapted to general edge devices through parameter compression or structural simplification, these methods are mostly designed for ground servers or natural image scenes. Their model structure, scanning method, and feature modeling mechanism have not fully considered the stringent resource constraints of satellite on-orbit platforms, nor have they adapted to the inherent characteristics of remote sensing images, such as arbitrary target orientation, rich oblique structure, and subtle inter-class differences.

[0005] Existing CNN-based remote sensing target detection methods typically follow a process of "image preprocessing—backbone network feature extraction—feature pyramid fusion—detection / classification head output." While these methods excel at extracting local textures and edges by using convolutional sliding windows to obtain local neighborhood information, they tend to lose global structural information in complex backgrounds, oblique ships, multi-scale targets, and elongated target scenes.

[0006] Existing Transformer-type methods typically segment images into a sequence of image patches and model global dependencies using a multi-head self-attention mechanism. Although these methods possess strong global modeling capabilities, the computational complexity of the self-attention mechanism increases quadratically with spatial size. Direct deployment on spaceborne embedded processors (GPUs) faces severe challenges, including high memory consumption, large computational latency, and high power consumption.

[0007] Existing visual Mamba-like methods typically flatten two-dimensional image features into a one-dimensional sequence and use horizontal or vertical Z-shaped scans for state space modeling. While this process can capture long-range dependencies with low complexity, conventional horizontal / vertical scans are insufficient to effectively characterize diagonal adjacency relationships in remote sensing imagery, such as diagonal runways, diagonal ships, road boundaries, and building outlines. Furthermore, relying solely on modeling in a single or a few directions of the spatial dimension makes it difficult to fully exploit the semantic response differences inherent in the channel dimension.

[0008] The shortcomings of existing technologies: (1) Insufficient computing resources: When the ground server model is directly migrated to the satellite on-orbit platform, the number of parameters, FLOPs and video memory are large, which makes it difficult to meet the requirements of low power consumption, low latency and long-term unattended operation on the satellite.

[0009] (2) Insufficient spatial adjacency modeling: Conventional visual Mamba often uses horizontal or vertical Z-shaped scanning, which cannot fully cover the diagonal adjacency relationships in remote sensing images, resulting in insufficient extraction of discriminative features for oblique target structures, oblique boundaries and obliquely arranged targets.

[0010] (3) Insufficient coupling between channel and spatial information: Some lightweight methods only focus on parameter compression or local operator replacement, lacking a joint scanning mechanism that simultaneously addresses height, width, diagonal and channel dimensions, making it difficult to take into account long-range spatial context and channel dependencies.

[0011] (4) It is difficult to balance accuracy and efficiency: Although lightweight networks with two stages or oversampling can achieve faster inference speed, they will lose fine-grained target features; although deeper or wider networks can slightly improve accuracy, they will significantly increase computational overhead and inference latency, and are not suitable for on-board deployment.

[0012] (5) High difficulty in remote sensing target identification: Targets in remote sensing images often have the characteristics of small scale, arbitrary posture, complex background, subtle differences between classes and dramatic changes within classes. Existing general lightweight networks are prone to missed detection, false detection and class confusion in fine-grained remote sensing target identification and complex scene detection. Summary of the Invention

[0013] In view of this, the present invention provides a method, system and storage medium for on-orbit ground target detection and recognition on satellites. It aims to address the problems of limited on-orbit computing resources of satellites, small scale and large attitude changes of remote sensing targets, limited receptive field of traditional visual models and insufficient accuracy of lightweight models, and to build a remote sensing image target detection and recognition model with low computational overhead, high recognition accuracy and fast reasoning ability. It provides algorithmic support for real-time intelligent interpretation, rapid screening and low-latency information downlink on satellites.

[0014] The first objective of this invention is to provide a method for detecting and identifying ground targets in orbit via satellite.

[0015] The second objective of this invention is to provide a satellite-based on-orbit ground target detection and identification system.

[0016] A third objective of this invention is to provide a storage medium.

[0017] The first objective of this invention can be achieved by adopting the following technical solution: A method for detecting and identifying ground targets in orbit via satellite, the method comprising: The remote sensing image is preprocessed, including normalization, cropping, and resizing. The preprocessed remote sensing image is input into a trained multi-stage network model to obtain target recognition or target detection results. The multi-stage network model includes an image patch embedding layer, multiple remote sensing state space modules, a global average pooling layer, and a classifier or feature pyramid and task head. The remote sensing state space module includes a remote sensing state space computation unit and a feedforward network embedded after deep convolution. The remote sensing state space computation unit is configured to perform multi-directional scanning, generate fused features, and includes a channel state space branch, a spatial state space branch, and a spatial-channel feature fusion module. The multi-directional scanning includes eight spatial scans: horizontal forward, horizontal reverse, vertical forward, vertical reverse, main diagonal forward, main diagonal reverse, anti-diagonal forward, and anti-diagonal reverse, as well as two channel scans: channel forward and channel reverse.

[0018] Preferably, the multi-stage network model is configured as follows: The preprocessed remote sensing image is processed by the image patch embedding layer to obtain the embedding features. The image patch embedding layer includes a convolution-batch normalization-activation function module and a convolution-batch normalization module. The embedded features are processed according to the three-stage structure to obtain lightweight features. The three-stage structure includes a first-stage structure, a second-stage structure, and a third-stage structure. The first-stage structure, the second-stage structure, and the third-stage structure include a remote sensing state space module. Depth-separable convolutional downsampling layers are set between adjacent stages. For the recognition task, the target recognition result is obtained based on the global average pooling layer, the classifier, and the lightweight features. For the detection task, the target detection result is obtained based on the feature pyramid, the task head, and the lightweight features.

[0019] Preferably, the remote sensing state space computing unit is configured to perform multi-directional scanning and generate fused features, including: Eight spatial scans are performed on the feature map, and the scans are stacked to obtain an eight-directional spatial scan feature. In the channel state space branch, the eight-directional spatial scan features are merged to obtain a four-dimensional feature representation. Global pooling is then performed on the four-dimensional feature representation in the height and width dimensions to obtain the channel feature vector. Perform two channel scans along the channel dimension on the channel feature vector to obtain multiple channel scan features; The stacked channel scanning features are fed into the state space model to model channel dependencies, and then merged to obtain channel modeling features. In the spatial state space branch, the eight-directional spatial scanning features are fed into the state space model to perform spatial multi-directional state modeling, obtain spatial modeling features, and merge them to obtain spatial enhancement features. The spatial enhancement feature and the channel modeling feature are modulated element-wise by multiplication, and then the modulation result is added element-wise to the spatial enhancement feature itself to obtain the fused feature.

[0020] Preferably, the remote sensing state space module further includes a first normalization layer and a second normalization layer.

[0021] Preferably, the remote sensing state space computation unit embedded after the depth convolution is located within the depth separable state space module, which further includes a first convolutional layer and a second convolutional layer. The first convolutional layer is set before the depth convolution, and the second convolutional layer is set after the remote sensing state space computation unit.

[0022] Preferably, the first normalization layer, the depth-separable state space module, and the first residual connection are arranged sequentially; the second normalization layer, the feedforward network, and the second residual connection are arranged sequentially.

[0023] Preferably, the feedforward network includes a third convolutional layer, two parallel deep convolutional branches, a multiplication unit for fusing the outputs of the two parallel deep convolutional branches, and a fourth convolutional layer connected in sequence.

[0024] Preferably, the steps for generating the main diagonal scan and the anti-diagonal scan include: Enumerate the diagonal numbers of the feature map, and extract the main diagonal feature sequence and the anti-diagonal feature sequence according to the preset index relationship; Perform reverse flipping operations on the extracted main diagonal feature sequence and antidiagonal feature sequence respectively to generate the corresponding reverse sequence; The main diagonal forward sequence, the main diagonal reverse sequence, the anti-diagonal forward sequence, and the anti-diagonal reverse sequence are stacked to obtain the four-directional diagonal scanning features.

[0025] The second objective of this invention can be achieved by adopting the following technical solution: A satellite-based on-orbit ground target detection and identification system, the system comprising: The preprocessing module is used to preprocess remote sensing images, including normalization, cropping, and resizing. The recognition or detection module is used to input preprocessed remote sensing images into a trained multi-stage network model to obtain target recognition or target detection results. The multi-stage network model includes an image patch embedding layer, multiple remote sensing state space modules, a global average pooling layer, a classifier or feature pyramid, and a task head. The remote sensing state space module includes a remote sensing state space computation unit and a feedforward network embedded after deep convolution. The remote sensing state space computation unit is configured to perform multi-directional scanning, generate fused features, and includes a channel state space branch, a spatial state space branch, and a spatial-channel feature fusion module. The multi-directional scanning includes eight spatial scans: horizontal forward, horizontal reverse, vertical forward, vertical reverse, main diagonal forward, main diagonal reverse, anti-diagonal forward, and anti-diagonal reverse, as well as two channel scans: channel forward and channel reverse.

[0026] The third objective of this invention can be achieved by adopting the following technical solution: A storage medium storing a program, which, when executed by a processor, implements the above-described method for detecting and identifying ground targets on orbit for satellites.

[0027] The present invention has the following advantages over the prior art: (1) Balancing linear complexity and long-range modeling capability. This invention inherits the linear complexity advantage of the Mamba state-space model, overcomes the problems of high computational complexity and large memory consumption faced by the Transformer self-attention mechanism when processing high-resolution remote sensing images, and at the same time maintains excellent long-range context modeling capability.

[0028] (2) Significantly enhances the modeling of diagonal spatial adjacency relationships. By introducing a main diagonal and anti-diagonal scanning mechanism, this invention explicitly captures the diagonal adjacency relationships in oblique ships, oblique roads, building boundaries, and arbitrarily rotated targets, effectively making up for the limitations of traditional horizontal and vertical scanning in extracting oblique structural features.

[0029] (3) Achieve synergistic enhancement of spatial and channel information. The HWDC scanning strategy integrates height, width, diagonal direction and channel dimension into the state space modeling framework. The RSSSM computing unit further integrates the spatial Mamba branch and the channel Mamba branch to achieve synergistic enhancement of spatial structure and channel semantics, which significantly improves the model's ability to extract discriminative features of complex remote sensing targets.

[0030] (4) Achieve a better balance between accuracy and efficiency in space scenarios. The three-stage minimalist network architecture avoids feature loss caused by oversampling in two stages while effectively avoiding computational redundancy caused by the four-stage structure. It achieves the best balance between accuracy and inference efficiency and is particularly suitable for the resource-constrained deployment requirements of spaceborne embedded GPUs.

[0031] (5) It has the characteristics of task versatility and engineering deployability. This invention can be used directly as an independent backbone network for target recognition, or it can be seamlessly embedded into rotating target detection frameworks such as AOPG, LO-Det, and S2ANet; and it supports deployment to spaceborne embedded platforms such as Jetson TX2 through optimization methods such as quantization, pruning and operator fusion, which has extremely high engineering practical value. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0033] Figure 1 This is a flowchart of a satellite-based on-orbit ground target detection and identification method according to Embodiment 1 of the present invention.

[0034] Figure 2 This is a schematic diagram of a LightMamba embodiment 1 of the present invention.

[0035] Figure 3 This is a schematic diagram of an RSSSM Block according to Embodiment 1 of the present invention.

[0036] Figure 4 This is a schematic diagram of an RSSSM calculation unit according to Embodiment 1 of the present invention.

[0037] Figure 5 This is a schematic diagram of an HWDC scanning strategy according to Embodiment 1 of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0039] For ease of understanding, some of the core terms used in this invention are defined as follows: LightMamba: refers to the ultra-lightweight visual Mamba model (referred to as the multi-stage network model) proposed in this invention for satellite on-orbit ground target detection and recognition, which serves as the backbone network architecture of this invention.

[0040] HWDC scanning strategy: The full name is Height-Width-Diagonal-Channel scanning strategy. It refers to a joint multi-directional scanning mechanism that covers the height, width, main / anti-diagonal lines and channel dimensions of the feature map. It is used to solve the problem of insufficient extraction of oblique target features in remote sensing images.

[0041] Remote Sensing State Space Module (RSSSM): This is the core algorithm unit integrated within the depth separable state space module. It is configured to execute the HWDC scanning strategy and spatial-channel feature fusion.

[0042] Depthwise Separable State Space Module (DSSSM): This is a lightweight structural unit that carries the remote sensing state space computing unit. It is composed of a depthwise separable convolution and a remote sensing state space computing unit connected in series.

[0043] Feedforward Network (EFFN): The full name is Efficient Feed-Forward Net. It is a nonlinear enhancement unit in the Remote Sensing State Space Block (RSSSM Block). It realizes feature interaction through parallel deep convolution branches and multiplication units.

[0044] This invention proposes an ultra-lightweight visual Mamba scheme for on-orbit target detection and recognition on satellites. Based on the linear complexity and long-range dependency modeling capabilities of the Mamba model, this scheme introduces a diagonal scanning method tailored to the characteristics of remote sensing images, a HWDC ten-direction scanning strategy, and an RSSSM computational unit. Furthermore, it achieves a balance between high accuracy and low overhead through a three-stage lightweight network structure. The specific implementation scheme is as follows: Example 1: like Figure 1 As shown, this embodiment provides a method for detecting and identifying ground targets in orbit via satellite. The method includes: S101: Preprocess the remote sensing image, the preprocessing including normalization, cropping and resizing.

[0045] In this embodiment, the remote sensing image to be processed is acquired and preprocessed, including normalization, cropping, and resizing, to obtain the input image tensor X∈R. {H×W×3} In target recognition tasks, the input can be a slice of the target; in target detection tasks, the input can be an image patch cropped from a large remote sensing image.

[0046] S102: Input the preprocessed remote sensing image into the trained multi-stage network model to obtain target recognition or target detection results. The multi-stage network model includes an image patch embedding layer, multiple remote sensing state space modules, and a global average pooling layer and a classifier or feature pyramid and task head. The remote sensing state space module includes a remote sensing state space computation unit and a feedforward network embedded after deep convolution. The remote sensing state space computation unit is configured to perform multi-directional scanning, generate fused features, and includes a channel state space branch, a spatial state space branch, and a spatial-channel feature fusion module. The multi-directional scanning includes eight spatial scans: horizontal forward, horizontal reverse, vertical forward, vertical reverse, main diagonal forward, main diagonal reverse, anti-diagonal forward, and anti-diagonal reverse, as well as two channel scans: channel forward and channel reverse (i.e., state space modeling in ten directions).

[0047] In some embodiments, as shown in Table 1 and as shown in Table 2 Figure 2 As shown, the multi-stage network model is configured as follows: S11: The preprocessed remote sensing image is processed according to the image patch embedding layer to obtain the embedding features. The image patch embedding layer includes a convolution-batch normalization-activation function (Conv-BN-ReLU) module and a convolution-batch normalization (Conv-BN) module.

[0048] In this step, the input image tensor is fed into the Patch Embedding layer. This layer consists of three Conv-BN-ReLU modules and one Conv-BN module cascaded together. Each convolutional module has a stride of S=2, which progressively downsamples the input features to H / 16×W / 16×C1 to reduce the computational cost of subsequent state space modeling.

[0049] S12: The embedded features are processed according to the three-stage structure to obtain lightweight features. The three-stage structure includes a first stage (Stage 1) structure, a second stage (Stage 2) structure and a third stage (Stage 3) structure. The first stage structure, the second stage structure and the third stage structure include a remote sensing state space module. Depth-separable convolutional downsampling layers are set between adjacent stages.

[0050] In this step, Stage 1, Stage 2, and Stage 3 contain L1, L2, and L3 RSSSM Blocks, respectively, typically set to L1=1, L2=2, and L3=2; Downsampling is set between adjacent stages to make the feature scales H / 16×W / 16, H / 32×W / 32, and H / 64×W / 64, respectively.

[0051] S13: For the recognition task, the target recognition results (ship1~shipN) are obtained based on the GlobalAvgPool layer, the classifier, and the lightweight features.

[0052] S14: For the detection task, the target detection result is obtained based on the feature pyramid, the task head, and the lightweight features.

[0053] Steps S13 and S14 are AND / OR relationships; in this embodiment, different output heads are connected according to the task type. For example, in the target recognition task, the target category is output through the GlobalAvgPool layer and the fully connected Classifier; in the target detection task, LightMamba is embedded as the backbone network into the rotating target detection framework, and the category, confidence and rotation box parameters are output through the task head.

[0054] Table 1. LightMamba Network Structure Configuration Diagram

[0055] It is worth noting that LightMamba employs a minimalist architecture with a single path and three concatenated stages. Input features are processed by an image patch embedding layer and then fed into the first stage to extract shallow semantic and local structural features. Subsequently, they are downsampled and fed into the second stage to construct mid-level contextual features. Finally, they are downsampled again and fed into the third stage to form high-level semantic features. In the target recognition task, the features output from the third stage are sequentially processed by a global average pooling layer and a classifier to output the target category. In the target detection task, the global average pooling layer and classifier are selectively replaced by a feature pyramid network and a corresponding detection head.

[0056] The aforementioned three-stage network architecture is the result of a comprehensive trade-off between recognition accuracy, number of parameters, computational cost (FLOPs), and inference speed. Compared to a two-stage architecture, this structure can effectively avoid the loss of small targets and fine-grained features due to oversampling; compared to a four-stage architecture, this structure can significantly suppress the inappropriate growth of computational cost and number of parameters while maintaining a diminishing marginal gain in accuracy, thus adapting to application scenarios where satellite on-orbit resources are limited.

[0057] In some embodiments, such as Figure 3 As shown, the remote sensing state space module further includes a first normalization (LN) layer and a second normalization layer. The remote sensing state space computation unit embedded after the depthwise convolution (DConv) is located within the depthwise separable state space module, which also includes a first convolution (Conv) layer and a second convolution layer. The first convolution layer is placed before the depthwise convolution, and the second convolution layer is placed after the remote sensing state space computation unit. The first normalization layer, the depthwise separable state space module, and the first residual connection are arranged sequentially; the second normalization layer, the feedforward network, and the second residual connection are arranged sequentially. The feedforward network includes a third convolution layer connected in sequence, two parallel depthwise convolution branches, a multiplication unit for fusing the outputs of the two parallel depthwise convolution branches, and a fourth convolution layer.

[0058] In this embodiment, the overall computation process of the RSSSM Block includes: the input features are first processed by LN and DSSSM to obtain enhanced features, and then the residuals are added to the original input; subsequently, LN and EFFN are used for further channel mixing and local structure enhancement, and the residuals are added to the features from the previous stage to obtain the feature output. This structure improves feature representation ability and training stability while maintaining lightweight design.

[0059] It is worth noting that DSSSM is built on a depthwise separable convolution framework, embedding RSSSM computation units after depthwise convolution. This design utilizes depthwise separable convolution to reduce computational cost, while leveraging RSSSM to enhance multi-directional spatial and channel dependency modeling capabilities. EFFN also adopts a lightweight design. Its overall computation process includes: first, projecting the input to a lower dimension through a 1×1 convolution, then feeding it into two parallel depthwise convolution branches; performing element-wise multiplication on the outputs of the two branches to enhance non-linear interactions, and finally restoring the target channel dimension through a 1×1 convolution.

[0060] In this embodiment, the features after multi-directional scanning are enhanced by the DSSSM module and EFFN module in the RSSSM Block, and the stable propagation of features is maintained by residual connection, resulting in lightweight features with strong spatial-channel context representation capabilities.

[0061] In some embodiments, such as Figure 4 As shown, the remote sensing state space computing unit is configured to perform multi-directional scanning and generate fused features, including: S21: Perform eight flat full scans (FFS) on the feature map and stack them to obtain an eight-directional flat scan feature x. s =FFS(x), where x s ∈R {B×8C×HW} .

[0062] In this embodiment, the input to the RSSSM calculation unit can be a remote sensing image feature map x∈R {B×C×H×W} The output is an enhanced feature with the same spatial size as the input.

[0063] S22: In the Channel Mamba branch, the eight-directional spatial scan features are merged to obtain a four-dimensional feature representation x. m ∈R {B×C×H×W} Global pooling is performed on the four-dimensional feature representation in the height and width dimensions to obtain the channel feature vector x. p ∈R {B×1×C} This eliminates the dependence on fixed spatial dimensions and reduces computational load.

[0064] In this step, in Channel Mamba, features x from eight spatial directions s After being segmented, the segments are added together and finally merged into a unified feature representation x. m =Merge(x s ).

[0065] S23: Perform two channel scans along the channel dimension on the channel feature vector to obtain the channel scan feature x. c ∈R {B×2×C} , where x c =CS(x p ), CS(·) represents the scan operation.

[0066] In this step, for x p Perform forward and reverse scans respectively.

[0067] S24: The stacked channel scan features are fed into the state-space model (SSM module) for channel dependency modeling, and the channel modeling features f are obtained by merging. m1 ∈R {B×C×1×1} .

[0068] In this step, these x's are stacked. c Input into the Mamba module SSM(·) to extract channel-level dependencies, resulting in f1=SSM(x c ), f m1 =Merge(f1).

[0069] S25: In the Spatial Mamba branch, the eight-directional spatial scan features are fed into the State Space Model (SSM module) for multi-directional spatial state modeling to obtain spatial modeling feature f2, which is then merged to obtain spatial enhancement feature f. m2 ∈R {B×C×H×W} .

[0070] In this step, f2 = SSM(x) s ), f m2 =Merge(f2).

[0071] S26: Apply the spatial enhancement feature f m2 With the channel modeling feature f m1 Element-wise multiplication is performed, and the modulation result is then combined with the spatial enhancement feature f. m2 The fusion feature f is obtained by performing element-wise multiplication addition. f =f m2 f m1 f m2 This feature fusion method preserves the multi-directional spatial structure while using channel weights to emphasize key semantic responses.

[0072] Furthermore, joint scanning is performed in the height, width, diagonal, and channel directions. Specifically, such as... Figure 5 As shown, HWDC comprises ten directions: two directions for horizontal Z-scanning (positive and negative), two directions for vertical Z-scanning (positive and negative), two directions for main diagonal scanning (positive and negative), two directions for anti-diagonal scanning (positive and negative), and two directions for channel dimension (positive and negative). The first eight spatial directions constitute Flat Full Scanning, used to cover multi-directional spatial adjacency relationships in remote sensing imagery; channel scanning is used to model response dependencies between different semantic channels.

[0073] It is worth noting that, compared with methods that only use horizontal or vertical scanning, HWDC can explicitly capture oblique structures and target contours in arbitrary directions that are common in remote sensing images; compared with methods that only use spatial scanning, HWDC further introduces channel dependency modeling, which enables spatial structural features and channel semantic responses to be synergistically enhanced within the state space framework.

[0074] Compared to methods that only use horizontal or vertical scanning, HWDC can explicitly capture oblique structures and target contours in arbitrary directions that are common in remote sensing images; compared to methods that only use spatial scanning, HWDC further introduces channel dependency modeling, which enables spatial structural features and channel semantic responses to be synergistically enhanced within the state space framework.

[0075] Furthermore, given that conventional visual Mamba models rely solely on horizontal or vertical Z-shaped scans, which are insufficient to fully capture the unique diagonal adjacency relationships of oblique targets in remote sensing images, this invention proposes a diagonal scanning mechanism. This mechanism encompasses both main diagonal scanning and anti-diagonal scanning: main diagonal scanning is configured to extract feature positions along a physical path from the upper left to the lower right, while anti-diagonal scanning is configured to extract feature positions along a physical path from the upper right to the lower left. To further enhance the bidirectional dependency of state-space modeling, forward and reverse traversals are performed on the main diagonal and anti-diagonal sequences, respectively, thereby constructing four diagonal scanning sequences covering the main diagonal forward, main diagonal reverse, anti-diagonal forward, and anti-diagonal reverse directions.

[0076] In some embodiments, the steps of generating the main diagonal scan and the anti-diagonal scan include: S31: Enumerate the diagonal numbers of the feature map, and extract the main diagonal feature sequence and the anti-diagonal feature sequence according to the preset index relationship.

[0077] S32: Perform reverse flipping operations on the extracted main diagonal feature sequence and antidiagonal feature sequence respectively to generate the corresponding reverse sequence.

[0078] S33: Stack the main diagonal forward sequence, the main diagonal reverse sequence, the anti-diagonal forward sequence, and the anti-diagonal reverse sequence to obtain the four-direction diagonal scanning features.

[0079] Specifically, the input feature map is defined as x∈R^{B×C×H×W}, where B is the batch size, C is the number of channels, and H and W are the height and width of the feature map, respectively. First, the diagonal numbers are enumerated according to s=0,1,...,H+W-2, and the coordinates (i,j) on each diagonal are calculated, where i represents the row index and j represents the column index. For the main diagonal direction, the column index satisfies the relation j=si; for the anti-diagonal direction, the column index satisfies the relation j=W-1-(si). Then, the two-dimensional feature is flattened into a one-dimensional feature vector xflat∈R^{B×C×HW}. Feature selection (IndexSelect) is performed according to the main diagonal index and the anti-diagonal index to obtain the main diagonal feature sequence ymain and the anti-diagonal feature sequence yanti. Then, a flip operation is performed on these two sequences to obtain the main diagonal reverse sequence ymain_rev and the anti-diagonal reverse sequence yanti_rev. Finally, the Stack is a four-directional diagonal scan feature y∈R^{B×4×C×HW}. This process maintains linear complexity, making it suitable for scenarios with limited onboard resources.

[0080] After training, validation, and compression deployment preparation are completed on the ground server, the aforementioned model will be deployed to a spaceborne embedded GPU or edge computing board to achieve on-orbit target detection or recognition of remote sensing images. Specifically, the training process of the model is executed on the ground server. For target detection tasks, large-scale fine-grained remote sensing target detection datasets such as FAIR1M can be used for training, and the input images are cropped into 1024×1024 image blocks; for target recognition tasks, fine-grained ship recognition datasets such as FGSC-23 can be used for training. After the model is trained, its weights are deployed to an NVIDIA Jetson TX2 or a spaceborne analog embedded GPU platform with equivalent computing power to perform inference validation. Performance evaluation metrics include: mean accuracy (mAP), frames per second (FPS), floating-point operations (FLOPs), and number of parameters for target detection tasks; and recognition accuracy, macro-average precision, macro-average recall, and macro-average F1 score for target recognition tasks. The experimental results aim to demonstrate the excellent balance achieved by this invention between extremely low computational overhead and high-precision recognition performance. It should be understood that the implementation of this invention is not limited to the specific datasets, training hyperparameters, or hardware platforms described above, and any equivalent substitutions based on the concept of this invention should fall within the protection scope of this invention.

[0081] Specifically, this embodiment uses the publicly available benchmark dataset FGSC-23, specifically designed for fine-grained ship classification of optical remote sensing images. All model training was performed on a high-performance server with the following hardware configuration: an Intel Xeon E5-2640v4 @ 2.40GHz processor (40 threads), two NVIDIA GeForce RTX 3090 graphics cards, and a 64-bit Ubuntu 24.04.3LTS operating system. For the software environment, Python 3.11.12 and the PyTorch 2.1.2 deep learning framework were used. The model hyperparameters were set as follows: the optimizer was AdamW, with momentum parameters β1=0.9, β2=0.999; the initial learning rate was 0.02, multiplied by a decay factor of 0.1 at the 200th and 300th epochs; the batch size was set to 64, and the total number of training epochs was 400; the cross-entropy loss function was used; and the random seed was fixed at 42 to ensure experimental reproducibility. The experimental results are as follows: Table 2. The Balance Advantages of the Three-Phase Architecture Compared to Two-Phase and Four-Phase Architectures

[0082] As shown in Table 2, the three-stage structure increases the computational load by only a small amount compared to the two-stage structure, but significantly improves the accuracy. Compared to the four-stage structure, the three-stage structure achieves similar recognition accuracy with lower FLOPs, higher throughput and fewer parameters, which is more in line with the comprehensive needs of satellite on-orbit computing.

[0083] Table 3. Effects of HWDC scanning strategy compared to single or partial combined scanning

[0084] As shown in Table 3, scanning in a single spatial direction is insufficient to fully characterize the structure of remote sensing targets; the accuracy is steadily improved after introducing channel scanning; the complete HWDC strategy achieves the highest accuracy, indicating that the comprehensive modeling of height H, width W, diagonal D and channel dimension C plays a key role in remote sensing target identification.

[0085] Table 4 shows the comparative results when detecting backbone networks.

[0086] As shown in Table 4, among the three rotating target detection frameworks—Adaptive Direction-Aware Progressive Growth Network (AOPG), Local Overlap-Aware Detection Network (LO-Det), and Spatial Alignment Single-Stage Anchor Frame Network (S2ANet)—LightMamba as the backbone network achieved the highest mAP. This indicates that the present invention is not only effective for a single classification task, but also has the adaptability to serve as a general remote sensing detection backbone network.

[0087] Table 5 Comparison of FGSC-23 fine-grained target recognition task performance

[0088] As shown in Table 5, LightMamba achieves an accuracy of 97.30% and a Macro F1 score of 97.29% on the FGSC-23 dataset, while maintaining 264 MFLOPs, 8.9M parameters, and a throughput of 111 images / s, demonstrating the combined advantages of high accuracy and high efficiency.

[0089] In some embodiments, the present invention provides an ultra-lightweight visual Mamba device for on-orbit target detection and recognition on satellites, the device comprising: (1) Remote sensing image access module, used to receive optical remote sensing images acquired by satellite payload, as well as pre-processed image slices or large image blocks; (2) Preprocessing module, used to perform scale normalization, pixel value normalization, image cropping and batch processing on the input image; (3) Patch Embedding module, used to map the original image into a low-resolution, high-dimensional feature representation through continuous convolutional downsampling operations; (4) Multi-directional scan generation module, used to generate multi-directional scan indexes and scan sequences covering horizontal, vertical, main diagonal, anti-diagonal and channel dimensions; (5) RSSSM feature extraction module, used to extract highly discriminative remote sensing target features through spatial Mamba branch, channel Mamba branch and spatial-channel feature fusion module; (6) Task output module, used to output classification results, target confidence, horizontal bounding box or rotated bounding box parameters according to the target task type; (7) On-board deployment and inference module, used to convert the model trained on the ground into an inference model adapted to the on-board embedded GPU, NPU, FPGA or edge AI acceleration processor, and perform on-orbit real-time inference.

[0090] In one optional implementation, the device is configured as a fine-grained target recognition model, directly outputting the remote sensing target category via the task output module. In another optional implementation, LightMamba is used as a backbone network to embed rotating target detection frameworks such as AOPG, LO-Det, and S2ANet, outputting the rotated bounding box parameters of the remote sensing targets via the task output module. The specific configuration of the task output module can be adaptively replaced according to the actual application requirements on the satellite.

[0091] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware, and the corresponding program can be stored in a computer-readable storage medium.

[0092] It should be noted that although the method operations of the above embodiments are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the order of execution of the described steps may be changed. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0093] Example 2: This embodiment provides a satellite-based on-orbit ground target detection and identification system, including a preprocessing module and an identification or detection module. The specific functions of each module are as follows: The preprocessing module is used to preprocess remote sensing images, including normalization, cropping, and resizing. The recognition or detection module is used to input preprocessed remote sensing images into a trained multi-stage network model to obtain target recognition or target detection results. The multi-stage network model includes an image patch embedding layer, multiple remote sensing state space modules, a global average pooling layer, a classifier or feature pyramid, and a task head. The remote sensing state space module includes a remote sensing state space computation unit and a feedforward network embedded after deep convolution. The remote sensing state space computation unit is configured to perform multi-directional scanning, generate fused features, and includes a channel state space branch, a spatial state space branch, and a spatial-channel feature fusion module. The multi-directional scanning includes eight spatial scans: horizontal forward, horizontal reverse, vertical forward, vertical reverse, main diagonal forward, main diagonal reverse, anti-diagonal forward, and anti-diagonal reverse, as well as two channel scans: channel forward and channel reverse.

[0094] Example 3: This embodiment provides a storage medium, which is a computer-readable storage medium, storing a computer program. When the computer program is executed by a processor, it implements the satellite-based on-orbit ground target detection and identification method of Embodiment 1 above, as follows: The remote sensing image is preprocessed, including normalization, cropping, and resizing. The preprocessed remote sensing image is input into a trained multi-stage network model to obtain target recognition or target detection results. The multi-stage network model includes an image patch embedding layer, multiple remote sensing state space modules, a global average pooling layer, and a classifier or feature pyramid and task head. The remote sensing state space module includes a remote sensing state space computation unit and a feedforward network embedded after deep convolution. The remote sensing state space computation unit is configured to perform multi-directional scanning, generate fused features, and includes a channel state space branch, a spatial state space branch, and a spatial-channel feature fusion module. The multi-directional scanning includes eight spatial scans: horizontal forward, horizontal reverse, vertical forward, vertical reverse, main diagonal forward, main diagonal reverse, anti-diagonal forward, and anti-diagonal reverse, as well as two channel scans: channel forward and channel reverse.

[0095] It should be noted that the computer-readable storage medium in this embodiment can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0096] In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this embodiment, the computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0097] The computer-readable storage medium described above can be used to write computer programs for executing this embodiment in one or more programming languages ​​or combinations thereof. These programming languages ​​include object-oriented programming languages—such as Java, Python, and C++—and conventional procedural programming languages—such as C or similar programming languages. The program can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0098] In summary, this invention addresses the challenges of limited on-orbit computing resources for satellites, small-scale and highly variable attitude of remote sensing targets, limited receptive fields of traditional visual models, and insufficient accuracy of lightweight models. It constructs a remote sensing image target detection and recognition model that combines low computational overhead, high recognition accuracy, and fast reasoning capabilities, providing algorithmic support for real-time intelligent interpretation, rapid filtering, and low-latency information downlink on satellites.

[0099] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope disclosed in the present invention, based on the technical solution and inventive concept of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A method for detecting and identifying ground targets in orbit using satellites, characterized in that, include: The remote sensing image is preprocessed, including normalization, cropping, and resizing. The preprocessed remote sensing image is input into a trained multi-stage network model to obtain target recognition or target detection results. The multi-stage network model includes an image patch embedding layer, multiple remote sensing state space modules, a global average pooling layer, and a classifier or feature pyramid and task head. The remote sensing state space module includes a remote sensing state space computation unit and a feedforward network embedded after deep convolution. The remote sensing state space computation unit is configured to perform multi-directional scanning, generate fused features, and includes a channel state space branch, a spatial state space branch, and a spatial-channel feature fusion module. The multi-directional scanning includes eight spatial scans: horizontal forward, horizontal reverse, vertical forward, vertical reverse, main diagonal forward, main diagonal reverse, anti-diagonal forward, and anti-diagonal reverse, as well as two channel scans: channel forward and channel reverse.

2. The satellite-based on-orbit ground target detection and identification method according to claim 1, characterized in that, The multi-stage network model is configured as follows: The preprocessed remote sensing image is processed by the image patch embedding layer to obtain the embedding features. The image patch embedding layer includes a convolution-batch normalization-activation function module and a convolution-batch normalization module. The embedded features are processed according to the three-stage structure to obtain lightweight features. The three-stage structure includes a first-stage structure, a second-stage structure, and a third-stage structure. The first-stage structure, the second-stage structure, and the third-stage structure include a remote sensing state space module. Depth-separable convolutional downsampling layers are set between adjacent stages. For the recognition task, the target recognition result is obtained based on the global average pooling layer, the classifier, and the lightweight features. For the detection task, the target detection result is obtained based on the feature pyramid, the task head, and the lightweight features.

3. The satellite-based on-orbit ground target detection and identification method according to claim 1, characterized in that, The remote sensing state space computing unit is configured to perform multi-directional scanning and generate fused features, including: Eight spatial scans are performed on the feature map, and the scans are stacked to obtain an eight-directional spatial scan feature. In the channel state space branch, the eight-directional spatial scan features are merged to obtain a four-dimensional feature representation. Global pooling is then performed on the four-dimensional feature representation in the height and width dimensions to obtain the channel feature vector. Perform two channel scans along the channel dimension on the channel feature vector to obtain multiple channel scan features; The stacked channel scanning features are fed into the state space model to model channel dependencies, and then merged to obtain channel modeling features. In the spatial state space branch, the eight-directional spatial scanning features are fed into the state space model to perform spatial multi-directional state modeling, obtain spatial modeling features, and merge them to obtain spatial enhancement features. The spatial enhancement feature and the channel modeling feature are modulated element-wise by multiplication, and then the modulation result is added element-wise to the spatial enhancement feature itself to obtain the fused feature.

4. The satellite-based on-orbit ground target detection and identification method according to claim 1, characterized in that, The remote sensing state space module also includes a first normalization layer and a second normalization layer.

5. The satellite-based on-orbit ground target detection and identification method according to claim 4, characterized in that, The remote sensing state space computation unit embedded after the depth convolution is located within the depth separable state space module. The depth separable state space module also includes a first convolutional layer and a second convolutional layer. The first convolutional layer is set before the depth convolution, and the second convolutional layer is set after the remote sensing state space computation unit.

6. The satellite-based on-orbit ground target detection and identification method according to claim 5, characterized in that, The first normalization layer, the depth-separable state space module, and the first residual connection are arranged sequentially; the second normalization layer, the feedforward network, and the second residual connection are arranged sequentially.

7. The satellite-based on-orbit ground target detection and identification method according to any one of claims 1 to 6, characterized in that, The feedforward network includes a third convolutional layer, two parallel deep convolutional branches, a multiplication unit for fusing the outputs of the two parallel deep convolutional branches, and a fourth convolutional layer connected in sequence.

8. The satellite-based on-orbit ground target detection and identification method according to any one of claims 1 to 6, characterized in that, The steps for generating the main diagonal scan and anti-diagonal scan include: Enumerate the diagonal numbers of the feature map, and extract the main diagonal feature sequence and the anti-diagonal feature sequence according to the preset index relationship; Perform reverse flipping operations on the extracted main diagonal feature sequence and antidiagonal feature sequence respectively to generate the corresponding reverse sequence; The forward sequence of the main diagonal, the reverse sequence of the main diagonal, the forward sequence of the anti-diagonal, and the reverse sequence of the anti-diagonal are stacked to obtain the four-directional diagonal scanning features.

9. A satellite-based on-orbit ground target detection and identification system, characterized in that, include: The preprocessing module is used to preprocess remote sensing images, including normalization, cropping, and resizing. The recognition or detection module is used to input preprocessed remote sensing images into a trained multi-stage network model to obtain target recognition or target detection results. The multi-stage network model includes an image patch embedding layer, multiple remote sensing state space modules, a global average pooling layer, a classifier or feature pyramid, and a task head. The remote sensing state space module includes a remote sensing state space computation unit and a feedforward network embedded after deep convolution. The remote sensing state space computation unit is configured to perform multi-directional scanning, generate fused features, and includes a channel state space branch, a spatial state space branch, and a spatial-channel feature fusion module. The multi-directional scanning includes eight spatial scans: horizontal forward, horizontal reverse, vertical forward, vertical reverse, main diagonal forward, main diagonal reverse, anti-diagonal forward, and anti-diagonal reverse, as well as two channel scans: channel forward and channel reverse.

10. A readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the satellite-oriented on-orbit ground target detection and identification method according to any one of claims 1 to 8.

Images