A method and apparatus for semantic segmentation of spatial targets based on ISAR echoes

Abstract

This invention discloses a spatial target semantic segmentation method and apparatus based on ISAR echoes, belonging to the field of radar target recognition technology. The method first uses a trainable domain alignment module (DAM) to directly perform a learnable orthogonal transformation on the original ISAR echo without imaging transformation. While maintaining the same amount of information, the echo domain of the ISAR echo is mapped to a feature domain compatible with the mask domain of the semantic segmentation generation mask, obtaining a first feature. This first feature is then fed into a complex domain encoder for scattering feature extraction. The scattering feature is further processed by a complex domain decoder to achieve the recognition of the semantic segmentation mask. This invention differs from the traditional "image first, segment later" process, avoiding potential information loss and time consumption during the imaging stage, and achieving a direct mapping from the original ISAR echo to pixel-level segmentation.

Description

Technical Field

[0001] This invention relates to the field of radar target recognition technology, and specifically to a spatial target semantic segmentation method and apparatus based on ISAR echoes. Background Technology

[0002] In recent years, the rapid development of civilian space missions worldwide has led to a significant increase in the number of satellites in orbit. Simultaneously, the risk of space collisions has continued to rise, with frequent on-orbit impacts that can cause structural damage to satellite platforms or critical payloads, seriously threatening the safe on-orbit operation and long-term service capability of spacecraft. Against this backdrop, achieving precise perception and status identification of space targets has become a core issue urgently needing to be addressed in the field of on-orbit safety maintenance of spacecraft. Inverse synthetic aperture radar (ISAR), with its advantages of all-weather, all-day operation, long range, and high resolution, has become an important technical means for maintaining the safe on-orbit operation of space targets. ISAR-based semantic segmentation of space targets can not only accurately estimate component size information and determine the main structure, but also further determine their operational status and perform precise positioning of spacecraft, providing strong data support for collision damage assessment and fault diagnosis.

[0003] Traditional semantic segmentation methods struggle to accurately segment complex targets. However, the development of deep learning, with its powerful nonlinear modeling and feature mining capabilities, offers a new solution for spatial target semantic segmentation. These methods, through end-to-end pixel-level classification and training on large-scale data, can automatically learn and extract high-level semantic features of images, thus significantly distinguishing different targets from the background. Compared to traditional algorithms, deep neural networks can fully utilize high-order semantic information, maintaining high segmentation accuracy even in complex environments and with changing targets.

[0004] Existing methods primarily draw upon mature models from the field of optical image processing and have achieved initial success on ISAR data. However, these methods still have several significant limitations, restricting the robustness and fine-grained resolution capabilities of the models in practical applications. Specifically, existing methods fail to fully exploit the radar physics characteristics inherent in ISAR data, particularly neglecting the complex domain phase information based on scattering mechanisms, resulting in an inability to accurately model the complete electromagnetic scattering characteristics of the target. Furthermore, most methods process only real-valued images, relying on specific imaging operations and limiting their application potential in real-time processing scenarios. Simultaneously, prior knowledge of the spatial target structure has not been effectively integrated into the current segmentation framework. These factors collectively constrain the improvement of high-precision segmentation performance for spatial targets.

[0005] Therefore, traditional spatial target segmentation algorithms based on ISAR images have the following shortcomings:

[0006] (1) Lack of modeling of radar physical characteristics, real-valued images lose some scattering information and phase details; (2) The process of “image first, segmentation later” is complex and requires multiple rounds of algorithm processing, resulting in high processing complexity.

[0007] (3) The prior information on the spatial target structure was not effectively utilized, and the network could not be effectively guided to focus on the spatial target segmentation subject and edge; (4) Insufficient edge segmentation accuracy for space targets. Summary of the Invention

[0008] In view of this, the present invention provides a spatial target semantic segmentation method and apparatus based on ISAR echoes, realizing a direct mapping from raw ISAR echoes to pixel-level segmentation. This network differs from the traditional "image first, segment later" approach, avoiding information loss and time consumption that may occur during the imaging stage. Its multi-branch fusion encoder-decoder architecture fully mines and utilizes the complete electromagnetic scattering features contained in the echoes, achieving efficient and accurate conversion from the data domain to the task domain, providing a novel processing paradigm for semantic segmentation tasks based on ISAR data.

[0009] To solve the above-mentioned technical problems, the present invention is implemented as follows.

[0010] A spatial target semantic segmentation method based on ISAR echoes includes: Step 1: Without performing imaging transformation, the domain alignment module (DAM) is used to directly perform an orthogonal transformation with learnable transformation parameters on the original ISAR echo. This maps the echo domain of the ISAR echo to a feature domain compatible with the mask domain of the semantic segmentation generation mask while maintaining the information content, thus obtaining the first feature. Step 2: The first feature is fed into the complex domain encoder to extract the scattering feature; Step 3: The scattering features are fed into the complex domain decoder to achieve the recognition of the semantic segmentation mask.

[0011] Preferably, step 1 specifically includes the following steps: Step 11: For the parameter matrix of the orthogonal transformation Normalization is performed, and then a unit modulus constraint is applied to obtain the parameter matrix. ; Step 12: Using the parameter matrix Construct a rotation matrix to perform an orthogonal transformation on the ISAR echo.

[0012] Preferably, step 1 further includes: Step 13: Apply learnable scaling factors to the real and imaginary components of the features obtained from the orthogonal transformation in Step 12. Perform feature amplitude scaling.

[0013] Preferably, step 1 further includes: Step 14: Apply a learnable spatial weight matrix to the features obtained by scaling the feature magnitudes in Step 13. Implement feature space modulation.

[0014] Preferably, the complex domain encoder has at least one layer of multi-branch coding module (MEB); The MEB adopts a heterogeneous feature fusion architecture containing at least three parallel branches, where two branches respectively incorporate a sliding attention (SCA) module and a subdomain balanced attention (SBA) module to form a multi-view attention framework (MPA). The SCA module performs global shifting and edge filling on the complex conjugate of its input feature map to obtain a shift-filled feature map; it then performs edge filling on the input feature map to achieve size alignment, obtaining the original filled feature map; the original filled feature map and the shift-filled feature map are used to segment local feature blocks through a sliding window operation, and the autocorrelation coefficient of each pair of corresponding local feature blocks is calculated to express the correlation of the spacecraft's space structure in orbit; the autocorrelation coefficients of each pair of local feature blocks constitute a structural similarity intensity map, which is then processed through equalization and normalization to obtain sliding correlation attention enhancement features. ; The SBA module divides its input feature map into amplitude feature map and phase feature map for separate processing; in the amplitude attention channel, the input amplitude feature map is processed using a value matrix. Key matrix and query matrix Combined calculation of amplitude attention features To adaptively focus on the characteristics of strong scattering regions; in the phase attention channel, input the phase feature map and calculate the phase attention features. To enhance the perception of micro-moving components' structure; convolutional layers are used to implement amplitude attention features. Phase attention features Dynamic fusion generates spatially discriminative subdomain attention-enhanced features. .

[0015] Preferably, in the SBA module, the value matrix Key matrix and query matrix Each layer uses a 1×1 convolutional layer to maintain the original spatial structure relationship; The amplitude attention feature is implemented using convolutional layers. Phase attention features The dynamic fusion is specifically as follows:

[0016] in, All represent 3×3 convolutional layers. For amplitude-based attention features Phase attention features The real part of the synthesized subdomain attention-enhanced feature. For amplitude-based attention features Phase attention features The imaginary part of the synthesized subdomain attention-enhanced features; It is the imaginary unit.

[0017] Preferably, in the complex domain encoder, key local features are first extracted in the feature domain through a cascaded structure of a 3×3 complex domain convolutional layer CConv, a complex domain batch normalization layer CBN, and a complex domain ReLU function. The key local features are input into a processing unit composed of multiple stacked MEBs to capture multi-scale scattering features layer by layer, and residual connections are added between multiple MEBs. Subsequently, global features are integrated and compressed through a complex domain fully connected layer CFc. Finally, multiple 1×1 complex domain convolutional layers CConv enrich the feature expression in the encoding stage and output scattering features that fuse global semantics.

[0018] Preferably, the MEB includes three heterogeneous parallel branches, which are the original transform feature branches. Relevance attention feature enhancement branch Subdomain attention feature enhancement branch ; The original transform feature branch It consists of a single 3×3 complex convolutional layer; The relevance attention feature enhancement branch It consists of a first 3×3 complex convolutional layer, a first complex ReLU activation layer, a second 3×3 complex convolutional layer, and an SCA module connected in sequence; The subdomain attention feature enhancement branch It consists of a third 3×3 complex convolutional layer, a complex batch normalization layer, a second complex ReLU activation layer, a fourth 3×3 complex convolutional layer, and an SBA module connected in sequence; The heterogeneous features of the three heterogeneous parallel branch outputs are spliced ​​along the channel dimension to generate combined features. Then, cross-channel feature fusion and dimensionality compression are performed through 1×1 complex convolutional layers to output the optimized representation of MEB. .

[0019] Preferably, in step 3, the complex domain decoder uses a multi-level multi-branch decoding module (MDB) to upsample the scattering features step by step to restore the resolution of the feature map, and simultaneously aggregates the feature maps of the same size from the complex domain encoder; finally, a 1×1 convolutional layer is used to combine the features, and after being transformed by the Sigmoid function, a pixel-level semantic segmentation mask is output.

[0020] The present invention also provides a spatial target semantic segmentation device based on ISAR echo, comprising: a domain alignment module (DAM), a complex domain encoder, and a complex domain decoder; The DAM is used to directly perform a learnable orthogonal transformation on the original ISAR echo without performing ISAR echo imaging transformation, thereby mapping the echo domain of the ISAR echo to a feature domain compatible with the mask domain of the semantic segmentation generation mask under the constraint of keeping the amount of information unchanged, and obtaining the first feature. The complex domain encoder is used to process the first feature to extract scattering features; The complex domain decoder is used to process the scattering features to achieve the recognition of semantic segmentation masks.

[0021] Beneficial effects: (1) Traditional methods often use real-valued images for related research, but lack modeling of complete physical electromagnetic scattering information. This invention proposes an end-to-end complex domain ISAR echo semantic segmentation network, which adopts an encoder-decoder structure to realize the mapping from the original ISAR echo to the pixel-level segmentation result, making full use of the complete amplitude and phase information of the original physical electromagnetic scattering information, and avoiding the information loss caused by the imaging process, so as to achieve effective modeling of radar physical characteristics.

[0022] (2) Traditional spatial target segmentation methods require imaging operations first, followed by segmentation. This invention proposes a Domain Alignment Module (DAM) to bridge the data distribution differences between the echo domain and the mask domain, effectively solving the inter-domain mismatch problem caused by data distribution differences between the original echo signal and the image mask. By constructing an adaptive feature transformation function with learnable parameters, the input echo data can be dynamically mapped to a feature space compatible with the mask domain. At the same time, the model convergence is accelerated by compressing the despace, improving segmentation accuracy and enhancing generalization ability. Moreover, this invention starts from ISAR echoes, eliminating the need for imaging operations, resulting in a simpler process and lower computational overhead.

[0023] (3) This invention proposes a multi-branch coding module MEB, which enhances the diversity and discriminativeness of scattering features through the complementary features of multiple perspectives within the hierarchy, breaks through the representation limitations of a single path, and effectively preserves the spatial continuity of the target structure while improving feature diversity and ensuring edge positioning accuracy.

[0024] (4) Traditional algorithms lack the use of prior information on the structure of space targets and fail to fully utilize prior information to guide model design. This invention introduces structural priors into model design and proposes a sliding attention module (SCA). In response to the problems of weakened perception of the overall structure of spacecraft and inaccurate edge positioning in existing models, the spatial structure correlation of feature maps is calculated by local autocorrelation operators to generate a structural spatial attention map to enhance the feature response of key components. This enables the model to accurately focus on highly correlated structural main bodies and edge features, improves the accuracy of structural perception, and makes the model more realistic with higher segmentation accuracy.

[0025] (5) This invention proposes a subdomain balanced attention module SBA. In view of the failure of existing models to effectively establish a representation mechanism for amplitude and phase coordination, an amplitude attention mechanism and a phase attention mechanism are designed respectively. The amplitude attention mechanism adaptively focuses on strong scattering features, and the phase attention mechanism enhances the perception of fine parts. Finally, the constraint balance of amplitude and phase features is achieved through the dual-path coordination mechanism. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the complex domain ISAR echo semantic segmentation network OSSNet of the present invention.

[0027] Figure 2 This is a schematic diagram of the DAM structure of the present invention.

[0028] Figure 3 This is a schematic diagram of the multi-branch coding module (MEB) in a complex domain encoder.

[0029] Figure 4 This is a schematic diagram of the Sliding Attention Module (SCA) in MEB.

[0030] Figure 5 This is a schematic diagram of the Subdomain Balanced Attention Module (SBA) in MEB.

[0031] Figure 6 This is a schematic diagram of the complex domain decoder and its MDB module.

[0032] Figure 7 Construct ISAR images corresponding to representative ISAR echo samples in the dataset.

[0033] Figure 8The following are schematic diagrams illustrating the experimental results: (a) ISAR image corresponding to the echo; (b) ground truth label; (c) segmentation result of the baseline OSSNet architecture without DAM, SCA, and SBA; (d) segmentation result of the OSSNet architecture without SBA; (e) segmentation result of the OSSNet architecture without SCA; (f) segmentation result of the OSSNet architecture without MPA; (g) segmentation result of the OSSNet architecture without DAM; and (h) segmentation result of the complete OSSNet architecture. Detailed Implementation

[0034] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0035] This invention provides a spatial target semantic segmentation device based on ISAR echoes, which is an end-to-end complex domain ISAR echo semantic segmentation network OSSNet (One-stop Semantic Segmentation Network) that achieves direct mapping from raw ISAR echoes to pixel-level segmentation. The key features of this invention are: considering the domain mismatch problem caused by the data distribution difference between the echo domain and the mask domain, a Domain Alignment Module (DAM) is designed; in the encoder, a Multi-branch Encoding Module (MEB) is designed to capture multi-scale scattering features hierarchically. The proposed Multi-perspective Attention (MPA) framework includes a Sliding Correlation-based Attention (SCA) module and a Subdomain Balance Attention (SBA) module to guide the network in extracting and fusing different features; finally, a multi-branch decoder without an attention mechanism is used to fuse features at different scales in the encoder to output a high-precision segmentation mask.

[0036] Figure 1A schematic diagram of the complex domain ISAR echo semantic segmentation network OSSNet in an embodiment of the present invention is shown. As shown in the figure, the network takes the original ISAR echo signal as input. First, without performing imaging transformation, a trainable domain alignment module (DAM) is used to directly perform an orthogonal transformation on the original ISAR echo with learnable transformation parameters. Under the constraint of keeping the amount of information unchanged, the echo domain of the ISAR echo is mapped to a feature domain compatible with the mask domain of the semantic segmentation generation mask, effectively solving the domain mismatch problem between the echo and the mask label. The transformed features are then fed into the complex domain encoder to realize scattering feature extraction. Finally, a complex domain decoder is used to realize the recognition of the semantic segmentation mask.

[0037] In this embodiment, the complex domain encoder employs a multi-branch encoder that incorporates a multi-view attention framework (MPA). This encoder extracts scattering features at different levels through parallel paths and incorporates SBA and SCA modules to enhance the representation of the spacecraft's main structure and detailed features. Correspondingly, the decoder adopts a multi-branch structure without an attention mechanism. By progressively upsampling and skip connections, it fuses features at different scales from the encoder, fully utilizing shallow high-resolution information to improve spatial positioning accuracy. Simultaneously, it combines deep semantic features to achieve accurate classification, ultimately outputting a high-precision semantic mask.

[0038] The following is a detailed description of each module.

[0039] (1) Domain Alignment Module (DAM) Traditional ISAR segmentation masks are generated based on target-background edge localization in the image domain, sharing spatial domain characteristics with ISAR images. Therefore, neural networks can directly learn pixel spatial relationships to generate masks. However, ISAR echoes require complex imaging transformations to obtain images, leading to significant differences in data distribution between the echo domain and the mask domain, a phenomenon known as "domain mismatch." Directly transferring image segmentation algorithms to the echo domain results in model incompatibility due to domain mismatch, thus limiting segmentation accuracy. This is a key design feature of this invention.

[0040] Furthermore, ISAR images are essentially a mapping of echo data to the pixel domain, but existing image-based segmentation methods only utilize amplitude information while discarding the phase component, lacking in-depth modeling of the original scattering information. However, the amplitude and phase of the echo together carry the target scattering characteristics, and the lack of phase information will lead to the loss of key physical features, thus limiting the improvement of segmentation accuracy.

[0041] Therefore, to address the aforementioned issues of echo-mask domain mismatch and incomplete utilization of physical scattering features, this invention, without performing imaging transformation, considers the "domain mismatch" problem that occurs when directly encoding ISAR images and adds a DAM module before the encoder. In this embodiment, the DAM not only performs orthogonal transformation but also incorporates amplitude and phase scaling, as well as feature space adjustment, to reduce the problems of unrealistic amplitude and phase magnitudes and overly uniform feature output caused by orthogonal transformation alone.

[0042] In this preferred embodiment, the preferred process for DAM to achieve domain alignment is as follows: Step 11: For the parameter matrix of the orthogonal transformation Normalization is performed, and then a unit modulus constraint is applied to obtain the parameter matrix. .

[0043] Step 12: Using the parameter matrix Construct a rotation moment matrix and perform an orthogonal transformation on the ISAR echo.

[0044] Step 13: Apply a learnable scaling factor to the real and imaginary components of the features obtained from the orthogonal transformation in Step 12. Perform feature magnitude scaling Step 14: Apply a learnable space weight matrix to the features obtained by adjusting the feature magnitude in Step 13. Implement feature space modulation.

[0045] Steps 13 and 14 are preferred steps.

[0046] Figure 2 The specific structure of the DAM in this embodiment is shown. As illustrated, it utilizes a learnable parameter matrix. and learnable parameters By constructing an adaptive feature transformation function, the input echo data can be dynamically mapped to a feature space (i.e., feature domain) that is compatible with the mask domain, thereby effectively bridging the data distribution differences between the echo domain and the mask domain. At the same time, by compressing the solution space, the model convergence is significantly accelerated, ultimately improving the segmentation accuracy and enhancing the model's generalization ability.

[0047] Assume the input to the DAM is an ISAR echo. , among which dimension These correspond to the number of channels, the number of sampling points in the orientation dimension, and the number of sampling points in the distance dimension, respectively. The DAM implementation process is as follows: First, a learnable transformation matrix is ​​introduced. Construct a feature space mapping and employ... The function, acting as a normalization function, constrains the matrix range to... Between these parameters, range normalization eliminates amplitude scale differences to improve optimization stability, a sign-preserving mechanism maintains the discriminability of phase relationships, and finally, a robust feature representation basis is established in the transform domain. The relevant mathematical operations are as follows: (1) in, Each is a matrix Middle position The element at that location, Matrix obtained for normalization Middle position The element at that location. This indicates normalization processing; here we choose... function.

[0048] Based on this, the transformation matrix Applying a unit modulus constraint, as shown in equation (2), forces the basis vectors to lie on the unit circle in the complex plane to maintain the numerical stability of the complex transformation. Then, using... The transformation matrix performs a rotational transformation on the input echo, preserving the signal's phase characteristics and geometric structure while extracting global features and constraining the total energy of the signal before and after the transformation. This design enables the model to adaptively learn the optimal complex basis space, achieving optimization of feature representation. (2) in, It represents the Hadamah accumulation. It is a very small positive number used to maintain numerical stability.

[0049] ISAR echoes are essentially single-channel complex-valued matrices. After channel compression It can be decomposed into real components. R and imaginary part quantity I ,Right now ,in By combining and reconstructing the real and imaginary features through adaptive transformation (as shown in Equation (3)), the optimized representation of complex field information is achieved.

[0050] (3) Where: Let the rotation matrix ,but: (4) From formula (2), we can see that, (5) in, Each is a matrix Middle position The element at that location, These are the real and imaginary component matrices, respectively. Middle position The element at that location, These are the real and imaginary components after the characteristic rotation transformation, respectively; Represents the identity matrix; matrix Since it is a rotation matrix, it needs to be an orthogonal matrix, and the constraint of formula (2) just satisfies the condition of an orthogonal matrix. Therefore, the above transformation process is an adaptive orthogonal transformation of the echo signal.

[0051] Simultaneously, through the learnable space weight matrix Implement feature space modulation to enhance the model's ability to select features in key regions, and simultaneously introduce a learnable scaling factor. Dynamically adjust the characteristic amplitude of the transform domain to maintain numerical stability.

[0052] (6) Finally, adjust the transformed feature dimensions, i.e. By combining the complex features after the transformation, the features of the transform domain are obtained. .

[0053] (2) Complex domain encoder The amplitude and phase perturbations in the echo caused by changes in the relative angle between the radar and the target manifest as artifacts such as blurred edges and strong scattering point sidelobe interference in ISAR images. Their physical essence stems from distortions in the scattering characteristics of the echo domain. Therefore, how to combine the rigid structural characteristics of spacecraft with the ISAR scattering mechanism to extract efficient and robust features with spatial discriminative capabilities from the echo has become a crucial issue for improving semantic segmentation accuracy.

[0054] To address the above issues and construct discriminative feature representations for accurate semantic segmentation, the inventors propose a complex domain encoder. This complex domain encoder incorporates at least one layer of a multi-branch coding module (MEB). The MEB employs a heterogeneous feature fusion architecture with at least three parallel branches, where two branches respectively incorporate a sliding attention module (SCA) and a subdomain balanced attention module (SBA), forming a multi-view attention framework (MPA). Specifically, the SCA module performs global shifting and edge filling on the complex conjugate feature map of the input feature map to obtain a shift-filled feature map; it performs edge filling on the input feature map to achieve size alignment and obtains the original filled feature map; it performs local feature block segmentation on the original filled feature map and the shift-filled feature map through a sliding window operation, and calculates the autocorrelation coefficient of each group of two corresponding local feature blocks to express the correlation of the space structure of the spacecraft in orbit; the autocorrelation coefficients of each group of local feature blocks constitute the structural space attention map.

[0055] The SBA module divides the input feature map into amplitude feature map and phase feature map for separate processing; in the amplitude attention channel, the input amplitude feature map is processed using a value matrix. Key matrix and query matrix Combined calculation of amplitude attention features To adaptively focus on the characteristics of strong scattering regions; in the phase attention channel, input the phase feature map and calculate the phase attention features. To enhance the perception of micro-moving components' structure; then, convolutional layers are used to implement amplitude attention features. Phase attention features Dynamic fusion generates spatially discriminative subdomain attention-enhanced features. .

[0056] See also Figure 1 This diagram illustrates the architecture of a complex domain encoder according to this embodiment. As shown in the figure, the network of this complex domain encoder first extracts key local features and enhances nonlinear modeling capabilities in the feature domain through a cascaded structure of a 3×3 complex domain convolutional layer (CConv), a complex domain batch normalization layer (CBN), and a complex domain ReLU (CReLU) function. These features are then input into a multi-level multi-branch coding module (MEB) to capture multi-scale scattering features layer by layer. Shallow features retain detailed information, while deep features contain high-level semantic information. In this embodiment, residual connections are added to the multi-layer MEB, with 1×1 complex domain convolutional layers added to these residual connections to adjust the number of channels, maintain dimensionality consistency, and achieve cross-channel information fusion. Subsequently, a fully connected complex domain layer (CFc) is used for global feature integration and compression. Finally, multiple layers of 1×1 complex domain convolutional layers (CConv) enrich the feature representation during the encoding stage, outputting a scattering point feature representation that integrates global semantics. Therefore, the complex domain encoder is mainly constructed by complex domain convolutional layers, complex domain batch normalization layers, complex domain ReLU, MEB, and complex domain fully connected layers.

[0057] In this embodiment, the MEB has a 4-layer structure, with residual connections added to the multiple MEB layers. Finally, the output part enriches the feature representation during the encoding stage through 3 layers of 1×1 complex field convolutional layers.

[0058] Figure 3The preferred MEB structure of this embodiment is shown. The MEB employs a multi-branch feature fusion architecture. The difference between this invention and existing technologies lies in designing the MEB as a multi-view feature complementarity architecture, namely a multi-view attention framework (MPA). This significantly enhances the diversity and discriminative power of scattering features through multi-view feature complementarity within layers. This MPA proposes a combination mechanism of SCA and SBA modules, which can suppress irrelevant local interference and dynamically balance the weights of foreground and background amplitude and phase subdomains. It guides the network to focus on key features of the main structure and edge details in different dimensions, thereby improving semantic segmentation accuracy and model interpretability. Only two of the SCA and SBA modules need to be embedded in the branches. The number of branches in the MEB is unlimited; except for branches with SCA and SBA modules, other branches can adopt existing designs.

[0059] To achieve maximum performance with relatively low computational load, the MEB architecture of this invention employs a three-way heterogeneous feature fusion architecture, consisting of the original transformed feature branch... Relevance attention feature enhancement branch Subdomain attention feature enhancement branch The design employs a complementary fusion of scattering representations by extracting original transform features, correlation-enhanced features, and subdomain-enhanced features in parallel. This design overcomes the limitations of single-path representations, effectively preserving the spatial continuity of the target structure and ensuring edge localization accuracy while enhancing feature diversity. Specifically... Figure 3 As shown.

[0060] Let the input features of MEB be... ,in Input the number of channels, feature map height, and feature map width for MEB.

[0061] Original feature transformation branch It consists of a single 3×3 complex convolutional layer, generating the original transformed features. Complex convolutional layers are represented as: (7) in, This represents a regular convolutional layer. This represents the output characteristics of the original transform characteristic branch. , Features The real and imaginary parts. These are the real and imaginary parts of the features after processing by complex convolutional layers.

[0062] Relevance attention feature enhancement branch It consists of a first 3×3 complex convolutional layer, a first complex ReLU activation layer, a second 3×3 complex convolutional layer, and an SCA module connected in sequence, and outputs relevant attention features. Its operation process is expressed as follows: (8) in, Represents convolution over complex fields. This indicates that the SCA module is handling the process; This is the ReLU function for complex numbers. The function's operation is shown below, calculating the complex input value. Amplitude and phase: (9) in, Indicates amplitude. Indicates phase, This represents the arctangent function in the four quadrants.

[0063] The complex ReLU function is expressed as: (10) Subdomain attention feature enhancement branch It consists of a third 3×3 complex convolutional layer, a complex batch normalization layer, a second complex ReLU activation layer, a fourth 3×3 complex convolutional layer, and an SBA module, and outputs subdomain attention features. Its operation process is expressed as follows: (11) SBA() is the SBA module processing. This is the batch normalization function for complex numbers, and its operation process is shown below. The complex input is calculated according to formula (9). The amplitude and phase are then calculated as shown in formula (12).

[0064] (12) in, It is the extra offset, and BN() is the complex batch normalization process.

[0065] Three heterogeneous features are spliced ​​along the channel dimension ( concatenate Generate combined features Then, cross-channel feature fusion and dimensionality compression are performed through 1×1 complex convolutional layers to output the optimized representation of MEB. .

[0066] (13) In this embodiment, the encoder input feature map is set as follows: After extraction via CConv-CBN-CBN cascade, preliminary features are output. Subsequently, a double spatial downsampling is performed step-by-step through multiple MEB stages to maintain a constant number of output channels. , Indicates the number of MEB output channels. Indicates the first The output feature map of MEB. To fuse multi-scale features, and Through 2×2 CConv cross-layer connections, feature reuse and deep information fusion are achieved to construct a more discriminative feature representation; then, feature aggregation is performed. And adjust dimensions After transformation by a complex fully connected layer, the dimensions are adjusted. Finally, after three levels of 1×1 CConv channel expansion, a multi-level scattering feature representation with low spatial resolution and high semantic density is output. .

[0067] (3) Sliding Attention Module (SCA) in Complex Domain Encoder Spacecraft in orbit have highly regular geometric structures, and the arrangement of their key components exhibits strong symmetry, providing prior geometric constraints for models to capture the main structure and fine boundaries. However, the complex connection structures between components and the uneven distribution of scattering point intensity weaken the model's perception of the overall spacecraft structure and blur the boundaries.

[0068] To address the aforementioned issues of weakened structure perception and inaccurate edge localization, this invention proposes an SCA module inspired by the autocorrelation function. This module calculates the spatial structural correlation of feature maps using a local autocorrelation operator, generating a structural spatial attention map to selectively enhance the feature responses of key components. This enables the model to accurately focus on highly correlated structural elements and edge features, significantly improving the accuracy of structure perception.

[0069] The structure of the SCA in this embodiment is as follows: Figure 4 As shown.

[0070] Step a1: Find the complex conjugate of the input feature map, and perform a global shift on the complex conjugate feature map to obtain the shifted feature map.

[0071] First, let the input feature map of the SCA module be... Construct its complex conjugate feature map ,in yes The complex conjugate of . (See diagram) This indicates finding the conjugate of the input complex feature map.

[0072] Secondly, a shift feature map of the generated complex conjugate feature map. subscript , These represent the number of sampling points in the orientation and distance dimensions of the shift feature map, respectively. (See figure.) This indicates a shift operation. In this embodiment, shifting refers to moving the input feature map to the lower right by a set distance. The formula for the shift operation is as follows: The height dimension shift mapping relationship is as follows: (14) The width dimension shift mapping relationship is as follows: (15) in, and These represent the height shift step and the width shift step, respectively. , , respectively, represent the number of sampling points in the orientation and distance dimensions of the shift feature map. H and W represent the number of sampling points in the input feature map. J Number of sampling points in the orientation and distance dimensions.

[0073] Step a2: Align the size of the shifted feature map and the input feature map to obtain the shifted zero-padded feature map and the original zero-padded feature map.

[0074] After shifting in step a1, the original image expanded to the lower right. lines and This column leaves the top left corner of the original image empty. lines and To maintain the same dimensions as the input feature map, both the input and shifted feature maps need to undergo boundary padding to ensure they are the same size.

[0075] This embodiment of the invention uses a method of filling the four sides with 0 values. Let the length of the fill on one side be... ,but ,in, It's the size of the sliding window. This indicates rounding down. The filled feature map. .

[0076] Step a3: Perform local feature block segmentation on the original filled feature map and the shifted filled feature map using a sliding window operation to obtain local feature blocks, denoted as follows: .

[0077] This step involves extracting a sliding window, as shown in the image. This indicates a sliding window operation. In this embodiment, the height dimension and width dimension of the feature map are the same, i.e. Using a size of The square sliding window has a step size of 1, therefore the fill lengths for the height and width dimensions of the feature map are the same, both being... A sliding window decomposition is performed, yielding a series of feature blocks in both the original filled feature map and the shifted filled feature map. Each sliding window position corresponds to a spatial location in the feature map, generating a corresponding local feature block group. The sliding window constructs a local neighborhood context for each location.

[0078] Step a4: Calculate the autocorrelation coefficient of each local feature block group to represent the spatial structural correlation of the spacecraft in orbit. The autocorrelation coefficients of each local feature block group constitute the structural spatial attention map.

[0079] This step performs local feature autocorrelation calculation. Let the similarity strength between the original feature map and the shifted feature be... The calculation process is shown in formula (16).

[0080] (16) in, It represents the Hadamah accumulation. Indicates the modulo value. yes M Middle position cij element at ； This refers to the autocorrelation coefficient. The autocorrelation coefficients of each group of local feature blocks constitute the structural similarity intensity map. Rf .

[0081] Step a5: To suppress the weight imbalance caused by excessive regional similarity intensity due to strong scattering points, logarithmic compression (log) is used to equalize the similarity intensity distribution and improve the visibility of microstructures; subsequently... function( The normalized similarity intensity is mapped to attention weights. The original feature map is spatially weighted to ultimately generate a sliding correlation attention-enhanced feature map. The formula is expressed as: (17) The diagram uses This represents similarity calculation and equalization (log) and normalization (log). )operate.

[0082] (4) Subdomain Balanced Attention Module (SBA) in Complex Domain Encoder In ISAR echoes, amplitude information characterizes the distribution of electromagnetic scattering intensity of the target, while phase information is highly sensitive to subtle changes in the target's micro-motions and local structures, providing complementary discriminative features. Most existing segmentation methods process only amplitude images, failing to fully establish a feature representation mechanism that synergistically enhances amplitude and phase, thus limiting the model's ability to perceive and utilize the key structures of complex targets.

[0083] To address the aforementioned challenges, this invention proposes an SBA module. This module divides the input feature map into amplitude feature maps and phase feature maps for separate processing. In the amplitude attention channel, the input amplitude feature map is processed using a value matrix. Key matrix and query matrix Combined calculation of amplitude attention features To adaptively focus on the characteristics of strong scattering regions; in the phase attention channel, input the phase feature map and calculate the phase attention features. To enhance the perception of the structure of micro-moving components; finally, convolutional layers are used to implement amplitude attention features. Phase attention features The dynamic fusion achieves a constrained balance of amplitude and phase features and a physically consistent model of scattering features across all dimensions, generating spatially discriminative subdomain attention-enhanced features. .

[0084] The structure of SBA is as follows: Figure 5 As shown.

[0085] First, calculate the amplitude attention features. Let the input feature map of the SBA module be... Calculate the magnitude features of the input feature map. Inspired by the Transformer's self-attention mechanism, this invention introduces three 1×1 convolutional layers to construct the value matrix. Key matrix and query matrix Compared to the traditional Transformer structure In ISAR semantic segmentation, 1×1 convolutional layers can maintain the original spatial structure correlation, avoiding a surge in the number of parameters in a fixed-size weight matrix while modeling the correlation of channel-dimensional scattering features. The calculation process for matrix values, keys, and queries is as follows: (18) Adjust dimensions Attention score and amplitude attention features The calculation process is as follows: (19) in, express function, Represents matrix multiplication. Adjusts the magnitude of attention feature dimension. .

[0086] Next, calculate the phase attention features. Phase feature map. Phase attention features The calculation process is as follows: (20) in, express function, It represents the Hadamah accumulation. This represents the k-th convolutional layer.

[0087] A dynamic weighted fusion strategy is used to balance amplitude and phase attention features, generating spatially discriminative subdomain attention enhancement features. : (twenty one) in, Both represent 3×3 convolutional layers; For amplitude-based attention features Phase attention features The real part of the synthesized subdomain attention-enhanced feature. For amplitude-based attention features Phase attention features The imaginary part of the synthesized subdomain attention-enhanced features; It is the imaginary unit.

[0088] (5) Complex domain decoder The complex domain decoder in this embodiment uses a multi-branch decoding module (MDB) as its core architecture. It employs multi-level MDBs to upsample step by step to restore the resolution of the feature map. It also uses parallel branches within the MDB to fuse complementary information at multiple scales at the same level to achieve refined feature representation. The complex domain decoder also simultaneously aggregates feature maps of the same size as the complex domain encoder and supplements high-resolution spatial detail information through cross-layer connections. In the process of gradually restoring spatial resolution, it significantly suppresses the boundary blurring effect and improves the segmentation accuracy.

[0089] See Figure 1 In this embodiment, the decoder input feature map is represented as follows: The feature map resolution is restored through multi-level MDB upsampling. This embodiment includes 5 levels of MDB. , , , Indicates the number of MDB output channels. Indicates the first Output feature maps of each MDB. and Through a 2×2 complex transposed convolutional layer ( Complex ConvTranspose Connectivity enables feature reuse and integrates multi-scale contextual information to improve segmentation accuracy. Finally... The features are combined using a 1×1 convolutional layer (Conv), and after transformation by the Sigmoid function, a pixel-level semantic segmentation map Mask is output.

[0090] (twenty two) like Figure 6 As shown, the MDB employs a dual-path feature fusion architecture, consisting of shallow and deep feature branches. By designing parallel branch structures of varying depths, this module effectively fuses spatial information at different levels, taking into account both local detailed features and global semantic features, significantly improving the segmentation accuracy of the spacecraft's main structure and boundaries.

[0091] The shallow feature branches consist of a 2×2 complex transposed convolutional layer followed by a CBN layer. Given the input feature map of the MDB, The output feature map of this branch can be represented as: (twenty three) in, Let CBN be the representation. The calculation process of the complex transposed convolutional layer is shown in formula (24): (twenty four) In the formula, This indicates a complex transpose convolutional layer. This indicates a normal transposed convolutional layer.

[0092] The deep feature branches consist of two 3×3 complex transposed convolutional layers, two complex batch normalization layers, and one complex ReLU layer. Given the same input feature map... The output feature map of this branch can be represented as: (25) In the formula, Represents the ReLU function for complex numbers.

[0093] First, the dual-path features are concatenated along the channel dimension. concatenate ), forming combined features Subsequently, the combined features Through a 1×1 complex convolutional layer Efficient fusion and dimensionality reduction are performed to ultimately generate the output feature map of MDB. .

[0094] (26) This concludes the process.

[0095] The model training process is described below.

[0096] (1) Dataset construction This invention uses an ISAR dataset generated based on a ground-based radar model simulation. The dataset contains 1793 sample pairs, each consisting of raw ISAR echo data and its corresponding label mask. The dataset is divided into training and test sets in a 7:3 ratio. The input data is a 512×512 echo matrix; to visually demonstrate the data characteristics, Figure 7 The ISAR images corresponding to representative echo samples are shown.

[0097] (2) Model training and validation All experiments were conducted using the PyTorch framework on an NVIDIA GeForce RTX 3090 equipped with a single 24GB VRAM. During training, the Adam optimizer was used, and differentiated initial learning rates were set based on parameter characteristics: given the small proportion of the spacecraft target in the mask leading to foreground-background pixel imbalance, the loss function BCEWithLogitsLoss was chosen, and the weights of positive samples were increased to improve the model's learning ability for key target regions.

[0098] To comprehensively evaluate model performance, we use the mean Intersection over Union (mIoU) and F1 score as the core evaluation metrics. IoU measures the overlap between predicted and ground truth segments; it is the ratio of the intersection to the union of predicted and ground truth values. mIoU is the average IoU across all classes in the dataset. On the other hand, the F1 score provides a comprehensive evaluation of recall and precision for each class. mF1 is the average F1 score across all classes in the dataset.

[0099] To verify the superiority of OSSNet and the effectiveness of its key modules, ablation experiments were conducted: using a basic OSSNet architecture without DAM, SCA, and SBA as the baseline model, the impact of DAM, SBA, and SCA on segmentation performance was evaluated. The comprehensive quantitative evaluation results are shown in Table 1, and the corresponding visualization results are as follows: Figure 8 As shown.

[0100] Table 1 Verification Results

[0101] The specific embodiments described above only illustrate the design principles of the present invention. The shapes and names of the components in this description may differ and are not limited. Therefore, those skilled in the art can modify or make equivalent substitutions to the technical solutions described in the foregoing embodiments; and these modifications and substitutions do not depart from the inventive spirit and technical solutions of the present invention, and should all fall within the protection scope of the present invention.

Claims

1. A spatial target semantic segmentation method based on ISAR echoes, characterized in that, include: Step 1: Without performing imaging transformation, the domain alignment module (DAM) is used to directly perform an orthogonal transformation with learnable transformation parameters on the original ISAR echo. This maps the echo domain of the ISAR echo to a feature domain compatible with the mask domain of the semantic segmentation generation mask while maintaining the information content, thus obtaining the first feature. Step 2: The first feature is fed into the complex domain encoder to extract the scattering feature; Step 3: The scattering features are fed into the complex domain decoder to achieve the recognition of the semantic segmentation mask.

2. The spatial target semantic segmentation method based on ISAR echo as described in claim 1, characterized in that, Step 1 specifically includes the following steps: Step 11: For the parameter matrix of the orthogonal transformation Normalization is performed, and then a unit modulus constraint is applied to obtain the parameter matrix. ; Step 12: Using the parameter matrix Construct a rotation matrix to perform an orthogonal transformation on the ISAR echo.

3. The spatial target semantic segmentation method based on ISAR echo as described in claim 2, characterized in that, Step 1 further includes: Step 13: Apply learnable scaling factors to the real and imaginary components of the features obtained from the orthogonal transformation in Step 12. Perform feature amplitude scaling.

4. The spatial target semantic segmentation method based on ISAR echo as described in claim 3, characterized in that, Step 1 further includes: Step 14: Apply a learnable spatial weight matrix to the features obtained by scaling the feature magnitudes in Step 13. Implement feature space modulation.

5. The spatial target semantic segmentation method based on ISAR echo as described in claim 1, characterized in that, The complex domain encoder has at least one layer of multi-branch coding module (MEB); The MEB adopts a heterogeneous feature fusion architecture containing at least three parallel branches, where two branches respectively incorporate a sliding attention (SCA) module and a subdomain balanced attention (SBA) module to form a multi-view attention framework (MPA). The SCA module performs global shifting and edge filling on the complex conjugate of its input feature map to obtain a shift-filled feature map; it then performs edge filling on the input feature map to achieve size alignment, obtaining the original filled feature map; the original filled feature map and the shift-filled feature map are used to segment local feature blocks through a sliding window operation, and the autocorrelation coefficient of each pair of corresponding local feature blocks is calculated to express the correlation of the spacecraft's space structure in orbit; the autocorrelation coefficients of each pair of local feature blocks constitute a structural similarity intensity map, which is then processed through equalization and normalization to obtain sliding correlation attention enhancement features. ; The SBA module divides its input feature map into amplitude feature map and phase feature map for separate processing; in the amplitude attention channel, the input amplitude feature map is processed using a value matrix. Key matrix and query matrix Combined calculation of amplitude attention features To adaptively focus on the characteristics of strong scattering regions; in the phase attention channel, input the phase feature map and calculate the phase attention features. To enhance the perception of micro-moving components' structure; convolutional layers are used to implement amplitude attention features. Phase attention features Dynamic fusion generates spatially discriminative subdomain attention-enhanced features. .

6. The spatial target semantic segmentation method based on ISAR echo as described in claim 5, characterized in that, In the SBA module, the value matrix Key matrix and query matrix Each layer uses a 1×1 convolutional layer to maintain the original spatial structure relationship; The amplitude attention feature is implemented using convolutional layers. Phase attention features The dynamic fusion is specifically as follows: in, All represent 3×3 convolutional layers. For amplitude-based attention features Phase attention features The real part of the synthesized subdomain attention-enhanced feature. For amplitude-based attention features Phase attention features The imaginary part of the synthesized subdomain attention-enhanced features; It is the imaginary unit.

7. The spatial target semantic segmentation method based on ISAR echo as described in claim 5, characterized in that, In the complex domain encoder, key local features are first extracted in the feature domain through a cascaded structure of a 3×3 complex domain convolutional layer CConv, a complex domain batch normalization layer CBN, and a complex domain ReLU function. Key local features are input into a processing unit composed of multiple layers of MEB stacks, which captures multi-scale scattering features in layers, and residual connections are added between multiple MEB layers. Then, global features are integrated and compressed through a complex domain fully connected layer CFc. Finally, multiple 1×1 complex domain convolutional layers CConv enrich the feature expression in the encoding stage and output scattering features that fuse global semantics.

8. The spatial target semantic segmentation method based on ISAR echo as described in claim 5, characterized in that, The MEB contains three heterogeneous parallel branches, namely the original transform feature branch. Relevance attention feature enhancement branch Subdomain attention feature enhancement branch ; The original transform feature branch It consists of a single 3×3 complex convolutional layer; The relevance attention feature enhancement branch It consists of a first 3×3 complex convolutional layer, a first complex ReLU activation layer, a second 3×3 complex convolutional layer, and an SCA module connected in sequence; The subdomain attention feature enhancement branch It consists of a third 3×3 complex convolutional layer, a complex batch normalization layer, a second complex ReLU activation layer, a fourth 3×3 complex convolutional layer, and an SBA module connected in sequence; The heterogeneous features of the three heterogeneous parallel branch outputs are spliced ​​along the channel dimension to generate combined features. Then, cross-channel feature fusion and dimensionality compression are performed through 1×1 complex convolutional layers to output the optimized representation of MEB. .

9. The spatial target semantic segmentation method based on ISAR echo as described in claim 1, characterized in that, In step 3, the complex domain decoder uses a multi-level multi-branch decoding module (MDB) to upsample the scattering features step by step to restore the resolution of the feature map, and simultaneously aggregates the feature maps of the same size from the complex domain encoder; finally, a 1×1 convolutional layer is used to combine the features, and after being transformed by the Sigmoid function, a pixel-level semantic segmentation mask is output.

10. A spatial target semantic segmentation device based on ISAR echo, characterized in that, include: Domain alignment module DAM, complex domain encoder and complex domain decoder; The DAM is used to directly perform a learnable orthogonal transformation on the original ISAR echo without performing ISAR echo imaging transformation, thereby mapping the echo domain of the ISAR echo to a feature domain compatible with the mask domain of the semantic segmentation generation mask under the constraint of keeping the amount of information unchanged, and obtaining the first feature. The complex domain encoder is used to process the first feature to extract scattering features; The complex domain decoder is used to process the scattering features to achieve the recognition of semantic segmentation masks.

Images