ISAR image rotation speed estimation method based on sequence continuity

Abstract

The invention discloses an ISAR (Inverse Synthetic Aperture Radar) image rotation speed estimation method based on sequence continuity, which depends on a SeqFormer network architecture and is combined with a decomposable instance query mechanism and a cross-frame feature modeling strategy to fully mine rotation consistency and geometric relevance between adjacent frames. According to the method, the rotation dynamic constraint is established among multiple frames of images, so that unified modeling and accurate analysis of continuous rotation features of the target are realized. In the feature extraction stage, a cross-frame feature interaction and fusion mechanism is utilized, the problem of local structure discontinuity caused by noise and shielding is effectively weakened, the morphological integrity of a target in a time sequence is kept, and stable geometric support is provided for rotation estimation. Furthermore, modeling and regression learning are carried out on rotation difference features between adjacent frames, the network can directly output the rotation speed of the target, the complex process depending on calibration or key point matching in a traditional method is avoided, and estimation precision and stability are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of inverse synthetic aperture radar technology, specifically relating to a method for estimating the rotational velocity of ISAR images based on sequence continuity. Background Technology

[0002] Inverse Synthetic Aperture Radar (ISAR) imaging, with its advantages of all-weather, all-time, long-range, and high-resolution capabilities, has become an important means of space target surveillance. Contour extraction from ISAR images can reveal the morphological and structural information of targets, which is crucial for space target identification, 3D reconstruction, and attitude estimation. Furthermore, by combining the time-series information of multiple ISAR images, the rotational motion and attitude change parameters of the target between consecutive observation frames can be estimated, enabling the inference from two-dimensional scattering images to three-dimensional motion states, providing a key basis for attitude inversion of space targets. However, due to the anisotropy of electromagnetic scattering and the target's own occlusion, the target structure in ISAR images is sparse and incomplete, and the degree of structural loss varies with the viewing angle. This makes traditional contour extraction methods for single-frame images unrobust, posing a challenge to robust target contour extraction. Furthermore, existing methods for estimating rotation angles or attitudes mostly rely on manually labeled features or matching calculations based on geometric models, which are highly dependent on prior knowledge of the target and image quality. Under complex backgrounds or low signal-to-noise ratio conditions, the estimation accuracy of these methods drops significantly, and they may even fail to obtain an effective rotation vector. Therefore, how to effectively fuse motion information and morphological features between multiple frames in ISAR image sequences to achieve robust extraction of target contours and accurate estimation of rotation speed has become a core technical problem that urgently needs to be solved in the field of ISAR imaging and spatial target recognition.

[0003] Accurate extraction of target contours provides geometric priors and structural constraints for rotation estimation, forming a crucial foundation for high-precision pose inversion. With the rapid development of artificial intelligence, deep learning networks have become the primary means of extracting image target contours. In the field of structural recognition of ISAR images of space targets, researchers have successively proposed various contour segmentation models based on convolutional neural networks, such as fully convolutional networks (FCNs), the Deeplab series, and the U-Net series. These networks achieve high boundary segmentation accuracy through pixel-level feature learning. A traditional ISAR image segmentation method based on Pix2pixGAN treats ISAR image segmentation as a style transfer process, utilizing conditional generative adversarial networks to establish a mapping relationship between the original image and the segmentation result under global constraints, thereby obtaining more detailed boundary structures. The ISAR image semantic segmentation method proposed by Kou et al., combining contrastive learning and nonlocal U-Net, removes sidelobe noise through coarse segmentation and then uses a nonlocal attention mechanism to capture the structural symmetry of the image, improving segmentation quality and morphological consistency to a certain extent. Based on the contour structural features of sequential ISAR images, existing research has attempted to invert the pose of space targets using single-station ISAR image sequences. Kou et al. modeled the target motion as a complex tumbling motion involving spin and precession around an axis. Based on the projection feature matching of typical components, they optimized the attitude parameters and successfully achieved on-orbit attitude estimation of the Tiangong-1 space station. Wang et al., under the fixed-axis spin model, extracted key points such as the corner points of the solar panels and the endpoints of the main axis through an hourglass-shaped network. They then used key point factorization constraints to jointly solve for attitude and lateral calibration parameters, effectively reducing the impact of calibration errors on attitude estimation.

[0004] Existing contour extraction methods for estimating target rotation parameters in ISAR image sequences mostly focus on structural extraction from static single-frame images, lacking modeling of temporal consistency and rotational correlation between consecutive frames. Therefore, they are difficult to directly apply to the task of estimating the rotational velocity of space targets. The morphological changes of targets in ISAR images not only reflect the distribution of scattering features but also carry information about the target's rotational motion. Relying solely on single-frame segmentation cannot accurately capture these dynamic features. While existing methods have made some progress in rotation vector estimation accuracy, their performance is highly dependent on the completeness of contour extraction and keypoint detection. When ISAR images are affected by noise, scattering characteristics, or changes in viewing angle, resulting in structural incompleteness, rotation parameter inversion is prone to instability or bias. Therefore, how to jointly model contour changes and rotational continuity across multiple ISAR images becomes crucial for improving the accuracy and robustness of rotation estimation. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides a method for estimating the rotation velocity of ISAR images based on sequence continuity. The technical problem to be solved by this invention is achieved through the following technical solution: In a first aspect, the present invention provides a method for estimating the rotation velocity of ISAR images based on sequence continuity, the method comprising: Acquire the image to be estimated; the image to be estimated is a continuous ISAR image; A pre-trained rotational velocity estimation model is used to extract contours and estimate rotational velocity in the image to be estimated, thereby obtaining the effective rotational velocity of the ISAR image; wherein, The pre-trained rotational velocity estimation model includes: The backbone network, encoder, decoder, and output head are set in sequence. The backbone network extracts hierarchical and multi-scale semantic features from the image to be estimated by introducing residual connections between convolutional layers, thereby obtaining a multi-scale feature map corresponding to each frame of the image to be estimated. The encoder performs dimensional expansion and deformable attention on the multi-scale feature map of each frame image to obtain the flattened features corresponding to each frame image. The decoder processes the flattened features of each frame image based on instance query and a multi-scale deformable attention module to obtain the instance features, position information and rotation information corresponding to each frame image. The output head includes a bounding box head, a mask head, and a rotation head. The bounding box head processes the positional information of each frame using ReLU activation and a linear projection layer, and outputs the predicted normalized bounding box information. The mask head fuses and stitches the feature map sequence corresponding to each frame image based on instance features and bounding box information to generate an instance mask sequence. The rotation head stitches, fuses, and compresses the rotation information corresponding to adjacent frames images based on a lightweight regression head, and outputs the effective rotation speed of the ISAR image.

[0006] In one embodiment of the present invention, the backbone network includes: The initial convolutional layer and four residual stages; among which, Each residual stage includes a bottleneck module, and each bottleneck module consists of several convolutions of different dimensions.

[0007] In one embodiment of the present invention, in the decoder, the instance query is a decomposable instance query, used to extract features of the same target in multiple frames of images; each instance query is decomposed into several box queries, and each box query serves as an anchor point for locating features of the current frame, and feature retrieval is performed independently on the current frame.

[0008] In one embodiment of the present invention, the expression for the instance feature is as follows: ; Indicates the first Instance features of all frames generated by the layer decoder. Indicates generation from output features linear layers, Indicates the first Output features after the layer executes the attention mechanism; The expression for the location information is as follows: ; Indicates the first The position information of all frames generated by the layer decoder Indicates generation from output features Linear layers; The expression for the rotation information is as follows: ; Indicates the first Rotation information for all frames generated by the layer decoder. Indicates generation from output features Linear layers.

[0009] In one embodiment of the present invention, the bounding box header consists of three layers of feedforward network, and the information of the normalized bounding box includes: the center coordinates, height and width of the bounding box.

[0010] In one embodiment of the present invention, the mask head fuses and splices the feature map sequence corresponding to each frame image based on instance features and bounding box information to generate an instance mask sequence, including: The mask head uses a feedforward network to use instance features as mask head parameters; Multi-scale features are extracted from the encoder to generate a sequence of feature maps for each frame of the image. The feature map sequence corresponding to each frame image is concatenated with the relative coordinate map in the bounding box information of that frame image to obtain the concatenated feature map; Based on the mask head parameters and the spliced ​​feature map, an instance mask sequence is generated through convolution.

[0011] In one embodiment of the present invention, the rotation head, based on a lightweight regression head, stitches, fuses, and compresses the rotation information corresponding to adjacent frames of images to output the effective rotation speed of the ISAR image, including: The rotating head uses a lightweight regression head to stitch together the rotation information corresponding to adjacent frames to form temporal features; The temporal features are reduced in dimensionality through linear mapping to obtain the dimensionality-reduced features; The dimensionality-reduced features are concatenated and fused along the feature channel dimension to obtain the fused features; The fused features are compressed through a two-layer linear network, and the output is... N One prediction result; Using the matcher based on N The prediction results output the effective rotation speed of the ISAR image.

[0012] In one embodiment of the present invention, the loss function of the pre-trained rotational velocity estimation model during the training process is the Hungarian loss function, the expression of which is as follows: ; in, Indicates border loss. Indicates mask loss. Represents a set of real instances. Represents the predicted instance set, This indicates the loss of rotational speed. Represents the actual bounding box, Indicates the optimal match. Represents the predicted bounding box. Represents the actual mask. Indicates the predicted mask, Represents the actual rotational speed. This indicates the predicted rotational speed. This indicates the number of predicted instances.

[0013] Secondly, the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs. When the processor executes the program stored in the memory, it implements the steps of the ISAR image rotation velocity estimation method based on sequence continuity as described in the first aspect.

[0014] Thirdly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the ISAR image rotation velocity estimation method based on sequence continuity as described in the first aspect.

[0015] The beneficial effects of this invention are: The solution provided in this invention combines a decomposable instance query mechanism with a cross-frame feature modeling strategy to fully exploit the rotational consistency and geometric correlation between adjacent frames. By establishing dynamic rotational constraints across multiple frames, this invention achieves unified modeling and accurate analysis of continuous rotational features of the target. In the feature extraction stage, this invention utilizes a cross-frame feature interaction and fusion mechanism to effectively mitigate local structural discontinuities caused by noise and occlusion, maintaining the morphological integrity of the target in the time series and providing stable geometric support for rotation estimation. Furthermore, by modeling and regressing the rotational difference features between adjacent frames, the network can directly output the target's rotational speed, avoiding the complex process of calibration or keypoint matching in traditional methods, significantly improving estimation accuracy and stability. Attached Figure Description

[0016] Figure 1 This is a schematic diagram illustrating the steps of an ISAR image rotation velocity estimation method based on sequence continuity provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a pre-trained rotation velocity estimation model in an ISAR image rotation velocity estimation method based on sequence continuity provided in an embodiment of the present invention; Figure 3 A schematic diagram of a three-dimensional model of Tiangong-1 provided in an embodiment of the present invention; Figure 4 A schematic diagram of the geometric relationship of a slow-spinning spatial target rotation constructed by an ISAR image rotation velocity estimation method based on sequence continuity, provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the imaging plane turntable model in an ISAR image rotation velocity estimation method based on sequence continuity provided in an embodiment of the present invention; Figure 6 A schematic diagram of the projection geometry of an ISAR target slowly rotating around a fixed axis, constructed using an ISAR image rotation velocity estimation method based on sequence continuity, as provided in an embodiment of the present invention. Figure 7 This is a comparison of the rotation velocity estimation model in the ISAR image rotation velocity estimation method based on sequence continuity provided in this embodiment of the invention with the segmentation results of UNET and U2NET when there are missing images; Figure 8 This is a comparison of the rotation velocity estimation model in the ISAR image rotation velocity estimation method based on sequence continuity provided in this embodiment of the invention with the segmentation results of UNET and U2NET when the image has noise (0dB); Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0017] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0018] To address the problem of accurately estimating the rotation angle of spatial targets in existing ISAR images, this invention provides an ISAR image rotation velocity estimation method, electronic device, and storage medium based on sequence continuity.

[0019] It should be noted that the execution entity of the ISAR image rotation velocity estimation method based on sequence continuity provided in this embodiment of the invention can be a device, which can run in an electronic device. This electronic device can be a server or a terminal device, but is not limited to these.

[0020] Below, we will first introduce a method for estimating the rotation velocity of ISAR images based on sequence continuity, as provided in the embodiments of the present invention.

[0021] The present invention provides a method for estimating the rotation velocity of ISAR images based on sequence continuity, such as... Figure 1 As shown, it may include the following steps: S1, Obtain the image to be estimated; the image to be estimated is a continuous ISAR image; S2, using a pre-trained rotational velocity estimation model, performs contour extraction and rotational velocity estimation on the image to be estimated, obtaining the effective rotational velocity of the ISAR image; where, Pre-trained rotational velocity estimation models, such as Figure 2 As shown, it may include: The backbone network, encoder, decoder, and output head (BoxHead) are configured sequentially. The backbone network extracts hierarchical and multi-scale semantic features from the image to be estimated by introducing residual connections between convolutional layers, thus obtaining the multi-scale feature map corresponding to each frame of the image to be estimated. The encoder performs dimensionality unpacking and deformable attention on the multi-scale feature map of each frame image to obtain the flattened features corresponding to each frame image. The decoder processes the flattened features of each frame image based on instance query and the multi-scale deformable attention module to obtain the instance features, position information and rotation information corresponding to each frame image. The output head includes a bounding box head, a mask head, and a rotation head. The bounding box head processes the positional information of each frame using ReLU activation and a linear projection layer, and outputs the predicted normalized bounding box information. The mask head fuses and stitches the feature map sequence corresponding to each frame image based on instance features and bounding box information, and outputs an instance mask sequence. The rotation head stitches, fuses, and compresses the rotation information corresponding to adjacent frames images based on a lightweight regression head, and outputs the effective rotation speed of the ISAR image.

[0022] The purpose of this invention is: ISAR image sequences exhibit motion and morphological continuity. By mining the continuity features of adjacent images and using them to constrain the overall contour extraction results of adjacent images, the fluctuations in extraction results caused by changes in the quality of single-frame images can be effectively suppressed, improving the robustness of the algorithm. In recent years, the field of video image segmentation has seen the emergence of contour segmentation networks represented by SeqFormer (Sequence-to-Sequence Transformer for Video Object Segmentation). SeqFormer is an efficient Transformer architecture for instance segmentation of sequential images. It achieves information interaction between sequential images through an independent attention mechanism within each frame and the learning of shared instance query vectors between sequential images. Based on this, through an instance query decomposition mechanism, it effectively aligns target features in different frames, achieving the tracking and extraction of image sequence contours.

[0023] First, based on the temporal features of consecutive multi-frame ISAR images, this invention establishes an explicit mapping relationship between contour morphology changes and target rotation parameters between adjacent frames. This constrains the feature distribution of adjacent frames with rotational motion continuity, achieving integrated joint modeling of contour extraction and rotation estimation. This constraint not only improves the stability of contour extraction but, more importantly, provides a continuous and discernible geometric constraint foundation for rotation vector inversion. Second, this invention simultaneously executes contour segmentation and rotation estimation tasks within the same network framework. Utilizing a cross-task shared spatiotemporal feature representation and interaction mechanism, it significantly enhances the model's robustness in low signal-to-noise ratio environments, avoiding error accumulation caused by feature propagation in traditional multi-stage algorithms. Furthermore, by learning and regressing the rotation difference features between adjacent frames, this invention can directly output rotation speed without additional lateral calibration or manual keypoint detection, overcoming calibration errors and feature extraction uncertainties. Finally, thanks to the cross-frame feature fusion and rotation consistency constraint mechanism, this invention can still achieve stable and high-precision rotation speed estimation under complex noise and occlusion conditions, providing reliable technical support for ISAR image attitude inversion of space targets.

[0024] This invention addresses the difficulty in accurately estimating the rotation angle of spatial targets in existing ISAR images by proposing a rotation velocity estimation method based on sequence continuity. This method utilizes the SeqFormer network architecture and introduces a decomposable instance query mechanism and a cross-frame feature modeling strategy to establish an intrinsic mapping relationship between target morphological changes and rotational motion in continuous ISAR images. To improve the stability and geometric discriminability of rotation estimation, this invention integrates a multi-frame contour joint extraction module in the feature extraction stage. Through information interaction and feature fusion between multiple frames, it effectively suppresses the problem of local structure loss caused by noise and occlusion, providing continuous and complete geometric constraints for rotation estimation. By jointly modeling and learning the morphological changes and rotational differences between adjacent frames, high-precision inversion of the spatial target rotation velocity can be achieved, thus providing reliable support for attitude estimation of sequential spatial target ISAR images.

[0025] This invention utilizes a constructed ISAR electromagnetic simulation dataset to train a rotational velocity estimation model. Electromagnetic simulation data is used for experimental verification and performance analysis. During the electromagnetic simulation process, [the following is a continuation of the previous sentence, but the context is unclear]. Figure 3 The Tiangong-1 space station shown is the target model. The ISAR simulation signal is set in the form of linear frequency modulation, with a carrier frequency of 30 GHz, a bandwidth of 3 GHz, and a signal pulse repetition frequency of 50 Hz.

[0026] The target's on-orbit attitude was set to slow rotation around a fixed axis, with the maximum velocity of the rotation vector in each dimension set to 0.1 rad / s. The rotation vectors were set to [0, 0.01, 0] rad / s and [0, 0, 0.01] rad / s. Sub-aperture segmentation and imaging processing were performed on the simulated echoes to obtain ISAR image sequences under the slow rotation around a fixed axis.

[0027] The rotation velocity estimation model proposed in this embodiment of the invention relates to the ISAR rotation relationship of the space target sequence and the ISAR imaging projection geometry model of the slowly rotating space target around a fixed axis as follows: The ISAR rotation relationship of the space target sequence is as follows: Three-axis stability and slow rotation around a fixed axis are two typical on-orbit states of space targets. To facilitate analysis and derivation, this embodiment of the invention uses the slow rotation around a fixed axis to uniformly describe the on-orbit state of the target, and combines the principle of range Doppler imaging and spatial geometric relationship analysis to model the rotational motion relationship between adjacent images.

[0028] A schematic diagram of the geometric relationship of the rotation of a target in slow-rotation space, as shown below. Figure 4 As shown, O-XYZ is the orbital plane coordinate system of the space target. OZ points from the center of mass of the space target to the Earth's center, and OX points to the direction of the space target's movement and is parallel to the orbital plane. Let LOS be the target rotation vector, and LOS be the line-of-sight direction of the ground-based radar for observing the space target. The azimuth and elevation angles of the radar line of sight in the coordinate system are respectively... and , This refers to the radar observation time.

[0029] Ground-based ISAR performs long-term continuous observations of space targets, obtaining continuous echoes from them. Through sub-aperture segmentation and ISAR imaging processing, sequential ISAR images of the space targets can be obtained. For a single-frame ISAR image, assuming its coherence accumulation time is... The observation times of the initial echo and the final echo are respectively and The azimuth and elevation angles of the radar line of sight in the orbital plane coordinate system at the observation times corresponding to the start and end echoes of this frame are respectively... , and , Radar line-of-sight vector at the start of observation. and radar line-of-sight vector at the termination time They are as follows: ; ; Changes in the radar line of sight cause changes in the attitude of a space target relative to the radar line of sight. The direction of the equivalent rotational angular velocity of the space target caused by the change in the radar line of sight is defined as... , unit vector It can be calculated using the following formula: ; Besides the equivalent rotational motion of the space target caused by changes in the radar line of sight, the space target itself also exhibits rotational motion. This rotational motion, perpendicular to the radar line of sight, causes a change in the distance between the space target's scattering center and the radar, thus generating Doppler radiation, which is the basis for ISAR azimuth high-resolution imaging. Therefore, considering the space target's own rotational motion... Decomposed into components parallel to the radar line of sight. and the component perpendicular to the radar line of sight During coherent accumulation time Inner, effective rotational angular velocity of space target The expression is as follows: ; Throughout the ISAR observation process, as the distance between the radar and the space target changes and the radar's line of sight shifts, and It is also constantly changing. However, for two adjacent image frames, the total observation time is relatively short, at which point the ISAR imaging model of space targets can be simplified to a planar turntable model, such as... Figure 5 As shown. OU and OV represent the Doppler axis and range axis of the imaging projection plane, respectively. Let the observation time interval between two adjacent ISAR frames be denoted as . The effective rotational angular velocity of the target within the imaging CPI is The rotation of the space target on the imaging projection plane is , If this holds true, then the imaging projection planes of two adjacent ISAR frames can be considered approximately coplanar.

[0030] The projection model of the spatial target sequence ISAR image is constructed as follows: The ISAR imaging projection model for space targets establishes a mapping relationship between attitude parameters and images, supporting high-precision in-orbit attitude inversion. Three-axis stabilization and slow rotation around a fixed axis are two typical on-orbit states, with the latter encompassing the former (equivalent when the rotation speed is zero). By analyzing the range-Doppler imaging characteristics of the scattering center of a slow-rotating target, the established model can uniformly describe the imaging relationship between these two states.

[0031] The projection geometry of ISAR imaging for slowly rotating space targets around a fixed axis is as follows: Figure 6 As shown, ISAR obtains a sequence of L frames through long-term continuous observation, with the imaging time of each frame corresponding to its CPI midpoint. Because the target moves along its orbit, the radar line of sight changes over time, resulting in different imaging lines of sight for each frame. Let the... The imaging line of view of the frame is The radar line-of-sight rotation vector is The two satisfy the rotation vector relationship: Among them, the target itself rotates. It remains unchanged during observation. According to the Doppler effect, rotation only perpendicular to the radar line of sight will cause Doppler modulation at the scattering center, thus affecting azimuth resolution; while rotation along the radar line of sight does not produce Doppler on the echo. Therefore, The rotation component of the vertical imaging line of sight is defined as the first... The effective rotation components of a frame are expressed as follows: .

[0032] Let the attitude at the start of ISAR imaging observation be the initial attitude. Then the rotation transformation relationship between the main body attitude target and the initial attitude target is: ;in, and Represent the initial attitude and the body attitude of the target respectively. The coordinate vectors of the scattering centers Let be the attitude rotation matrix, and its expression is as follows: ; in, , , These are rotation matrices around the X-axis, Y-axis, and Z-axis, respectively.

[0033] No. The formula for calculating the coordinate vector of the nth scattering center of the target at the moment of frame image imaging is as follows: ; in, This represents the angle the target has rotated from the initial moment to the imaging moment. Let the rotation matrix be the rotation matrix corresponding to the target rotation vector. ; express , and Then in the first At the time of frame imaging, the target's first The expression for the theoretical projection position of each scattering center on the imaging plane is as follows: ; in, Indicates the wavelength of the radar signal. and These are Doppler resolution units and range resolution units, respectively. Indicates the first The imaging projection matrix of a frame image.

[0034] The rotational speed estimation model of this invention embodiment, such as Figure 2 As shown, the input consists of three consecutive frames of ISAR images. The backbone network and encoder extract feature maps for each frame, and the decoder generates a representation of the satellite's global information and rotation state within the image sequence. Different output heads are then used to perform mask segmentation, bounding box prediction, and rotation vector prediction for the satellite. Each part of the network will be described in detail below.

[0035] Specifically, the backbone network may include: The initial convolutional layer and four residual stages; among which, Each residual stage includes a bottleneck module, and each bottleneck module consists of several convolutions of different dimensions.

[0036] In this embodiment of the invention, ResNet-50 is used as the backbone network for feature extraction. ResNet-50 is a 50-layer residual network that effectively alleviates the gradient vanishing and performance degradation problems common in deep convolutional networks by introducing residual connections between convolutional layers. The network consists of an initial 7×7 convolutional layer and four residual stages, each containing multiple bottleneck modules composed of 1×1, 3×3, and 1×1 convolutions. This structure can efficiently extract hierarchical and multi-scale semantic features from the input image. Multi-scale feature maps for each frame of the image are obtained through the backbone network.

[0037] The feature maps processed by the backbone network are first standardized in terms of channel count before entering the encoder. To perform an attention mechanism separately for each frame, the feature maps are flattened only in the length and width dimensions of the image, preserving the temporal and spatial dimensions of the feature maps. Deformable attention is applied to the feature maps at each encoder layer without altering their shape, thus obtaining the flattened features corresponding to each frame.

[0038] When observing continuous information, the same instance is treated as a whole, making it easy to identify instances in each frame and automatically link them together. This is the basis for the decomposable instance query used by the decoder. This embodiment of the invention introduces a set of learnable fixed embedding vectors, called instance queries, used to extract features of the same target across multiple frames. To handle changes in target appearance and location, each instance query is decomposed into E frame-specific bounding box queries. Each bounding box query serves as an anchor point for locating features in the current frame, allowing for independent feature retrieval within that frame.

[0039] In the decoder, the instance query is a decomposable instance query, used to extract features of the same target in multiple frames of images; each instance query is decomposed into several bounding box queries, and each bounding box query serves as an anchor point for locating features in the current frame, and feature retrieval is performed independently on the current frame.

[0040] In the first layer, instance queries and bounding box queries use the same initialization. The bounding box query contains the instance's location information, which is used to generate the sampling locations for deformable attention. The initial bounding box query independently queries instance features on each frame's feature map. ; in, Indicates the first Initial frame query, This represents a multi-scale deformable attention module, which focuses on only a small set of key sampling points. Indicates the first Feature map of the frame For the first layer to the first Output features after the frame performs the attention mechanism.

[0041] In the decoder, the instance query is a decomposable instance query, used to extract features of the same target in multiple frames of images; each instance query is decomposed into several bounding box queries, and each bounding box query serves as an anchor point for locating features in the current frame, and feature retrieval is performed independently on the current frame.

[0042] The output features will generate features with different emphases after passing through different projection layers, which can be used for different tasks: The expression for instance features is as follows: ; Indicates the first Instance features of all frames generated by the layer decoder. Indicates generation from output features linear layers, Indicates the first Output features after the layer executes the attention mechanism; The expression for location information is as follows: ; Indicates the first The position information of all frames generated by the layer decoder Indicates generation from output features Linear layers; The expression for rotation information is as follows: ; Indicates the first Rotation information for all frames generated by the layer decoder. Indicates generation from output features Linear layers.

[0043] Instance queries aggregate temporal features at the end of each decoder layer by weighted summation of all box queries. These weights are obtained through end-to-end learning of box embeddings. ; In the N ( N >1) Layer, the box query obtained from the previous layer will be used as input: ; go through After the layer is decoded, each instance corresponds to an instance query. T Individual query and T Each element has a rotation query. The instance query represents the overall features in the image sequence, the bounding box query contains positional information for predicting the bounding boxes in each frame, and the rotation query contains difference information for predicting rotational changes between multiple frames. The instance query, bounding box query, and rotation query output by the last decoder layer are defined as follows: , and .

[0044] This invention utilizes the rotation consistency and deformation correlation among three consecutive ISAR images to establish a cross-frame feature interaction structure, rather than the traditional single-frame independent segmentation; through a decomposable instance query mechanism, the same target can share semantic query vectors in different frames, achieving consistent identification and associated tracking of the target across consecutive images; and in the same neural network, contour segmentation is used to assist in the rotation speed regression task, improving the accuracy of rotation feature extraction, estimating rotation speed with higher precision, and avoiding feature duplication and error accumulation.

[0045] The output heads include a bounding box head, a mask head, and a rotation head; the decoder output will pass through three output heads, namely the bounding box head, the mask head, and the rotation head.

[0046] The bounding box header uses ReLU activation and a linear projection layer to process the positional information of each frame and outputs the predicted normalized bounding box information.

[0047] The bounding box header consists of three layers of feedforward networks. The information of the normalized bounding box can include: the center coordinates, height, and width of the bounding box.

[0048] The bounding box header consists of a three-layer feedforward network, using ReLU activation and linear projection layers for each frame. Output the center coordinates, height, and width of the predicted normalized bounding box. Therefore, for an index of... For example, let the predicted box sequence be denoted as . .

[0049] The masking head fuses and stitches together the feature map sequence corresponding to each frame of the image based on instance features and bounding box information to generate an instance mask sequence.

[0050] The masking head fuses and stitches together the feature map sequence corresponding to each frame of the image based on instance features and bounding box information to generate an instance mask sequence, which may include: The mask head uses a feedforward network to use instance features as mask head parameters; Multi-scale features are extracted from the encoder to generate a sequence of feature maps for each frame of the image. The feature map sequence corresponding to each frame image is concatenated with the relative coordinate map in the bounding box information of that frame image to obtain the concatenated feature map; Based on the mask head parameters and the spliced ​​feature map, an instance mask sequence is generated through convolution.

[0051] Specifically, the instance query output by the decoder incorporates cross-frame information, which can be used to efficiently generate the entire mask sequence. First, a three-layer feedforward network encodes the instance query into a corresponding index. Mask head parameters The feedforward network consists of three 1×1 convolutional layers, each containing 8 channels, and all except the last layer are activated using ReLU.

[0052] The network includes a masking branch that provides feature maps to the masking head for instance mask prediction. This branch employs an FPN-like structure, extracting multi-scale features from the encoder and generating independent feature map sequences for each frame. The resolution is 1 / 8 of the input, and the number of channels is 8. Then, the feature map of each frame... With the predicted bounding box in the corresponding frame The relative coordinate maps of the center are stitched together to provide positional guidance information for instance mask prediction, thus obtaining... The mask head then generates an instance mask sequence through convolution: .

[0053] The mask head employs a dynamic convolutional structure, performing three 1×1 convolutions on the input feature map. Since the same instance shares mask head parameters across different frames, this method can efficiently achieve cross-frame instance segmentation, improving segmentation efficiency. Similar to DETR, this embodiment adds an output head and Hungarian matching loss after each decoder layer as auxiliary supervision to improve training performance.

[0054] The rotation head, based on a lightweight regression head, stitches, fuses, and compresses the rotation information corresponding to adjacent frames to output the effective rotation speed of the ISAR image, which may include: The rotation head uses a lightweight regression head to stitch together the rotation information corresponding to adjacent frames to form temporal features; The temporal features are reduced in dimensionality through linear mapping to obtain the dimensionality-reduced features; The dimensionality-reduced features are concatenated and fused along the feature channel dimension to obtain the fused features; The fused features are compressed through a two-layer linear network, and the output is... N One prediction result; Using the matcher based on N The prediction results output the effective rotation speed of the ISAR image.

[0055] Rotation within three consecutive frames of an ISAR image can be considered as rotation within the same plane due to the short time interval. To estimate the effective rotational velocity of the target across these three frames, a lightweight regression head module is employed. This module uses the decoder output... Features are used as input, combining the first frame with the second frame, and the second frame with the third frame. Two sets of temporal features are concatenated and then fed into a shared intra-frame dimensionality reduction network for encoding. Each concatenated feature set undergoes dimensionality reduction via a linear mapping (Linear + LayerNorm + ReLU + Dropout), and the two dimensionality-reduced feature sets are then further concatenated along the feature channel dimension. The fused features are further compressed via two layers of linear networks, outputting rotation velocity estimates for the corresponding three frames. This structure effectively captures inter-frame temporal variations while maintaining network simplicity and low computational overhead. All weight parameters of the network structure are learned end-to-end during training. Modeled according to the aforementioned projection relationship, the effective rotation velocity output by this module is further estimated using the PSO search method.

[0056] The ISAR image rotation velocity estimation method proposed in this embodiment generates a fixed number of [items] each time the image passes through the decoder. A One prediction, of which A The number of instances is much larger than the number of instances. To train the network, bipartite graph matching between the prediction set and the ground truth labels is required first. Let the ground truth instance set be... y The predicted instance set is , Each real instance can be represented as ,in, Indicates the first The center coordinates, relative height, and width of the predicted bounding box and the ground truth bounding box in the frame. For index 1 Predicted instances, defining the true labels The matching cost between the prediction result and the actual result is: .

[0057] Because pixel-by-pixel comparison is computationally expensive, the matching process does not consider the similarity between the predicted mask and the ground truth mask. To find the optimal match between the predicted and ground truth instances, a permutation of N elements... The middle search minimizes the total matching cost: .

[0058] This invention employs the Hungarian algorithm to calculate the optimal match. The network is trained using Hungarian loss for all optimal matching pairs.

[0059] The loss function of the pre-trained rotational velocity estimation model during training is the Hungarian loss function, which is expressed as follows: ; in, Indicates border loss. Indicates mask loss. Represents a set of real instances. Represents the predicted instance set, This indicates the loss of rotational speed. Represents the actual bounding box, Indicates the optimal match. Represents the predicted bounding box. Represents the actual mask. Indicates the predicted mask, Represents the actual rotational speed. This indicates the predicted rotational speed. This indicates the number of predicted instances.

[0060] Border loss Depend on The loss is a weighted combination of the generalized IoU loss. This is due to the mask sequence output by the mask head. Since the resolution is 1 / 8 of the original video, details may be lost. Therefore, this embodiment of the invention upsamples the predicted mask to 1 / 4 and then downsamples the real mask to the same resolution to calculate the mask loss. Mask loss It combines Dice loss and Focal loss. (The remaining text appears to be incomplete and possibly contains errors.) Defined as the squared difference between the predicted and actual values. Finally, for each frame, the bounding box, mask, and rotation speed losses are calculated separately, and the average is used for the Hungarian matching loss.

[0061] The specific parameter settings for the rotational velocity estimation model are as follows: This embodiment of the invention uses ResNet-50 as the backbone network and extracts features {C3, C4, C5} from its last three stages, with strides of {8, 16, 32}. Furthermore, a minimum resolution feature C6 is constructed on C5 using a 3×3 convolution with a stride of 2. The multi-scale deformable attention module sets the number of keypoints K=4 and uses 8 attention heads. Both the encoder and decoder are set to 6 layers, with a hidden dimension of 256 and 300 instance queries.

[0062] During the training of the rotational velocity estimation model, this embodiment of the invention uses the AdanW optimizer, with 16 training epochs and a base learning rate of 2×10⁻⁶. -5 A multi-step learning rate decayer is used, with a decay factor set to 0.5, and the learning rate decays in rounds 4 and 12. The weight decay is 10. -4 The learning rate for the linear projection in the backbone network and deformable attention is scaled to 0.1 of its original value. The model trains with T=3 frames per session, consisting of three consecutive imaging frames. The model runs on PyTorch 1.7 and uses an NVIDIA RTX 4090 GPU.

[0063] The pre-trained rotational velocity estimation model can directly process ISAR simulation image sequences of arbitrary length without dividing them into subsequences. During mask inference, the entire sequence is input, and the model learns the overall representation of the instances, which is then used for dynamic segmentation in each frame. The bounding box sequence is directly generated by the decoder, requiring no additional instance association post-processing.

[0064] Table 1 shows a comparison of the average cross-union ratio (CUI) results of the rotational velocity estimation model proposed in this embodiment of the invention with those of UNET and U2NET for segmenting four sets of ISAR electromagnetic simulation images.

[0065] Table 1

[0066] As can be seen from Table 1, the rotation speed estimation model proposed in the embodiments of the present invention has the highest average value and the best segmentation effect.

[0067] A comparison of the rotation velocity estimation model proposed in this embodiment of the invention with the segmentation results of UNET and U2NET when images have missing parts is shown in the figure. Figure 7 As shown, from Figure 7 As can be seen, when there are missing or occluded frames in consecutive frames, UNET and U2NET cannot segment them well, resulting in incomplete segmentation results. However, the rotational velocity estimation model proposed in this embodiment can achieve good segmentation using the learned overall features. A comparison of the segmentation results of the rotational velocity estimation model with UNET and U2NET when the image has noise (0dB) is shown in the figure. Figure 8 As shown, from Figure 8 As can be seen, UNET and U2NET cannot effectively segment the target contour in the presence of noise, while the model proposed in this embodiment of the invention can still obtain relatively complete feature extraction results.

[0068] The average intersection-over-union ratio (IoU) of the rotation speed estimation model proposed in this embodiment of the invention with that of UNET and U2NET when the image has missing parts or noise (0dB) is compared in Table 2.

[0069] Table 2

[0070] As can be seen from Table 2, when adding noise to the image, the average cross-union ratio of the traditional method is greatly reduced, while the average cross-union ratio of the rotation speed estimation model proposed in this embodiment of the invention is not reduced.

[0071] The results of the effective rotational velocity estimation error and the effective rotational vector estimation error proposed in the embodiments of the present invention are shown in Table 3.

[0072] Table 3

[0073] As can be seen from Table 3, the errors of the rotational speed estimation model proposed in this embodiment are all at a low level.

[0074] The rotational velocity estimation model proposed in this invention combines a decomposable instance query mechanism with a cross-frame feature modeling strategy to fully exploit the rotational consistency and geometric correlation between adjacent frames. By establishing dynamic rotational constraints across multiple frames, this invention achieves unified modeling and accurate analysis of continuous rotational features of the target. In the feature extraction stage, this invention utilizes a cross-frame feature interaction and fusion mechanism to effectively mitigate local structural discontinuities caused by noise and occlusion, maintaining the morphological integrity of the target in the time series and providing stable geometric support for rotational estimation. Furthermore, by modeling and regressing the rotational difference features between adjacent frames, the network can directly output the target's rotational velocity, avoiding the complex process of calibration or keypoint matching in traditional methods, significantly improving estimation accuracy and stability.

[0075] Secondly, embodiments of the present invention also provide an electronic device, such as... Figure 9 As shown, it includes a processor 001, a communication interface 002, a memory 003, and a communication bus 004, wherein the processor 001, the communication interface 002, and the memory 003 communicate with each other through the communication bus 004. The memory is used to store computer programs; When the processor executes the program stored in the memory, it implements the steps of any of the ISAR image rotation velocity estimation methods based on sequence continuity provided in the first aspect of the present invention.

[0076] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0077] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0078] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0079] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0080] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0081] Thirdly, corresponding to the ISAR image rotation velocity estimation method based on sequence continuity provided in the first aspect, this embodiment of the invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the ISAR image rotation velocity estimation methods based on sequence continuity provided in the first aspect of this invention.

[0082] For the electronic device / storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and relevant details can be found in the description of the method embodiments.

[0083] It should be noted that, in the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0084] The various embodiments in this specification are described in a related manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0085] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for estimating the rotational velocity of ISAR images based on sequence continuity, characterized in that, include: Obtain the image to be estimated; The image to be estimated is a continuous ISAR image; A pre-trained rotational velocity estimation model is used to extract contours and estimate rotational velocity in the image to be estimated, thereby obtaining the effective rotational velocity of the ISAR image; wherein, The pre-trained rotational velocity estimation model includes: The backbone network, encoder, decoder, and output head are set in sequence. The backbone network extracts hierarchical and multi-scale semantic features from the image to be estimated by introducing residual connections between convolutional layers, thereby obtaining a multi-scale feature map corresponding to each frame of the image to be estimated. The encoder performs dimensional expansion and deformable attention on the multi-scale feature map of each frame image to obtain the flattened features corresponding to each frame image. The decoder processes the flattened features of each frame image based on instance query and a multi-scale deformable attention module to obtain the instance features, position information and rotation information corresponding to each frame image. The output head includes a bounding box head, a mask head, and a rotation head. The bounding box head processes the positional information of each frame using ReLU activation and a linear projection layer, and outputs the predicted normalized bounding box information. The mask head fuses and stitches the feature map sequence corresponding to each frame image based on instance features and bounding box information, and outputs an instance mask sequence. The rotation head stitches, fuses, and compresses the rotation information corresponding to adjacent frames images based on a lightweight regression head, and outputs the effective rotation speed of the ISAR image.

2. The method for estimating the rotation velocity of ISAR images based on sequence continuity according to claim 1, characterized in that, The backbone network includes: The initial convolutional layer and four residual stages; among which, Each residual stage includes a bottleneck module, and each bottleneck module consists of several convolutions of different dimensions.

3. The method for estimating the rotation velocity of ISAR images based on sequence continuity according to claim 1, characterized in that, In the decoder, the instance query is a decomposable instance query, used to extract features of the same target in multiple frames of images; each instance query is decomposed into several bounding box queries, and each bounding box query serves as an anchor point for locating features in the current frame, and feature retrieval is performed independently on the current frame.

4. The method for estimating the rotation velocity of ISAR images based on sequence continuity according to claim 1, characterized in that, The expression for the instance feature is as follows: ; Indicates the first Instance features of all frames generated by the layer decoder. Indicates generation from output features linear layers, Indicates the first Output features after the layer executes the attention mechanism; The expression for the location information is as follows: ; Indicates the first The position information of all frames generated by the layer decoder Indicates generation from output features Linear layers; The expression for the rotation information is as follows: ; Indicates the first Rotation information for all frames generated by the layer decoder. Indicates generation from output features Linear layers.

5. The method for estimating the rotation velocity of ISAR images based on sequence continuity according to claim 1, characterized in that, The bounding box header consists of three layers of feedforward network, and the information of the normalized bounding box includes: the center coordinates, height, and width of the bounding box.

6. The method for estimating the rotation velocity of ISAR images based on sequence continuity according to claim 1, characterized in that, The mask head fuses and stitches together the feature map sequence corresponding to each frame image based on instance features and bounding box information to generate an instance mask sequence, including: The mask head uses a feedforward network to use instance features as mask head parameters; Multi-scale features are extracted from the encoder to generate a sequence of feature maps for each frame of the image. The feature map sequence corresponding to each frame image is concatenated with the relative coordinate map in the bounding box information of that frame image to obtain the concatenated feature map; Based on the mask head parameters and the spliced ​​feature map, an instance mask sequence is generated through convolution.

7. The method for estimating the rotation velocity of ISAR images based on sequence continuity according to claim 1, characterized in that, The rotating head, based on a lightweight regression head, stitches, fuses, and compresses the rotation information corresponding to adjacent frames of the image, outputting the effective rotation speed of the ISAR image, including: The rotating head uses a lightweight regression head to stitch together the rotation information corresponding to adjacent frames to form temporal features; The temporal features are reduced in dimensionality through linear mapping to obtain the dimensionality-reduced features; The dimensionality-reduced features are concatenated and fused along the feature channel dimension to obtain the fused features; The fused features are compressed through a two-layer linear network, and the output is... N One prediction result; Using the matcher based on N The prediction results output the effective rotation speed of the ISAR image.

8. The method for estimating the rotation velocity of ISAR images based on sequence continuity according to claim 1, characterized in that, The loss function used in the training process of the pre-trained rotational velocity estimation model is the Hungarian loss function, and the expression of the Hungarian loss function is as follows: ； in, Indicates border loss. Indicates mask loss. Represents a set of real instances. Represents the predicted instance set, This indicates the loss of rotational speed. Represents the actual bounding box, Indicates the optimal match. Represents the predicted bounding box. Represents the actual mask, The mask representing the prediction. Represents the actual rotational speed. This indicates the predicted rotational speed. This indicates the number of predicted instances.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs. When the processor executes the program stored in the memory, it implements the steps of the ISAR image rotation velocity estimation method based on sequence continuity as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the ISAR image rotation velocity estimation method based on sequence continuity as described in any one of claims 1-8.

Images