Space target recognition method based on complex domain multi-scale visual transformer

Abstract

The method for space target recognition based on complex domain multi-scale visual transformer belongs to the field of ISAR image processing. In order to solve the problem of defocusing of the ISAR image of a mobile space target and the problem of poor recognition effect of a single image, the method comprises the following steps: acquiring ISAR motion space sample data, grouping every three continuous image frames into an image sequence sample, and dividing the ISAR defocusing image sequence into image sequence blocks of different sizes; inputting the image sequence blocks of different sizes after division into two complex domain convolution modules of different sizes for feature extraction, respectively obtaining the feature embedding of the image sequence of two sizes in the complex domain, adding the feature embedding and the position coding to obtain a feature vector F after addition, combining the feature vector F after addition with a recognition coding vector, and then inputting the combined output into a complex domain CVMVT network based on a transformer model to realize target recognition. The method is used for space target recognition of ISAR images.

Description

Technical Field

[0001] This invention relates to a spatial target recognition method based on a complex domain multi-scale visual transformer, belonging to the field of ISAR image processing. Background Technology

[0002] In recent years, research on inverse synthetic aperture radar (ISAR) image recognition has deepened, but most methods only utilize the amplitude information of ISAR image data. Furthermore, higher-order terms in the complex-valued (CV) received signal of maneuvering space targets can cause ISAR images to defocus, affecting recognition accuracy. Simultaneously, single-frame images are not effective at recognizing space targets with similar appearances, while the phase information between frames in an image sequence can aid in recognition.

[0003] To address this, this invention proposes an end-to-end recognition framework for the complex domain based on the transformer model, CVMVT (Complex-valued Multi-scale Vision Transformer). It uses ISAR image sequences as input, avoiding the limitations of single-frame image feature extraction, and employs a CV attention mechanism to capture information between image frames from the input sequence to aid recognition. Furthermore, combining multi-scale feature extraction methods with CV-CNN allows for better extraction of local features from ISAR images. Finally, by calculating the covariance of the real and imaginary parts, it innovatively uses defocus information from higher-order terms fed forward by the CV Layernorm (LN) layer to enhance recognition. Summary of the Invention

[0004] To address the issues of defocusing in ISAR images of maneuvering space targets and poor recognition performance in single-frame images, this invention provides a space target recognition method based on a complex-domain multi-scale visual transformer.

[0005] A spatial target recognition method based on a complex-domain multi-scale visual transformer includes the following steps:

[0006] ISAR motion space sample data is acquired. The ISAR motion space sample is a sequence of ISAR defocused images of a moving target. Every three consecutive image frames form an image sequence sample. The ISAR defocused image sequence is divided into image sequence blocks of different scales.

[0007] The segmented image sequence blocks of different sizes are input into two complex domain convolution modules of different scales for feature extraction, respectively, to obtain the feature embeddings of the image sequences of the two scales in the complex domain. Then, this feature embedding is added to the position code to obtain the summed feature vector F, where the position code represents the position of the segmented small image block in the original image. The feature vector F is then combined with the recognition code vector, which represents the classification of the target.

[0008] The combined output is input into a complex domain CVMVT network for target recognition.

[0009] The complex-domain CVMVT network comprises two processing paths, each used to process the combined output corresponding to image sequence blocks of different sizes. Each processing path includes a complex-domain convolutional module and a complex-domain transformer encoder module. Therefore, the two processing paths contain two complex-domain convolutional modules and two complex-domain transformer encoder modules with different parameters.

[0010] The first complex-domain convolutional module consists of two convolutional layers, each followed by a complex batch normalization layer, a complex max pooling layer, and a complex ReLU activation layer. Then, the output feature map dimensions are expanded and the dimensions of the image patches are reconstructed. Finally, a complex-domain fully connected layer is added.

[0011] The first complex-domain transformer encoder module contains two complex-domain layer normalization layers, one complex-domain attention mechanism layer, and one complex-domain feedforward layer. The complex-domain attention mechanism layer contains eight attention modules, each containing four real-domain fully connected layers. The first three real-domain fully connected layers extract features from the transformer model's query matrix, keyword matrix, and value matrix, respectively, and the fourth fully connected layer outputs the features. The complex-domain feedforward layer contains two complex-domain fully connected layers and one complex ReLU activation layer.

[0012] The second complex domain convolutional module consists of two convolutional layers, each followed by a complex batch normalization layer, a complex max pooling layer, and a complex ReLU activation layer. Then, the output feature map dimensions are expanded and the dimensions of the image patch are reconstructed. Finally, a complex domain fully connected layer is added.

[0013] The second complex-domain transformer encoder module contains two complex-domain layer normalization layers, one complex-domain attention mechanism layer, and one complex-domain feedforward layer. The complex-domain attention mechanism layer contains eight attention modules, each containing four real-domain fully connected layers. The first three real-domain fully connected layers extract features from the transformer model's query matrix, keyword matrix, and value matrix, respectively, and the fourth fully connected layer outputs the features. The complex-domain feedforward layer contains two complex-domain fully connected layers and one complex ReLU activation layer.

[0014] In the complex domain transformer encoder module, the input and the complex domain attention mechanism layer are residually connected, and the output after the residual connection is then residually connected to the complex domain feedforward layer.

[0015] Finally, the outputs of the complex domain transformer encoder modules at both scales are concatenated in the last dimension and passed through a fully connected complex domain layer with an output dimension of 8 to obtain the output vector of the CVMVT model; thus realizing spatial target recognition.

[0016] Furthermore, the processing procedure for the complex field layer normalization layer includes the following steps:

[0017] For input complex CV data CV_LN is represented as:

[0018]

[0019] Where i represents the order of the feature maps, This is the output of this layer, where B represents the batch size and N represents the output of this layer. s It is the number of samples, and D is the dimension of the one-dimensional vector obtained after feature extraction from the input complex data; It is a virtual part unit. and represent the real and imaginary parts of the CV data, respectively; E(·) represents the average value of the data calculated in the sample dimension;

[0020] Covariance matrix V l Represented as

[0021]

[0022] in

[0023]

[0024]

[0025]

[0026] Furthermore, the complex domain attention mechanism layer, namely the complex domain transformer attention mechanism for sequences of two-dimensional images, includes the following steps in its processing:

[0027] For the output of the CV_LN layer a and b are used to represent First, a CV-Linear layer is used to add high-dimensional information to the component matrix of the transformer model, represented as follows:

[0028]

[0029] Among them, CL N This refers to a linear activation layer in the complex field, i.e., a CV-Linear layer; L N N represents the linear activation layer in the real number field; N = Q, K, V, where Q, K, and V represent the query matrix, keyword matrix, and value matrix, respectively.

[0030] The attention weight is obtained by multiplying the query matrix and the keyword matrix to calculate relevance, and is represented as follows:

[0031]

[0032] Wherein, CW represents the relevant weight;

[0033] Finally, the complex attention result is obtained by calculating the weighted sum of the value matrix and the relevant weights, which is then calculated as follows:

[0034]

[0035] Among them, I Q (n), I K (n), I V (n) represents the attention results corresponding to the query matrix, keyword matrix, and value matrix in the real-valued (RV) domain, where n = a, b.

[0036] Furthermore, the position encoding size is K×dim, where K is the number of small image blocks after image segmentation, and dim is the dimension size of the output of the convolution module.

[0037] Furthermore, the ISAR motion space sample data is obtained by simulation using the range-Doppler algorithm based on a three-dimensional spatial target model.

[0038] Preferably, the ISAR defocused image sequence data size is 3×224×224. After image segmentation, the large-scale image block sequence data size is 48×56×56, and the small-scale image block sequence data size is 147×32×32.

[0039] Preferably, the specific structural parameters of the first complex field convolution module are as follows:

[0040] The first convolutional layer has a kernel size of 3×3 pixels and a stride of 1 pixel, with a total of 16 kernels.

[0041] The stride of the complex max pooling layer is 2 pixels.

[0042] The second convolutional layer has 32 kernels with a kernel size of 3×3 pixels and a stride of 1 pixel.

[0043] The stride of the complex max pooling layer is 2 pixels.

[0044] The output dimension of the last fully connected layer in the complex field is 384.

[0045] Preferably, the specific structural parameters of the first complex domain transformer encoder module are as follows:

[0046] In the complex field attention mechanism layer, the output size of the first three of the four real field fully connected layers in each module is 512, and the output size of the fourth fully connected layer is 384.

[0047] The complex domain feedforward layer contains two fully connected complex domain layers with output dimensions of 512 and 384, respectively.

[0048] Preferably, the specific structural parameters of the second complex field convolution module are as follows:

[0049] The first convolutional layer has a kernel size of 3×3 pixels and a stride of 1 pixel, with a total of 16 kernels.

[0050] The stride of the complex max pooling layer is 2 pixels.

[0051] The second convolutional layer has 32 kernels with a kernel size of 3×3 pixels and a stride of 1 pixel.

[0052] The stride of the complex max pooling layer is 2 pixels.

[0053] The output dimension of the last fully connected layer in the complex field is set to 192.

[0054] Preferably, the specific structural parameters of the second complex domain transformer encoder module are as follows:

[0055] In the complex field attention mechanism layer, the output size of the first three of the four real field fully connected layers in each module is 512, and the output size of the fourth fully connected layer is 192.

[0056] The complex domain feedforward layer contains two fully connected complex domain layers with output dimensions of 512 and 192, respectively.

[0057] The beneficial effects of this invention are:

[0058] This invention proposes an end-to-end image recognition framework called CVMVT for recognizing ISAR image sequences of maneuvering space targets. First, the method trains an end-to-end recognition model, achieving fast recognition speed and high accuracy. Furthermore, this invention proposes an innovative complex-layer normalization layer that fully mines the implicit defocus information between the real and imaginary parts of the input by calculating their covariance, thereby improving recognition performance. In addition, this invention proposes a CV attention mechanism for sequences of two-dimensional images to capture the spatiotemporal continuity information between image frames in the input sequence. Finally, combining multi-scale feature extraction methods with CV-CNN can better extract local features of ISAR images, compensating for the disadvantage of the transformer model in extracting local image features poorly.

[0059] This invention utilizes deep learning technology to better meet the recognition requirements of ISAR defocused images of moving targets in different imaging scenarios. The network maintains a certain degree of stability for changes in viewpoint, affine transformation, and noise in the input image. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of generating a sequence of ISAR defocused images of a maneuvering space target;

[0061] Figure 2 This is a flowchart of the CVMVT framework.

[0062] Figure 3 This is a schematic diagram of the overall structure of a complex domain network;

[0063] Figure 4 It includes serial and parallel processing methods for image sequences of different spatial targets and flickering phenomena;

[0064] Figure 5 shows the visualization results of attention from different models; where Figures 5(a)-5(c) These are the original ISAR images of different space targets; Figures 5(d)-5(f) This is a visualization of the attention of the CVMVT model; Figures 5(g)-5(i) This is a visualization of the attention result of the RVVIT model. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] This invention primarily addresses the following issues: higher-order terms in the complex-valued (CV) received signals of maneuvering space targets can cause ISAR images to defocus, affecting recognition accuracy. Furthermore, single-frame images are ineffective for recognizing similar-looking space targets, while phase information between frames in a sequence is helpful for recognition. To address these issues, this invention proposes an end-to-end recognition framework in the CV field based on the transformer model, CVMVT. It uses ISAR image sequences as input, avoiding the limitations of single-frame image feature extraction, and employs a CV attention mechanism to capture information between image frames from the input sequence to aid in recognition. Moreover, combining multi-scale feature extraction methods with CV-CNN allows for better extraction of local features from ISAR images. Finally, by calculating the covariance of the real and imaginary parts, an innovative CV Layernorm (LN) layer is used to forward defocus information with higher-order terms, thus promoting recognition.

[0067] according to Figure 4 As can be seen, due to the angular scintillation characteristics of ISAR images, the position and intensity of scattering points between adjacent frames vary significantly. Generally, RNN methods for sequence recognition process the input sequence data in a concatenated manner. However, the variation in the position and intensity of scattering points between ISAR image frames makes this concatenated processing ineffective. But the transformer attention uses a parallel method, which can calculate the correlation between interval frames. Because the angular scintillation of ISAR images is random in practice, the correlation of target features between interval frames may be higher than that between adjacent frames. Therefore, this invention uses the transformer attention mechanism instead of RNN.

[0068] The reason for using a sequence-based approach for identification is that spatial targets have relatively specific functions, resulting in high similarity in their appearances. They all possess key components such as solar panels and antennas, making the appearance features of different targets within a single frame image highly similar. This makes identification based on a single frame image more difficult. Further observation... Figure 4During the maneuvering of space targets, the targets look similar in the first frame, making identification difficult. However, the differences between the two targets become more obvious in the third frame. Therefore, using a sequence method can reduce the difficulty of image recognition. Furthermore, during the sequence recognition process, the target's maneuvering also contains features that are helpful for recognition. The attention mechanism of the transformer can effectively extract the target's maneuvering feature information and defocus information, which further aids in recognition.

[0069] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0070] Specific Implementation Method 1: Combination Figures 1 to 4 As shown,

[0071] The spatial target recognition method based on a complex domain multi-scale visual transformer described in this embodiment includes the following steps:

[0072] Based on the simulation of the three-dimensional spatial target model and the range Doppler algorithm, a sample dataset of ISAR defocused image sequences of maneuvering targets was obtained, and the ISAR defocused image dataset was grouped into a training sample library and a test sample library.

[0073] An image sequence recognition network based on a complex domain multi-scale visual transformer (CVMVT) is constructed, which includes a multi-scale feature extraction network, an innovative complex domain layer normalization (LN) layer, and a transformer attention mechanism in the complex domain. The CVMVT is trained using defocused training sequence samples to obtain the training model.

[0074] The trained recognition model is used to test the defocused image sequence to obtain the recognition results of the test image sequence.

[0075] More specifically,

[0076] Step 1: Acquire ISAR moving target sample data, i.e., image data, and preprocess the image dataset:

[0077] ISAR moving spatial target samples were obtained by simulation using the range-Doppler (RD) algorithm based on a three-dimensional spatial target model. The ISAR moving spatial target samples were ISAR defocused image sequences of maneuvering targets, and the rotational acceleration of the target model was set to 5 rad / s². 2 In the time series (T1,...,T) n The ISAR defocused image sequence (S1,...,S) with a size of 224×224 was obtained from the image. n );

[0078] Every three consecutive image frames form an image sequence sample, such as seq1 = (S1, S2, S3) and seq2 = (S2, S3, S4). Due to the movement of the target, there will be differences in features between the image frames in the sequence. These differences are useful for target recognition, so it is necessary to combine the image frames into an image sequence.

[0079] The ISAR image sequence sample dataset is divided into a training set and a test set. 80% of the ISAR defocused image sequence samples are selected as the training set. The training set contains a label, which is a number representing the class to which the target belongs. The test set does not contain a label.

[0080] like Figure 1 As shown, the process of obtaining ISAR moving space target samples includes three parts: three-dimensional modeling of space targets, setting of electromagnetic parameters, and ISAR imaging.

[0081] The three-dimensional modeling of the space target is responsible for generating a three-dimensional model of the simulated space target; the electromagnetic parameter setting sets the frequency of the generated grid according to the size of the model, and the corner points of the grid lines on the target model are used as the spatial coordinates of the three-dimensional scattering points; the ISAR imaging part uses the three-dimensional information of the target's scattering points, and performs ISAR imaging simulation according to the set radar parameters and target motion parameters to obtain ISAR moving space target image sequences; then, the simulation is performed to obtain image sequences of 8 types of targets to form a simulation sample library.

[0082] ISAR defocused image segmentation: The ISAR defocused image sequence is segmented into image sequence blocks of size H1×H1 and H2×H2. For example, seq1 becomes after image segmentation. and in Let n represent the complex number field, n = 1, 2, 3, and N1 and N2 represent the number of small image blocks after image segmentation, respectively.

[0083] In this embodiment, the ISAR defocused image sequence data size is 3×224×224. The ISAR defocused images in the image sequence sample are segmented. After segmentation, the large-scale image patch sequence data size is 48×56×56, and the small-scale image patch sequence data size is 147×32×32, containing a total of 8 types of spatial targets. This segmentation transforms the sequence into a sequence with continuous spatiotemporal distribution characteristics.

[0084] Step 2: Since the transformer has a weak ability to extract local features of the target image, this invention adds a convolutional neural network in the complex domain to the front end of the transformer model to enhance the ability to extract local features. The input image sequence is a block of image sequence with size H1×H1 and H2×H2. Since the segmentation transforms the sequence into a sequence with spatiotemporal continuous distribution characteristics, the attention mechanism of the transformer module can extract the defocus information and spatial distribution information of the target. This information can facilitate the recognition of spatial targets.

[0085] A schematic diagram of the overall structure of complex domain networks is shown below. Figure 3 As shown, image sequence blocks of different sizes after segmentation are input into two complex domain convolutional modules of different scales for feature extraction, resulting in feature embeddings of the image sequences in the complex domain at the two scales. This feature embedding is then added to a positional encoding (a trainable vector, initialized before training; it can be initialized to 0, but after training, it can represent the position of the image block in the overall image; during testing, it is also initialized to a vector, which can be composed of 0s, but after previous training, the positional encoding becomes a representation of the position in the test image after passing through the trained transformer model). This encoding has a size of K×dim, where K is the number of small image blocks after segmentation, and dim is the dimension of the output of the convolutional module. This positional encoding represents the position of the segmented small image block in the original image, so its size corresponds to the number of image blocks. The summed feature vector F is then combined with the recognition encoding vector. This recognition encoding vector is trained together with the feature vector (the recognition encoding is a trainable vector; after being combined with the feature vector, it is trained through the network and ultimately represents the target category; it is not a label. During testing, it is also an initialized vector, which can consist of 0s, and is transformed into a numerical value representing the target category by the trained transformer model). This combination can represent the target classification. The combined output is then input into the complex domain CVMVT network for target recognition.

[0086] 2.1 Constructing a Layer normalization (LN) layer (CV_LN) for the complex field (CV)

[0087] Since the input sequence contains ISAR imaging results under continuous maneuvering of space targets, the defocus information of the images is also continuous. In order to better extract and forward the defocus information in the images to the network, it is necessary to maintain the consistent distribution within each sequence sample. Furthermore, the normalization process in the real domain ignores the correlation between the real and imaginary parts of the data, which has an adverse effect on the extraction of defocus information. Therefore, this invention innovatively proposes a complex-valued layer normalization layer, namely the CV_LN layer, which normalizes the image sequence in the sample dimension (the features become a 1×D vector). This layer can normalize the complex data to a normal complex distribution by calculating the covariance of the real and imaginary parts.

[0088] For the complex (CV) data of the input to the l-th layer of the network CV_LN can be expressed as formula (1):

[0089]

[0090] Where i represents the order of the feature maps, This is the output of this layer, where B represents the batch size and N represents the output of this layer. s is the number of samples, and D is the dimension of the one-dimensional vector obtained after feature extraction from the input complex data. It is a virtual part unit. and Let V represent the real and imaginary parts of the CV data, respectively. E(·) represents the mean of the data calculated across the sample dimension. Covariance matrix V l It can be represented as

[0091]

[0092] in

[0093]

[0094] and Alternatively, it can be calculated using the formula described above.

[0095]

[0096]

[0097] 2.2 Constructing a complex-domain transformer attention mechanism for sequences of two-dimensional images

[0098] This invention proposes a complex-domain transformer attention mechanism for sequences of two-dimensional images to fit complex-domain image data and better extract global information from the sequence. This includes spatial information between small image patches of different scales within the same ISAR defocused image, as well as implicit defocus information between image patches at the same location across frames of different ISAR defocused images—that is, continuous feature information in both the spatial and temporal domains (transformed to the frequency domain via Fast Fourier Transform). For the output of the CV_LN layer... First, a CV-Linear layer is used to add high-dimensional information to the component matrix of the transformer model, which can be represented as follows:

[0099]

[0100] Among them, CL N This refers to a linear activation layer in the complex field, i.e., a CV-Linear layer; L N N represents the linear activation layer in the real number field; N = Q, K, V, where Q, K, and V represent the query matrix, keyword matrix, and value matrix, respectively.

[0101] The attention weight is obtained by multiplying the query matrix and the keyword matrix to calculate relevance, which can be expressed as:

[0102]

[0103] Where CW represents the relevant weight.

[0104] Finally, by calculating the weighted sum of the value matrix and the relevant weights, the complex attention result is obtained, which can be calculated as follows:

[0105]

[0106] Among them, I Q (n), I K (n), I V (n) represents the attention results corresponding to the query matrix, keyword matrix, and value matrix in the real-valued (RV) domain, where n = a, b. This allows us to obtain better global features implicit in complex data.

[0107] 2.3 Constructing a multi-scale feature extraction network CVMVT

[0108] The CVMVT network comprises two processing paths, each designed to process image sequence blocks of different sizes. Each path includes a complex-domain convolutional module and a complex-domain transformer encoder module. Consequently, the two paths contain two complex-domain convolutional modules and two complex-domain transformer encoder modules with different parameters.

[0109] The first complex-domain convolutional module has two convolutional layers. Each convolutional layer is followed by a complex batch normalization layer (CV_BN2D, where batch normalization is performed on the batch dimension, a current technique), a complex max pooling layer, and a complex ReLU activation layer. The first convolutional layer has 16 kernels with a kernel size of 3×3 pixels and a stride of 1 pixel. The complex max pooling layer has a stride of 2 pixels. The second convolutional layer has 32 kernels with a kernel size of 3×3 pixels and a stride of 1 pixel. The complex max pooling layer also has a stride of 2 pixels. The output feature map is then expanded and the dimensions of the image patches are reconstructed. This is to separate the spatial and temporal features of the image sequence patches, allowing the attention mechanism to better extract the global features of the sequence data. Finally, a complex-domain fully connected layer is added, and the output dimension is set to 384.

[0110] The first complex-domain transformer encoder module contains two complex-domain layer normalization layers, one complex-domain attention mechanism layer, and one complex-domain feedforward layer. The complex-domain attention mechanism layer contains eight attention modules, each containing four real-domain fully connected layers. The first three real-domain fully connected layers extract features from the transformer model's query matrix, keyword matrix, and value matrix, respectively, with an output size of 512. The fourth fully connected layer outputs the features with a size of 384. The complex-domain feedforward layer contains two complex-domain fully connected layers and one complex ReLU activation layer. The output dimensions of the complex-domain fully connected layers are 512 and 384, respectively.

[0111] The second complex-domain convolutional module has two convolutional layers, each followed by a complex batch normalization layer, a complex max pooling layer, and a complex ReLU activation layer. The first convolutional layer has 16 kernels with a 3×3 pixel kernel size and a 1-pixel stride; the complex max pooling layer has a 2-pixel stride. The second convolutional layer has 32 kernels with a 3×3 pixel kernel size and a 1-pixel stride; the complex max pooling layer also has a 2-pixel stride. The output feature map is then expanded, and the dimensions of the image patches are reconstructed. Finally, a complex-domain fully connected layer is added, with the output dimension set to 192.

[0112] The second complex-domain transformer encoder module contains two complex-domain layer normalization layers, one complex-domain attention mechanism layer, and one complex-domain feedforward layer. The complex-domain attention mechanism layer contains eight attention modules, each containing four real-domain fully connected layers. The first three real-domain fully connected layers extract features from the transformer model's query matrix, keyword matrix, and value matrix, respectively, with an output size of 512. The fourth fully connected layer outputs the features with a size of 192. The complex-domain feedforward layer contains two complex-domain fully connected layers and one complex ReLU activation layer. The output dimensions of the complex-domain fully connected layers are 512 and 192, respectively.

[0113] In the complex domain transformer encoder module, the input and the complex domain attention mechanism layer are residually connected, and the output after the residual connection is then residually connected to the complex domain feedforward layer.

[0114] Finally, the outputs of the complex-domain transformer encoder modules at both scales are concatenated in the last dimension and passed through a fully connected complex-domain layer with an output dimension of 8 to obtain the output vector of the CVMVT model.

[0115] More specifically, the processing procedures for complex domain convolutional modules of different scales and the CVMVT network are as follows:

[0116] Add a dimension to the large-scale image sequence patch, with a size of 192×1×56×56 pixels; here, 192 is determined by 48*4, and 4 represents the number of training samples in each batch of 4;

[0117] Large-scale image sequence blocks are input into the first complex convolutional module and then passed to the first convolutional layer;

[0118] The first convolutional layer processes the input data and outputs two feature maps of size 192×16×54×54 pixels, one of which is the real part and the other is the imaginary part. The processing of the real part and the imaginary part is the same. After processing by the complex domain batch normalization layer (CV_BN2D), they are sent to the complex domain max pooling layer.

[0119] The complex domain max pooling layer outputs two feature maps of size 192×16×27×27 pixels, which are processed by the complex domain ReLU activation layer (CV_ReLU) and then fed into the second convolutional layer.

[0120] The second convolutional layer outputs two feature maps of size 192×32×25×25 pixels, which are processed by the CV_BN2D layer and then fed into the complex domain max pooling layer.

[0121] The complex domain max pooling layer outputs two feature maps of size 192×32×12×12 pixels, which are then flattened after being processed by the CV_ReLU layer.

[0122] After the dimensions are flattened, the feature map size is 192×4608. The dimensions of the feature map are then reconstructed.

[0123] The size of the feature map after reconstructing the feature map dimensions is 4×16×13824, and then it is fed into the complex domain fully connected layer.

[0124] The output size of the fully connected layer in the complex field is 4×16×384;

[0125] Add a dimension to the small-scale image sequence patch, with a size of 588×1×32×32 pixels;

[0126] Small-scale image sequence blocks are input into the first complex convolutional module and then passed to the first convolutional layer;

[0127] The first convolutional layer processes the input data and outputs two feature maps of size 588×16×29×29 pixels. After being processed by CV_BN2D, these feature maps are fed into the complex domain max pooling layer.

[0128] The complex domain max pooling layer outputs two feature maps of size 588×16×14×14 pixels, which are processed by CV_ReLU and then fed into the second convolutional layer.

[0129] The second convolutional layer outputs two feature maps of size 588×32×12×12 pixels, which are processed by the CV_BN2D layer and then fed into the complex domain max pooling layer.

[0130] The complex domain max pooling layer outputs two feature maps of size 588×32×6×6 pixels, which are then flattened after being processed by the CV_ReLU layer.

[0131] After the dimensions are flattened, the feature map size is 588×1152. The dimensions of the feature map are then reconstructed.

[0132] The feature map size after reconstructing the feature map dimensions is 4×49×3456, and then it is fed into the complex domain fully connected layer.

[0133] The output size of the fully connected layer in the complex field is 4×49×192;

[0134] The output of the large-scale complex convolution module is added to the position encoding, and its size is 4×16×384. After being combined with the recognition encoding vector, its size is 4×17×384. It is then fed into the first complex domain transformer encoder module, processed by CV_LN, and then fed into the complex domain attention mechanism module to obtain an output feature map of size 4×17×384. This feature map is then residually connected to the input to obtain an output feature map of size 4×17×384.

[0135] The output of the residual connection is fed into the CV_LN layer for processing and then into the complex domain feedforward layer to obtain an output feature map of size 4×17×384. It is then residually connected with the output of the previous residual connection to obtain an output feature map of size 4×17×384.

[0136] The output of the small-scale complex convolution module, after being added to the positional encoding, has a size of 4×49×192. After being combined with the recognition encoding vector, the size is 4×50×192. This is then fed into the first complex domain transformer encoder module, processed by CV_LN, and fed into the complex domain attention mechanism module to obtain an output feature map of size 4×50×192. This is then residually connected to the input to obtain an output feature map of size 4×50×192.

[0137] The output of the residual connection is fed into the CV_LN layer for processing and then into the complex domain feedforward layer to obtain an output feature map of size 4×50×192. It is then residually connected with the output of the previous residual connection to obtain an output feature map of size 4×50×192.

[0138] The recognition encoding vectors output by the complex domain transformer encoder modules at two scales are taken to obtain feature maps of sizes 4×384 and 4×192, respectively. They are concatenated in the last dimension to obtain a feature map of size 4×576. Then, a fully connected layer in the complex domain with an output dimension of size 8 is passed to obtain the output vector of the CVMVT model, which is 4×8.

[0139] Combination Figure 2 As shown, the target type estimation of the CVMVT network architecture includes two parts: training and testing. First, the training samples are input into the CVMVT architecture for training to obtain a training model for image sequence recognition; then, the defocused image sequences in the test dataset are input into the model to obtain the recognition results, and the method ends.

[0140] Combination Figure 4 This invention demonstrates the advantages of using sequential input and the transformer attention mechanism. Because spatial targets possess rich features, different individuals may exhibit similar structures under the same observation angle. For example... Figure 4The first frame images of the two sequences show that they belong to two different spatial targets but exhibit similar features. In this case, recognition based on a single frame image is not effective. In the last frame of the sequence, the two images show significant differences, indicating that inputting the sequence is more beneficial for recognition. Furthermore, the ISAR imaging mechanism suffers from scintillation, meaning that the intensity and location of scattering points in adjacent image frames can differ greatly. Therefore, the serial processing method of RNNs has limitations in ISAR image recognition. In contrast, the parallel processing approach of the transformer model can focus on the spatiotemporal information between all frames in the sequence, allowing for better extraction of spatial and defocus features from the image sequence.

[0141] Figure 5 shows the visualization results of attention from different models; where Figures 5(a)-5(c) These are the original ISAR images of different space targets; Figures 5(d)-5(f) This is a visualization of the attention of the CVMVT model; Figures 5(g)-5(i) This is a visualization of the attention result of the RVVIT model.

[0142] The above examples of the present invention are merely illustrative of the computational model and process of the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is impossible to exhaustively list all possible implementations here. Any obvious variations or modifications derived from the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A spatial target recognition method based on a complex-domain multi-scale visual transformer, characterized in that, Includes the following steps: ISAR motion space sample data is acquired. The ISAR motion space sample is a sequence of ISAR defocused images of a moving target. Every three consecutive image frames form an image sequence sample. The ISAR defocused image sequence is divided into image sequence blocks of different scales. The segmented image sequence blocks of different sizes are input into two complex-domain convolutional modules of different scales for feature extraction, yielding feature embeddings of the image sequences at the two scales in the complex domain. These feature embeddings are then added to the positional encodings to obtain the summed feature vector. The positional encoding represents the location of the segmented small image patch in the original image; feature vector This is then combined with the identification encoding vector, which represents the target's classification; The combined output is input into a complex domain CVMVT network for target recognition. The complex domain CVMVT network contains two processing paths, which are used to process the combined output corresponding to image sequence blocks of different sizes. Each processing path includes a complex domain convolution module and a complex domain transformer encoder module. The complex domain convolution module includes two convolutional layers, each followed by a complex batch normalization layer, a complex max pooling layer, and a complex ReLU activation layer; then the output feature map dimensions are expanded and the dimensions of the image patch are reconstructed, and finally a complex domain fully connected layer is added. The complex-domain transformer encoder module comprises two complex-domain layer normalization layers, one complex-domain attention mechanism layer, and one complex-domain feedforward layer. The complex-domain attention mechanism layer contains eight attention modules, each containing four real-domain fully connected layers. The first three real-domain fully connected layers extract features from the transformer model's query matrix, keyword matrix, and value matrix, respectively, while the fourth fully connected layer outputs these features. The complex-domain feedforward layer comprises two complex-domain fully connected layers and one complex ReLU activation layer. In the complex-domain transformer encoder module, the input is residually connected to the complex-domain attention mechanism layer, and the output after the residual connection is then residually connected to the complex-domain feedforward layer. The CVMVT network includes two complex-domain convolutional modules and two complex-domain transformer encoder modules with different parameters in its two processing paths. These include a first complex-domain convolutional module and a first complex-domain transformer encoder module, and a second complex-domain convolutional module and a first complex-domain transformer encoder module. Finally, the outputs of the two complex-domain transformer encoder modules are concatenated in the last dimension and passed through a fully connected complex-domain layer with an output dimension of 8 to obtain the output vector of the CVMVT model, thereby achieving spatial target recognition.

2. The spatial target recognition method based on a complex-domain multi-scale visual transformer according to claim 1, characterized in that, The processing steps of the complex field layer normalization layer include the following steps: For input complex CV data CV_LN is represented as: (1) in, Indicates the order of feature maps. It is the output of this layer; It is a virtual part unit. and These represent the real and imaginary parts of the CV data, respectively. This indicates that the average value of the data is calculated in the sample dimension; covariance matrix Represented as (2) in It is the dimension of the one-dimensional vector obtained by feature extraction from the input complex data.

3. The spatial target recognition method based on a complex-domain multi-scale visual transformer according to claim 2, characterized in that, The complex domain attention mechanism layer is a complex domain transformer attention mechanism for sequences of two-dimensional images, and its processing includes the following steps: For the output of the CV_LN layer , Used to represent First, a CV-Linear layer is used to add high-dimensional information to the component matrix of the transformer model, represented as follows: (4) in, It refers to the linear activation layer in the complex field, i.e., the CV-Linear layer; Represents a linear activation layer in the real number field; ,in , and These represent the query matrix, keyword matrix, and value matrix, respectively. The attention weight is obtained by multiplying the query matrix and the keyword matrix to calculate relevance, and is represented as follows: (5) in, Indicates the relevant weights; Finally, the complex attention result is obtained by calculating the weighted sum of the value matrix and the relevant weights, which is then calculated as follows: (6) in, This represents the attention results corresponding to the query matrix, keyword matrix, and value matrix in the real-valued (RV) domain. .

4. The spatial target recognition method based on a complex-domain multi-scale visual transformer according to claim 3, characterized in that, The location code size is ,in It is the number of small image patches after image segmentation. It is the dimension size of the output of the convolution module.

5. The spatial target recognition method based on a complex-domain multi-scale visual transformer according to claim 4, characterized in that, ISAR motion space sample data are obtained by simulation using a range-Doppler algorithm based on a three-dimensional spatial target model.

6. The spatial target recognition method based on a complex-domain multi-scale visual transformer according to claim 5, characterized in that, The ISAR defocused image sequence data size is 3×224×224. After image segmentation, the large-scale image block sequence data size is 48×56×56, and the small-scale image block sequence data size is 147×32×32.

7. The spatial target recognition method based on a complex-domain multi-scale visual transformer according to any one of claims 1 to 6, characterized in that, The specific structural parameters of the first complex field convolution module are as follows: The first convolutional layer has a kernel size of 3×3 pixels and a stride of 1 pixel, with a total of 16 kernels. The stride of the complex max pooling layer is 2 pixels. The second convolutional layer has 32 kernels with a kernel size of 3×3 pixels and a stride of 1 pixel. The stride of the complex max pooling layer is 2 pixels. The output dimension of the last fully connected layer in the complex field is 384.

8. The spatial target recognition method based on a complex-domain multi-scale visual transformer according to claim 7, characterized in that, The specific structural parameters of the first complex field transformer encoder module are as follows: In the complex field attention mechanism layer, the output size of the first three of the four real field fully connected layers in each module is 512, and the output size of the fourth fully connected layer is 384. The complex domain feedforward layer contains two fully connected complex domain layers with output dimensions of 512 and 384, respectively.

9. The spatial target recognition method based on a complex-domain multi-scale visual transformer according to claim 8, characterized in that, The specific structural parameters of the second complex field convolution module are as follows: The first convolutional layer has a kernel size of 3×3 pixels and a stride of 1 pixel, with a total of 16 kernels. The stride of the complex max pooling layer is 2 pixels. The second convolutional layer has 32 kernels with a kernel size of 3×3 pixels and a stride of 1 pixel. The stride of the complex max pooling layer is 2 pixels. The output dimension of the last fully connected layer in the complex field is set to 192.

10. The spatial target recognition method based on a complex-domain multi-scale visual transformer according to claim 9, characterized in that, The specific structural parameters of the second complex domain transformer encoder module are as follows: In the complex field attention mechanism layer, the output size of the first three of the four real field fully connected layers in each module is 512, and the output size of the fourth fully connected layer is 192. The complex domain feedforward layer contains two fully connected complex domain layers with output dimensions of 512 and 192, respectively.

Images