SAR target recognition method and device based on deep Grassmann manifold space

Abstract

The present application relates to a SAR target recognition method and device based on a deep Grassmann manifold space. The method uses a trained target recognition network to identify targets in an image. In the network, the target category in the image is predicted by extracting the deep features of the input image by a feature extractor and the projected distance between each subspace in the Grassmann manifold space. If the target in the image is of an unknown category, the network is trained online with small sample increments using the image, wherein the training is performed using a deep subspace preservation loss function, a deep subspace discrimination loss function, a semantic instance boundary loss function, and a semantic aggregation loss function. When using this method to identify targets in SAR images, it is possible to accurately identify targets of known categories while also using a small number of SAR images of targets of unknown categories for online learning of the target recognition network.

Description

Technical Field

[0001] The present application relates to the technical field of SAR target recognition, and in particular to a SAR target recognition method and device based on deep Grassmann manifold space. Background Art

[0002] SAR ATR (SAR Automatic Target Recognition), a long-standing and challenging problem in SAR image interpretation, has been the subject of extensive research over the past few decades. Advances in SAR imaging technology have led to an exponential growth in the number of high-resolution SAR images, spurring research on specific SAR ATR issues, such as small-sample learning and incremental learning.

[0003] In recent years, deep learning has achieved tremendous progress in many computer vision problems (such as detection, recognition, and segmentation) thanks to its powerful feature extraction and discrimination capabilities. Small-Sample Class Incremental (FSCIL) learning aims to update knowledge about new categories from a very small number of labeled samples without forgetting previously learned information, making it more relevant to real-world applications. In particular, due to the unique nature of SAR imaging, the number of examples of targets of interest is very limited under non-cooperative conditions. Furthermore, since the interpretation of targets in SAR images relies heavily on expert knowledge, obtaining large-scale labeled samples is expensive and impractical, significantly limiting the practical application of data-driven deep learning techniques. Therefore, research on deep learning-based SAR ATR algorithms capable of continuous learning from small samples is urgently needed. Summary of the Invention

[0004] Based on this, it is necessary to provide a SAR target recognition method and device based on deep Grassmann manifold space that can accurately identify base class targets while also performing small sample augmentation learning to address the above technical problems.

[0005] A SAR target recognition method based on deep Grassmann manifold space, the method comprising:

[0006] Acquire a SAR image to be identified;

[0007] Inputting the SAR image into a trained target recognition network, the target recognition network comprising a feature extractor and a Grassmann manifold space, wherein the Grassmann manifold space comprises a plurality of subspaces corresponding to different known target categories, the feature extractor extracting depth features of the input SAR image, and then predicting the target category in the SAR image based on the projected distance between the depth features and each subspace in the Grassmann manifold space;

[0008] If the target in the SAR image is of an unknown category, the SAR image is used as a small sample data to perform online iterative training on the target recognition network. In each iterative training, the subspace in the Grassmann manifold space is constrained by using a deep subspace preservation loss function and a deep subspace discrimination loss function, and the deep feature space is constrained by using a semantic instance boundary loss function and a semantic aggregation loss function. The parameters of the feature extractor are adjusted accordingly until all loss functions converge, thereby obtaining a target recognition network capable of recognizing the target of the unknown category.

[0009] If the target in the SAR image is of a known category, the prediction result of the target is output to achieve recognition of the target in the SAR image.

[0010] In one embodiment, after the target recognition network is trained online using SAR images with unknown target categories, the Grassmann manifold space in the target recognition network contains a subspace corresponding to the unknown category.

[0011] In one embodiment, the deep subspace preserving loss function is used to constrain the distance drift between a subspace in the current trainable Grassmann manifold space and the corresponding subspace in the Grassmann manifold space in the previous iterative training. The deep subspace preserving loss function is expressed as:

[0012] ;

[0013] In the above formula, Represents the category in the Grassmann manifold space i The dimension of the feature, Indicates the number of old categories in the current stage, , Respectively represent i The representation of categories in the history and current manifold space, subscript F Represents the matrix norm.

[0014] In one embodiment, the cosine similarity between two subspaces in the Grassmann manifold space is constrained using the deep subspace discrimination loss function, and the deep subspace discrimination loss function is expressed as:

[0015] ;

[0016] In the above formula, the first term is the classification loss based on cross entropy, and the second term is the constraint loss of subspace separation, where Represents the total number of samples in the current training stage, Represents the current training sample The category to which it belongs, For the The total number of category samples in the training phase, , Represents categories With category The subspace of Indicates the balance factor, subscript F Represents the matrix norm.

[0017] In one embodiment, a semantic instance boundary loss function is used to constrain the distance between feature points of unknown category objects and other known category objects in the deep feature space. The semantic instance boundary loss function is expressed as:

[0018] ;

[0019] In the above formula, The number of samples representing unknown category targets, Indicates the number of negative class samples participating in the loss, Represents the cosine distance between the anchor sample and the positive sample, Represents anchor samples and negative samples The cosine distance, Indicates the distance boundary.

[0020] In one embodiment, a semantic aggregation loss function is used to constrain the distance between target feature points of unknown categories in the deep feature space. The semantic aggregation loss function is expressed as:

[0021] ;

[0022] In the above formula, Indicates belonging to a category No. Samples are passed through the feature extractor The semantic features obtained after mapping, Measuring the same category Different samples of The cosine distance between them.

[0023] In one embodiment, the SAR target recognition method based on deep Grassmann manifold space further includes a training method for the target recognition network, and the training method includes:

[0024] Acquire a SAR training data set, wherein the SAR training data set includes a plurality of SAR training image sets of targets of known categories, and the training image set corresponding to each known category of target includes a plurality of SAR training images;

[0025] Constructing a base class training data set for a base class training phase and a plurality of small sample training data sets for a small sample incremental learning phase based on the SAR training data set;

[0026] In the base class training stage, the base class training data set is input into the target recognition network for iterative training, and the cosine cross entropy loss function is used to adjust the parameters of the feature extractor in the target recognition network in each iterative training until convergence, thereby obtaining a target recognition network capable of recognizing base class targets, and constructing multiple subspaces in the Grassmann manifold space corresponding to different categories of targets in the base class training data set;

[0027] In the small sample incremental learning stage, multiple small sample training data sets are sequentially input into the target recognition network capable of recognizing base class targets to train it;

[0028] Before using the first small sample training data set to train the target recognition network capable of identifying base class targets, the small sample training data set is merged with the base class training data set to obtain a historical training data set, and the historical training data set is input into the target recognition network capable of identifying base class targets until the target recognition network converges. The next small sample training data set is then merged with the historical training data set to obtain a current historical training data set, and the target recognition network is trained using the current historical training data set until all small sample training data sets have trained the target recognition network, thereby obtaining a trained target recognition network.

[0029] In one embodiment, during the small sample incremental learning stage, the deep subspace preservation loss function, the deep subspace discrimination loss function, the semantic instance boundary loss function and the semantic aggregation loss function are also used to train the object recognition network.

[0030] In one embodiment, constructing a plurality of subspaces in the Grassmann manifold space corresponding to different categories of targets in the base class training dataset comprises:

[0031] In the base class training stage, the feature extractor is used to project target images of each known category in the base class training data set into a deep semantic space to extract deep features of each image;

[0032] The deep features of each known category target are divided into matrix to construct a corresponding subspace in the Grassmann manifold space.

[0033] The present application also provides a SAR target recognition device based on a deep Grassmann manifold space, the device comprising:

[0034] SAR image acquisition module, used to acquire SAR images to be identified;

[0035] a target category prediction module, configured to input the SAR image into a trained target recognition network, the target recognition network comprising a feature extractor and a Grassmann manifold space, the Grassmann manifold space comprising a plurality of subspaces corresponding to different known target categories, the feature extractor extracting depth features of the input SAR image and then predicting the target category in the SAR image based on the projected distance between the depth features and each subspace in the Grassmann manifold space;

[0036] A small sample incremental online training module is configured to, if the target in the SAR image is of an unknown category, use the SAR image as small sample data to perform online iterative training on the target recognition network. In each iterative training, a deep subspace preservation loss function and a deep subspace discrimination loss function are used to constrain the subspace in the Grassmann manifold space, a semantic instance boundary loss function and a semantic aggregation loss function are used to constrain the deep feature space, and the parameters of the feature extractor are adjusted accordingly until all loss functions converge, thereby obtaining a target recognition network capable of recognizing targets of the unknown category.

[0037] The target recognition result acquisition module is used to output the target prediction result if the target in the SAR image is of a known category, so as to realize the recognition of the target in the SAR image.

[0038] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0039] Acquire a SAR image to be identified;

[0040] Inputting the SAR image into a trained target recognition network, the target recognition network comprising a feature extractor and a Grassmann manifold space, wherein the Grassmann manifold space comprises a plurality of subspaces corresponding to different known target categories, the feature extractor extracting depth features of the input SAR image, and then predicting the target category in the SAR image based on the projected distance between the depth features and each subspace in the Grassmann manifold space;

[0041] If the target in the SAR image is of an unknown category, the SAR image is used as a small sample data to perform online iterative training on the target recognition network. In each iterative training, the subspace in the Grassmann manifold space is constrained by using a deep subspace preservation loss function and a deep subspace discrimination loss function, and the deep feature space is constrained by using a semantic instance boundary loss function and a semantic aggregation loss function. The parameters of the feature extractor are adjusted accordingly until all loss functions converge, thereby obtaining a target recognition network capable of recognizing the target of the unknown category.

[0042] If the target in the SAR image is of a known category, the prediction result of the target is output to achieve recognition of the target in the SAR image.

[0043] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0044] Acquire a SAR image to be identified;

[0045] Inputting the SAR image into a trained target recognition network, the target recognition network comprising a feature extractor and a Grassmann manifold space, wherein the Grassmann manifold space comprises a plurality of subspaces corresponding to different known target categories, the feature extractor extracting depth features of the input SAR image, and then predicting the target category in the SAR image based on the projected distance between the depth features and each subspace in the Grassmann manifold space;

[0046] If the target in the SAR image is of an unknown category, the SAR image is used as a small sample data to perform online iterative training on the target recognition network. In each iterative training, the subspace in the Grassmann manifold space is constrained by using a deep subspace preservation loss function and a deep subspace discrimination loss function, and the deep feature space is constrained by using a semantic instance boundary loss function and a semantic aggregation loss function. The parameters of the feature extractor are adjusted accordingly until all loss functions converge, thereby obtaining a target recognition network capable of recognizing the target of the unknown category.

[0047] If the target in the SAR image is of a known category, the prediction result of the target is output to achieve recognition of the target in the SAR image.

[0048] The above-mentioned SAR target recognition method and device based on deep Grassmann manifold space recognizes targets in SAR images using a trained target recognition network. In this network, the target category in the SAR image is predicted by the projected distance between the deep features of the input SAR image extracted by the feature extractor and each subspace in the Grassmann manifold space. If the target category in the image is unknown, the network is trained online with small sample increments using the SAR image. The deep subspace preservation loss function and the deep subspace discrimination loss function are used to constrain the Grassmann manifold space and adjust the parameters accordingly. The semantic instance boundary loss function and the semantic aggregation loss function are used to constrain the deep feature space and adjust the parameters in the feature extractor until the loss functions converge, completing the online training. When using this method to recognize targets in SAR images, it can not only accurately recognize targets of known categories, but also use a small number of SAR images of targets of unknown categories for online learning of the target recognition network. By using the four loss functions proposed in this method, the target recognition network has the ability to convert unknown categories into known categories when trained with a small number of unknown samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 1 is a flow chart of a SAR target recognition method based on a deep Grassmann manifold space in one embodiment;

[0050] Figure 2 Schematic diagram of a flow chart of a method for training a target recognition network in one embodiment;

[0051] Figure 3 A schematic diagram of a training framework for a target recognition network according to an embodiment;

[0052] Figure 4 A schematic diagram of constructing subspaces in a Grassmann manifold space in one embodiment;

[0053] Figure 5 A schematic diagram of a target recognition network testing phase according to an embodiment;

[0054] Figure 6 A schematic diagram of a depth subspace preservation loss function in one embodiment;

[0055] Figure 7 Schematic diagram of a deep subspace discrimination loss function in one embodiment;

[0056] Figure 8 Schematic diagram of a semantic instance boundary loss function in one embodiment;

[0057] Figure 9 Schematic diagram of a semantic aggregation loss function in one embodiment;

[0058] Figure 10 1 is a structural block diagram of a SAR target recognition device based on deep Grassmann manifold space in one embodiment;

[0059] Figure 11 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0061] In view of the urgent problem in the prior art, a deep learning-based SAR automatic target recognition algorithm with small sample continuous learning capability is studied. In one embodiment, Figure 1 As shown, a SAR target recognition method based on deep Grassmann manifold space is provided, comprising the following steps:

[0062] Step S100, obtaining a SAR image to be identified;

[0063] Step S110: Inputting the SAR image into a trained target recognition network, the target recognition network including a feature extractor and a Grassmann manifold space, wherein the Grassmann manifold space includes multiple subspaces corresponding to different known target categories. The feature extractor extracts depth features of the input SAR image and then predicts the target category in the SAR image based on the projected distance between the depth features and each subspace in the Grassmann manifold space.

[0064] Step S120: If the target in the SAR image is of an unknown category, the SAR image is used as a small sample data to perform online iterative training on the target recognition network. In each iterative training, the deep subspace preservation loss function and the deep subspace discrimination loss function are used to constrain the Grassmann manifold space and the parameters of the Grassmann manifold space are adjusted accordingly. The semantic instance boundary loss function and the semantic aggregation loss function are used to constrain the deep feature space and the parameters of the feature extractor are adjusted accordingly until all loss functions converge. Thus, a target recognition network capable of recognizing the target of the unknown category is obtained.

[0065] Step S130: If the target in the SAR image is of a known category, the prediction result of the target is output to realize recognition of the target in the SAR image.

[0066] In practice, deep learning models capable of continuous learning and target recognition with small samples must achieve a balance between stability and plasticity. Stability refers to the model's ability to preserve old knowledge, while plasticity refers to its ability to continuously learn new knowledge. Furthermore, due to the unique nature of SAR imaging and the similarities between target configurations and components, targets in SAR images contain limited information, exhibiting significant intra-class differences and similar inter-class features, and exhibiting significant background interference. To address these issues, this method proposes a learning framework based on Deep Grassmann Subspace Preservation and Separation (DGSPS).

[0067] Compared to the classic classification method that uses the point-to-point distance between the test sample and the class features as a metric, this method models the features of different target categories as multiple separable subspaces on the Grassmann manifold, ensuring the model's robust representation of existing knowledge and rapid adaptation to new class features. When using the model derived from this framework for SAR target recognition, the classification result is obtained by calculating the minimum distance between the original image features and their projections in the class subspace.

[0068] In SAR images, target features are characterized by a diverse distribution of backscatter points, exhibiting azimuth-dependence and component sparsity. Unlike optical images, which offer rich representations of target discriminative features, target information in SAR images is often sparse, unstable, and easily affected by the surrounding environment due to the specific nature of SAR imaging mechanisms and the diversity and complexity of target poses. Therefore, this method accurately identifies targets by constructing a feature space with high inter-class separation.

[0069] Compared to classifying and recognizing objects in the original feature space, representing classes as subspaces and classifying and recognizing objects based on subspaces offers many significant advantages, including compactness, scalability, and discriminability. Compactness is manifested in that subspaces can express class characteristics more compactly and comprehensively than single feature points, thereby improving classification accuracy. Compared to a single class vector (e.g., prototypes or fully connected (FC) nodes), subspaces consist of a collection of features representing representative examples of that class. These features can be obtained through feature decomposition, resulting in more accurate high-dimensional information about the class. Scalability is manifested in that when a new class arrives, knowledge about the new class can be updated by directly appending its subspace to the current subspace without changing the structure of the current subspace. Discriminability is manifested in that the loss function constructed based on the subspace provides a direct optimization signal to the feature extractor, avoiding the proxy task of a fully connected classifier. Specifically, by explicitly constraining the drift of the subspace of the same class across sessions and amplifying the differences between subspaces with different labels, the subspaces can preserve old knowledge and rapidly represent new knowledge.

[0070] The above steps S100-S130 describe the process of identifying targets in SAR images using a target recognition network trained based on the DGSPS framework proposed in this application. Specifically, this process involves accurately identifying known target images using the target recognition network, and performing small-sample incremental online training of the target recognition network using unknown target images. After online training, the target recognition model is capable of identifying targets of unknown categories. Enabling the target recognition network to simultaneously identify and learn with small samples online is inseparable from the specific framework of the target recognition model and the four loss functions used during online training.

[0071] In this embodiment, four losses—Deep Subspace Preservation (DSP), Deep Subspace Discriminative (DSD), Semantic Instance Boundary (SIM), and Semantic Aggregation (SA)—are applied during both online and pre-training of the object recognition network to enhance the model's ability to retain important knowledge about old categories (i.e., retain the ability to recognize objects of known categories) and adapt to new categories (i.e., enable online training using unknown small samples). The DSP loss aims to suppress the drift of features from old categories by minimizing the distance between the data points projected on the Grassmann manifold by the model in the previous training iteration and the model in the current training iteration. DSD aims to increase the variance between different categories on the Grassmann manifold. SIM is an improved boundary-constrained loss that expands the decision boundary between different categories based on the cosine distance between samples from the old and new classes. SA is a class-aware distance loss that aims to make the semantic features of samples with the same label more compact.

[0072] In this embodiment, after the target recognition network is trained online using SAR images of unknown target categories, the Grassmann manifold space in the target recognition network contains a subspace corresponding to the unknown category. In other words, the unknown category targets are converted into known category targets that can be recognized by the target recognition network.

[0073] In this embodiment, the Few-Shot Class-Incremental Learning (FSCIL) approach is used to train the target class network. The FSCIL learning approach typically consists of a base class learning phase and multiple incremental learning phases, or sessions. In a base session, each class contains a large number of training samples, which can be viewed as knowledge collected in a collaborative environment. In an incremental session, new class samples appear continuously in a non-cooperative setting and contain only a small number of samples. During this phase, the model should adapt quickly by using samples from the new class as well as some examples from the old classes. During testing at the end of each session, the model should be able to recognize all known classes.

[0074] Furthermore, the small sample incremental learning problem can be formulated as follows. Assume that there is a series of labeled datasets ,in Include For samples and the corresponding labels .for The first dataset Represents basic conversation data and provides large-scale basic class training samples. It is incremental session data, containing finite samples of some new categories. Stage, Model Based on the current dataset and some old class samples retained from previous sessions (i.e. ). During the testing phase, the model needs to be able to distinguish the categories learned in all sessions.

[0075] In this embodiment, a training method for a target recognition network is also provided. Figure 2 Shown, including:

[0076] Step S200, obtaining a SAR training data set, wherein the SAR training data set includes a plurality of SAR training image sets of targets of known categories, and the training image set corresponding to each known category of target includes a plurality of SAR training images;

[0077] Step S210, constructing a base class training data set for the base class training phase and a plurality of small sample training data sets for the small sample incremental learning phase based on the SAR training data set;

[0078] Step S220: In the base class training phase, the base class training data set is input into the target recognition network for iterative training, and the cosine cross entropy loss function is used to adjust the parameters of the feature extractor in the target recognition network in each iterative training until convergence, thereby obtaining a target recognition network capable of recognizing base class targets, and constructing multiple subspaces in the Grassmann manifold space corresponding to different categories of targets in the base class training data set;

[0079] Step S230, in the small sample incremental learning phase, multiple small sample training data sets are sequentially input into an object recognition network capable of recognizing base class objects to train it;

[0080] In step S240, before using the first small sample training data set to train the target recognition network capable of recognizing base class targets, the small sample training data set is merged with the base class training data set to obtain a historical training data set, and the historical training data set is input into the target recognition network capable of recognizing base class targets until the target recognition network converges. The next small sample training data set is then merged with the historical training data set to obtain the current historical training data set, and the target recognition network is trained using the current historical training data set until the target recognition network is trained with all small sample training data sets, thereby obtaining a trained target recognition network.

[0081] In this embodiment, the training framework of the base class training phase and the small sample incremental learning phase is also as follows: Figure 3 shown.

[0082] In step S200 and step S210, a large number of labeled data sets are first prepared. ,in Include For samples and the corresponding labels . Each data set are sample data of different target categories. If the condition is always true, the first data set Represents basic conversation data, providing large-scale basic class training samples, which include multiple target categories, and each category includes multiple sample images. It is incremental session data, containing some small samples of new categories, that is, N categories, K data for each category, and the values of N and K are very small.

[0083] In the base class training phase, a parameterized feature extractor is first trained , so that it can accurately distinguish the base category targets and have the ability to generalize the features of unknown new categories. This can be obtained by constructing a proxy classifier and calculating the cosine cross entropy loss. Based on this feature extractor, the base class is constructed in the category subspace of the Grassmann manifold. .

[0084] Compared with traditional classifiers with point-to-point criteria, subspace classifiers can comprehensively represent the characteristics of classes. Considering the effective discrimination ability of targets and robustness to outliers based on the point-to-space metric, a class-specific subspace is constructed on the Grassmann manifold.

[0085] In this embodiment, multiple subspaces corresponding to different categories of targets in the base class training data set are constructed in the Grassmann manifold space, such as Figure 4As shown, it includes: in the base class training stage, through the feature extractor The target images of each known category in the base class training dataset are projected into the deep semantic space, the deep features of each image are extracted, and the corresponding subspace of the deep features of each known category target is constructed in the Grassmann manifold space using matrix division.

[0086] Specifically, first use the training Extract the high-dimensional feature set of the category sample, which can be expressed as , and represent them as matrices ,in and Refers to the number of instances of the same class and the dimension of the sample features. In order to meet the Grassmann geometry constraints, the singular value decomposition (SVD) technique is used to convert Decompose into Thus, the category On a Grassmann manifold The subspace of Specific columns of .

[0087] Specifically, the base class sample set For example, the class instance samples are first trained Map it to the feature space to obtain its high-dimensional feature vector. For each category set feature, the category-related feature vector is obtained through matrix decomposition (such as QR, SVD decomposition), and this vector is used as the discriminant feature of the category. At the same time, based on the herding strategy proposed by the iCaRL algorithm, the representative examples (Exemplar) of the old category are saved. , to provide necessary supervision signals for subsequent models.

[0088] In the small sample incremental learning phase, in the incremental training phase ( ), taking the second session (t=2) as an example, first the new class data and sample set Combined to obtain a sample set for training in the current stage (that is, the historical training data set in step S240). In order to alleviate the model stability-plasticity problem, four loss functions, DSR, DSP, SIM and SA, are designed in this learning stage, so that the old category features are retained in the Grassmann manifold space and the semantic space, and the new category features are adaptively learned. Among them, DSR and DSP act on the Grassmann manifold space, and SIM and SA losses act on the original feature space. Among them, DSP loss aims to suppress the drift of old category features by minimizing the distance between the data points projected by the previous model and the current model on the Grassmann manifold. DSD aims to expand the differences between different categories on the Grassmann manifold. SIM is an improved boundary constraint loss that expands the decision boundaries of different categories based on the cosine distance of samples of the new and old classes. SA is a class-aware distance loss that aims to make the semantic features of samples with the same label more compact. After the learning is completed, the subspace of the new class is constructed with the features of the new class.

[0089] By iteratively repeating the above process until all categories are learned, a feature extractor with the ability to recognize all categories is obtained. and lies in the category subspace of the Grassmann manifold.

[0090] After the target recognition network is trained using the above training method, it needs to be tested. In the testing phase, if Figure 5 As shown, the embedded target recognition network constructed based on The prediction of unknown SAR image target samples can be regarded as finding the high-dimensional features of the sample and its projection measurement on different class subspaces, and selecting the shortest distance as the category prediction result of the sample.

[0091] Specifically, the Grassmann-based classification can be shown as the following formula (1), which obtains the test sample category by finding the shortest distance between the original embedded feature and its reconstructed data point in a specific subspace. Represents high-dimensional features, Here it is expressed as a class subspace. Further, the distance can be expressed as formula (2), where is the transfer matrix. Finally, the sample Belong to category The category classification result can be expressed as formula (3).

[0092] (1)

[0093] (2)

[0094] (3)

[0095] Next, the four loss functions in this method are explained.

[0096] In this embodiment, a deep subspace preserving loss function is used to constrain the distance drift between a subspace in the current trainable Grassmann manifold space and the corresponding subspace in the Grassmann manifold space in the previous iterative training.

[0097] In order to prevent the classification performance of the object recognition network for old classes from dropping sharply in each incremental session (i.e., small-sample incremental online learning), the structure and position of the previously constructed subspace must be stabilized. To this end, the Deep Subspace Preservation (DSP) loss is proposed.

[0098] like Figure 6 shown ， is the class subspace of the historical model projection, is the same type of subspace that the current trainable model projects. Therefore, overcoming the catastrophic forgetting of old knowledge requires minimizing the and Based on the projection metric theory of Grassmann manifold, formula (4) calculates the distance between two subspaces on the Grassmann manifold. Finally, the loss of DSP can be expressed as formula (5).

[0099] (4)

[0100] Furthermore, the depth subspace preservation loss function is expressed as:

[0101] (5)

[0102] In formula (5), using Represents the category in the Grassmann manifold space i The dimension of the feature, Indicates the number of old categories in the current stage, , Respectively represent i The representation of categories in the history and current manifold space, subscript F Representation matrix norm deep subspace discrimination loss function constrains the cosine similarity between two subspaces in the Grassmann manifold space.

[0103] According to the definition of the principle angle of the Grassmann manifold space, the principle angle based on cosine It is calculated based on the nearest normalized similarity of two basis vectors from different subspaces, which implicitly expresses the correlation between the key features of heterogeneous subspaces. The smaller the value, the more obvious the difference between targets of different categories. Therefore, by reducing the cosine similarity between the bases corresponding to heterogeneous subspaces, the separation of heterogeneous classes can be improved, thereby improving the model's ability to discriminate between categories. Therefore, in order to accurately distinguish different categories in the incremental learning process, a deep subspace discrimination (DSD) loss is proposed, which combines the regularization constraint of subspace orthogonality and classification based on constructed subspaces, such as Figure 7 shown.

[0104] In incremental training, the DSD loss can be expressed as formula (6), where It's a conversation The number of test samples used for training. , It is from The class index selected in . Hyperparameters Used to balance the contributions of the two loss terms.

[0105] Furthermore, the deep subspace discrimination loss function is expressed as:

[0106] (6)

[0107] In formula (6), the first term is the classification loss based on cross entropy, and the second term is the constraint loss of subspace separation, where Represents the total number of samples in the current training stage, Represents the current training sample The category to which it belongs, For the The total number of category samples in the training phase, Represents categories With category The subspace of Indicates the balance factor, subscript F Represents the matrix norm.

[0108] In this embodiment, a semantic instance boundary loss function is used to constrain the distance between feature points of unknown category targets and other known category targets in the deep feature space.

[0109] In addition to directly using the DSD loss to make different subspaces more orthogonal on the Grassmann manifold, the clear edges between the semantic features of samples between categories of objects should also be maintained to further improve the separability of inter-class subspaces. To this end, the Semantic Instance Margin (SIM) loss is introduced, which takes into account the distance between sample-level semantic features.

[0110] like Figure 8 As shown in the figure, among the sample feature points of the same category as the anchor sample, the anchor point that can be compared with the anchor sample can be regarded as a positive sample. The starting point of the three arrows in the figure is the anchor sample, and the origin pointed by the arrow is the positive sample. The farthest distance between the anchor sample and the positive sample For comparison, the distance between negative samples and anchor samples is less than , the distance is greater than The positive examples do not contribute to the SIM loss.

[0111] The SIM loss can be expressed as formula (7), where and They refer to the number of samples of unknown categories and the number of negative category samples participating in the loss, respectively. Represents the distance between the anchor sample and the positive sample, Represents anchor point samples and a negative example The cosine distance between them. control With the previous The distance between the negative pairs of other distances is bounded.

[0112] Furthermore, the semantic instance boundary loss function is expressed as:

[0113] (7)

[0114] In this embodiment, a semantic aggregation loss function is used to constrain the distance between target feature points of unknown categories in the deep feature space.

[0115] Due to the diversity of backscattering features of targets in SAR images, it is necessary to improve the compactness of the class subspace to accurately represent the target information. To solve this problem, the semantic aggregation (SA) loss acting on the deep semantic feature space is proposed, such as Figure 9 As shown in the figure, solid points of the same color represent the normalized embedding features of instances with the same label. represents the pairwise cosine distance between instances of the same class. It can be formulated as (8).

[0116] Furthermore, the semantic aggregation loss function is expressed as:

[0117] (8)

[0118] In formula (8), and Is selected from the new class training set Test samples Here is the normalized embedding feature of of of norm, Measures the cosine similarity between two normalized features.

[0119] The aforementioned SAR target recognition method based on the deep Grassmann manifold space represents the knowledge of different target categories as a semantic subspace on the Grassmann manifold, effectively preserving the current category information and scalable to unknown categories. To address the challenges of similarity between target feature categories, significant intra-class differences, and severe background interference in SAR images, the authors proposed deep subspace preservation (DSP), deep subspace discrimination (DSD), semantic instance boundary (SIM), and semantic aggregation (SA) losses to mitigate the drift of old knowledge feature space and enhance the differentiation of information between different categories in the Grassmann manifold and semantic space. Experiments on three SAR ATR datasets (MSTAR, SARSIM, and SAMPLE) demonstrate that the proposed DGSPS framework outperforms methods for classical classification, incremental learning (IL), and few-shot class incremental learning (FSCIL) tasks. Furthermore, ablation experiments demonstrate the effectiveness of the designed loss function.

[0120] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0121] In one embodiment, Figure 10 As shown, a SAR target recognition device based on a deep Grassmann manifold space is provided, comprising: a SAR image acquisition module 300, a target category prediction module 310, a small sample incremental online training module 320, and a target recognition result acquisition module 330, wherein:

[0122] SAR image acquisition module 300, used to acquire SAR images to be identified;

[0123] a target category prediction module 310 configured to input the SAR image into a trained target recognition network, wherein the target recognition network includes a feature extractor and a Grassmann manifold space, wherein the Grassmann manifold space includes multiple subspaces corresponding to different known target categories; the feature extractor extracts depth features of the input SAR image and then predicts the target category in the SAR image based on the projected distance between the depth features and each subspace in the Grassmann manifold space;

[0124] A small sample incremental online training module 320 is configured to perform online iterative training on the target recognition network using the SAR image as small sample data if the target in the SAR image is of an unknown category. In each iterative training, a deep subspace preservation loss function and a deep subspace discrimination loss function are used to constrain the subspace in the Grassmann manifold space, and a semantic instance boundary loss function and a semantic aggregation loss function are used to constrain the deep feature space. Parameters of the feature extractor are adjusted accordingly until all loss functions converge, thereby obtaining a target recognition network capable of recognizing targets of the unknown category.

[0125] The target recognition result acquisition module 330 is configured to output a prediction result of the target if the target in the SAR image is of a known category, so as to realize recognition of the target in the SAR image.

[0126] The specific limitations of the SAR target recognition device based on the deep Grassmann manifold space can be found in the limitations of the SAR target recognition method based on the deep Grassmann manifold space above and will not be repeated here. Each module in the above-mentioned SAR target recognition device based on the deep Grassmann manifold space can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0127] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 11As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a SAR target recognition method based on deep Grassmann manifold space is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0128] Those skilled in the art will understand that Figure 11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0129] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0130] Acquire a SAR image to be identified;

[0131] Inputting the SAR image into a trained target recognition network, the target recognition network comprising a feature extractor and a Grassmann manifold space, wherein the Grassmann manifold space comprises a plurality of subspaces corresponding to different known target categories, the feature extractor extracting depth features of the input SAR image, and then predicting the target category in the SAR image based on the projected distance between the depth features and each subspace in the Grassmann manifold space;

[0132] If the target in the SAR image is of an unknown category, the SAR image is used as a small sample data to perform online iterative training on the target recognition network. In each iterative training, the subspace in the Grassmann manifold space is constrained by using a deep subspace preservation loss function and a deep subspace discrimination loss function, and the deep feature space is constrained by using a semantic instance boundary loss function and a semantic aggregation loss function. The parameters of the feature extractor are adjusted accordingly until all loss functions converge, thereby obtaining a target recognition network capable of recognizing the target of the unknown category.

[0133] If the target in the SAR image is of a known category, the prediction result of the target is output to achieve recognition of the target in the SAR image.

[0134] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0135] Acquire a SAR image to be identified;

[0136] Inputting the SAR image into a trained target recognition network, the target recognition network comprising a feature extractor and a Grassmann manifold space, wherein the Grassmann manifold space comprises a plurality of subspaces corresponding to different known target categories, the feature extractor extracting depth features of the input SAR image, and then predicting the target category in the SAR image based on the projected distance between the depth features and each subspace in the Grassmann manifold space;

[0137] If the target in the SAR image is of an unknown category, the SAR image is used as a small sample data to perform online iterative training on the target recognition network. In each iterative training, the subspace in the Grassmann manifold space is constrained by using a deep subspace preservation loss function and a deep subspace discrimination loss function, and the deep feature space is constrained by using a semantic instance boundary loss function and a semantic aggregation loss function. The parameters of the feature extractor are adjusted accordingly until all loss functions converge, thereby obtaining a target recognition network capable of recognizing the target of the unknown category.

[0138] If the target in the SAR image is of a known category, the prediction result of the target is output to achieve recognition of the target in the SAR image.

[0139] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0140] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0141] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A SAR target recognition method based on deep Grassmann manifold space, characterized in that: The method comprises: Acquire a SAR image to be identified; Inputting the SAR image into a trained target recognition network, the target recognition network comprising a feature extractor and a Grassmann manifold space, wherein the Grassmann manifold space comprises a plurality of subspaces corresponding to different known target categories, the feature extractor extracting depth features of the input SAR image, and then predicting the target category in the SAR image based on the projected distance between the depth features and each subspace in the Grassmann manifold space; If the target in the SAR image is of an unknown category, the SAR image is used as a small sample data to perform online iterative training on the target recognition network. In each iterative training, the subspace in the Grassmann manifold space is constrained by using a deep subspace preservation loss function and a deep subspace discrimination loss function. The distance between the feature points of the unknown category target and other known category targets in the deep feature space is constrained by using a semantic instance boundary loss function. At the same time, the distance between the feature points of the unknown category target in the deep feature space is constrained by using a semantic aggregation loss function. The parameters of the feature extractor are adjusted accordingly until all loss functions converge. Then, a target recognition network with the ability to recognize the unknown category target is obtained. The deep subspace preservation loss function is used to constrain the distance drift between a subspace in the current trainable Grassmann manifold space and the corresponding subspace in the Grassmann manifold space in the previous iterative training. The deep subspace preservation loss function is expressed as: In the above formula, Represents the category in the Grassmann manifold space The dimension of the feature, Indicates the number of old categories in the current stage, , Respectively represent The representation of categories in the history and current manifold space, subscript F represents the matrix norm. At the same time, the cosine similarity between the two subspaces in the Grassmann manifold space is constrained by the deep subspace discrimination loss function. The deep subspace discrimination loss function is expressed as: In the above formula, the first term is the classification loss based on cross entropy, and the second term is the constraint loss of subspace separation, where Represents the total number of samples in the current training stage, Represents the current training sample The category to which it belongs, For the The total number of category samples in the training phase, Represents categories With category The subspace of Indicates the balance factor, subscript F represents the matrix norm; If the target in the SAR image is of a known category, the prediction result of the target is output to achieve recognition of the target in the SAR image.

2. The SAR target recognition method according to claim 1, characterized in that: After the target recognition network is trained online using SAR images with unknown target categories, the Grassmann manifold space in the target recognition network contains a subspace corresponding to the unknown category.

3. The SAR target recognition method according to claim 1, wherein: The semantic instance boundary loss function is expressed as: In the above formula, The number of samples representing unknown category targets, Indicates the number of negative class samples participating in the loss, Represents the cosine distance between the anchor sample and the positive sample, Represents anchor samples and negative samples The cosine distance, Indicates the distance boundary.

4. The SAR target recognition method according to claim 1, wherein: The semantic aggregation loss function is expressed as: In the above formula, Indicates belonging to a category No. The semantic features obtained after the samples are mapped by the feature extractor, Measuring the same category Different samples of The cosine distance between them.

5. The SAR target recognition method according to any one of claims 1 to 4, characterized in that: The SAR target recognition method based on deep Grassmann manifold space also includes a training method for the target recognition network, and the training method includes: Acquire a SAR training data set, wherein the SAR training data set includes a plurality of SAR training image sets of targets of known categories, and the training image set corresponding to each known category of target includes a plurality of SAR training images; Constructing a base class training data set for a base class training phase and a plurality of small sample training data sets for a small sample incremental learning phase based on the SAR training data set; In the base class training stage, the base class training data set is input into the target recognition network for iterative training, and the cosine cross entropy loss function is used to adjust the parameters of the feature extractor in the target recognition network in each iterative training until convergence, thereby obtaining a target recognition network capable of recognizing base class targets, and constructing multiple subspaces in the Grassmann manifold space corresponding to different categories of targets in the base class training data set; In the small sample incremental learning stage, multiple small sample training data sets are sequentially input into the target recognition network capable of recognizing base class targets to train it; Before using the first small sample training data set to train the target recognition network capable of identifying base class targets, the small sample training data set is merged with the base class training data set to obtain a historical training data set, and the historical training data set is input into the target recognition network capable of identifying base class targets until the target recognition network converges. The next small sample training data set is then merged with the historical training data set to obtain a current historical training data set, and the target recognition network is trained using the current historical training data set until all small sample training data sets have trained the target recognition network, thereby obtaining a trained target recognition network.

6. The SAR target recognition method according to claim 5, characterized in that: In the small sample incremental learning stage, the deep subspace preservation loss function, the deep subspace discrimination loss function, the semantic instance boundary loss function and the semantic aggregation loss function are also used to train the object recognition network.

7. The SAR target recognition method according to claim 6, characterized in that: The step of constructing a plurality of subspaces in the Grassmann manifold space corresponding to different categories of targets in the base class training data set includes: In the base class training stage, the feature extractor is used to project each known category target image in the base class training data set into the deep semantic space to extract the deep features of each image; The deep features of each known category target are divided into matrix to construct a corresponding subspace in the Grassmann manifold space.

8. A SAR target recognition device based on deep Grassmann manifold space, characterized in that: The SAR target recognition method based on deep Grassmann manifold space described in claim 1 is implemented in the device, and the device comprises: SAR image acquisition module, used to acquire SAR images to be identified; a target category prediction module, configured to input the SAR image into a trained target recognition network, the target recognition network comprising a feature extractor and a Grassmann manifold space, the Grassmann manifold space comprising a plurality of subspaces corresponding to different known target categories, the feature extractor extracting depth features of the input SAR image and then predicting the target category in the SAR image based on the projected distance between the depth features and each subspace in the Grassmann manifold space; A small sample incremental online training module is configured to, if the target in the SAR image is of an unknown category, use the SAR image as small sample data to perform online iterative training on the target recognition network. In each iterative training, a deep subspace preservation loss function and a deep subspace discrimination loss function are used to constrain the subspace in the Grassmann manifold space, a semantic instance boundary loss function and a semantic aggregation loss function are used to constrain the deep feature space, and the parameters of the feature extractor are adjusted accordingly until all loss functions converge, thereby obtaining a target recognition network capable of recognizing targets of the unknown category. The target recognition result acquisition module is used to output the target prediction result if the target in the SAR image is of a known category, so as to realize the recognition of the target in the SAR image.

Images