SAR (Synthetic Aperture Radar) image classification method and device, equipment and storage medium

Abstract

The invention discloses an SAR image classification method and device, equipment and a storage medium. Firstly, an SAR data set to be classified is obtained, features of the SAR data set are extracted, an embedded feature matrix set is formed, and the embedded feature matrix set comprises a query feature matrix set and a support feature matrix set. Performing multi-prototype learning on the support feature matrix set to generate a plurality of matrix prototypes, and subtracting each feature matrix to obtain a target feature matrix; thirdly, singular value decomposition is carried out on the target feature matrix to obtain a plurality of subspaces, and a Softmax function is utilized to be combined with the subspaces corresponding to each image category to calculate the classification probability of a query feature matrix set; and finally, through the exchange of the feature matrixes, executing the whole process again until a new process of querying the feature matrix category probability in the feature matrix set is obtained. According to the method, the problems of small inter-class difference and large intra-class difference in the SAR image can be effectively relieved, so that the risk of misclassification of the SAR image in the classification process is reduced.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a SAR image classification method, device, equipment and storage medium. Background Art

[0002] Synthetic Aperture Radar (SAR) is an active detection system with all-weather, all-day, high spatial and temporal resolution, and is widely used in defense and livelihood fields such as environmental monitoring, disaster assessment, urban planning, target detection, and land cover mapping. Usually, SAR equipment is carried on payload platforms such as satellites and aircraft, and can provide continuous and reliable data collection under various weather conditions, so that it can still play an important role in complex and changeable natural environments.

[0003] Unlike optical images, SAR images have unique physical properties and imaging mechanisms, which lead to unique challenges in the classification process. SAR images have the characteristics of small inter-class differences and large intra-class differences, which means that the similarity between different image categories is high, while within the same image category, there may be large differences between images. This small inter-class difference makes it difficult to accurately distinguish different ground object categories during SAR image classification. For example, the reflection characteristics of forests and grasslands in SAR images may be very similar, which requires the classification algorithm to have stronger differentiation ability to effectively identify the differences between them. The large intra-class difference means that the internal differences of samples under the same image category are large. This internal difference may be affected by many factors, including occlusion, the complexity of ground micro-topography, and the scattering effect during electromagnetic wave propagation. These factors will cause images of the same category to show large visual differences, increasing the difficulty of classification. For example, the SAR image performance of the same forest area may be very different due to different tree density, height and type. However, the small inter-class difference and large intra-class difference in SAR images increase the risk of misclassification of SAR images during classification.

[0004] Therefore, how to effectively alleviate the problem of small inter-class differences and large intra-class differences in SAR images, thereby reducing the risk of misclassification in the SAR image classification process, is a technical problem that technicians in this field urgently need to solve. Summary of the invention

[0005] Based on the above problems, the present application provides a SAR image classification method, apparatus, device and storage medium, which can effectively alleviate the problem of small inter-class differences and large intra-class differences in SAR images, thereby reducing the risk of misclassification in the SAR image classification process.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] A SAR image classification method, the method comprising:

[0008] Acquire a synthetic aperture radar (SAR) data set to be classified, and perform feature extraction on the SAR data set to be classified to obtain an embedded feature matrix set; the embedded feature matrix set includes a query feature matrix set and a support feature matrix set; the query feature matrix set and the support feature matrix set have the same category of images; the query feature matrix set and the support feature matrix set both include multiple feature matrices;

[0009] Performing multi-prototype learning on the feature matrix in the support feature matrix set to obtain multiple matrix prototypes;

[0010] Subtracting each feature matrix in the supporting feature matrix set from the multiple matrix prototypes in turn to obtain multiple target feature matrices; the target feature matrices have a corresponding relationship with columns of feature matrices in the supporting feature matrix set;

[0011] Performing singular value decomposition on the plurality of target feature matrices respectively to obtain a plurality of subspaces; each image category corresponds to the plurality of subspaces;

[0012] Utilizing the Softmax function in combination with each image category corresponding to a plurality of subspaces, the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs is calculated;

[0013] The query feature matrix set and the feature matrix in the support feature matrix set are exchanged to obtain a new query feature matrix set and a new support feature matrix set, and all the above steps are performed again using the new query feature matrix set and the new support feature matrix set until the probability that the feature matrix in the new query feature matrix set belongs to the image category to which it belongs is obtained.

[0014] In a possible implementation, the using of the Softmax function in combination with each image category corresponding to a plurality of subspaces to calculate the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs includes:

[0015] The projection distances of each feature matrix in the query feature matrix set to the corresponding multiple subspaces are calculated respectively; the feature matrix in the query feature matrix set has a corresponding relationship with the multiple subspaces in the same image category;

[0016] In the query feature matrix set, the projection distances of the plurality of subspaces corresponding to the feature matrix are averaged to obtain an average distance; the average distance indicates the distance from the feature matrix in the query feature matrix set to the image category to which it belongs;

[0017] The Softmax function is combined with the average distance to calculate the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs.

[0018] In a possible implementation, performing multi-prototype learning on the feature matrix in the support feature matrix set to obtain multiple matrix prototypes includes:

[0019] Flattening all feature matrices in the support feature matrix set one by one to obtain a plurality of flattened matrices;

[0020] Based on the multiple image categories supported by the supporting feature matrix set, the multiple flattened matrices are divided into image categories to obtain multiple similar feature matrix sets; the multiple flattened matrices contained in each similar feature matrix set all belong to the same image category;

[0021] Performing feature aggregation on a plurality of flattened matrices in the same set of similar feature matrices to obtain an aggregation matrix of each set of similar feature matrices;

[0022] Based on the number of matrix partitions, S matrix prototypes of the clustering matrix are generated; S is the number of partitions; and S is a positive integer greater than 1.

[0023] In a possible implementation, generating S matrix prototypes of the clustering matrix based on the number of matrix partitions includes:

[0024] Based on the number of partitions, the clustering matrix is ​​partitioned by columns to obtain S partition matrices; each clustering matrix corresponds to S partition matrices; and the partition matrices belonging to the same column of each clustering matrix have a one-to-one correspondence;

[0025] The partition matrices with one-to-one correspondence are averaged to obtain S matrix prototypes.

[0026] In a possible implementation manner, respectively calculating the projection distances of each feature matrix in the query feature matrix set to the corresponding multiple subspaces includes:

[0027] According to the projection formula, each feature matrix in the query feature matrix set is projected onto the corresponding subspace to obtain multiple projection matrices of each feature matrix in the query feature matrix set; the projection formula is y=p T ·(f-μ), where y is the projection matrix; f is the feature matrix in the query feature matrix set; p is any one of the subspaces corresponding to the feature matrix in the query feature matrix set; p T is the transpose of p; μ is the matrix prototype;

[0028] Based on the projection matrix and the distance calculation formula, the projection distance from each feature matrix in the query feature matrix set to the corresponding subspace is calculated; the distance calculation formula is d=||f-μ-p·y||.

[0029] In a possible implementation, extracting features from the SAR data set to be classified to obtain an embedded feature matrix set includes:

[0030] The pre-trained feature extractor is used to extract features from the SAR data set to be classified to obtain the embedded feature matrix set.

[0031] A SAR image classification device, the device comprising:

[0032] An acquisition unit, used for acquiring a SAR data set to be classified;

[0033] A feature extraction unit is used to extract features from the SAR data set to be classified to obtain an embedded feature matrix set; the embedded feature matrix set includes a query feature matrix set and a support feature matrix set; the query feature matrix set is consistent with the category of the image included in the support feature matrix set; the query feature matrix set and the support feature matrix set both include multiple feature matrices;

[0034] A multi-prototype learning unit, used for performing multi-prototype learning on the feature matrix in the support feature matrix set to obtain multiple matrix prototypes;

[0035] A first calculation unit is used to sequentially subtract each feature matrix in the support feature matrix set from the multiple matrix prototypes to obtain multiple target feature matrices; the target feature matrices have a one-to-one correspondence with the feature matrices in the support feature matrix set;

[0036] A singular value decomposition unit is used to perform singular value decomposition on the plurality of target feature matrices to obtain a plurality of subspaces; each image category corresponds to the plurality of subspaces;

[0037] The second calculation unit is used to calculate the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs by using the Softmax function in combination with the multiple subspaces corresponding to each image category;

[0038] A feature matrix exchange unit, used for exchanging the query feature matrix set with the feature matrix in the support feature matrix set to obtain a new query feature matrix set and a new support feature matrix set;

[0039] The repetitive execution unit is used to execute all the above steps again using the new query feature matrix set and the new support feature matrix set until the probability that the feature matrix in the new query feature matrix set belongs to the image category to which it belongs is obtained.

[0040] In a possible implementation manner, the second computing unit specifically includes:

[0041] A third calculation unit is used to respectively calculate the projection distance of each feature matrix in the query feature matrix set to the corresponding multiple subspaces; the feature matrix in the query feature matrix set has a corresponding relationship with the multiple subspaces in the same image category;

[0042] A fourth calculation unit is used to average the projection distances of the plurality of subspaces corresponding to the feature matrix in the query feature matrix set to obtain an average distance; the average distance indicates the distance from the feature matrix in the query feature matrix set to the image category to which it belongs;

[0043] The fifth calculation unit is used to calculate the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs by using the Softmax function in combination with the average distance.

[0044] A SAR image classification device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the SAR image classification method described above is implemented.

[0045] A computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the SAR image classification method as described above.

[0046] Compared with the prior art, this application has the following beneficial effects:

[0047] The present application provides a SAR image classification method, device, equipment and storage medium. Specifically, when executing the SAR image classification method provided by the embodiment of the present application, first obtain a SAR data set to be classified, and perform feature extraction on the SAR data set to be classified to obtain an embedded feature matrix set, wherein the embedded feature matrix set includes a query feature matrix set and a support feature matrix set of the same image category. Perform multi-prototype learning on the feature matrix in the support feature matrix set to obtain multiple matrix prototypes, and subtract each feature matrix in the support feature matrix set from the multiple matrix prototypes in turn to obtain multiple target feature matrices. Then perform singular value decomposition on the multiple target feature matrices to obtain multiple subspaces, and use the Softmax function to combine each image category to correspond to multiple subspaces to calculate the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs. Finally, the query feature matrix set is exchanged with the feature matrix in the support feature matrix set to obtain a new query feature matrix set and a new support feature matrix set, and all the above steps are performed again using the new query feature matrix set and the new support feature matrix set until the probability that the feature matrix in the new query feature matrix set belongs to the image category to which it belongs is obtained.

[0048] The embedded feature matrix set including the query feature matrix set and the support feature matrix set in the present application makes full use of multi-level feature information, so as to capture the scale and complexity information of different images, so as to more accurately identify the subtle differences between different image categories. By performing multi-prototype learning on the support feature matrix set, multiple matrix prototypes can be generated, which improves the adaptability to complex intra-class differences. Subsequently, each support feature matrix is ​​subtracted from multiple matrix prototypes and singular value decomposition is performed to further extract the key subspace reflecting the inter-class differences. By effectively alleviating the problem of small inter-class differences and large intra-class differences in SAR images, the risk of misclassification of SAR images in the classification process can be reduced.

[0049] In addition, using the Softmax function to calculate the probability of belonging to the image category to which it belongs can effectively normalize the classification results and provide a probabilistic output for easy decision-making. This step not only improves the credibility of the classification, but also facilitates subsequent data processing and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1A method flow chart of a SAR image classification method provided in an embodiment of the present application;

[0052] Figure 2 A method flow chart of a multi-prototype learning method provided in an embodiment of the present application;

[0053] Figure 3 A method flow chart of a matrix prototype generation method provided in an embodiment of the present application;

[0054] Figure 4 A flow chart of a method for calculating the probability of an image category provided in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of the structure of a SAR image classification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the background technology involved in the embodiments of the present application will be described below.

[0057] SAR is an active detection system with all-weather, all-day, high spatial and temporal resolution, which is widely used in defense and livelihood fields such as environmental monitoring, disaster assessment, urban planning, target detection and land cover mapping. Usually, SAR equipment is carried on payload platforms such as satellites and aircraft, and can provide continuous and reliable data collection under various weather conditions.

[0058] Unlike optical images, SAR images have small inter-class differences and large intra-class differences, which poses a challenge to SAR image classification. Small inter-class differences mean that there are large similarities between different image categories, making it difficult to accurately distinguish them during SAR image classification. Large intra-class differences mean that there are large internal differences between images of the same image category. Such internal differences may be affected by factors such as occlusion and illumination, resulting in large differences between images of the same category. These characteristics increase the risk of misclassification of SAR images during classification.

[0059] In order to solve this problem, a SAR image classification method, device, equipment and storage medium are provided in the embodiment of the present application. First, a SAR data set to be classified is obtained, and features are extracted to obtain an embedded feature matrix set. The embedded feature matrix set includes a query feature matrix set and a support feature matrix set, wherein the query feature matrix set has the same image category as the support feature matrix set. Next, multi-prototype learning is performed on the support feature matrix set to generate multiple matrix prototypes, and each support feature matrix is ​​subtracted from these matrix prototypes to obtain multiple target feature matrices. Then, singular value decomposition is performed on these target feature matrices to obtain multiple subspaces corresponding to each image category. The Softmax function is used to combine these subspaces to calculate the probability that each feature matrix in the query feature matrix set belongs to the image category to which it belongs. Finally, the query feature matrix set is exchanged with the support feature matrix set to form a new query feature matrix set and a new support feature matrix set, and all the above steps are repeated until the probability that the feature matrix in the new query feature matrix set belongs to the image category to which it belongs is obtained. This application adopts the strategy of multi-prototype learning and singular value decomposition for SAR image classification, which can effectively alleviate the problem of small inter-class differences and large intra-class differences in SAR images, thereby reducing the risk of misclassification of SAR images during the classification process.

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0061] See also Figure 1 , which is a flow chart of a SAR image classification method provided in an embodiment of the present application, such as Figure 1 As shown, the SAR image classification method may include steps S101-S106:

[0062] S101: Acquire a SAR data set to be classified, and perform feature extraction on the SAR data set to be classified to obtain an embedded feature matrix set.

[0063] Acquiring a SAR dataset to be classified means collecting raw SAR image data from various sources such as satellites, aircraft, or ground stations, which usually contains a large number of image samples. For this data, feature extraction is a key step that aims to convert the raw image into a more informative representation. The feature extraction process can capture key information and features in the image. These features include edges and textures, which help describe the structure and content of the image.

[0064] After acquiring and extracting the features of the SAR dataset, an embedding feature matrix set can be constructed. The embedding feature matrix set consists of two parts: a query feature matrix set and a support feature matrix set. The image categories in the query feature matrix set and the support feature matrix set should be consistent, which means that they both contain image features from the same category. The feature matrices in these two feature matrix sets can be regarded as representations of image samples after feature extraction, and they may contain multiple visual features of the image, such as shape and texture, for subsequent classification and recognition tasks.

[0065] In a possible implementation, feature extraction of the SAR training data to obtain an embedded feature matrix is ​​achieved by using a pre-trained backbone network Conv4 as a feature extractor. The backbone network Conv4 consists of four convolution blocks with the same structure for feature embedding. Each convolution block in the backbone network Conv4 includes a four-layer structure, and its specific structure is as follows: the first layer is a 3×3 kernel, the second layer is a convolution layer with a step of 1, and the third layer immediately following it is composed of a batch normalization layer and a maximum pooling layer with a 2×2 kernel. The fourth layer is an activation function layer using a rectified linear unit (Rectified Linear Unit, ReLU), which is an activation function commonly used in the hidden layer of a neural network. It is worth noting that, except for the first convolution block with an input channel of 1 and an output channel of 64, the input and output channels of the other three convolution blocks are all 64. When the input image size is 84×84, this design will eventually generate an embedded feature matrix of 64×5×5 dimensions.

[0066] The convolutional layer of the backbone network Conv4 generates feature maps by sliding and performing convolution operations on the input data through a series of filters. The batch normalization layer normalizes the input of each feature dimension so that its mean is close to 0 and its standard deviation is close to 1. This makes the input distribution of the backbone network Conv4 more stable before passing through the activation function, which helps to avoid problems such as gradient vanishing or gradient exploding. The ReLU activation function sets the output to the same value as the input when the input is positive and to zero when the input is negative. This property gives it a simple and effective nonlinear characteristic. The main function of the maximum pooling layer is to downsample the input feature map. This operation can effectively reduce the size of the feature map and retain the most representative features, which helps to improve the computational efficiency and performance of the model.

[0067] There are two main reasons for choosing this backbone network as a feature extractor. On the one hand, Conv4 has been widely used in the field of image classification and has also shown good performance in feature extraction. On the other hand, considering its simplicity and effectiveness, this network does not introduce additional modifications such as dropout or wider channels, and is easier to train and optimize than other classic networks such as ResNet50 and VGG16.

[0068] In a possible implementation, the backbone network Conv4 as a feature extractor needs to be trained. To implement the training of the feature extractor, it is first necessary to obtain a SAR training sample data set, and perform feature extraction on the SAR training sample data set to obtain a training embedded feature matrix set; the SAR training sample data set includes a query sample data set and a support sample data set; the embedded feature matrix set includes a query sample feature matrix set and a support sample feature matrix set; the query sample data set is consistent with the category of the image contained in the support sample data set; the query sample feature matrix set and the support sample feature matrix set both include multiple feature matrices. Then, multi-prototype learning is performed on the feature matrix in the support sample feature matrix set to obtain multiple matrix prototypes. Each feature matrix in the support sample feature matrix set is sequentially subtracted from the multiple matrix prototypes to obtain multiple subtracted feature matrices; the subtracted feature matrix has a one-to-one correspondence with the feature matrix in the support sample feature matrix set. Then, singular value decomposition is performed on the multiple subtracted feature matrices to obtain multiple subspaces; each image category corresponds to multiple subspaces. Then, the Softmax function is used to combine multiple subspaces corresponding to each image category to calculate the probability that the feature matrix in the query sample feature matrix set belongs to the image category to which it belongs. Finally, the query sample feature matrix set is exchanged with the feature matrix in the support sample feature matrix set to obtain a new query sample feature matrix set and a new support sample feature matrix set, and all the above steps are performed again using the new query sample feature matrix set and the new support sample feature matrix set until the probability that the feature matrix in the new query sample feature matrix set belongs to the image category to which it belongs is obtained.

[0069] It should be noted that the steps of multi-prototype learning (refer to steps A1-A4 and steps B1-B2 for details), subspace projection distance (refer to the specific calculation process of subspace projection distance in step C1 for details), and calculation of the probability that the feature matrix in the query sample feature matrix set belongs to the image category to which it belongs (refer to steps C1-C3 for details) involved in the training process of the feature extractor are similar to the classification process of SAR images. Please refer to the classification process of SAR images for details and will not be repeated here.

[0070] In a possible implementation, the method further includes: adjusting parameters of the trained feature extractor using a loss function. The loss function used here consists of two parts, namely a cross-entropy loss function (Cross-Entropy Loss) and a subspace discriminative loss function (Subspace Discriminative Loss). The details are as follows:

[0071] l total = l ce +λl sdl ;

[0072] Among them, l ce represents the cross entropy loss; l sdl Represents the subspace discriminant loss function; λ is a hyperparameter that controls the proportion of adversarial loss in the overall loss function.

[0073] In classification tasks, the Softmax activation function is often used to convert the output of the model into a probability distribution. The cross entropy loss function is used in conjunction with the Softmax activation function to directly calculate the difference between the probability distribution predicted by the model and the true label. In addition, the cross entropy loss function has a large penalty for incorrect predictions. When the model's prediction for a certain sample is very inaccurate, that is, when the predicted probability differs greatly from the actual label, the loss value will increase significantly, resulting in a large gradient, which in turn drives the model's weights to adjust quickly. This helps the model learn from errors quickly in the early stages and accelerates the convergence process. For multi-classification problems, the formula for the cross entropy loss function is as follows:

[0074]

[0075] Where N is the total number of samples and C is the total number of categories. i,c is a unique encoding vector that indicates whether sample i belongs to category c. If the sample belongs to this category, then y i,c =1, otherwise 0. i,c represents the probability that the model predicts that sample i is of category c, i.e., the output of the Softmax function. It should be noted that the cross entropy loss function implemented in PyTorch requires the true label to be provided in the form of category index, rather than in the form of independent encoding. This means that for each sample, only the index value of the category needs to be provided, and PyTorch will automatically handle the label conversion and cross entropy calculation.

[0076] During the training process, the purpose of introducing subspace discrimination loss is to maximize the distance between subspaces of different categories, thereby improving the distinguishability of subspaces between different categories. This application uses the projection metric of Grassmannian geometry to calculate the distance between subspaces. The projection metric reflects the "closeness" of one subspace projected onto another subspace. If two subspaces are very close, the projection metric between them will be very small. If they are far apart, this metric will be larger. If two subspaces P are given, i and P j , then the projection metric is defined as:

[0077]

[0078] So, if we want to make the subspace P i and P j The distance is farther, just by minimizing In the process of back propagation, as As the value of gets smaller, the value of the projection metric gets larger, that is, the distance between subspaces gets farther and farther. The formula of the subspace discriminant loss function is:

[0079]

[0080] Because multiple subspaces are generated for each category in the support set during training, if the discriminant loss is calculated for all subspaces of different categories, the amount of calculation will be very large. Therefore, back propagation is performed by calculating the discriminant loss between the query image and the nearest subspace of its category and the query image and the nearest subspace of each other category. This can reduce the amount of calculation on the one hand, and achieve the desired effect on the other.

[0081] During training, each time after training, the total loss l is calculated. total Backpropagation is performed on the feature extractor to update the parameters of the feature extractor. Through continuous training and backpropagation, the loss value will gradually decrease until the feature extractor reaches the desired performance level.

[0082] In a possible implementation, before extracting features from the SAR data set to be classified, the SAR data set to be classified needs to be preprocessed to obtain a preprocessed data set, and then feature extraction is performed on the preprocessed data set. The preprocessing may be resizing the SAR images in the SAR data set to be classified (e.g., changing a SAR image of size 256*256 to a SAR image of size 84*84) and normalizing.

[0083] S102: Perform multi-prototype learning on the feature matrices in the support feature matrix set to obtain multiple matrix prototypes.

[0084] Multi-prototype learning of feature matrices in the support feature matrix set means discovering the potential structure and patterns in the data through a learning method and extracting multiple representative "prototypes" from them. These matrix prototypes can be regarded as abstract representations of different categories, features or patterns in the data, which helps to better capture and understand the information in the support feature matrix set. By learning multiple matrix prototypes, different features can be more effectively distinguished and represented, providing more accurate and effective feature expressions for further data analysis and classification. The purpose of this method is to improve the expressiveness and classification performance of the data by learning multiple prototypes, so that different feature categories or patterns can be better identified and distinguished when processing complex data.

[0085] See also Figure 2 , Figure 2 A method flow chart of a multi-prototype learning method provided in an embodiment of the present application. Accordingly, step S102 performs multi-prototype learning on the feature matrix in the support feature matrix set to obtain multiple matrix prototypes, which can be implemented through steps A1-A4:

[0086] A1: Flatten all feature matrices in the support feature matrix set one by one to obtain multiple flattened matrices.

[0087] Flattening all feature matrices in the supporting feature matrix set one by one means converting the two-dimensional or multi-dimensional data in these feature matrices into a one-dimensional form, thereby obtaining a plurality of flattened matrices.

[0088] In order to flatten all feature matrices in the support feature matrix set, the following steps can be followed: First, traverse each feature matrix in the support feature matrix set. For each feature matrix, rearrange its internal multidimensional data (such as image pixel values) into a continuous one-dimensional array. This is called a flattening operation, which expands the original two-dimensional or multidimensional data into a one-dimensional vector. Repeat this process until the flattening operation is completed for all feature matrices, and multiple flattened matrices are obtained, each representing the flattened result of a feature matrix.

[0089] The flattening operation helps to simplify the data structure and reduce the data dimension while retaining the key features of the original data, making the data easier to understand and process.

[0090] A2: Based on a plurality of image categories supported by the supporting feature matrix set, the plurality of flattened matrices are divided into image categories to obtain a plurality of feature matrix sets of the same type.

[0091] Based on the multiple image categories supported by the supporting feature matrix set, the multiple flattened matrices are divided into image categories to obtain multiple homogeneous feature matrix sets, which means that the flattened feature matrices are classified and combined according to the real categories of the images to form feature matrix subsets with the same category labels. In other words, the flattened matrices contained in the same homogeneous feature matrix set all belong to the same image category, which facilitates subsequent processing and analysis.

[0092] In order to classify the flattened matrices based on the multiple image categories supported by the feature matrix set, the following steps can be taken: First, each flattened matrix is ​​associated with its corresponding real image category according to the image category information corresponding to the feature matrix. Then, according to the category information, the flattened matrices are grouped, and the flattened matrix sets of the same category are combined together to form a similar feature matrix set. In this way, the flattened matrices in each similar feature matrix set correspond to the same image category, so that the data is sorted and classified by category.

[0093] Through this process, the feature matrices can be integrated together according to the true category to form a set of feature matrices with the same category label.

[0094] A3: performing feature aggregation on multiple flattened matrices in the same feature matrix set to obtain an aggregation matrix of each feature matrix set of the same type.

[0095] Performing feature aggregation on multiple flattened matrices in the same feature matrix set to obtain the clustered matrix of each feature matrix set means aggregating the flattened matrices under the same category and integrating their feature information to generate a clustered matrix that comprehensively represents the category. This operation aims to extract common features in the data of the same category and combine them into a single matrix with more representative and high-dimensional information for subsequent data analysis.

[0096] To perform feature aggregation on multiple flattened matrices in the same feature matrix set and obtain the aggregation matrix of each feature matrix set, the following steps can be followed: First, collect the flattened matrices under the same image category to ensure that these matrices belong to the same category. Then, for these flattened matrices, various aggregation methods (such as average, maximum, weighted average, etc.) can be applied to integrate their feature information. By calculating the feature aggregation results of these flattened matrices, a new aggregation matrix is ​​obtained, which contains the comprehensive feature representation of all flattened matrices in the same category.

[0097] Through feature aggregation operations, key feature information can be extracted from the flattened matrix of the same category, while reducing the dimension of the data and generating a more concise and representative aggregation matrix. This processing method helps to reduce redundant information, highlight the common features of the category, and make the data easier to analyze, understand and use in machine learning tasks.

[0098] A4: Generate S matrix prototypes of the clustering matrix based on the number of matrix partitions.

[0099] Generating S matrix prototypes of the clustered matrix based on the number of matrix partitions means further dividing the clustered matrix into multiple different parts or sub-matrices to form S independent matrix prototypes. This operation aims to split the overall clustered matrix into multiple parts according to the pre-specified number of partitions S, each of which represents an independent prototype structure. Here, S is defined as the number of partitions, which is a positive integer greater than 1 and is used to control the decomposition and representation of the clustered matrix.

[0100] By generating S matrix prototypes based on the number of matrix partitions, a more fine-grained view of the clustering matrix can be obtained. Such a partitioning process helps to split the complex clustering matrix into more manageable and understandable parts, making the characteristics and structures between different parts more clearly visible. Each matrix prototype represents a different aspect or feature in the clustering matrix, thus providing a more explanatory and abstract view of the data.

[0101] See also Figure 3 , Figure 3 A method flow chart of a method for generating a matrix prototype provided in an embodiment of the present application. Accordingly, step A4 generates S matrix prototypes of the clustered matrix based on the number of matrix partitions, which can be specifically implemented through steps B1-B2:

[0102] B1: Based on the number of partitions, the clustering matrix is ​​partitioned by columns to obtain S partition matrices.

[0103] Based on the number of partitions, each clustering matrix is ​​partitioned by column to obtain S partition matrices. Each original clustering matrix will generate S partition matrices, ensuring the integrity and relevance of the data structure. Importantly, there is a one-to-one correspondence between the partition matrices on the same column, which means that the data in each partition matrix has a direct correspondence with the data in the same column of the original matrix, ensuring accurate mapping and comparison between data.

[0104] In order to determine the number of matrix partitions, we first need to consider the characteristics of the data set, task requirements, and actual analysis needs. Based on these factors, we select an appropriate number of partitions S. Usually, this number can be determined by the dimension of the data, the number of features, or other relevant domain knowledge. Once the number of partitions S is determined, we can perform column-by-column partitioning operations on each clustering matrix. By dividing the clustering matrix by columns, we obtain S partition matrices, ensuring that each partition matrix has a clear correspondence with the corresponding column of the original data. This method helps to observe and analyze data from different angles and dimensions, improve the accuracy and interpretability of data processing, and is of great significance for subsequent tasks such as data mining and feature extraction.

[0105] B2: Calculate the average of the partition matrices with one-to-one correspondence to obtain S matrix prototypes.

[0106] The purpose of averaging the segmentation matrices with one-to-one correspondence and obtaining S matrix prototypes is to extract common features from the segmentation matrices at each corresponding position and generate a more comprehensive and representative data structure. Through this process, multiple data with similar structures can be integrated and averaged to obtain a more comprehensive and representative data representation.

[0107] To implement the process of averaging and generating matrix prototypes, we first need to ensure that there is a one-to-one correspondence between the segmentation matrices in each clustering matrix. Then, for these segmentation matrices with corresponding relationships, the average values ​​of the elements at the corresponding positions are calculated one by one. Specifically, for the segmentation matrices at each position (i.e., the same column), the element values ​​corresponding to these positions are averaged to form a new matrix prototype. This process will obtain S matrix prototypes, each of which represents the mean of the corresponding position in all segmentation matrices. Doing so helps to extract common features in the data, reduce the impact of noise and individual differences, and thus generate a more robust and comprehensive data structure. These matrix prototypes can be used for further analysis, feature extraction, or machine learning tasks.

[0108] For example, assuming that there are two clustering matrices A and B, and the determined number of partitions is 4, then the matrices A and B are divided into 4 blocks according to the columns, then the matrix A has 4 partition matrices A1, A2, A3 and A4, and the matrix B has 4 partition matrices B1, B2, B3 and B4. A1 and B1 belong to the first column in their respective clustering matrices, so A1 and B1 have a one-to-one correspondence; A2 and B2 belong to the second column in their respective clustering matrices, so A2 and B2 have a one-to-one correspondence; A3 and B3 belong to the third column in their respective clustering matrices, so A3 and B3 have a one-to-one correspondence; A4 and B4 belong to the fourth column in their respective clustering matrices, so A4 and B4 have a one-to-one correspondence. Therefore, the average of A1 and B1 is C1, the average of A2 and B2 is C2, the average of A3 and B3 is C3, and the average of A4 and B4 is C4. Therefore, four matrix prototypes C1, C2, C3 and C4 are obtained.

[0109] S103: Subtract each feature matrix in the supporting feature matrix set from the multiple matrix prototypes in turn to obtain multiple target feature matrices.

[0110] Each feature matrix corresponds to an image category, and the matrix prototype under the corresponding image category is the matrix prototype corresponding to the feature matrix. The operation of subtracting multiple matrix prototypes from each feature matrix in the support feature matrix set to obtain multiple target feature matrices is intended to highlight and extract differences at the data level by subtracting common features. This process can help us separate individual-specific information from the more abstract common features in the data, while ensuring the correspondence between the target feature matrix and the support feature matrix to ensure the comparability and continuity of the data.

[0111] To achieve the process of sequentially subtracting to generate the target feature matrix, it is first necessary to clarify the relationship between the columns of each support feature matrix and multiple matrix prototypes, and ensure that they have corresponding structures. Then, each support feature matrix is ​​subtracted from multiple matrix prototypes one by one in order to generate the corresponding number of target feature matrices. The specific operation is to subtract the elements at the same position in turn, and obtain the new value as the element at the corresponding position in the target feature matrix. This operation will make each feature in the target feature matrix subtract the averaged common features from the corresponding original features, thereby highlighting the individual-specific difference features. These target feature matrices can be used for further data analysis, feature selection or pattern recognition tasks, providing more specific and personalized data features for subsequent processing.

[0112] For example, assume that the feature matrix C is 4 matrix prototypes C1, C2, C3 and C4: C1 is C2 is C3 C4 C1 corresponds to the first column of A, C2 corresponds to the second column of A, C3 corresponds to the third column of A, and C4 corresponds to the fourth column of A. Therefore, the target feature matrix obtained by subtracting the clustering matrix A from multiple matrix prototypes C1, C2, C3 and C4 is

[0113] It should be noted that each feature matrix in the support feature matrix set needs to be subtracted from multiple matrix prototypes in turn, and the number of target feature matrices is consistent with the number of feature matrices in the support feature matrix set. Assuming that there are three feature matrices in the support feature matrix set, namely feature matrices C, D and E, and four matrix prototypes C1, C2, C3 and C4 are calculated, then C needs to be subtracted from the four matrix prototypes C1, C2, C3 and C4 in turn, D needs to be subtracted from the four matrix prototypes C1, C2, C3 and C4 in turn, and E needs to be subtracted from the four matrix prototypes C1, C2, C3 and C4 in turn, so that three target feature matrices can be obtained.

[0114] S104: Perform singular value decomposition on the plurality of target feature matrices respectively to obtain a plurality of subspaces.

[0115] The operation of performing singular value decomposition on multiple target feature matrices to obtain multiple subspaces aims to extract the main features and structural information in the data by decomposing the original data. Each subspace corresponds to a specific aspect or dimension of the data. This decomposition helps to reduce the dimension of the data, extract key features, and reveal the potential patterns and structures in the data. When the target feature matrices corresponding to different image categories are mapped to multiple subspaces respectively, it means that each image category has different representations and feature combinations in the data space, which facilitates subsequent classification tasks.

[0116] To achieve the process of generating subspaces by singular value decomposition, it is first necessary to perform singular value decomposition on each target feature matrix and decompose it into the product form of singular values, left singular vectors and right singular vectors. Singular value decomposition is an effective matrix decomposition method that can extract important information and structural features of the data. The specific steps are to perform singular value decomposition on each target feature matrix to obtain a corresponding number of subspaces. Each subspace reflects the changes and important features of the data in different directions. These subspaces can be used to represent different attributes of the data, such as texture and shape, and also help to distinguish the differences between different categories of images in the feature space. In this way, the inherent structure of the data can be better understood and characterized.

[0117] S105: Utilizing the Softmax function in combination with the plurality of subspaces corresponding to each image category, the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs is calculated.

[0118] The Softmax function is used to combine multiple subspaces corresponding to each image category to calculate the probability that the feature matrices in the query feature matrix set belong to their respective image categories in order to evaluate the possibility of a feature matrix belonging to different categories through a probability model. The Softmax function can map a set of numerical values ​​to a probability distribution, thereby quantitatively expressing the degree of association between a feature matrix and each image category. This helps the classification model assign appropriate category labels to feature vectors based on their performance in different categories, thereby improving accuracy and generalization.

[0119] For example, assume that there are image category A and image category B, image category A corresponds to subspace 1 and subspace 2; and image category B corresponds to subspace 3 and subspace 4, and the image category of feature matrix C in the query feature matrix set is image category A, then feature matrix C corresponds to subspace 1 and subspace 2.

[0120] The Softmax function works by converting the input real value into a probability distribution, where each category corresponds to a probability value and the sum of the probabilities of all categories is 1. In this case, multiple subspaces corresponding to each image category are used as feature inputs, and then the Softmax function is used to calculate a probability distribution, which shows the probability that the query feature matrix belongs to different image categories. This method allows us to classify the query feature matrix and determine its most likely category label for further recognition, retrieval or other visual tasks. Therefore, the Softmax function combines multiple subspaces and provides an effective means to infer the correlation between the query feature matrix and each image category and obtain the corresponding classification results.

[0121] See also Figure 4 , Figure 4 A flow chart of a method for calculating the probability of an image category provided in an embodiment of the present application. Accordingly, step S105 uses a Softmax function in combination with multiple subspaces corresponding to each image category to calculate the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs. Specifically, steps C1-C3 can be implemented:

[0122] C1: respectively calculating the projection distance of each feature matrix in the query feature matrix set to its corresponding multiple subspaces.

[0123] The projection distance of each feature matrix in the query feature matrix set to its corresponding multiple subspaces is calculated. This step aims to measure the projection position and relative relationship of each feature matrix in different subspaces, and help understand the distribution and structure of data in the feature space. In addition, the correspondence between the feature matrix and multiple subspaces of the same image category indicates that there is a correlation and connection between these feature matrices and the categories to which they belong. This correspondence helps to accurately classify and identify image data.

[0124] In a possible implementation, the specific calculation process of the projection distance of the subspace includes: when calculating the projection distance, each feature matrix in the query feature matrix set needs to be projected onto its corresponding subspace through a projection formula to obtain multiple projection matrices for each feature matrix. In this process, the projection formula is y=pT·(f-μ), where y represents the projection matrix, f represents the feature matrix in the query feature matrix set, p represents the subspace corresponding to any feature matrix, and p T is the transpose of p, and μ is the matrix prototype. This step actually maps the data into different feature spaces, helping us better understand the distribution and structure of the data.

[0125] Then, based on the projection matrix and the distance calculation formula, the projection distance of each feature matrix to its corresponding subspace can be calculated. The distance calculation formula is d = ||f-μ-p·y||, where d represents the projection distance, |||| represents the norm operation, and f-μ-p·y represents the calculated distance value. This calculation process is actually a quantitative measurement of the similarity or difference between each feature matrix and its subspace, using distance to represent the degree of closeness or offset between them.

[0126] Through the above steps, the query feature matrix can be mapped to a specific subspace, and then the relationship between each feature matrix and its corresponding subspace can be effectively evaluated by calculating the projection distance. This analysis helps to reveal the hidden features of the data, provide a deeper understanding of the data, and provide strong support and guidance for subsequent data processing, classification or recognition tasks.

[0127] C2: In the query feature matrix set, the projection distances of the plurality of subspaces corresponding to the feature matrix are averaged to obtain an average distance.

[0128] In the query feature matrix set, the average distance is obtained by averaging the projection distances of multiple subspaces corresponding to the feature matrix. This average distance reflects the overall distance from the feature matrix in the query feature matrix set to the image category to which it belongs. Specifically, the average distance represents the average similarity or difference between each feature matrix and its category, which helps to measure the relationship between different features and their categories in the data space.

[0129] When averaging the projection distances of multiple subspaces corresponding to the feature matrices in the query feature matrix set to obtain the average distance, it is first necessary to calculate the projection distance of each feature matrix to each subspace. Then, these projection distances are summed and divided by the number of subspaces to obtain the average distance of each feature matrix to its corresponding subspace. This process actually comprehensively considers the position information and feature distribution of each feature matrix in different subspaces, thereby obtaining a more comprehensive and integrated metric to measure the overall similarity or distance between the feature matrix and its category. This method can provide more in-depth feature analysis and classification information, providing strong support for further data processing and feature recognition tasks.

[0130] C3: Utilizing the Softmax function in combination with the average distance, calculate the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs.

[0131] By using the Softmax function combined with the average distance, the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs can be calculated. The purpose of this step is to estimate the probability that the feature matrix belongs to each category based on the average distance of the feature matrix in different subspaces, so as to achieve the prediction and analysis of data classification. The Softmax function converts the average distance into a probability value between 0 and 1, and ensures that the sum of the probabilities of all categories is 1.

[0132] To use the Softmax function combined with the average distance to calculate the probability that a feature matrix belongs to the image category to which it belongs, we first need to obtain the average distance of each feature matrix in different subspaces. Then, these average distances are used as input and processed by the Softmax function to obtain the probability distribution of each category. The Softmax function exponentializes the scores of each category and then normalizes them to probability values, indicating the possibility that the feature matrix belongs to each category. In this way, the possibility of the feature matrix belonging to different categories can be inferred based on the average distance, thereby achieving effective classification and recognition of data, providing an important reference for further image processing and analysis.

[0133] S106: Exchange the query feature matrix set with the feature matrix in the support feature matrix set to obtain a new query feature matrix set and a new support feature matrix set, and use the new query feature matrix set and the new support feature matrix set to perform the above steps S101-S105 again until the probability that the feature matrix in the new query feature matrix set belongs to the image category to which it belongs is obtained.

[0134] By exchanging the feature matrices in the query feature matrix set with those in the support feature matrix set, a new query feature matrix set and a new support feature matrix set are obtained. Subsequently, the aforementioned steps (S101-S105) are performed using these new data sets, including calculating the projection distance, finding the average distance, and calculating the probability using the Softmax function, etc., until the probability that the feature matrix in the new query feature matrix set belongs to the image category to which it belongs is obtained.

[0135] Based on the contents of S101-S106, it can be known that the embedded feature matrix set is first obtained by extracting the features of the SAR data set to be classified, which includes the query and support feature matrix sets with the same image category. The support feature matrix is ​​processed by multi-prototype learning to generate multiple matrix prototypes, and the target feature matrix is ​​obtained by subtracting these prototypes. Next, different subspaces are obtained by singular value decomposition, and then the probability that the feature matrix in the query feature matrix set belongs to its category is calculated by combining the Softmax function and the corresponding subspace of each image category. Finally, by exchanging the feature matrix, the whole process is executed again until the process of obtaining the feature matrix category probability in the new query feature matrix set is obtained. The embedded feature matrix set (including the query feature matrix set and the support feature matrix set) in this application makes full use of multi-level feature information, can capture the scale and complexity of different images, and more accurately identify the subtle differences between image categories. By performing multi-prototype learning on the support feature matrix set, multiple matrix prototypes are generated, the adaptability to complex intra-class differences is improved, and the key subspace reflecting the inter-class differences is extracted by singular value decomposition, which effectively alleviates the problem of small inter-class differences and large intra-class differences in SAR images, and reduces the risk of misclassification. In addition, the Softmax function is used to calculate the classification probability and normalize the output results, which improves the credibility of the classification and the convenience of subsequent data processing.

[0136] See also Figure 5 , Figure 5 The schematic diagram of the structure of a SAR image classification device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the SAR image classification device includes:

[0137] An acquisition unit 501 is used to acquire a SAR data set to be classified;

[0138] A feature extraction unit 502 is used to extract features from the SAR data set to be classified to obtain an embedded feature matrix set; the embedded feature matrix set includes a query feature matrix set and a support feature matrix set; the query feature matrix set is consistent with the category of the image included in the support feature matrix set; the query feature matrix set and the support feature matrix set both include multiple feature matrices;

[0139] A multi-prototype learning unit 503, used for performing multi-prototype learning on the feature matrix in the support feature matrix set to obtain multiple matrix prototypes;

[0140] A first calculation unit 504 is used to sequentially subtract each feature matrix in the support feature matrix set from the multiple matrix prototypes to obtain multiple target feature matrices; the target feature matrices have a one-to-one correspondence with the feature matrices in the support feature matrix set;

[0141] A singular value decomposition unit 505 is used to perform singular value decomposition on the plurality of target feature matrices respectively to obtain a plurality of subspaces; each image category corresponds to the plurality of subspaces;

[0142] The second calculation unit 506 is used to calculate the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs by using the Softmax function in combination with the multiple subspaces corresponding to each image category;

[0143] A feature matrix exchange unit 507, used to exchange the query feature matrix set with the feature matrix in the support feature matrix set to obtain a new query feature matrix set and a new support feature matrix set;

[0144] The repetitive execution unit 508 is used to execute all the above steps again using the new query feature matrix set and the new support feature matrix set until the probability that the feature matrix in the new query feature matrix set belongs to the image category to which it belongs is obtained.

[0145] In a possible implementation, the second calculating unit 506 specifically includes:

[0146] A third calculation unit is used to respectively calculate the projection distance of each feature matrix in the query feature matrix set to the corresponding multiple subspaces; the feature matrix in the query feature matrix set has a corresponding relationship with the multiple subspaces in the same image category;

[0147] A fourth calculation unit is used to average the projection distances of the plurality of subspaces corresponding to the feature matrix in the query feature matrix set to obtain an average distance; the average distance indicates the distance from the feature matrix in the query feature matrix set to the image category to which it belongs;

[0148] The fifth calculation unit is used to calculate the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs by using the Softmax function in combination with the average distance.

[0149] In a possible implementation, the multi-prototype learning unit specifically includes:

[0150] A matrix flattening unit, used for flattening all feature matrices in the support feature matrix set one by one to obtain a plurality of flattened matrices;

[0151] A division unit, configured to divide the plurality of flattened matrices into image categories based on the plurality of image categories supported by the supporting feature matrix set to obtain a plurality of similar feature matrix sets; the plurality of flattened matrices contained in each similar feature matrix set all belong to the same image category;

[0152] A feature aggregation unit, used for performing feature aggregation on a plurality of flattened matrices in the same set of similar feature matrices to obtain an aggregation matrix of each set of similar feature matrices;

[0153] The matrix prototype generation unit is used to generate S matrix prototypes of the clustered matrix based on the number of matrix partitions; S is the number of partitions; S is a positive integer greater than 1.

[0154] In a possible implementation, the matrix prototype generation unit specifically includes:

[0155] A segmentation unit, configured to segment the clustering matrix according to columns based on the segmentation quantity to obtain S segmentation matrices; each clustering matrix corresponds to the S segmentation matrices; and the segmentation matrices belonging to the same column of each clustering matrix have a one-to-one correspondence;

[0156] The sixth calculation unit is used to calculate the average of the segmentation matrices with a one-to-one correspondence to obtain S matrix prototypes.

[0157] In a possible implementation manner, the third computing unit specifically includes:

[0158] A projection unit is used to project each feature matrix in the query feature matrix set onto the corresponding subspace according to a projection formula to obtain multiple projection matrices of each feature matrix in the query feature matrix set; the projection formula is y=p T ·(f-μ), where y is the projection matrix; f is the feature matrix in the query feature matrix set; p is any one of the subspaces corresponding to the feature matrix in the query feature matrix set; p T is the transpose of p; μ is the matrix prototype;

[0159] The seventh calculation unit is used to calculate the projection distance from each feature matrix in the query feature matrix set to the corresponding subspace based on the projection matrix and the distance calculation formula; the distance calculation formula is d=||f-μ-p·y||.

[0160] In a possible implementation, the feature extraction unit 502 is specifically configured to:

[0161] The pre-trained feature extractor is used to extract features from the SAR data set to be classified to obtain the embedded feature matrix set.

[0162] In addition, an embodiment of the present application also provides a SAR image classification device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the SAR image classification method described above is implemented.

[0163] In addition, an embodiment of the present application further provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes the SAR image classification method as described above.

[0164] The embodiment of the present application provides a SAR image classification device, which first uses an acquisition unit 501 to acquire a SAR data set to be classified, and uses a feature extraction unit 502 to extract features from the SAR data set to be classified to obtain an embedded feature matrix set, wherein the embedded feature matrix set includes a query feature matrix set and a support feature matrix set. The multi-prototype learning unit 503 performs multi-prototype learning on the feature matrix in the support feature matrix set to obtain multiple matrix prototypes, and uses a first calculation unit 504 to subtract each feature matrix in the support feature matrix set from the multiple matrix prototypes in turn to obtain multiple target feature matrices. Then, the singular value decomposition unit 505 performs singular value decomposition on the multiple target feature matrices to obtain multiple subspaces, and the second calculation unit 506 uses a Softmax function to combine each image category to correspond to multiple subspaces to calculate the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs. Finally, the feature matrix exchange unit 507 exchanges the query feature matrix set with the feature matrix in the support feature matrix set to obtain a new query feature matrix set and a new support feature matrix set, so that the repeated execution unit 508 can use the new query feature matrix set and the new support feature matrix set to perform all the above steps again until the probability that the feature matrix in the new query feature matrix set belongs to the image category to which it belongs is obtained. The embedded feature matrix set (including the query feature matrix set and the support feature matrix set) in this application makes full use of multi-level feature information, captures the scale and complexity of different images, and accurately identifies the subtle differences between image categories. Multiple matrix prototypes are generated through multi-prototype learning to improve the adaptability to complex intra-class differences, and the key subspace is extracted through singular value decomposition, which effectively alleviates the problem of small inter-class differences and large intra-class differences in SAR images and reduces the risk of misclassification. In addition, the Softmax function is used to calculate the classification probability, normalize the output results, improve the classification credibility and facilitate subsequent data processing and application.

[0165] The above is a detailed introduction to a SAR image classification method, device, equipment and storage medium provided by the present application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can refer to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

[0166] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0167] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. A SAR image classification method, characterized in that: The method comprises: Acquire a synthetic aperture radar (SAR) data set to be classified, and perform feature extraction on the SAR data set to be classified to obtain an embedded feature matrix set; the embedded feature matrix set includes a query feature matrix set and a support feature matrix set; the query feature matrix set and the support feature matrix set have the same category of images; the query feature matrix set and the support feature matrix set both include multiple feature matrices; Performing multi-prototype learning on the feature matrix in the support feature matrix set to obtain multiple matrix prototypes; Subtracting each feature matrix in the supporting feature matrix set from the multiple matrix prototypes in turn to obtain multiple target feature matrices; the target feature matrices have a corresponding relationship with columns of feature matrices in the supporting feature matrix set; Performing singular value decomposition on the plurality of target feature matrices respectively to obtain a plurality of subspaces; each image category corresponds to the plurality of subspaces; Utilizing the Softmax function in combination with each image category corresponding to a plurality of subspaces, the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs is calculated; The query feature matrix set and the feature matrix in the support feature matrix set are exchanged to obtain a new query feature matrix set and a new support feature matrix set, and all the above steps are performed again using the new query feature matrix set and the new support feature matrix set until the probability that the feature matrix in the new query feature matrix set belongs to the image category to which it belongs is obtained.

2. The method according to claim 1, characterized in that The method of using the Softmax function in combination with each image category corresponding to a plurality of subspaces to calculate the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs includes: The projection distances of each feature matrix in the query feature matrix set to the corresponding multiple subspaces are calculated respectively; the feature matrix in the query feature matrix set has a corresponding relationship with the multiple subspaces in the same image category; In the query feature matrix set, the projection distances of the plurality of subspaces corresponding to the feature matrix are averaged to obtain an average distance; the average distance indicates the distance from the feature matrix in the query feature matrix set to the image category to which it belongs; The Softmax function is combined with the average distance to calculate the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs.

3. The method according to claim 1, characterized in that The performing multi-prototype learning on the feature matrix in the support feature matrix set to obtain multiple matrix prototypes includes: Flattening all feature matrices in the support feature matrix set one by one to obtain a plurality of flattened matrices; Based on the multiple image categories supported by the supporting feature matrix set, the multiple flattened matrices are divided into image categories to obtain multiple similar feature matrix sets; the multiple flattened matrices contained in each similar feature matrix set all belong to the same image category; Performing feature aggregation on a plurality of flattened matrices in the same set of similar feature matrices to obtain an aggregation matrix of each set of similar feature matrices; Based on the number of matrix partitions, S matrix prototypes of the clustering matrix are generated; S is the number of partitions; and S is a positive integer greater than 1.

4. The method according to claim 3, characterized in that The step of generating S matrix prototypes of the clustering matrix based on the number of matrix partitions includes: Based on the number of partitions, the clustering matrix is ​​partitioned by columns to obtain S partition matrices; each clustering matrix corresponds to S partition matrices; and the partition matrices belonging to the same column of each clustering matrix have a one-to-one correspondence; The partition matrices with one-to-one correspondence are averaged to obtain S matrix prototypes.

5. The method according to claim 2, characterized in that: The respectively calculating the projection distances of each feature matrix in the query feature matrix set to the corresponding multiple subspaces includes: According to the projection formula, each feature matrix in the query feature matrix set is projected onto the corresponding subspace to obtain multiple projection matrices of each feature matrix in the query feature matrix set; the projection formula is y=p T ·(f-μ), where y is the projection matrix; f is the feature matrix in the query feature matrix set; p is any one of the subspaces corresponding to the feature matrix in the query feature matrix set; p T is the transpose of p; μ is the matrix prototype; Based on the projection matrix and the distance calculation formula, the projection distance from each feature matrix in the query feature matrix set to the corresponding subspace is calculated; the distance calculation formula is d=||f-μ-p·y||.

6. The method according to claim 1, characterized in that The step of extracting features from the SAR data set to be classified to obtain an embedded feature matrix set includes: The pre-trained feature extractor is used to extract features from the SAR data set to be classified to obtain the embedded feature matrix set.

7. A SAR image classification device, characterized in that: The device comprises: An acquisition unit, used for acquiring a SAR data set to be classified; A feature extraction unit is used to extract features from the SAR data set to be classified to obtain an embedded feature matrix set; the embedded feature matrix set includes a query feature matrix set and a support feature matrix set; the query feature matrix set is consistent with the category of the image included in the support feature matrix set; the query feature matrix set and the support feature matrix set both include multiple feature matrices; A multi-prototype learning unit, used for performing multi-prototype learning on the feature matrix in the support feature matrix set to obtain multiple matrix prototypes; A first calculation unit is used to sequentially subtract each feature matrix in the support feature matrix set from the multiple matrix prototypes to obtain multiple target feature matrices; the target feature matrices have a one-to-one correspondence with the feature matrices in the support feature matrix set; A singular value decomposition unit is used to perform singular value decomposition on the plurality of target feature matrices to obtain a plurality of subspaces; each image category corresponds to the plurality of subspaces; The second calculation unit is used to calculate the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs by using the Softmax function in combination with the multiple subspaces corresponding to each image category; A feature matrix exchange unit, used for exchanging the query feature matrix set with the feature matrix in the support feature matrix set to obtain a new query feature matrix set and a new support feature matrix set; The repetitive execution unit is used to execute all the above steps again using the new query feature matrix set and the new support feature matrix set until the probability that the feature matrix in the new query feature matrix set belongs to the image category to which it belongs is obtained.

8. The device according to claim 7, characterized in that The second calculation unit specifically includes: A third calculation unit is used to respectively calculate the projection distance of each feature matrix in the query feature matrix set to the corresponding multiple subspaces; the feature matrix in the query feature matrix set has a corresponding relationship with the multiple subspaces in the same image category; A fourth calculation unit is used to average the projection distances of the plurality of subspaces corresponding to the feature matrix in the query feature matrix set to obtain an average distance; the average distance indicates the distance from the feature matrix in the query feature matrix set to the image category to which it belongs; The fifth calculation unit is used to calculate the probability that the feature matrix in the query feature matrix set belongs to the image category to which it belongs by using the Softmax function in combination with the average distance.

9. A SAR image classification device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the SAR image classification method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the SAR image classification method according to any one of claims 1 to 6.

Images