Multi-spectral target identification method based on double-branch multi-view spatial-spectral feature interaction

Abstract

The invention discloses a multi-spectral target recognition method based on double-branch multi-view spatial-spectral feature interaction, which comprises the following steps: performing principal component analysis operation on a multi-spectral image to perform spectral-spectral dimension reduction processing, and extracting a three-dimensional image block as input data by using pixel-level segmentation operation; capturing shallow spectrum-space joint features of the input data; extracting local and global spectrum-space joint features of the input data by using a CNN (Convolutional Neural Network); extracting local and global correlation characteristics between a spectrum band and a space pixel by utilizing Transform; the self-adaptive interaction of local and global spectrum-space joint features captured by CNN and Transform is fused in an early stage, and interdependent attention with more discriminant characteristics is generated; and an output module is utilized to obtain a multi-spectrum target recognition result through mutual dependence on attention.

Description

Technical Field

[0001] The present invention belongs to the field of image processing and multi-spectral target recognition, and in particular relates to a multi-spectral target recognition method based on dual-branch multi-viewing angle spatial spectrum feature interaction. Background Art

[0002] Multi-spectral images organically combine high spatial and high spectral resolution, remote sensing macro and micro, spatial information and spectral information. They are no longer traditional two-dimensional images, but three-dimensional data cubes. The number of spectral bands in multi-spectral images can range from a few to hundreds, and the imaging range covers the ultraviolet, visible light, near-infrared to mid-infrared electromagnetic spectrum regions, with the characteristics of wide spectral coverage and large number. The fine spectral information of multi-spectral images can reflect the intrinsic physical and chemical properties of targets that cannot be detected by the naked eye and ordinary optical sensors, as well as the subtle differences between different targets; the fine spatial information can reflect the external quality characteristics of the target, such as the physical position, structural relationship, and shape proportion, and has the characteristics of "space-spectrum integration", providing rich data support for target identification, making target information acquisition more comprehensive, and beneficial to improving the accuracy of target identification.

[0003] At present, there are two main methods for multi-spectral target recognition: 1) Traditional multi-spectral target recognition methods use feature engineering to reduce the dimension of multi-spectral images and obtain features or spectra with discriminative characteristics, and transfer these features or spectra to classifiers for optimization training. However, traditional multi-spectral target recognition methods rely heavily on manual features or shallow descriptions. Most manual features or shallow descriptions are usually designed for specific tasks, and they need to rely on experienced experts to design feature extraction methods during the parameter setting stage, which makes these methods only applicable to certain specific scenarios, limiting their applicability to different scenarios; 2) Multi-spectral target recognition methods based on deep learning simulate the information processing structure of the human brain by building multiple hierarchical network structures, and automatically learn the raw data through training methods to obtain the information characteristics of these data. These methods can automatically learn important features from the raw data using hierarchical structures, and gradually abstract them into higher-level feature representations without the need for manual feature engineering. They have strong generalization capabilities and universal characteristics and are strong candidates for the development of multi-spectral target recognition methods.

[0004] However, due to the essential differences in structural paradigms, there is an insurmountable perceptual gap between the spectral-spatial joint features captured by different networks. In addition, the multi-spectral target recognition method based on deep learning focuses on improving the recognition accuracy by increasing the depth or width of the network model, which causes the spectral-spatial joint features with discriminative characteristics to disappear as the model becomes deeper or wider during the training process. Therefore, a special multi-spectral target recognition network model is needed to solve the above problems. Summary of the invention

[0005] In view of the shortcomings of the existing multi-spectral target recognition methods based on deep learning, the present invention provides a multi-spectral target recognition method based on dual-branch multi-view spatial-spectral feature interaction. This method gives full play to the complementary advantages of CNN and Transformer networks in structural paradigms, and adopts an early fusion strategy to realize the adaptive interaction of local and global spectral-spatial joint features captured by CNN and Transformer dual networks, thereby obtaining multi-spectral target recognition results.

[0006] The technical solution adopted by the present invention is as follows: a multi-spectral target recognition method based on dual-branch multi-view spatial spectrum feature interaction includes the following six steps:

[0007] Step 1: Perform principal component analysis on the multi-spectral image to reduce the spectral band dimension, and use pixel-level segmentation to extract three-dimensional image blocks as input data;

[0008] Step 2: Capture shallow spectral-spatial joint features of input data;

[0009] Step 3: Use CNN to extract local and global spectral-spatial joint features of input data;

[0010] Step 4: Use Transformer to extract local and global correlation characteristics between spectral bands and spatial pixels;

[0011] Step 5: The symmetric multi-dimensional awareness fusion mechanism uses the early fusion strategy to early fuse the adaptive interaction of local and global spectral-spatial joint features captured by CNN and Transformer to generate more discriminative interdependent attention;

[0012] Step 6: Use the output module to obtain multi-spectral target recognition results through interdependent attention.

[0013] The advantages and beneficial effects of the present invention compared with the prior art are:

[0014] The CNN designed in the present invention utilizes multiple spectral-spatial feature extraction modules and cross-layer transmission to locally and globally interpret multi-spectral images, effectively reducing the perception difference between CNN and Transformer.

[0015] The Transformer designed in the present invention integrates the distortion invariance of CNN and utilizes multiple composite encoder modules and cross-layer transmission to locally and globally encode the correlation characteristics between spectral bands and spatial pixels.

[0016] The symmetric multi-dimensional consciousness fusion mechanism designed in the present invention obtains the interdependent attention of CNN and Transformer, effectively deepening the cross-dimensional interaction of the two networks, thereby avoiding the loss of important information. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a specific flow chart of the multi-spectral target recognition method based on dual-branch multi-view spatial spectrum feature interaction of the present invention;

[0018] Figure 2 It is a structural diagram of a multi-spectral target recognition method based on dual-branch multi-view spatial spectrum feature interaction of the present invention;

[0019] Figure 3 This is a structural diagram of the symmetrical multi-dimensional consciousness fusion mechanism of the present invention;

[0020] Figure 4 The experimental data set used in this invention;

[0021] Figure 5 Figure 1 is an experimental result diagram of the present invention, where (a) shows the truth map, (b) shows the experimental result of 2DCNN (two-dimensional convolutional neural network), (c) shows the experimental result of 3DCNN (three-dimensional convolutional neural network), (d) shows the experimental result of HybridSN (hybrid three-dimensional-two-dimensional convolutional neural network), (e) shows the experimental result of RSSAN (residual spectral-spatial attention network), (f) shows the experimental result of DMCN (end-to-end multi-layer hybrid attention network), (g) shows the experimental result of MSDAN (multi-scale densely connected attention network), (h) shows the experimental result of ViT (visual Transformer), (i) shows the experimental result of Spectralformer (spectral Transformer), (j) shows the experimental result of SSFTT (spectral-spatial feature tagging Transformer), (k) shows the experimental result of GAHT (group awareness hierarchical Transformer), (l) shows the experimental result of CDCformer (densely connected Transformer based on cascaded convolution), (m) The experimental results of DLMFI of the present invention are shown. DETAILED DESCRIPTION

[0022] The specific flow chart and structure diagram of the multi-spectral target recognition method based on dual-branch multi-view spatial spectrum feature interaction of the present invention are as follows: Figure 1 and Figure 2 The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0023] like Figure 1As shown in FIG. 1 , the multi-spectral target recognition method based on dual-branch multi-view spatial spectrum feature interaction realizes multi-spectral target recognition through the following six steps:

[0024] Step 1: Perform principal component analysis on the multi-spectral image to reduce the spectral band dimension, and use pixel-level segmentation to extract three-dimensional image blocks (consisting of the target pixel and its surrounding neighborhood pixels) as input data;

[0025] Step 2: Capture shallow spectral-spatial joint features of input data;

[0026] Step 3: Use CNN to extract local and global spectral-spatial joint features of the input data;

[0027] Step 4: Use Transformer to extract local and global correlation characteristics between the spectral band and the spatial pixels of the 3D image block;

[0028] Step 5: Use the symmetric multi-dimensional awareness fusion mechanism to fuse the local and global correlation features captured by CNN and Transformer at an early stage to generate more discriminative interdependent attention;

[0029] Step 6: Use the output module to obtain multi-spectral target recognition results through interdependent attention.

[0030] In one embodiment, in step 1, the multi-spectral image is subjected to spectral band dimension reduction processing using principal component analysis, including: removing high-correlation spectral bands; extracting target pixels and their neighboring pixels to form a three-dimensional image block using pixel-level segmentation operation as input data of the present invention; .

[0031] In one embodiment, the initialization module in step 2 is composed of two kernels of size and It consists of a two-dimensional convolutional layer, two batch normalization layers, and two ReLU activation functions to obtain input data The shallow spectral-spatial joint features of are calculated as follows:

[0032] (1)

[0033] The meanings of the symbols in formula (1) are:

[0034] : Input data;

[0035] : The middle layer outputs the result;

[0036] : Initialize module output results;

[0037] : ReLU activation function;

[0038] : Batch normalization layer;

[0039] : The kernel size is A two-dimensional convolutional layer;

[0040] : The kernel size is A two-dimensional convolutional layer.

[0041] In one embodiment, the CNN described in step 3 is composed of three spectral-spatial feature extraction modules (SSFEMS) and a cross-layer transfer operation, which is used to mine local and global spectral-spatial joint features. The calculation process is as follows:

[0042] (2)

[0043] The meanings of the symbols in formula (2) are:

[0044] : Initialize module output results;

[0045] : Dimension reshaping operation;

[0046] : Spectral-spatial feature extraction module;

[0047] : The output result of the first spectral-spatial feature extraction module;

[0048] : Cross-layer transfer operation;

[0049] : The output result of the first cross-layer transfer operation;

[0050] : The output result of the second spectral-spatial feature extraction module;

[0051] : The second cross-layer transfer operation outputs the result;

[0052] : Output result of the third spectral-spatial feature extraction module;

[0053] : The output result of the third cross-layer transfer operation;

[0054] : CNN branch output results;

[0055] : ReLU activation function;

[0056] : Batch normalization layer;

[0057] :The kernel is A 2D convolutional layer of size .

[0058] In one embodiment, the Transformer described in step 4 is composed of four composite encoders (CombinedEncoders, CEncoders) and a cross-layer transfer operation, such as Figure 3 As shown, it is used to capture the local and global correlation characteristics between spectral bands and spatial pixels. The calculation process is as follows:

[0059] (3)

[0060] The meanings of the symbols in formula (3) are:

[0061] : Initialize module output results;

[0062] : Dimension reshaping operation;

[0063] : Cross-layer transfer operation;

[0064] : Image block operation;

[0065] : Composite encoder;

[0066] : The first composite encoder output result;

[0067] : The first cross layer transfers the output result;

[0068] : The second composite encoder output result;

[0069] : The second cross layer transfers the output result;

[0070] : The output result of the third composite encoding module;

[0071] : The third cross layer transfers the output result;

[0072] : The fourth composite coding module outputs the result;

[0073] : The fourth cross layer transfers the output result;

[0074] : ReLU activation function;

[0075] : Batch normalization layer;

[0076] : Transformer branch output result.

[0077] In one embodiment, the symmetric multi-dimensional consciousness fusion mechanism described in step 5 utilizes the early fusion strategy to early fuse the adaptive interaction of local and global spectral-spatial joint features captured by the CNN and Transformer dual networks to generate more discriminative interdependent attention, and the calculation process is as follows:

[0078] Step a: Features of CNN branch output And the characteristics of the Transformer network branch output The global maximum pooling operation and the global average pooling operation are performed along the row direction and the column direction respectively, so as to obtain the correlation of the three dimensions of height, width and spectrum. The calculation formula is as follows:

[0079] (4)

[0080] The meanings of the symbols in formula (4) are:

[0081] : and of high;

[0082] : and width;

[0083] : The output result of performing the global maximum pooling operation along the row direction;

[0084] : The output result of performing the global maximum pooling operation along the row direction;

[0085] : The output result of performing the global maximum pooling operation along the column direction;

[0086] : Output of global max pooling along the column direction

[0087] : The output result of performing global average pooling operation along the row direction;

[0088] : The output result of performing global average pooling operation along the row direction;

[0089] : The output of the global average pooling operation along the column direction;

[0090] : Output of a global average pooling operation along the column direction.

[0091] Step b: Aggregate the eight feature maps obtained in step a through a cascade operation to generate global cross-dimensional awareness information. The calculation formula is as follows:

[0092] (5)

[0093] The meanings of the symbols in formula (5) are:

[0094] : Cascade operation;

[0095] : Cascade result of global average pooling and global maximum pooling output features along the row direction;

[0096] : Cascade result of global average pooling and global maximum pooling output features along the row direction;

[0097] : and Cascading results;

[0098] : The cascade result of global average pooling and global maximum pooling output features along the column direction;

[0099] : The cascade result of global average pooling and global maximum pooling output features along the column direction;

[0100] : and Cascading results;

[0101] : and Cascading results.

[0102] Step c: Global cross-dimensional awareness information Perform custom transformations to further refine , the calculation formula is as follows:

[0103] (6)

[0104] The meanings of the symbols in formula (6) are:

[0105] : Batch normalization layer;

[0106] : The kernel size is A two-dimensional convolutional layer;

[0107] : Customized conversion operations;

[0108] : Element-wise addition operation;

[0109] : Fine-grained global cross-dimensional awareness information.

[0110] Step d: Transform along the height and width dimensions Divided into two different feature maps, namely and ,right and Perform additional transformation operations and element-wise multiplication operations to obtain the correlation between high and wide dimensions. The calculation formula is as follows:

[0111] (7)

[0112] The meanings of the symbols in formula (7) are:

[0113] : Feature partitioning operation;

[0114] : Additional conversion operations;

[0115] : Element-wise multiplication operation;

[0116] : Correlation between high and wide dimensions.

[0117] Step e: Utilization and Reweighting along high, wide, and spectral dimensions and Acquire multidimensional consciousness characteristics and , and use element-wise addition to , , and Aggregate to obtain more comprehensive and discriminative multi-dimensional consciousness fusion features , the calculation formula is as follows:

[0118] (8)

[0119] The meanings of the symbols in formula (8) are:

[0120] : Element-wise multiplication operation;

[0121] : Element-wise addition operation.

[0122] In one embodiment, the multi-spectral target recognition result is obtained by using the output module in step 6, wherein the output module is composed of a two-dimensional global average pooling, three fully connected layers and a softmax function, and the calculation formula is as follows:

[0123] (9)

[0124] The meanings of the symbols in formula (9) are:

[0125] : 2D global average pooling;

[0126] : Fully connected layer;

[0127] : Multi-spectral target recognition results.

[0128] The above method is composed of a convolutional neural network (CNN) and a transformer dual-branch network and an early fusion strategy (symmetric multi-dimensional awareness fusion mechanism), which gives full play to the complementary advantages of CNN and Transformer networks in structural paradigms, and adopts an early fusion strategy to achieve the adaptive interaction of local and global spectral-spatial joint features captured by the CNN and Transformer dual networks, thereby obtaining multi-spectral target recognition results. Specifically, CNN uses multiple spectral-spatial feature extraction modules and cross-layer transmission to locally and globally interpret multi-spectral images, effectively reducing the perception difference between CNN and Transformer; Transformer integrates the distortion invariance of CNN, and uses multiple composite Encoder modules and cross-layer transmission to locally and globally encode the correlation characteristics between spectral bands and spatial pixels; in addition, the present invention adopts a symmetric multi-dimensional awareness fusion mechanism to obtain the mutual dependence of CNN and Transformer attention, effectively deepening the cross-dimensional interaction of the two networks, thereby avoiding the loss of important information.

[0129] Example:

[0130] 1) Experimental conditions: The computer used in the experiment is NVIDIA RTX 2080Ti GPU, the memory is 16.0 GB, and the programming platform is Tensorflow2.1.0. The size of the dataset used in the experiment is , with 200 spectral bands and 16 target categories.

[0131] 2) Experimental content:

[0132] The method proposed in the present invention is compared with the existing multi-spectral target recognition algorithms, among which the commonly used algorithms mainly include 2DCNN (two-dimensional convolutional neural network), 3DCNN (three-dimensional convolutional neural network), HybridSN (hybrid three-dimensional-two-dimensional convolutional neural network), RSSAN (residual spectral-spatial attention network), DMCN (end-to-end multi-layer hybrid attention network), MSDAN (multi-scale densely connected attention network), ViT (visual Transformer), Spectralformer (spectral Transformer), SSFTT (spectral-spatial feature tagging Transformer), GAHT (group awareness hierarchical Transformer), and CDCformer (densely connected Transformer based on cascaded convolution).

[0133] Figure 4 The experimental data set used in the present invention is given. Figure 5The visual results of 12 methods on the experimental data are given, where (a) shows the truth map, (b) shows the experimental results of 2DCNN (two-dimensional convolutional neural network), (c) shows the experimental results of 3DCNN (three-dimensional convolutional neural network), (d) shows the experimental results of HybridSN (hybrid three-dimensional-two-dimensional convolutional neural network), (e) shows the experimental results of RSSAN (residual spectral-spatial attention network), (f) shows the experimental results of DMCN (end-to-end multi-layer hybrid attention network), (g) shows the experimental results of MSDAN (multi-scale densely connected attention network), (h) shows the experimental results of ViT (visual Transformer), (i) shows the experimental results of Spectralformer (spectral Transformer), (j) shows the experimental results of SSFTT (spectral-spatial feature tagging Transformer), (k) (l) shows the experimental results of GAHT (Group Awareness Hierarchical Transformer), (m) shows the experimental results of DLMFI (Densely Connected Transformer Based on Cascaded Convolution), and (d) shows the experimental results of DLMFI of the present invention. The experimental results show that the visual result graph of the present invention is the clearest and cleanest, containing more detailed information and less noise; except for the present invention, the visual result graphs of other methods contain salt and pepper noise (such as 2DCNN) or the edges tend to be smooth (such as HybridSN, MSDAN), which proves the superiority and robustness of the present method.

[0134] This experiment uses overall accuracy (OA), average accuracy (AA) and Kappa coefficient as evaluation indicators to reflect the target recognition ability and accuracy of the algorithm. Table 1 shows the OA, AA and Kappa of the existing multi-spectral target recognition method and the present invention (DLMFI shown in Table 1). The experimental results show that the present invention obtains the best OA, AA and Kappa on the experimental data set, indicating that the target recognition ability of the present invention is better than that of the existing multi-spectral target recognition method.

[0135] Table 1

[0136]

[0137] The parts not described in detail in the present invention belong to the known technologies in the technical field. The above description is only a specific example of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-spectral target recognition method based on dual-branch multi-view spatial spectrum feature interaction, characterized in that: include: Step 1: Perform principal component analysis on the multi-spectral image to reduce the spectral band dimension, and use pixel-level segmentation to extract three-dimensional image blocks as input data; Step 2: Capture shallow spectral-spatial joint features of input data; Step 3: Use CNN to extract local and global spectral-spatial joint features of the input data; Step 4: Use Transformer to extract local and global correlation characteristics between spectral bands and spatial pixels; Step 5: The symmetric multi-dimensional awareness fusion mechanism uses the early fusion strategy to early fuse the adaptive interaction of local and global spectral-spatial joint features captured by CNN and Transformer to generate more discriminative interdependent attention; Step 6: Use the output module to obtain multi-spectral target recognition results through interdependent attention.

2. The multi-spectral target recognition method based on dual-branch multi-view spatial spectrum feature interaction according to claim 1 is characterized in that: In step 1, the principal component analysis operation is performed on the multi-spectral image to reduce the spectral band dimension, including: removing the high-correlation spectral band; using the pixel-level segmentation operation to extract the target pixel and its neighboring pixels to form a three-dimensional image block as the input data. .

3. The multi-spectral target recognition method based on dual-branch multi-view spatial spectrum feature interaction according to claim 2 is characterized in that: The initialization module described in step 2 is composed of two kernels of size and It consists of a two-dimensional convolutional layer, two batch normalization layers, and two ReLU activation functions to obtain input data The shallow spectral-spatial joint features of are calculated as follows: (1) The meanings of the symbols in formula (1) are: : Input data of the present invention; : The middle layer outputs the result; : Initialize module output results; : ReLU activation function; : Batch normalization layer; : The kernel size is A two-dimensional convolutional layer; : The kernel size is A two-dimensional convolutional layer.

4. The multi-spectral target recognition method based on dual-branch multi-view spatial spectrum feature interaction according to claim 1 is characterized in that: The CNN described in step 3 is composed of three spectral-spatial feature extraction modules and cross-layer transfer operations to mine local and global spectral-spatial joint features. The calculation process is as follows: (2) The meanings of the symbols in formula (2) are: : Initialize module output results; : Dimension reshaping operation; : Spectral-spatial feature extraction module; : The output result of the first spectral-spatial feature extraction module; : Cross-layer transfer operation; : The output result of the first cross-layer transfer operation; : The output result of the second spectral-spatial feature extraction module; : The second cross-layer transfer operation outputs the result; : Output result of the third spectral-spatial feature extraction module; : The output result of the third cross-layer transfer operation; : CNN branch output results; : ReLU activation function; : Batch normalization layer; :The kernel is A 2D convolutional layer of size .

5. The multi-spectral target recognition method based on dual-branch multi-view spatial spectrum feature interaction according to claim 1 is characterized in that: The Transformer described in step 4 consists of four composite encoders and cross-layer transfer operations to capture the local and global correlation characteristics between spectral bands and spatial pixels. The calculation process is as follows: (3) The meanings of the symbols in formula (3) are: : Initialize module output results; : Dimension reshaping operation; : Cross-layer transfer operation; : Image block operation; : Composite encoder; : The first composite encoder output result; : The first cross layer transfers the output result; : The second composite encoder output result; : The second cross layer transfers the output result; : The output result of the third composite encoding module; : The third cross layer transfers the output result; : The fourth composite coding module outputs the result; : The fourth cross layer transfers the output result; : ReLU activation function; : Batch normalization layer; : Transformer branch output result.

6. The multi-spectral target recognition method based on dual-branch multi-view spatial spectrum feature interaction according to claim 1 is characterized in that: The symmetric multi-dimensional awareness fusion mechanism described in step 5 uses the early fusion strategy to early fuse the adaptive interaction of local and global spectral-spatial joint features captured by CNN and Transformer to generate more discriminative interdependent attention. The calculation process is as follows: Step a: Features of CNN output And the features of Transformer output The global maximum pooling operation and the global average pooling operation are performed along the row direction and the column direction respectively, so as to obtain the correlation of the three dimensions of height, width and spectrum. The calculation formula is as follows: (4) The meanings of the symbols in formula (4) are: : sum Target height; : and width; : The output result of performing global average pooling operation along the row direction; : The output result of performing global average pooling operation along the row direction; : The output of the global average pooling operation along the column direction; : The output of the global average pooling operation along the column direction; : The output result of performing the global maximum pooling operation along the row direction; : The output result of performing the global maximum pooling operation along the row direction; : The output result of performing the global maximum pooling operation along the column direction; : The output result of performing the global maximum pooling operation along the column direction; Step b: Aggregate the eight feature maps obtained in step a through a cascade operation to generate global cross-dimensional awareness information. The calculation formula is as follows: (5) The meanings of the symbols in formula (5) are: : Cascade operation; : Cascade result of global average pooling and global maximum pooling output features along the row direction; : Cascade result of global average pooling and global maximum pooling output features along the row direction; : and Cascading results; : The cascade result of global average pooling and global maximum pooling output features along the column direction; : The cascade result of global average pooling and global maximum pooling output features along the column direction; : and Cascading results; : and Cascading results; Step c: Global cross-dimensional awareness information Perform custom transformations to further refine , the calculation formula is as follows: (6) The meanings of the symbols in formula (6) are: : Batch normalization layer; : The kernel size is A two-dimensional convolutional layer; : Customized conversion operations; : Element-wise addition operation; : Fine-grained global cross-dimensional awareness information; Step d: Transform along the height and width dimensions Divided into two different feature maps, namely and ,right and Perform additional transformation operations and element-wise multiplication operations to obtain the correlation between high and wide dimensions. The calculation formula is as follows: (7) The meanings of the symbols in formula (7) are: : Feature partitioning operation; : Additional conversion operations; : Element-wise multiplication operation; : Correlation between high and wide dimensions; Step e: Utilization and Reweighting along high, wide, and spectral dimensions and Gaining multidimensional consciousness characteristics and , and use element-wise addition to , , and Aggregate to obtain more comprehensive and discriminative multi-dimensional consciousness fusion features , the calculation formula is as follows: (8) The meanings of the symbols in formula (8) are: : Element-wise multiplication operation; : Element-wise addition operation.

7. The multi-spectral target recognition method based on dual-branch multi-view spatial spectrum feature interaction according to claim 1 is characterized in that: The multi-spectral target recognition result is obtained by using the output module described in step 6, where the output module is composed of a two-dimensional global average pooling, three fully connected layers and a softmax function, and the calculation formula is as follows: (9) The meanings of the symbols in formula (9) are: : 2D global average pooling; : Fully connected layer; : Multi-spectral target recognition results.

Images