Complex target identification method based on deep twin self-attention network

Abstract

The invention discloses a complex target recognition method based on a deep twin self-attention network, and relates to the technical field of target recognition methods. According to the method, a twin network based on a sliding window multi-head self-attention transformation mechanism is applied, the global modeling capability of a feature extraction network and the acquisition efficiency of long-distance semantic information are enhanced, and the bottom-layer feature extraction capability and the abstract effect of advanced semantic information are improved; the twin network personalized modeling improves the feature segmentation effect; the whole network forms a U-shaped structure, a local symmetric encoder and decoder structure is adopted, global context information is supplemented, and semantic information space association is enhanced; a residual link is added, so that the deep network is trained efficiently, and the recognition precision is enhanced; and adaptive image feature scale input processing is added, and adaptive scale transformation is performed on the input image, so that an accurate end-to-end semantic segmentation result is realized, and multi-scale input is realized to enhance the generalization ability and accuracy of a network model.

Description

Technical Field

[0001] The present invention relates to the technical field of target recognition methods, and in particular to a complex target recognition method based on a deep twin self-attention network. Background Art

[0002] In complex image scenes, the visual space of the image has different spectral characteristics due to the phenomenon of different objects with the same spectrum or different spectra for the same object. The spectral characteristics of different objects overlap with each other, making the intra-class variance larger and the inter-class variance smaller. This will confuse the image feature details with high-level semantic information, which cannot be solved by relying solely on expert human eye recognition. Traditional algorithms, such as color clustering, cannot mine the deeper high-level semantic information behind the image, which results in a shallow understanding of the local and global aspects of the image and low efficiency in completing the task. Therefore, it is necessary to learn from artificial intelligence deep learning algorithms to complete the task and identify the high-level information contained in the image.

[0003] At the same time, with the development of digital vision technology, images in various fields are developing towards large data volumes and diverse types of information. Image recognition and semantic segmentation tasks have great application value in many fields such as land and resources surveys, geographic information surveying and mapping, autonomous driving navigation, and medical image detection. There is a practical need to accurately and effectively identify precise detail features from images and extract accurate semantic information while balancing information types.

[0004] Artificial intelligence algorithms can extract and analyze specific explicit features, abstract and summarize them into high-level semantic information, and systematize the process of specific extraction and abstraction, thereby enhancing efficiency and accuracy. At the same time, feature details will be lost in the process. The main task of the visual space is to segment and locate feature details, while the main task of the semantic space is to summarize and learn high-level conceptual information. This is the contradiction.

[0005] In a large number of complex image scenes, there will be problems such as imbalance between the underlying features and high-level semantic information of the image, confusion in approximate feature recognition, inaccurate local positioning and segmentation, neglect of small-scale features, insufficient exchange of global context information due to multi-scale changes, and processing of large sample data. These problems make the semantic segmentation task more challenging in terms of grasping the detailed features at multiple scales and the balance of semantic information.

[0006] The common complex target recognition method based on convolutional neural network currently on the market uses cascaded convolution and pooling operations for feature extraction. Since the size of the convolution kernel is usually small, such as 3×3 or 5×5, the receptive field of the convolution operation is limited, which is equivalent to each convolution operation being an operation on a very small area in the image or feature map. In the shallower convolution layer, only small-scale features can be extracted, and effective long-distance dependencies cannot be established. In the deeper convolution layer, global semantic information can be obtained, but some details are lost. In order to solve the above problems, the present invention proposes a complex target recognition method based on a deep twin self-attention network. Summary of the invention

[0007] The purpose of this invention is to propose a complex target recognition method based on a deep twin self-attention network to solve the following problems existing in the prior art: (1) Complex image scenes have features of different scales, and convolutional neural networks have difficulty balancing large-scale and small-scale features, which ultimately affects the accuracy of complex target recognition tasks; (2) The imbalance between the underlying features and high-level semantic information of the image leads to inaccurate local positioning and segmentation, and detailed features are ignored; (3) Unbalanced classification and recognition due to insufficient exchange of global context information and large sample data size.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions: A complex target recognition method based on a deep twin self-attention network specifically includes the following steps: S1. Separate the high-dimensional image information into two channel dimensions, and input each channel into the sub-encoding network for feature extraction. First, input the features of each channel into the convolution module of the twin encoding network to extract the image features. Then input the encoder (RSwinT-Encoder) with multiple residual module structures containing residual connections. Down modules, the input image is first scaled adaptively; S2: Input the image processed by S1 into a series of sliding window self-attention transformation modules (Swin Transformer) blocks to further extract and process features, and then restore the output feature map size after processing by the image size adaptation module (Resize); S3, the twin network hierarchical encoder generates feature maps of different depths by downsampling at each stage, and records the features in each encoding stage and the original image resolution features for the sub-encoding network with richer information input channels; S4, the encoding stage is over, the two sub-network branch channels are fused, and the feature maps extracted from each channel are fused and spliced; S5, input the fused image obtained in S4 into the decoder (RSwinT-Decoder) with multiple residual module structures Up modules containing residual connections. First, the image is upsampled and then gradually restored to the feature map of the input size; S6, uses feature fusion technology at each decoding stage, combining the high-resolution features of the hierarchical encoder in S3 and the upsampled features of the decoder (RSwinT-Decoder) in S5; S7, decoding stage, after the feature fusion in S6, repeat the operations described in S1-S2; S8, inputting the feature map of the final stage obtained in S7 into the segmentation head, classifying each pixel, dividing the pixels into predetermined categories, and obtaining a semantic segmentation recognition result; S9. Use the fusion loss function to calculate the semantic segmentation recognition results obtained in S8, and back-propagate to improve the network parameters.

[0009] Preferably, the S1 specifically includes the following contents: The input high-dimensional image is separated into two channels and input into the same sub-encoding network to form a twin network. In the sub-encoding network, the image is first input into a 2-layer CBL module for preliminary feature extraction. The encoder (RSwinT-Encoder) includes multiple Down modules with residual module structures containing residual connections. First, an adaptive scaleadaptive operation is performed on the input image to complete the adaptive interpolation calculation, and the image size is transformed into a common multiple of 2 and 7 to be suitable for subsequent sliding window self-attention mechanism calculations.

[0010] Preferably, S2 specifically includes the following contents: S2.1. The input image is divided into multiple small blocks, which are then flattened and converted into a vector of fixed dimension that can be processed through a linear layer, and position information is added to the feature vector of each block to ensure that the model can understand the spatial relationship between different blocks; the linearly embedded feature vector is further divided into multiple non-overlapping windows of fixed size, and self-attention calculation is performed independently within each window; S2.2: Keep the sequence with position encoding in S2.1 as the residual and input it into the normalization layer LN , for each element of the input sequence, W - MSA Multi-head self-attention calculation is added to the residual amount. The specific formula is expressed as: (1) In formula (1), Output sequence, x l-1 represents the input sequence; S2.3, retain the output obtained in S2.2 as the residual and input it into the normalization layer in turn LN , Multilayer Perceptron Network MLP , and then add it to the residual, the specific formula is expressed as: (2) In formula (2), x l represents the output sequence; S2.4, retain the output sequence in S2.3 as the residual and input it into the normalization layer LN , for each element of the input sequence, SW - SWA The moving window self-attention calculation is added to the residual amount. The specific formula is expressed as: (3) In formula (3), The output sequence represented by S2.5. The output obtained in S2.4 is used as the residual and input. The process is the same as S2.3. The specific formula is: (4) In formula (4), x l+1 represents the output sequence; The calculation method of self-attention calculation inside the window is as follows: (5) In formula (5), for each element of the input sequence, the query vector matrix, key vector matrix and value vector matrix are obtained by multiplying with different matrices. Q , K , V ∈ , B The value of is taken from the bias matrix ∈ (2M+1)(2M-1) , M 2 Indicates the number of patches in the window. d express Q or K The dimension of S2.6. After passing through a series of sliding window self-attention transformation modules (Swin Transformer), the image size adaptation module (Resize) is used to perform interpolation calculations to adaptively restore the image size to the input size, which is conducive to the design of the hierarchical feature extraction structure.

[0011] Preferably, S3 specifically includes the following contents: Feature downsampling is implemented through a convolutional layer with a stride of 2 and a 3×3 kernel, including a regularization layer and an activation function LeakRelu , the hierarchical encoder generates four feature maps of different depths of 1 / 2, 1 / 4, 1 / 8, and 1 / 16 by downsampling at each stage. And for the channels separated in S1 that contain richer information, the 1 / 2, 1 / 4, 1 / 8 features and the original image resolution features in each encoding stage of the encoding network are recorded as high-resolution features of the jump connection supplementary decoder.

[0012] Preferably, the S4 specifically includes the following contents: At the feature level, the feature maps from the two channel branches of the twin network are merged by splicing, and the similar characteristic features learned by the twin network are used to improve the segmentation performance, and it is highly interpretable when processing highly similar images; Preferably, the S5 specifically includes the following contents: Feature upsampling is performed through a convolutional layer with a stride of 2 and a 3×3 kernel, including a regularization layer and an activation function LeakRelu Each shallower layer has a width and height that is twice that of the previous layer, and a number of channels that is 1 / 2 of the previous layer, restoring feature maps at 1 / 8, 1 / 4, and 1 / 2 times the original resolution. The feature maps with restored resolution contain both rich detail feature information and high-level semantic information extracted from deep feature maps.

[0013] Preferably, the S6 specifically includes the following contents: Deep Features Figure 1 / 16 is restored to 1 / 8 of the size of the previous layer through upsampling, and is combined with the high-resolution features in the corresponding encoding stage Figure 1 / 8 Fusion, deep features Figure 1 / 8 is restored to 1 / 4 of the size of the previous layer through upsampling, and is matched with the high-resolution feature of the corresponding encoding stage Figure 1 / 4 Fusion, deep features Figure 1 / 4 is restored to 1 / 2 the size of the previous layer through upsampling, and is combined with the high-resolution features in the corresponding encoding stage Figure 1 / 2 Fusion, deep features Figure 1 / 2 is restored to the original image resolution size 1 through upsampling and fused with the selected and recorded high-resolution feature map in the encoding stage. This jump connection uses splicing rather than simple addition to ensure the integrity and richness of information during feature transmission and improve the accuracy of segmentation and edge details.

[0014] Preferably, the output resolution obtained by the segmentation head processing in S8 is the same as the original resolution. Figure 1 The value of each pixel is its category probability.

[0015] Preferably, the S9 specifically includes the following contents: Fusion of two loss functions can improve the robustness and generalization ability of the model, reduce overfitting, provide more accurate gradient signals, and help the model learn and adjust parameters better. At the same time, because Lovasz Loss has mathematically differentiable properties, it is easier to use in combination with other loss functions to further improve model performance. At the same time, when fusing two loss calculations, the weights between different objectives can be balanced by adjusting the weights between them. This allows the model to be flexibly optimized on different task indicators to meet actual needs. The loss function uses Soft Cross Entropy Loss and Lovasz Loss with a 1:1 weight fusion. Among them, soft cross entropy, that is, the cross entropy with label smoothing, will increase generalization. The calculation formula of soft cross entropy is as follows: (6) In formula (6), L SCE Represents the value of the Soft Cross Entropy Loss function; y i Indicates the label of the i-th category in the true label, set to soft label, y i ∈(0,1); P ( x i ) represents the probability of the i-th category predicted by the model; The meaning of Lovasz Loss is to gradually optimize the prediction results so that the prediction results are closer to the true labels. It not only focuses on correctly classified samples, but also considers boundary samples and misclassified samples. Therefore, it can handle the problems of imbalanced categories and imbalanced difficult and easy samples. The calculation formula is as follows: (7) (8) (9) (10) In formula (7), Δ Jc represents the loss function to be optimized; y* represents the ground truth; c represents the set of predicted error pixels; M c Indicates the set where the network segmentation results and labels do not match. M c The domain is {0,1} p, p Indicates the number of pixels; In formula (8), lovasz extension is used for smooth extension and is specifically implemented in multi-class segmentation; f i ( c ) indicates the c Probability value after class softmax; use scoring function f i ( c ) to construct a pixel errors m i ( c )vector; In formula (9), errors are used m ( c ) vector construction replaces Δ Jc The loss function; In formula (10), in order to optimize the evaluation of the mIoU index of all categories, the above average loss ( f ( c )).

[0016] Compared with the prior art, the present invention provides a complex target recognition method based on a deep twin self-attention network, which has the following beneficial effects: Compared with the common image semantic segmentation methods on the market, the present invention uses a new U-shaped symmetric codec structure based on the residual connection deep sliding window self-attention mechanism transformation construction, embeds scale adaptation, and combines fusion loss calculation to achieve long-term long-distance semantic information interaction, supplement contextual information, effectively balance the underlying features and high-level features, and greatly improve the prediction efficiency of the network. It not only solves the problem of input size limitation of the Swin Transformer sliding window self-attention mechanism, but also improves the generalization, accuracy and category balance indicators of the network, enhances the semantic space association without losing the feature space details, and balances the global and local recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The grid structure diagram mentioned in Example 1 of the present invention; Figure 2 Schematic diagram of important modules of the network structure mentioned in Example 1 of the present invention; Figure 3 This is a schematic diagram of the calculation of the self-attention mechanism mentioned in Example 1 of the present invention; Figure 4 This is a comparative qualitative experimental result diagram mentioned in Example 1 of the present invention. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0019] Embodiment 1: Based on the summary of the existing complex target recognition methods based on convolutional neural networks and symmetric encoding and decoding structure networks, this invention proposes an image semantic segmentation method based on a deep self-attention transform twin network. More specifically, this invention proposes a complex target recognition method based on a deep twin self-attention network. The overall framework adopts a U-shaped structure, combining the twin encoder network, the decoder frame and the improved Swin Transformer for feature extraction. The network comprehensively considers the global information and local feature relationship of the input data, uses the characteristics of the twin network to strengthen personalized modeling, further supplements the target positioning details, and establishes long-distance connections and dependencies between features, so that a wider range of contextual semantic information can be discovered, with stronger segmentation capabilities, specifically including the following contents: like Figure 1 , Figure 2 As shown in the figure, the U-shaped network based on the twin deep self-attention residual is mainly divided into the Down module in the encoder (RSwinT-Encoder) and the Up module in the decoder (RSwinT-Decoder). In addition, the twin network is innovatively used in the encoder (RSwinT-Encoder) to extract high-dimensional image features. Each stage is based on the improved SwinTransformer for feature extraction, and feature fusion is performed at the end of the encoding stage, and feature fusion is performed in the jump connection part. Finally, the segmentation head is input, and the result is to mark the semantic information for each pixel. Starting from the input picture, the specific steps are as follows: Step 1: Separate the high-dimensional image information into two channel dimensions. Each channel image is first input into a 2-layer CBL module, that is, a convolution layer with a convolution kernel of 3×3, a regularization layer, and an activation layer for preliminary feature extraction. The input image in the sub-encoding network then enters the encoder Down module. The encoder (RSwinT-Encoder) has multiple residual module structure Down modules with residual connections. First, the image is adaptively scaled. The adaptive interpolation calculation must transform the image size into a common multiple of 2 and 7, which meets the fixed size requirements of the Swin Transformer and is suitable for subsequent sliding window self-attention mechanism calculations.

[0020] Step 2: Since the input of Swin Transformer is a fixed one-dimensional sequence rather than a two-dimensional image, the image needs to be converted into an input sequence. At the same time, in order to reduce the computational complexity and avoid the input image sequence being too long, the input image is divided into several small blocks, which are flattened and converted into a fixed-dimensional vector that can be processed through a linear layer. Position information is added to the feature vector of each block to ensure that the model can understand the spatial relationship between different blocks. The feature vectors that have been linearly embedded and positionally embedded are further divided into multiple non-overlapping windows. The size of these windows is fixed, and self-attention calculations will be performed independently within each window. Figure 1 As shown, the structural residual is designed at the same time, the sequence with position encoding is retained as the residual amount and input into the normalization layer , for each element of the input sequence, The multi-head self-attention calculation, added with the residual, can be expressed as: (1) In formula (1), Output sequence, x l-1 represents the input sequence; Further, The output is retained as the residual and is fed into the normalization layer in turn. LN , Multilayer Perceptron Network MLP , and then add the residual to express the specific formula as follows: (2) Formula (2) Represents the output sequence.

[0021] Further, the output sequence x l Keep as residual and input into normalization layer LN , for each element of the input sequence, SW - MSA The moving window self-attention calculation is added to the residual amount. The specific formula is expressed as: (3) In formula (3), Represents the output sequence.

[0022] Furthermore, the output Reserved as the residual, the input process is also input into the normalization layer in turn LN , Multilayer Perceptron Network MLP , and then add it to the residual, the specific formula is expressed as: (4) In formula (4), x l+1 Represents the output sequence.

[0023] The calculation method of the above self-attention calculation inside the window is as follows: (5) In formula (5), for each element of the input sequence, the query vector matrix, key vector matrix and value vector matrix are obtained by multiplying with different matrices. Q , K , V ∈ B The value of is taken from the bias matrix ∈ (2M+1)(2M-1) , M 2 Indicates the number of patches in the window. d express Q or K The dimension of the self-attention calculation mechanism is as follows Figure 3 After a series of improved Swin Transformers, the image is restored to its input size through the Resize module for interpolation calculation, which is beneficial to the design of hierarchical feature extraction structure.

[0024] Step 3: Use the feature extraction network to extract feature maps of four different depths: 1 / 4, 1 / 8, 1 / 16, and 1 / 32, for feature map fusion. Feature downsampling is implemented through a convolutional layer with a step size of 2 and a convolution kernel of 3×3, which includes a regularization layer and an activation function. LeakRelu , the hierarchical encoder generates feature maps of four different depths of 1 / 2, 1 / 4, 1 / 8, and 1 / 16 by downsampling at each stage. The encoding stage cascades to generate 1 / 2, 1 / 4, and 1 / 8 features. For the sub-coding network containing richer information channels, its feature map of each depth will be retained as a high-resolution feature of the skip connection to supplement the decoder.

[0025] Step 4: At the feature level, the feature maps from the two channel branches of the twin encoding network are merged by splicing, and the similar characteristic features learned by the twin network are used to improve the segmentation performance and have high interpretability when processing highly similar images; Step 5: The deepest 1 / 16 feature map fused from step 4 is upsampled through a convolutional layer with a stride of 2 and a 3×3 convolution kernel, including a regularization layer and an activation function. LeakRelu, and then upsample layer by layer. Each shallower layer has a width and height twice that of the previous layer, and a number of channels 1 / 2 of the previous layer. The feature maps are restored at 1 / 8, 1 / 4, and 1 / 2 times, and the fused feature maps of each scale are also retained. The feature maps with restored resolution contain both rich detail feature information and high-level semantic information extracted from the deep feature maps.

[0026] Step 6: Deep Features Figure 1 / 16 is restored to 1 / 8 of the size of the previous layer through upsampling, and the high-resolution feature of the corresponding encoding stage is selected Figure 1 / 8 Fusion, deep features Figure 1 / 8 is restored to 1 / 4 of the size of the previous layer through upsampling, and the high-resolution feature of the corresponding encoding stage is selected Figure 1 / 4 Fusion, deep features Figure 1 / 4 is restored to 1 / 2 the size of the previous layer through upsampling, and the high-resolution feature of the corresponding encoding stage is selected Figure 1 / 2 Fusion, deep features Figure 1 / 2 is restored to the original image resolution size 1 through upsampling and fused with the high-resolution feature map of the corresponding encoding stage. This jump connection uses splicing rather than simple addition to ensure the integrity and richness of information during feature transmission, thereby improving segmentation accuracy and edge details; Step 7: After upsampling layer by layer, enter the improved Swin Transformer layer as in step 2, which includes adaptive size adjustment, a series of sliding window self-attention calculations, a series of residual connections, image restoration size, and feature and semantic information extraction; Step 8: Input the feature map of the last stage in step 6 into the convolutional layer segmentation head. The output resolution of the network is the same as the original Figure 1 The value of each pixel is its category probability.

[0027] Step 9: The network output is used to calculate the loss function for back propagation and to change the network parameters, including MLP parameters, self-attention matrix weight parameters, etc. Fusion of the two loss function calculations can improve the robustness and generalization ability of the model, reduce overfitting, provide more accurate gradient signals, and help the model learn and adjust parameters better. At the same time, because LovaszLoss has mathematically differentiable properties, it is easier to use in combination with other loss functions to further improve model performance. At the same time, when fusing the two loss calculations, the weights between different objectives can be balanced by adjusting the weights between them. This allows the degree of optimization of the model on different task indicators to be flexibly controlled to meet actual needs.

[0028] The loss function uses Soft Cross Entropy Loss and Lovasz Loss with a 1:1 weight fusion. Soft cross entropy, that is, the cross entropy with label smoothing, will increase generalization. The calculation formula of soft cross entropy is as follows: (6) In formula (6), L SCE The value of the Soft Cross Entropy Loss function; y i The label of the i-th category in the true label is set as soft label, y i ∈(0,1); P ( x i ) represents the probability of the i-th category predicted by the model.

[0029] The meaning of Lovasz Loss is to gradually optimize the prediction results so that the prediction results are closer to the true label. It not only focuses on correctly classified samples, but also considers boundary samples and misclassified samples, so it can handle the problems of class imbalance and imbalance of difficult and easy samples. The calculation formula is as follows: (7) (8) (9) (10) In formula (7), Δ Jc oss function; y* round truth; c represents the set of predicted error pixels; M c The network segmentation results and labels do not match the set, M c 0,1} p , p Indicates the number of pixels; In formula (8), lovasz extension is used for smooth extension and is specifically implemented in multi-class segmentation; f i ( c ) indicates the c Probability value after class softmax; use scoring function fi ( c ) to construct a pixel errors m i ( c )vector; In formula (9), errors are used m ( c ) Δ Jc The loss function; In formula (10), in order to optimize the evaluation of the mIoU index of all categories, the above average loss ( f ( c )).

[0030] In summary, compared with the previous convolutional feature extraction network and U-shaped network, the semantic segmentation efficiency of the present invention is improved, and the accuracy of segmentation and recognition results is improved, and it can be widely used in fields that require semantic segmentation, such as medicine and remote sensing. The twin network has model interpretability. By explaining the correlation between input and output, the essence of the segmented image can be understood. In addition, the twin network can be personalized for modeling, understand feature differences, and make personalized predictions based on these differences. By applying the sliding window multi-head self-attention transformation mechanism, the problem of shallow texture information and deep semantic information not being able to be taken into account at the same time is improved, the global modeling ability of the feature extraction network and the efficiency of obtaining long-distance semantic information are enhanced, and the underlying feature extraction ability and the abstract effect of high-level semantic information are improved. The overall network has a U-shaped structure and a locally symmetric encoder-decoder structure, which supplements global context information and strengthens the spatial association of semantic information. Residual links are added to efficiently train deep networks and enhance recognition accuracy. By adding adaptive image feature scale input processing, the input image is adaptively scaled to solve the redundant image preprocessing process caused by network input limitations, achieve end-to-end semantic segmentation accuracy, and realize multi-scale input to enhance the generalization ability and accuracy of the network model. The network uses a fusion loss function to simultaneously improve segmentation accuracy and recognition accuracy. In terms of application, the network targets the semantic segmentation task of complex scenes and solves the problems of inaccurate and unclear local positioning and segmentation, neglected small-scale feature recognition, and low efficiency in global category information recognition caused by the multi-scale features in the image and the imbalance between the underlying features and the high-level semantic information.

[0031] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A complex target recognition method based on deep twin self-attention network, characterized in that: The deep self-attention transformation network includes: The twin network can interpret high-dimensional image information to enhance individual feature modeling, improve segmentation performance, and enhance model generalization; A sliding window self-attention transformation module based on residual connections is used to enhance the network model's ability to recognize objects of different sizes and enhance the relevance of the global long-distance semantic information space; Image size adaptation module to solve the fixed limitation of sliding window self-attention mechanism; Locally symmetric encoder-decoder structure network based on hierarchical deep sliding window self-attention mechanism transformation; Multiple skip connections are used to directly connect the corresponding layers of the encoder and decoder to preserve and transfer high-resolution feature information; Fusion loss function to improve both recognition accuracy and classification balance; The method specifically comprises the following steps: S1, first separate the high-dimensional image information into two channel dimensions, and input each channel into the sub-encoding network for feature extraction; first, each channel feature is input into the convolution module of the twin encoding network to preliminarily extract the image features; then input into the encoder's multiple residual module structure Down module containing residual connections, and the input image is first scale-adaptive; S2, input the image processed by S1 into a series of sliding window self-attention transformation modules to further extract and process features, and then restore the output feature map size after processing by the image size adaptation module; S3, the twin network hierarchical encoder generates feature maps of different depths by downsampling at each stage, and records the features in each encoding stage and the original image resolution features for the sub-encoding network with richer information input channels; S4, the encoding stage is over, the two sub-network branch channels are fused, and the feature maps extracted from each channel are fused and spliced; S5, inputting the fused image obtained in S4 into a plurality of Up modules of a decoder including residual module structures of residual connections, and gradually restoring the image to a feature map of the input size after upsampling; S6, uses feature fusion technology at each decoding stage, combining the high-resolution features of the hierarchical encoder in S3 and the upsampled features of the decoder in S5; S7, decoding stage, after the feature fusion in S6, repeat the operations described in S1-S2; S8, inputting the feature map of the final stage obtained in S7 into the segmentation head, classifying each pixel, dividing the pixels into predetermined categories, and obtaining a semantic segmentation recognition result; S9. Use the fusion loss function to calculate the semantic segmentation recognition results obtained in S8, and back-propagate to improve the network parameters.

2. According to claim 1, a complex target recognition method based on deep twin self-attention network is characterized in that: The function of the Down module of the residual module structure in S1 is to adjust the image size, and use the sliding window self-attention transformation module to perform feature extraction and down sampling; S1 specifically includes the following contents: S1.1, separate the input high-dimensional image into two channels and input them into the same sub-encoding network to form a twin network; S1.2, adjust the Scale Adaptive module through adaptive size adjustment to meet the fixed size requirements of the sliding window self-attention transformation module; S1.3, input the image processed by S1.2 into the subsequent sliding window self-attention transformation module for calculation.

3. The complex target recognition method based on deep twin self-attention network according to claim 1, characterized in that: The S2 specifically includes the following contents: S2.1, split the image into multiple small blocks, flatten the small blocks and convert them into vectors of fixed dimension through a linear layer, and add position information to the feature vector of each small block to ensure that the model understands the spatial relationship between different blocks; S2.

2. After processing in S2.1, the sequence with position encoding information is retained as the residual and input into the normalization layer LN , for each element of the input sequence, multi-head self-attention calculation is performed and added to the residual. The specific formula is expressed as: （1） In formula (1), represents the output sequence, x l-1 represents the input sequence; S2.3, retain the output obtained in S2.2 as the residual and input it into the normalization layer in turn LN , Multilayer Perceptron Network MLP , and then add it to the residual, the specific formula is expressed as: （2） In formula (2), x l represents the output sequence; S2.4, retain the output sequence in S2.3 as the residual and input it into the normalization layer LN , for each element of the input sequence, a moving window self-attention calculation is performed and added to the residual. The specific formula is expressed as: （3） In formula (3), The output sequence represented by S2.

5. Use the output obtained in S2.4 as the residual and input, and repeat the operation described in S2.

3. The specific formula is: （4） In formula (4), x l+1 represents the output sequence; S2.

6. After a series of sliding window self-attention transformation modules, the image size is restored to the input size using the image size adaptation module to facilitate the design of the hierarchical feature extraction structure.

4. The complex target recognition method based on deep twin self-attention network according to claim 1, characterized in that: The S3 specifically includes the following contents: The twin network also implements feature downsampling through a convolutional layer with a stride of 2 and a convolution kernel of 3×3. Each deeper layer has a width and height of 1 / 2 of the previous layer, and the number of channels is twice that of the previous layer, forming feature maps of four different depths: 1 / 4, 1 / 8, 1 / 16, and 1 / 32. For images with more channels separated from high-dimensional images in S1, the input subnetwork sampled features layer by layer are recorded, where the shallow feature maps contain detailed feature information, and the deep feature maps contain high-level semantic information.

5. The complex target recognition method based on deep twin self-attention network according to claim 1, characterized in that: The S4 specifically includes the following contents: At the feature level, the feature maps from the two branches of the twin network are merged by splicing, and the similar characteristic features learned by the twin network are used to improve the segmentation performance, ensuring that it is highly interpretable when processing highly similar images.

6. The complex target recognition method based on deep twin self-attention network according to claim 1, characterized in that: The S5 specifically includes the following contents: After the two-channel images are fused by S4, feature upsampling is performed through a convolution layer with a stride of 2 and a convolution kernel of 3×3. Each shallower layer has a width and height that are twice that of the previous layer, and the number of channels is 1 / 2 of the previous layer, restoring feature maps of three depths: 1 / 8, 1 / 4, and 1 / 2. The feature map with restored resolution contains both detailed feature information and high-level semantic information extracted from the deep feature map.

7. The complex target recognition method based on deep twin self-attention network according to claim 1, characterized in that: The S6 specifically includes the following contents: The deep feature map 1 / 16 after channel fusion is restored to the size of 1 / 8 of the previous layer through upsampling, and is fused with the corresponding high-resolution feature map 1 / 8 recorded in the S3 selected channel in the encoding stage to ensure the integrity and richness of information in the feature transmission process and improve the accuracy of segmentation and edge details.

8. The complex target recognition method based on deep twin self-attention network according to claim 1, characterized in that: The S9 specifically includes the following contents: Soft Cross Entrophy Loss and Lovza Loss are used to fuse the semantic segmentation recognition results with a weight of 1:1, perform back propagation, and change the network parameters to improve the image segmentation accuracy and classification accuracy.

Images