Cross-layer constraint Transform network implementation method based on layered feature compensation

Abstract

The invention discloses a cross-layer constraint Transform network implementation method based on layered feature compensation. The method comprises three modules: high-efficiency cross attention has linear calculation complexity, and the demand for computing power is reduced while the cross attention is used; the cross-layer constraint encoder module applies constraint from shallow layer features representing boundaries and textures to deep layer features representing semantic and structural information, so that key low-dimensional details are reserved in a down-sampling process; the cross-layer constraint decoder module utilizes efficient cross attention to optimize feature fusion between encoder low-level features and decoder high-level features to ensure that key detail information is reserved in an up-sampling process. Under the combined action of the modules, the medical image segmentation method is excellent in performance in various medical image segmentation tasks, and the wide application prospect of the medical image segmentation method in the field of medical image segmentation is shown.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and specifically to a cross-layer constrained Transformer network implementation method based on hierarchical feature compensation, CCFormer (Cross-Layer Constraint Transformer). CCFormer introduces efficient cross-attention with linear computational complexity, which imposes constraints from shallow features representing boundaries and textures on deep features representing semantic and structural information, thereby retaining key low-dimensional details during the downsampling process. In addition, CCFormer uses efficient cross-attention to optimize the feature fusion between the encoder's low-level features and the decoder's high-level features, ensuring that key detail information is retained during the upsampling process. The network can be used to detect and analyze a variety of medical image processing tasks such as diabetic retinopathy and malignant skin lesions, providing accurate and efficient auxiliary support for clinical diagnosis and treatment. Background Art

[0002] In the field of semantic segmentation, medical image segmentation faces more challenges than natural image segmentation due to the complex and varied anatomical structures and rich important details, the common low contrast and high noise in medical images, the extremely unbalanced ratio of segmented areas to background areas, and the scarcity of annotated data. It is difficult to achieve ideal results using traditional natural image segmentation methods. Convolutional Neural Network (CNN) processes short-term and long-term image features by superimposing convolution operations with different receptive fields, thereby extracting local information of the image and capturing fine-grained features at the pixel level. This capability is very effective in detecting and identifying small-sized lesions and subtle anatomical changes in medical imaging. The shared weights and local connection characteristics of the convolution kernel give it translation invariance, which is essential for capturing local texture and edge information in medical images. In addition, the multi-level architecture of CNN can gradually extract high-level semantic features from low-level features such as edges and textures, thereby obtaining features of different scales. This is of great significance for identifying complex anatomical structures and lesions and organs of various sizes and shapes.

[0003] U-Net uses a symmetrical encoder-decoder structure and introduces jump connections. It has shown good segmentation results on a variety of medical images such as MRI, CT, and microscopic images, especially in small sample data. Many improved versions of U-Net have emerged, such as UNet++, UNet3+, Attention U-Net, and nnUNet. These improved versions further improve the performance and versatility of CNN architectures such as U-Net in various medical image segmentation tasks. In medical image segmentation tasks, CNN-based architectures have excellent detailed pathological structure recognition and feature extraction capabilities, while Transformer-based architectures use multi-head self-attention to perform global feature modeling and can capture long-range dependencies. Therefore, a hybrid Transformer-CNN method that combines CNN and Transformer has emerged to take advantage of the advantages of both. TransUNet uses CNN to extract low-level features, and then uses Transformer to encode the features into context sequences for global modeling. TransFuse combines Transformer and CNN in a parallel manner, which can capture global dependencies and low-level spatial details at a shallower level. Compared with the pure Transformer architecture, the hybrid Transformer-CNN method has better detailed feature modeling capabilities. However, the current hybrid transformer-CNN method is limited to the fusion of CNN and Transformer by using multi-head self-attention to model features in each fixed stage of the CNN architecture. It does not consider the connection between the low-level edge, texture, and color information represented by the shallow features of the previous stage and the deep features of the next stage in the downsampling process, and the relationship between the shallow features from the encoder and the deep features in the decoder in the skip connection. The CNN-based segmentation architecture often uses skip connections to supplement the edge, texture, color and other information in the shallow features of the encoder for the deep features in the decoder, but this simple feature fusion lacks the modeling of the dependency between different features and cannot effectively constrain the deep features through the shallow features.

[0004] In order to address the above challenges, it is necessary to design a method to limit feature selection in the downsampling process and jump connections, and retain features that are helpful for fine-grained segmentation. To this end, the present invention designs a method that uses an attention mechanism to achieve feature expression by limiting deep features through shallow features, so that deep features can take into account low-dimensional details while representing high-dimensional semantic information. Furthermore, the present invention proposes CCFormer with efficient cross-attention of linear computational complexity, which uses efficient cross-attention to constrain feature information for shallow and deep features in encoders and jump connections, and uses shallow information (edges, colors, texture materials) to guide deep features representing semantics, structures, and local details to express the common important parts of shallow and deep layers, which to a certain extent compensates for the information loss caused by convolution to reduce feature size, and achieves high-performance segmentation with low parameter count and computational complexity in both 2D and 3D tasks.

[0005] The CNN-based segmentation network extracts deep high-dimensional spatial features by stacking pooling layers and convolutions for downsampling and supplements edge and texture information for the decoder deep features through jump connections. However, the simple information fusion of different features in the jump connection lacks modeling of the dependency relationship between features, and it is difficult to fully utilize low-dimensional information to optimize the deep feature expression. The present invention improves the accuracy of medical image semantic segmentation through efficient cross-attention, uses shallow features with richer detail information such as high-resolution edge texture in the downsampling stage to guide the low-resolution semantic features in the upsampling process, improves the attention of deep features to important edge and detail information, and enhances the global context modeling ability of the model. Similarly, the present invention uses shallow features that characterize image edges, colors, and textures in the downsampling process to guide the cross-layer constraint method of deep features that characterize category semantics, structural information, and local details. Based on this, the present invention proposes CCFormer, which is a linear inter-layer cross-attention Transformer that can maximize the use of feature information at each stage in the downsampling and upsampling processes, and limits the expression of deep high-level features by downsampling low-level features, supplementing the process of upsampling to restore image information with sufficient detail information such as texture and edges. Summary of the invention

[0006] The purpose of the present invention is to provide a cross-layer constrained Transformer network implementation method based on hierarchical feature compensation in view of the deficiencies of the prior art. The CNN-based segmentation network extracts deep high-dimensional spatial features by stacking pooling layers and convolutions for downsampling and supplements edge and texture information for the deep features of the decoder through jump connections. However, the simple information fusion of different features in the jump connection lacks modeling of the dependency relationship between features, and it is difficult to fully utilize low-dimensional information to optimize the expression of deep features. The present invention improves the accuracy of semantic segmentation of medical images through efficient cross-attention, and uses shallow features with richer detail information such as high-resolution edge texture in the downsampling stage to guide low-resolution semantic features in the upsampling process, thereby improving the attention of deep features to important edge and detail information and enhancing the global context modeling capability of the model. Similarly, the present invention uses shallow features that represent image edges, colors, and textures in the downsampling process to guide the cross-layer constraint method of deep features that represent category semantics, structural information, and local details. Based on this, the present invention proposes CCFormer, which is a linear inter-layer cross-attention Transformer network. By introducing the Cross-Layer Constraint in Encoder (CCE) module and the Cross-Layer Constraint in Decoder (CCD) module, it can maximize the use of feature information at each stage of the downsampling and upsampling process, and limit the deep high-level feature expression by downsampling low-level features, which supplements the process of upsampling and restoring image information with sufficient texture and edge details. The present invention can achieve more accurate and stable segmentation effects in a variety of 2D and 3D medical image segmentation tasks, showing its broad application prospects in the field of medical image segmentation.

[0007] The technical solution included in the present invention to solve the technical problem includes the following steps: Step 1: Collect medical image segmentation datasets for multiple different segmentation tasks; Step 2: Divide the medical image segmentation dataset into a training set, a validation set, and a test set in a ratio of 8:1:1; Step 3: Preprocess the data; Step 4: Perform data enhancement on the data in step 3; Step 5: Build a neural network for medical image segmentation; Step 6: Model training and reasoning and save the best model weights; Step 7: Model performance evaluation and result analysis.

[0008] Beneficial effects of the present invention The present invention takes into account that the traditional CNN model is prone to lose detail information when downsampling, and although the Transformer model can capture long-distance dependencies, it may also ignore spatial details. The CCFormer architecture proposed in the present invention adopts an efficient cross-layer feature constraint mechanism, which can effectively retain low-level detail features in medical image segmentation tasks and optimize the expression of deep features. By constraining low-level features such as boundaries and textures, the segmentation ability of complex anatomical structures and detail features is effectively improved, especially in high-precision edge recognition and fine-grained segmentation. CCFormer not only achieves significant performance improvements, but also effectively reduces computational complexity and video memory consumption. By introducing an efficient cross-attention mechanism with linear complexity, the computational resource requirements are kept low without sacrificing segmentation accuracy, making this method more suitable for medical image segmentation tasks with high computational resource requirements in practical applications. The CCFormer proposed in the present invention shows good performance in a variety of medical image segmentation tasks, showing the generalization ability and applicability of the medical image segmentation model, greatly improving the accuracy and efficiency of clinical diagnosis and treatment, and making important contributions to the development of medical imaging technology, thereby effectively improving the treatment effect of patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is the overall framework diagram of 3D CCFormer.

[0010] Figure 2 This is the structure diagram of the efficient cross-attention module.

[0011] Figure 3 This is the structural diagram of the cross-layer constrained encoder module for 3D data.

[0012] Figure 4 This is the structural diagram of the cross-layer constrained decoder module for 3D data.

[0013] Figure 5 This is the overall framework diagram of CCFormer in 2D.

[0014] Figure 6 It is a schematic diagram of the module of 2D data in the cross-layer constrained encoder.

[0015] Figure 7 It is a schematic diagram of the module of 2D data in the cross-layer constrained decoder. DETAILED DESCRIPTION

[0016] The present invention will be further explained below in conjunction with the accompanying drawings and embodiments.

[0017] The implementation method of cross-layer constrained Transformer network based on hierarchical feature compensation is as follows: Figure 1-4 As shown, the purpose is to segment medical images, which specifically includes the following steps: Step 1: Collect multiple medical image segmentation datasets for different segmentation tasks: ISIC2017, ISIC2018, DSB-18, OIMHS, and SegTHOR, covering different types of medical images including skin cancer, abdominal CT, retinal OCT, and pathological slice images.

[0018] Step 2: Dataset division: divide the data into training set, validation set and test set in a ratio of 8:1:1.

[0019] Step 3: Data preprocessing: Since 2D and 3D data are different, different processing is required. For 2D data, the image is scaled to a uniform size and its pixel values ​​are normalized to the range of [0, 1].

[0020] For 3D data, randomly cropped blocks of [96, 96, 96] are input to the designed segmentation network. For the cropped data, maximum and minimum normalization is used to scale it to between [0, 1].

[0021] Step 4: Perform data enhancement on the data preprocessed in step 3, using random flipping, random intensity transformation, random flipping, and random scaling to increase data diversity, which helps improve the generalization of the model.

[0022] Step 5. In order to achieve high-precision medical image segmentation, the present invention proposes a segmentation network (CCFormer), which includes three modules: an efficient cross-attention module, a cross-layer constraint encoder (CCE) module, and a cross-layer constraint decoder (CCD) module. The following describes the structure and function of CCFormer and its modules in detail, and gives the relevant mathematical formulas.

[0023] The overall structure of CCFormer is as follows Figure 1As shown in the figure. First, the input image is transformed by convolution operation to expand the channel depth, thereby achieving preliminary feature extraction. In order to reduce overfitting and enhance the generalization ability of the model, the DropBlock layer is applied after the preliminary convolution. Downsampling is performed by combining maximum pooling with average pooling, and the channel depth is adjusted by convolution operation with a sampling factor of 2. The upsampling path is mirrored by combining deconvolution with two layers of convolution, and upsampling with a sampling factor of 2 is also achieved. Efficient Cross-Attention is then applied in the cross-layer constrained encoder module CCE. In this process, shallow features help guide the focus of deep representations, and the weighted mapping is constrained to the range of (0, 1) through the Sigmoid activation function, and then applied to the deep feature representation.

[0024] In the upsampling stage, efficient cross attention is applied to the cross-layer constrained decoder module CCD; the jump connection encoder features are adaptively combined with the decoder features to form a fusion feature that is supported by the context in terms of structural and texture characteristics. In all convolutional layers in the downsampling and upsampling stages, the LeakyReLU activation function is applied, combined with instance normalization, to balance feature extraction and preserve spatial details, thereby providing the necessary detail information for reconstruction.

[0025] like Figure 2 As shown, in order to achieve linear computational complexity in CCFormer, a high-performance linearCross-Attention is required. The current Linear Attention uses a simple activation function similar to ReLU or PositiveOrthogonal Random Features (PRFs) to approximate the softmax kernel, reducing the explicit storage and calculation of the attention matrix. However, using an overly simple activation function will make it difficult for the final attention output to focus on the truly important parts, and using overly complex operations to approximate softmax will generate additional computational burden. In order to obtain the performance of the approximate softmax kernel while reducing the operational complexity and computational complexity, the present invention selects a simple activation function LeakyReLU as the basis, and adjusts the direction of each Q and K so that Q and K with similarity higher than the set threshold are closer, while Q and K with similarity lower than the set threshold are farther away, thereby restoring the original softmax-like centralized distribution to a certain extent, focusing attention on the most informative area, thereby reducing the problem of overly smooth weight distribution of attention. Therefore, the formula is defined as follows:

[0026] in, represents the similarity function, Represents an intermediate function, function Is the focusing function. It indicates the degree of attention paid to the matrix. Inspired by Flatten Transformer, it retains the norm of the input tensor and only adjusts its direction. By default For an input tensor , A learnable bias initialized to zero is introduced to enable the model to adapt and exploit positional features. is a learnable scaling factor that acts on each channel of Q and K to prevent computational overflow and gradient explosion, while Represents the LeakyReLU function. The diversity of features also limits the expressive power of linear attention. In order to further enhance the feature representation, the present invention introduces a residual connection to process V, as shown in the formula:

[0027] in, is the reshaped V. By introducing the convolution operation And the nonlinear activation function LeakyReLU enhances the output rank and improves the diversity of features by retaining part of the input information. The convolution operation promotes cross-channel information fusion, thereby strengthening the capture of global features, which is crucial for depicting complex and subtle structures in medical images. Therefore, the proposed efficient cross-attention mechanism provides comparable performance to the softmax scaled dot product attention while maintaining linear computational complexity.

[0028] For the cross-layer constrained encoder and cross-layer constrained decoder, there are some differences in the feature dimension size of the input 3D data and 2D data during the operation process. For 3D data: like Figure 1 and 3 As shown, in the cross-layer constrained encoder, the output of the initial convolutional layer is represented as , and the feature maps generated by each subsequent downsampling stage and cross-layer constrained encoder (CCE) are represented as and . Among them, each Indicates that each stage (i.e. ) of a specific channel configuration, , and Respectively represent The length, width and depth of the feature map. For each cross-layer constrained encoder (CCE), Indicates The shallow characteristics of the layer Since implementing multi-head cross-layer attention on these layers requires the input sequences Q, K, and V to have consistent channel depth, it is necessary to pass a preliminary Convolution to adjust To match Channel depth Then, adaptive maximum pooling and average pooling operations are used to convert Downsample to the specified resolution , and get the downsampled versions and , the formula is as follows:

[0029] in, represents the adaptive maximum pooling operation, represents the adaptive average pooling operation; this configuration significantly reduces the computational burden while maintaining the integrity of the performance. For the multi-head attention mechanism, and As a set reshaped to the same format as the input Align, where B represents the batch size.

[0030] Then, Used as ,and Used as , processed by an efficient cross-attention module After that, the weighted feature representation is generated , among which the first to fourth stages (i.e. - ) are 8, 16, 32, and 64 heads respectively. In order to allow shallow features ( ) for deep features ( ) Optimized pixel-by-pixel guided, weighted feature representation is reshaped into ,and Aligned and upsampled to , recorded as .

[0031] Finally, after Sigmoid activation function processing, the adjusted value is kept at range and with deep features Perform element-wise multiplication to regulate the magnitude and avoid exploding gradients: in, represents the upsampling operation, Represents shallow features, Represents deep features. This formula ensures the effective alignment and adaptive optimization of deep features through shallow features, thereby improving the effectiveness of the encoder in medical image segmentation.

[0032] like Figure 1 and Figure 4 As shown, in the step-by-step upsampling process of the cross-layer constrained decoder, the outputs of its four stages are named For example, The skip connection of the feature shallow and Perform stitching and generate stitching features Next, the feature shallow and Through Convolution increases the channel dimension, and then group normalization (eight channels per group) and LeakyReLU activation function are applied to extract finer features, and finally the fine features are obtained respectively. and , as shown below:

[0033] Then, similar to the downsampling strategy, and All are adjusted to the specified resolution through adaptive maximum pooling and adaptive average pooling operations. , thereby reducing the computational overhead and generating , and . To be consistent with the multi-head attention format, and was reshaped; therefore, and As ,and As .exist , Calculate cross-layer attention and generate attention maps respectively and , attention map and It is then resized using linear interpolation to match and Finally, the attention map and The value of is constrained by the Sigmoid activation function range, and respectively with and Perform element-by-element multiplication to obtain weighted features and . Weighted Features and The feature fusion is performed by the following formula:

[0034] This cross-layer constrained decoder structure enhances the integration of multi-scale features, thus promoting accurate segmentation through a layer-by-layer optimized attention mechanism, especially the fine attention calibration between hierarchically generated features.

[0035] For 2D data, such as Figure 5 and 6 As shown, in the cross-layer constrained encoder, the output of the initial convolutional layer is represented as , and the feature maps generated by each subsequent downsampling stage and cross-layer constrained encoder (CCE) are represented as and . Among them, each Indicates that each stage (i.e. ) of a specific channel configuration, , and Respectively represent The length, width and depth of the feature map. For each cross-layer constrained encoder (CCE), Indicates The shallow characteristics of the layer Since implementing multi-head cross-layer attention on these layers requires the input sequences Q, K, and V to have consistent channel depth, it is necessary to pass a preliminary Convolution to adjust To match Channel depth Then, adaptive maximum pooling and average pooling operations are used to convert Downsample to the specified resolution , and get the downsampled versions and , the formula is as follows:

[0036] in, represents the adaptive maximum pooling operation, represents the adaptive average pooling operation; this configuration significantly reduces the computational burden while maintaining the integrity of the performance. For the multi-head attention mechanism, and As a set reshaped to the same format as the input Align, where B represents the batch size.

[0037] Then, Used as ,and Used as , processed by an efficient cross-attention module After that, the weighted feature representation is generated , among which the first to fourth stages (i.e. - ) are 8, 16, 32, and 64 heads respectively. In order to allow shallow features ( ) for deep features ( ) Optimized pixel-by-pixel guided, weighted feature representation is reshaped into ,and Aligned and upsampled to , recorded as .

[0038] Finally, after Sigmoid activation function processing, the adjusted value is kept at range and with deep features Perform element-wise multiplication to regulate the magnitude and avoid exploding gradients: in, represents the upsampling operation, Represents shallow features, Represents deep features. This formula ensures the effective alignment and adaptive optimization of deep features through shallow features, thereby improving the effectiveness of the encoder in medical image segmentation.

[0039] like Figure 5 and Figure 7 As shown, in the step-by-step upsampling process of the cross-layer constrained decoder, the outputs of its four stages are named For example, The skip connection of the feature shallow and Perform stitching and generate stitching features Next, the feature shallow and Through Convolution increases the channel dimension, and then group normalization (eight channels per group) and LeakyReLU activation function are applied to extract finer features, and finally the fine features are obtained respectively. and , as shown below:

[0040] Then, similar to the downsampling strategy, and All are adjusted to the specified resolution through adaptive maximum pooling and adaptive average pooling operations. , thereby reducing the computational overhead and generating , and . To be consistent with the multi-head attention format, and was reshaped; therefore, and As ,and As .exist , Calculate cross-layer attention and generate attention maps respectively and Attention Map and It is then resized using linear interpolation to match and Finally, the attention map and The value of is constrained by the Sigmoid activation function range, and respectively with and Perform element-by-element multiplication to obtain weighted features and . Weighted Features and The fusion is performed by the following formula:

[0041] This cross-layer constrained decoder structure enhances the integration of multi-scale features, thus promoting accurate segmentation through a layer-by-layer optimized attention mechanism, especially the fine attention calibration between hierarchically generated features.

[0042] The present invention is aimed at 2D and 3D data, the network framework does not change, but the dimensions of the data are different.

[0043] Step 6: Use the training set divided in step 2 for training. The framework is implemented based on CUDA 11.8, Python3.9, PyTorch 2.0.0, and MONAI 1.3.1, and is trained on dual NVIDIA 4090 GPUs. In terms of hyperparameters, the learning rate is set to 1e-4, and the AdamW strategy is used. The training is verified once every 1000 samples, and the best weight is saved.

[0044] Step 7: Model performance evaluation and result analysis. Use the best weights on the validation set obtained in step 6 to calculate and analyze the results on the test set. The evaluation indicators are Dice Similarity Coefficient, Mean Intersection over Union (mIoU), and 95% Hausdorff Distance (HD95).

[0045] According to the index results, it can be seen that the present invention has achieved good results in various segmentation tasks, showing that it has strong competitiveness in medical image segmentation.

[0046] Table 1 Comparison of experimental results on ISIC2017 and ISIC2018 datasets

[0047] The test results of ISIC2017 and ISIC2018 datasets are shown in Table 1. CCFormer outperforms other methods in all indicators. The Dice index on the ISIC2017 dataset and ISIC2018 dataset is 0.39% and 0.89% higher than the second best model respectively. It is worth noting that compared with the second best model DconnNet, CCFormer has smaller model parameters, which is only 36.68% of that.

[0048] Table 2 Comparison of experimental results on the DSB-18 dataset

[0049] Table 2 shows the test results on the DSB-18 dataset. The mIoU of CCFormer is 88.04% and that of Dice is 93.46%, which are better than several existing methods.

[0050] Table 3. Comparison of experimental results on OIMHS dataset

[0051] The test results of the OIMHS dataset are shown in Table 3. CCFormer achieves the best performance with smaller parameters and FLOPs, with Dice reaching 93.71% and mIoU reaching 88.60%.

[0052] Table 4 Comparison of experimental results on SegTHOR dataset

[0053] The test results of the SegTHOR dataset are shown in Table 4. CCFormer has a Dice index of 90.96%, an mIoU of 83.93%, and an HD95 of 2.89, which is better than several existing methods. CFormer shows good performance in both region and boundary segmentation accuracy.

[0054] Table 5 Ablation experiment

[0055] To further verify the effectiveness and robustness of CCFormer, an ablation experiment was conducted on the DSB-18 dataset. The experimental results in Table 5 show that integrating cross-layer constraints into the encoder or decoder can improve performance to a certain extent. It is worth noting that introducing a cross-attention mechanism in the jump connection can bring more significant performance improvements. In addition, the trade-off between the computational overhead and performance improvement brought by the cross-layer constraints is within an acceptable range.

[0056] In summary, the CCFormer proposed in the present invention improves the accuracy and robustness of medical image segmentation by integrating cross-layer constraints into the encoder or decoder and using efficient cross-attention to reduce its computing power requirements, and has broad application prospects.

[0057] The above description is only a preferred embodiment of the present invention and does not limit the scope of protection of the present invention. It should be emphasized that any minor changes or optimizations made by those skilled in the art without departing from the basic idea of ​​the present invention should be regarded as the scope of protection of the present invention.

Claims

1. A cross-layer constrained Transformer network implementation method based on hierarchical feature compensation, characterized in that: The steps include: Step 1: Collect medical image segmentation datasets for multiple different segmentation tasks; Step 2: Divide the medical image segmentation dataset into a training set, a validation set, and a test set in a ratio of 8:1:1; Step 3: Preprocess the data; Step 4: Perform data enhancement on the data in step 3; Step 5, building a segmentation network for medical image segmentation; the segmentation network integrates an efficient cross attention module, a cross-layer constraint encoder module and a cross-layer constraint decoder module; and the efficient cross attention module is applied to the cross-layer constraint encoder module and the cross-layer constraint decoder module respectively; Step 6: Model training and reasoning and save the best model weights; Step 7: Model performance evaluation and result analysis.

2. The method for implementing a cross-layer constrained Transformer network based on hierarchical feature compensation according to claim 1, characterized in that: The efficient cross-attention module is specifically implemented as follows: We choose a simple activation function LeakyReLU as the basis, and adjust the direction of each Q and K so that Q and K with similarity higher than the set threshold are closer, while Q and K with similarity lower than the set threshold are farther away, thus restoring the original softmax-like centralized distribution and focusing on the most informative area. The formula is defined as follows: ； ； ； in, represents the similarity function, Represents an intermediate function, function is the focusing function, ; For the input tensor , A learnable bias initialized to zero is introduced, is a learnable scaling factor that acts on each channel of Q and K to prevent computational overflow and gradient explosion, while Represents the LeakyReLU function; A residual connection is introduced to process V, as shown in the formula: ； in, For the reshaped V, by introducing the convolution operation And the nonlinear activation function LeakyReLU enhances the output rank and improves the diversity of features by retaining part of the input information.

3. The method for implementing a cross-layer constrained Transformer network based on hierarchical feature compensation according to claim 1, characterized in that: The proposed cross-layer constrained encoder module imposes constraints from shallow features representing boundaries and textures on deep features representing semantic and structural information, thereby preserving key low-dimensional details during downsampling.

4. The method for implementing a cross-layer constrained Transformer network based on hierarchical feature compensation according to claim 3 is characterized in that: The cross-layer constraint encoder module is specifically implemented as follows: The output of the initial convolutional layer is represented as , and the feature maps generated by each subsequent downsampling stage and cross-layer constrained encoder are represented as and ; each represents the specific channel configuration for each stage, , and Respectively represent The length, width and depth of the feature map of the stage; for each cross-layer constrained encoder, Indicates The shallow characteristics of the layer It indicates the subsequent deep features; through the preliminary Convolution to adjust To match Channel depth ; Then, adaptive maximum pooling and average pooling operations are used to Downsample to the specified resolution , and get the downsampled versions and ; Then, Used as ,and Used as , processed by an efficient cross-attention module After that, the weighted feature representation is generated , in order to allow shallow features For deep features Optimized pixel-wise guided, weighted feature representation is reshaped into ,and Aligned and upsampled to , recorded as ; Finally, after Sigmoid activation function processing, the adjusted value is kept at range and with deep features Perform element-wise multiplication to regulate magnitude and avoid exploding gradients.

5. The method for implementing a cross-layer constrained Transformer network based on hierarchical feature compensation according to claim 4 is characterized in that: Adaptive max pooling and average pooling operations are used to Downsample to the specified resolution , the specific formula is as follows: ； in, represents the adaptive maximum pooling operation, Represents an adaptive average pooling operation.

6. The method for implementing a cross-layer constrained Transformer network based on hierarchical feature compensation according to claim 4, characterized in that: For the multi-head attention mechanism, and As a set reshaped to the same format as the input Align, where B represents the batch size.

7. The method for implementing a cross-layer constrained Transformer network based on hierarchical feature compensation according to claim 4, characterized in that: The first to fourth stages, namely As for The number of heads in the stages are 8, 16, 32, and 64, respectively.

8. The method for implementing a cross-layer constrained Transformer network based on hierarchical feature compensation according to claim 4, characterized in that: With deep features Perform element-by-element multiplication and implement the formula as follows: ； in, represents the upsampling operation, Represents shallow features, Represents deep features.

9. The method for implementing a cross-layer constrained Transformer network based on hierarchical feature compensation according to claim 4, characterized in that: During the stepwise upsampling process of the cross-layer constrained decoder, the outputs of its four stages are named ;No. The skip connection of the stage and Perform stitching and generate stitching features ; Next, the feature shallow and Through Convolution increases the channel dimension, and then group normalization and LeakyReLU activation function are applied to extract finer features, and finally the fine features are obtained respectively. and , as shown below: ； ； Then, and All are adjusted to the specified resolution through adaptive maximum pooling and adaptive average pooling operations. ,generate , and ; To be consistent with the multi-head attention format, and was reshaped; therefore, and As ,and As ;exist , Calculate cross-layer attention and generate attention maps respectively and , attention map and It is then resized using linear interpolation to match and The dimension of ; finally, the attention map and The value of is constrained by the Sigmoid activation function range, and respectively with and Perform element-by-element multiplication to obtain weighted features and ; Weighted Features and The feature fusion is performed by the following formula: 。 10. A system, characterized in that: The system includes the cross-layer constrained Transformer network implementation method based on hierarchical feature compensation described in claim 1.

Images