Space redundancy compression model of space channel attention mechanism based on neural network
Through the spatial redundancy compression model based on the spatial channel attention mechanism of neural network, the problems of complex and redundant characteristics of polyp segmentation boundaries in medical image segmentation are solved, and a more accurate and efficient polyp segmentation effect is achieved.
Patent Information
- Application Number
- CN202411434678.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-15
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-10-15
AI Technical Summary
Medical image segmentation, especially polyp segmentation, faces the problems of boundary complex diversity and redundant feature mapping, resulting in fuzzy boundaries and inefficiency.
The spatial redundant compression model adopts a neural network-based spatial channel attention mechanism, through the pre-trained Res2Net-50 backbone network, shallow feature fusion module, edge-enhanced multi-scale feature fusion module, spatial channel enhancement convolution module and reverse The gated channel transformation module constitutes an EMFFNet model, realizing multi-scale fusion and redundancy reduction of feature information.
The boundary accuracy and efficiency of polyp segmentation are improved, the redundant information in feature maps is reduced, and the model's attention and robustness to boundaries is enhanced.
Smart Images

Figure CN118941911A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a spatial redundancy compression model based on a spatial channel attention mechanism of a neural network. Background Art
[0002] Colonoscopy requires professional doctors to perform detection and segmentation, which is very time-consuming. However, manual colonoscopy is limited by the operator's condition and experience. Therefore, using machine deep learning to assist clinicians in polyp detection and segmentation is an effective solution. Since the size, shape, color and texture of polyps themselves are quite variable, and the boundary with the background intestinal wall is blurred, these complex situations make polyp segmentation very challenging.
[0003] In recent years, with the development of deep learning technology, the field of medical image segmentation has made significant progress. Convolutional neural networks (CNNs) have become the model of choice for many medical image segmentation tasks because they can automatically learn image features and effectively perform regional segmentation. In particular, in polyp segmentation, researchers have used the powerful ability of CNNs to identify subtle features of polyps, improving the accuracy and efficiency of segmentation. In addition, researchers have explored a variety of data augmentation and regularization techniques to improve the generalization ability of the model, as well as the use of multimodal data fusion to combine the advantages of different imaging technologies. These methods help to improve the robustness of segmentation algorithms in different patients and under different imaging conditions.
[0004] Despite these advances, medical image segmentation, especially polyp segmentation, still faces many challenges. The boundaries of polyps exist in mucosa of different shapes, states, and colors, which leads to complex diversity of polyp boundaries. This diversity, coupled with downgraded decoding, poses challenges to the segmentation of detailed parts of the boundaries during decoding, resulting in blurred polyp boundaries. In addition, in deep neural networks for polyp segmentation, there is often a lot of redundancy in the spatial and channel dimensions of feature maps. Summary of the invention
[0005] To solve these problems, a spatial redundancy compression model based on the spatial channel attention mechanism of neural network is proposed.
[0006] To this end, the present invention provides a spatial redundancy compression model based on a spatial channel attention mechanism of a neural network to solve the problems raised in the background technology.
[0007] In order to achieve the above object, the present invention provides the following technical solutions: a spatial redundancy compression model based on the spatial channel attention mechanism of a neural network, a pre-trained backbone network Res2Net-50, a shallow feature fusion module, an edge enhancement multi-scale feature fusion module, a spatial channel enhancement convolution module and a reverse gated channel transformation module constitute a compression model EMFFNet; Shallow feature fusion module (SFF), the backbone network Resnet-50 divides the input polyp image into 5 feature layers, and uses the spatial channel attention mechanism to convolve the shallow features f1 and f2 to obtain f1 ’ and f2 ’ , and then use the shallow feature fusion module to fuse the two to obtain the fused shallow feature sf1; Edge-enhanced multi-scale feature fusion module, which combines high-level features {f i ,i=3,4,5} three high-level feature layers and fused shallow features sf1, through the decoder PD, through parallel connection to aggregate all features, the decoder is composed of S g =PD(sf1,f3,f4,f5) to calculate and obtain the global mapping map S g ; After deep supervision, the network is constrained at different scales to gain more attention on the boundaries; finally, the Sigmoid function is used to obtain an image with enhanced boundaries; Shallow features f 2 , advanced features f 3 , f 4 , f 5 The four feature layers obtain multi-scale feature information through the spatial channel enhanced convolution module (SCEConv), and then use the reverse gated channel transformation module (RGCT) to gradually mine the recognition area, maintain global connection and effectively calculate; a deep supervision strategy is used in the training process to supervise the output of the four feature layers and the global features; the output prediction image is obtained through the Sigmoid function; The spatial channel enhanced convolution module includes: The SRU spatial reconstruction unit separates the feature map with rich information from the feature map with less information through a separation operation, and then adds these two parts of the feature map through a reconstruction operation to generate a feature map with rich information; The CRU channel reconstruction unit segments, transforms and fuses the feature map with rich information in sequence to obtain the original feature map with reduced feature channels; The CBAM convolutional block attention module includes a channel attention module and a spatial attention module. The channel attention module first performs global maximum pooling and global average pooling operations on each channel of the original feature map to capture the global characteristics of the channel; the pooled features are modeled through the fully connected layer and activation function to obtain channel attention weights suitable for applying to the feature map obtained by the CRU channel reconstruction unit; these attention weights are used to weight the channels of the original feature map to generate a weighted channel feature map; this step enhances the channel features that contribute to the task while reducing the impact of unimportant channels.
[0008] In the spatial attention module, for the original input feature map, the maximum pooling and average pooling are implemented along the channel direction to capture features of different scales, thereby obtaining contextual information; the two pooling results are spliced in the channel dimension to form a feature map that integrates multi-scale contextual features; these fused features are refined using convolution operations to generate spatial attention weights; the Sigmoid function is applied to activate these spatial attention weights to ensure that the weight values are in the range of 0 to 1; these spatial attention weights are multiplied by each pixel position of the original feature map to achieve weighted processing of the feature map. This process enhances the feature response of key areas in the image, while suppressing the features of those unimportant areas, thereby optimizing the expressiveness of the feature map. Finally, the weighted channel feature map and the weighted spatial feature map are multiplied element by element to obtain the attention-enhanced feature.
[0009] Preferably, the specific steps of the SRU spatial reconstruction unit generating a feature map with rich information include: First, SRU uses trainable parameters in the group normalization layer to evaluate the spatial information of different feature maps; Then, by performing sigmoid function and threshold processing on the trainable parameters, the weights are mapped to the range of (0,1) to obtain the informative weight W1 and the redundant weight W2; Finally, the informative weight W1 and the redundant weight W2 are multiplied by the input feature X to obtain the informative feature and redundant information features ; Through cross-reconstruction operation, The upper part and The lower part Performing element-by-element summation, we get , and then The upper part and The lower part Performing element-by-element summation, we get ; Finally, The two parts of features are combined to obtain a spatially refined feature map ; The entire reconstruction operation is expressed as: ; in, is element-wise multiplication, is the element-wise sum, For series connection.
[0010] Preferably, the input space is refined into feature maps It is divided into two parts, the upper part and the lower part; then, the upper part and the lower part are transformed respectively; the upper part uses efficient convolution operation to extract high-level representative information to obtain output Y1; the lower part generates a feature map with shallow hidden details through 1×1 convolution operation, and reuses the features of the original input, and takes the union of the two to obtain output Y2; finally, for the outputs Y1 and Y2, first use global average pooling to generate pooled features S1 and S2; then perform Softmax function on the pooled features S1 and S2 to obtain two sets of feature weight vectors β1 and β2, and finally use the feature weight vector to obtain the final output: .
[0011] Preferably, the output of the channel attention module Calculated by formula (3): (3); in, is the input feature map, is a multi-layer perceptron, and represents global average pooling and global maximum pooling operations, is the Sigmoid activation function.
[0012] Preferably, the output of the spatial attention module Calculated using the following formula: (4); in, Represents a 7x7 convolution operation, Indicates that the global average pooling and maximum pooling results are concatenated along the channel.
[0013] Preferably, assuming the input feature Represents the activation features in the convolutional network, where C represents the number of channels, H represents the height, and W represents the width. The reverse gated channel transformation module performs the following transformation on it: (5) Among them, GCTi It represents the gated channel transformation module. represents the trainable parameters, Helps to embed the adaptability of the output, Used to control the activation threshold and determine the behavior of the gated channel conversion module in each channel; Side output S i Perform Sigmoid operation, and then use the reverse operation to get the reverse output R i , where the reverse operation is: (6) in, This is the reverse operation; The overall reverse gated channel transformation formula is: (7) Among them, RGCT i stands for reverse gated channel transformation, It means that they will be multiplied element by element.
[0014] The present invention has the following advantages: The image is input into the model of the present invention, and the backbone network Resnet-50 divides the network into 5 feature layers. First, the shallow feature fusion module (SFF) is used to fuse the two shallow features of f1 and f2. Then, the fused shallow features and the three high-level features of f3, f4, and f5 are aggregated through parallel partial decoders to obtain a global mapping map. Then, the spatial channel enhancement convolution module (SCEConv) is used for the four feature layers of f2, f3, f4, and f5 to obtain multi-scale feature information. Then, the obtained feature information is used using the reverse gated channel transformation module (RGCT) to gradually mine the identification area, maintain global connection and effectively calculate. Then, a deep supervision strategy is used in the training process to supervise the side outputs of the four feature layers and the global features to optimize the model performance. Finally, the output prediction image is obtained through the Sigmoid function, which solves the problem of blurred boundaries of polyp segmentation and a large amount of redundancy in the spatial and channel dimensions of feature mapping. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 A1-A5 in the figure are pictures of serrated polyps, and B1-B5 are pictures of adenomatous polyps; Figure 2 The segmentation effect diagrams of the six networks U-Net, U-Net++, SFA, PraNet, SANet and EMFFNet provided in this embodiment on five public data sets, where GT is the correct segmentation result diagram; Figure 3The segmentation effect diagram of the three networks PraNet, SANet and EMFFNet provided in this embodiment on the private dataset, where GT is the correct segmentation result diagram; Figure 4 The ablation experiment results of the EMFFNet model provided in this embodiment, where GT is the correct segmentation result map; Figure 5 Schematic diagram of the compression model EMFFNet provided in this embodiment. DETAILED DESCRIPTION
[0016] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0017] The training set of this embodiment consists of 900 images of Kvasir-SEG and 550 images of CVC-ClinicDB. The images of the Kvasir and CVC-ClinicDB datasets are randomly divided into three parts, 80% for training, 10% for verification, and 10% for testing.
[0018] In order to verify the effectiveness of the model of the present invention, this embodiment conducted a comparative experiment on seven benchmark colonoscopy datasets. The specific situation of each dataset is as follows: Kvasir: This dataset contains 1,000 polyp images and their corresponding GroundTruth from colonoscopy video sequences. The Ground Truth is manually annotated by medical doctors and then verified by experienced gastroenterologists. The resolution of the images ranges from 332×487 to 1920×1072 pixels.
[0019] CVC-ClinicDB: This dataset contains 612 images collected from 29 colonoscopy video sequences with a resolution of 288 × 384. These images contain multiple examples of polyps, and masks corresponding to the areas covered by polyps in the images are provided to support image segmentation tasks.
[0020] CVC-ColonDB: This dataset consists of 380 polyp images and their corresponding ground truths with a resolution of 500 × 570. These images were generated from 15 different videos, and experts ensured that the extracted frames showed significantly different perspectives by excluding similar frames and manually annotated the 380 frames.
[0021] ETIS: This dataset contains 196 polyp images extracted from 34 colonoscopy videos with a resolution of 966 × 1225. The dataset includes variations in the size and shape of the coral polyps, blurred boundaries, and complex backgrounds, making the segmentation task extremely challenging.
[0022] CVC-300: This dataset is a test dataset from EndoScene, including 60 polyp images with a resolution of 500 × 574. This example selects CVC-300 as the test set, which contains a total of 60 samples for evaluation.
[0023] NICE_1 and NICE_2: These two data sets come from the A University Affiliated Hospital B and the C Traditional Chinese Medicine Hospital, respectively, and include 1459 intestinal polyp images, of which Dataset 1 contains 467 serrated lesion polyps and Dataset 2 contains 992 adenomatous polyps, which are used for external testing of the model. These data sets include images of different sizes, among which images of 1280x995 and 1504x1080 sizes are more common, accounting for 25.3% and 23.6% of the total, respectively. The external test set is specifically used to evaluate the performance of the model. It is not used in the training and parameter adjustment process to ensure the objectivity and independence of its evaluation results. In order to improve the adaptability and stability of the model to different situations, this embodiment selects endoscopic images containing different intestinal cleanliness, includes patients who underwent colonoscopy within the specified period, and all polyps found have been removed and histopathologically analyzed, excluding those with inflammatory bowel disease, active colitis, coagulopathy, familial polyposis, and those who undergo emergency colonoscopy.
[0024] In this embodiment, all polyps identified by colonoscopy were removed and subsequently subjected to histopathological examination. The results of the pathological diagnosis were fully recorded in the electronic medical record system. Except for those images showing no polyps, all images and videos recorded polyps were confirmed by pathology. If there was a discrepancy between the pathological results of the biopsy and resection specimens, this embodiment would give priority to the pathological results of the resection specimen.
[0025] like Figure 1As shown, this embodiment classifies colonoscopic images according to pathological results, and divides colon polyps into two categories: adenomatous polyps (including tubular adenomas, villous adenomas, and tubulovillous adenomas) and serrated lesions (including hyperplastic polyps, sessile serrated lesions, and serrated adenomas). This classification is based on three main considerations: first, most colon polyps can be clearly classified into two categories: adenomatous or serrated lesions; second, pathologists have high consistency in distinguishing these two major types of lesions, but low consistency in distinguishing hyperplastic polyps and sessile serrated lesions; finally, the number of other types of polyps in the dataset of this embodiment is small, which is not enough to support the development of a robust multi-class model. To ensure the quality of the image, this embodiment uses a regularly maintained high-resolution endoscope and performs bowel preparation before colonoscopy according to the bowel preparation guidelines. All images included in the study have been anonymized to remove all information that may identify the patient.
[0026] In this embodiment, in order to comprehensively evaluate the performance of the compression model EMFFNet, a variety of quantitative evaluation indicators are used. First, this embodiment uses two basic indicators, average Dice and average IoU. In addition, four metrics widely used in the field of object detection are further introduced, including weighted Dice metric , which corrects the problem of equal importance of all pixels in the traditional Dice metric; Mean absolute error (MAE), used to evaluate pixel-level accuracy, the lower the value, the better; Enhanced alignment metric , used to evaluate the similarity at the pixel level and the global level; and the structural metric Sα, used to measure the structural similarity between the predicted result and the true label. This embodiment follows the existing state-of-the-art (SOTA) method and uses these six metrics - mDice, mIoU, 、Sα、 and MAE to evaluate the performance of the method in this embodiment and facilitate comparative analysis with other existing technologies. These metrics have been applied in a large number of literatures and have been proven to be effective in polyp segmentation tasks. Through these comprehensive evaluations, this embodiment can fully understand the performance of the system relative to the ground truth labels and make effective comparisons with other technologies. The evaluation tools come from the PraNet Evaluation Toolbox.
[0027] In this example, the segmentation results are described by comparing the compression model EMFFNet with five state-of-the-art polyp segmentation methods. Based on the visual results, the results of the compression model EMFFNet of this example are closest to GroundTruth, and the compression model EMFFNet of this example outperforms the other compared methods in dealing with different challenging factors. Specifically, from Figure 2(GT is the correct segmentation result image) Let’s take a look. In the first row, the background image in the polyp image is not clear and there are water droplet-shaped interferences, which increases the challenge of segmentation. In this case, U-Net, PraNet and SANet will produce some errors of over-segmentation areas, while SFA cannot segment them at all. In the second and third rows, the polyps are almost fused with the background and are large in size, which makes it extremely difficult for the segmentation method to fully identify the boundaries of the polyps. Under this challenge, the model of this embodiment accurately segments the area. In the small-size segmentation process in the fourth row, and in the segmentation of multiple small-size polyps in the fifth row, the model of this embodiment still completes the task more accurately than other methods.
[0028] In this example, the datasets NICE_1 and NICE_2 (i.e. Figure 3 In Ours, Figure 3 This embodiment compares several extremely challenging segmentations including small polyps, large polyps, and multiple polyps. The compression model EMFFNet of this embodiment still performs well.
[0029] Overall, the quantitative comparison further confirms that the model of this embodiment exhibits excellent performance in dealing with various challenging factors encountered in the polyp segmentation process.
[0030] After the compression model EMFFNet completes the training data set, it is first evaluated on the Kvasir and CVC-ClinicDB data sets. Table 1 shows the quantitative performance comparison of the EMFFNet method of this embodiment with other SOTA methods. The results show that the performance of the compression model EMFFNet is significantly better than other models. These models include the two most classic methods in the field of medical segmentation, U-Net and U-Net++, ResUNet++, a variant of ResNet and U-Net++, SFA, the originator of the selective feature aggregation network, and two advanced PraNet and SANet methods. As shown in Table 1, a comprehensive comparison of the model of this embodiment with past SOTA methods under six metrics is shown, and all indicators of EMFFNet on the two datasets of Kvasir and CVC-ClinicDB have been improved. This is because the SCEConv module proposed in this embodiment can effectively extract feature information in the image considering the spatial and channel dimensions, and adaptively select and weight different parts of the feature map, thereby improving the model's ability to focus on the target area. In addition, the multi-scale edge enhancement feature fusion module proposed by the compression model EMFFNet can effectively enhance the edge information of the image, thereby capturing more and stronger edge features.
[0031] Table 1. Performance comparison of test datasets on Kvasir and CVC-ClinicDB
[0032] In order to evaluate the generalization ability of the model, this embodiment selects CVC-300, ETIS, and CVC-ColonDB datasets for comparative experiments. Table 2 shows the qualitative results of the compression model EMFFNet proposed in this embodiment and other state-of-the-art models on five datasets.
[0033] Table 2. Performance comparison of test datasets on CVC-300, ETIS, and CVC-ColonDB
[0034] Since U-Net, U-Net++ and SFA have poor effects on NICE_1 and NICE_2 datasets, their data will not be collected for comparison in this article, and only PraNet and SANet mentioned in the article will be used. Table 3 is the qualitative results of EMFF-Net compared with 5 existing methods on the datasets NICE1 and NICE2 of this embodiment. It can be seen from the table that the dataset of this embodiment has a certain stability in production, and the three models have good performance on the dataset. Among them, the accuracy of the EMFFNet model of this embodiment on the NICE_1 and NICE_2 datasets is higher than that of PraNet and SANet.
[0035] Table 3. Performance comparison of the test dataset on the private datasets NICE_1 and NICE_2
[0036] In this section of the experiment, this example describes in detail the impact of each module on the performance of the entire model. In terms of evaluation methods, two common evaluation criteria in the field of medical image segmentation, mDice and mIoU, are adopted. This example uses PraNet as the baseline and evaluates the effectiveness of the three modules SFF, SCEConv and RGCT proposed in this paper by replacing or deleting them. The test results are shown in the figure. Figure 4 As shown, the model of this embodiment is still effective.
[0037] SFF Validity: In the model of this embodiment, the SFF module is designed to fuse and extract shallow features, which are combined into the segmentation network to provide boundary context information. In order to study the effectiveness of the SFF module, this embodiment performs an ablation study by removing it from the model of this embodiment, denoted as "w / o SFF". In addition, this embodiment adopts a simple and effective gate fusion strategy to integrate low-level features (i.e. f1 and f 2 ). The experimental results of the ablation study are shown in Table 4. As shown in the table, by comparing the “w / o SFF” with the model, it can be observed that the method using boundary-aware features can improve the segmentation performance.
[0038] RGCT effectiveness: In the model of this embodiment, the RGCT module is designed to analyze and understand the local features of the polyp, which are applied to the segmentation network to enhance the polyp boundary information. In order to study the effectiveness of the RGCT module, this embodiment performs an ablation study by removing it from the model of this embodiment, denoted as "w / o RGCT". The experimental results of the ablation study are shown in the table. As shown in Table 4, by comparing "w / o RGCT" with the model of this embodiment, it can be observed that the method of the reverse gated channel of this embodiment can improve the segmentation performance.
[0039] SCEConv Effectiveness: In order to verify the effectiveness of SCEConv in the model of this embodiment, the boundary features are directly merged into the segmentation network using a cascade operation, which is represented as "w / o SCEConv". As shown in Table 4, the proposed SCEConv is effective in merging polyp edge features into the segmentation network, improving the performance of the segmentation network.
[0040] Table 4. Ablation experimental results of SFF, RGCT and SCEConv modules on 5 public datasets
[0041] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made on the basis of the present invention. Therefore, these modifications or improvements made on the basis of not departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.
Claims
1. A spatial redundancy compression model based on the spatial channel attention mechanism of a neural network, characterized by: The compression model EMFFNet is composed of the pre-trained backbone network Res2Net-50, the shallow feature fusion module, the edge-enhanced multi-scale feature fusion module, the spatial channel enhancement convolution module, and the reverse gated channel transformation module; Shallow feature fusion module: The pre-trained backbone network Resnet-50 divides the input polyp image into 5 feature layers, and uses the spatial channel attention mechanism to convolve the shallow features f1 and f2 to obtain f1 ’ and f2 ’ , and then use the shallow feature fusion module to convolute f1 ’ and f2 ’ The fusion is used to obtain the fused shallow feature sf1; Edge-enhanced multi-scale feature fusion module, which combines high-level features {f i ,i=3,4,5} three high-level feature layers and fused shallow features sf1, through the decoder PD, through parallel connection to aggregate all features, the decoder is composed of S g =PD(sf1,f3,f4,f5) to calculate and obtain the global mapping map S g ; After deep supervision, the network is constrained at different scales to gain more attention on the boundaries; finally, the Sigmoid function is used to obtain an image with enhanced boundaries; Shallow features f 2 , advanced features f 3 , f 4 , f 5 The four feature layers obtain multi-scale feature information through the spatial channel enhanced convolution module, and then use the reverse gated channel transformation module to gradually mine the recognition area, maintain global connection and effectively calculate; a deep supervision strategy is used in the training process to supervise the output of the four feature layers and the global features; the output prediction image is obtained through the Sigmoid function; The spatial channel enhanced convolution module includes: The SRU spatial reconstruction unit separates the feature map with rich information from the feature map with less information through a separation operation, and then adds these two parts of the feature map through a reconstruction operation to generate a feature map with rich information; The CRU channel reconstruction unit sequentially segments, transforms and fuses the feature map with rich information to obtain a feature map with reduced feature channels; The CBAM convolutional block attention module includes a channel attention module and a spatial attention module. The channel attention module first performs global maximum pooling and global average pooling operations on each channel of the original feature map to capture the global features of the channel; the pooled features are modeled through the fully connected layer and the activation function to obtain the channel attention weights suitable for the feature map obtained by the CRU channel reconstruction unit; these attention weights are used to weight the channels of the original feature map to generate a weighted channel feature map; the spatial attention module captures features of different scales for the input original feature map by implementing maximum pooling and average pooling along the channel direction to obtain contextual information; the features of the maximum pooling and the average pooling are spliced along the channel dimension to form a feature map that integrates multi-scale contextual features; the convolution operation is used to refine the fused features to generate spatial attention weights; the sigmoid function is applied to activate the spatial attention weights; the spatial attention weights are multiplied by each pixel position of the original feature map to achieve weighted processing of the spatial feature map, and finally the weighted channel feature map and the weighted spatial feature map are multiplied element by element to obtain the attention enhanced feature.
2. The spatial redundancy compression model based on the spatial channel attention mechanism of the neural network according to claim 1, characterized in that: The specific steps of the SRU spatial reconstruction unit to generate a feature map with rich information include: First, SRU uses trainable parameters in the group normalization layer to evaluate the spatial information of different feature maps; Then, by performing Sigmoid function and threshold processing on the trainable parameters, the weights are mapped to the range of (0,1) to obtain the informative weight W1 and the redundant weight W2; Finally, the informative weight W1 and redundant weight W2 are multiplied by the input feature X to obtain the informative feature and redundant information features ; Through cross-reconstruction operation, The upper part and The lower part Performing element-by-element summation, we get , and then The upper part and The lower part Performing element-by-element summation, we get ; Finally, The two parts of features are combined to obtain a spatially refined feature map ; The entire reconstruction operation is expressed as: ; in, is element-wise multiplication, is the element-wise sum, For series connection.
3. The spatial redundancy compression model based on the spatial channel attention mechanism of the neural network according to claim 1, characterized in that: Map the input spatial fine feature It is divided into two parts, namely the upper part and the lower part; then, the upper part and the lower part are transformed respectively; the upper part uses convolution operation to extract high-level representative information to obtain output Y1; the lower part generates a feature map with shallow hidden details through 1×1 convolution operation, and reuses the features of the original input, and takes the union of the two to obtain output Y2; finally, for the outputs Y1 and Y2, first use global average pooling to generate pooled features S1 and S2; then perform Softmax function on the pooled features S1 and S2 to obtain two sets of feature weight vectors β1 and β2, and finally use the feature weight vector to obtain the final output: 。 4. The spatial redundancy compression model based on the spatial channel attention mechanism of the neural network according to claim 1, characterized in that: Output of the channel attention module Calculated by formula (3): (3); in, is the input feature map, is a multi-layer perceptron, and represents global average pooling and global maximum pooling operations, is the Sigmoid activation function.
5. The spatial redundancy compression model based on the spatial channel attention mechanism of the neural network according to claim 4, characterized in that: Output of the spatial attention module Calculated using the following formula: (4); in, Represents a 7x7 convolution operation, Indicates that the global average pooling and maximum pooling results are concatenated along the channel.
6. The spatial redundancy compression model based on the spatial channel attention mechanism of the neural network according to claim 1, characterized in that: Assuming the input features Represents the activation features in the convolutional network, where C represents the number of channels, H represents the height, and W represents the width. The reverse gated channel transformation module performs the following transformation on it: (5) Among them, GCT i It represents the gated channel transformation module. represents the trainable parameters, Helps to embed the adaptability of the output, Used to control the activation threshold and determine the behavior of the gated channel conversion module in each channel; Side output S i Perform Sigmoid operation, and then use the reverse operation to get the reverse output R i , where the reverse operation is: (6) in, This is the reverse operation; The overall reverse gated channel transformation formula is: (7) Among them, RGCT i stands for reverse gated channel transformation, It means that they will be multiplied element by element.
Citation Information
Patent Citations
Colon polyp segmentation method fusing multi-class feature processing strategies
CN117173410A
Polyp segmentation method and system based on banded pooling
CN117726602A
UNet3 +-based attention mechanism stroke image segmentation method
CN118154860A
Global and local feature reconstruction network-based medical image segmentation method
US20230274531A1
Cited By
Facial expression recognition method, system and equipment based on space channel convolution and enhanced compression incentive attention
CN120452047A
