Feature extraction method and system integrating source end purification and end alignment of residual network
By introducing channel attention and spatial gating mechanisms to clean up the first-layer features in the residual network, and performing bilinear interpolation and channel mapping alignment at the end, the problems of noise interference and feature alignment in cross-layer residual networks are solved, thereby improving network performance and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- WUHAN UNIV OF TECH
- Filing Date
- 2026-05-28
- Publication Date
- 2026-06-23
AI Technical Summary
In existing cross-layer residual networks, early feature textures and noise accumulate step by step, interfering with high-level semantic discrimination. Furthermore, features at different network stages lack targeted alignment in terms of spatial resolution and channel dimension, leading to information dilution and overfitting.
Channel attention and spatial gating mechanisms are used to purify the first-layer features. Bilinear interpolation and efficient channel attention are combined to align the features at the end. Deep features are extracted through the residual network backbone and then accurately aligned and fused in the fusion stage.
It effectively suppresses noise propagation, ensures the stability of high-level semantic discrimination, improves the performance and robustness of residual networks, and maintains lightweight computational and storage overhead.
Smart Images
Figure CN122265667A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of deep learning technology, specifically to a feature extraction method and system that integrates source-end purification and end-end alignment of residual networks. Background Technology
[0002] In recent years, deep learning technology has made significant progress in the field of visual recognition. Residual networks effectively alleviate the gradient vanishing problem during the training process of deep networks by introducing cross-layer shortcut connections, which allows for a significant expansion of network depth and a substantial enhancement of expressive power. However, as research has deepened, the inherent defects of the cross-layer residual connection mechanism have gradually become apparent.
[0003] In traditional residual network architectures, cross-layer residual connections directly pass low-level features extracted from early convolutional layers to subsequent network layers. These early features typically contain a large amount of texture details and background noise information irrelevant to the classification task. As this redundant information accumulates step by step along the residual pathway, it continuously interferes with the semantic discrimination ability of deep networks, resulting in long-term noise persistence. Meanwhile, the feature maps output from different stages of the network exhibit significant differences in spatial resolution and channel dimension. Without targeted scale alignment and selective filtering, directly adding or concatenating heterogeneous features often dilutes effective semantic information, significantly reducing the feature fusion effect. Furthermore, in small-sample or multi-class fine-grained scenarios, such as the CIFAR-10 / 100 dataset, the mapping between low-level high-frequency details and class discrimination patterns is not always linear. If the cross-layer carrying strategy is not differentiated, it can easily introduce overfitting and training instability.
[0004] To address the aforementioned issues, existing technologies have proposed various improvement schemes. The squeeze-excitation attention mechanism models the relationships between channels through dimensionality reduction and expansion operations, but the dimensionality reduction process inevitably results in information compression loss. Convolutional Block Attention Modules (CBAM) simultaneously introduce both channel attention and spatial attention branches, offering stronger modeling capabilities, but the additional computational and parameter overhead is limited in lightweight scenarios. Efficient channel attention mechanisms employ one-dimensional convolution for a lightweight design, but if applied only to a single stage of the network, they still cannot fundamentally constrain the propagation of cross-layer noise across the entire network.
[0005] More critically, most existing methods focus on feature enhancement in later layers of the network, neglecting the importance of the source layers. If the first layer does not filter out irrelevant components, noise will permeate the entire network along the residual path. Simultaneously, in the final layer fusion stage, if resolution matching and channel alignment are not performed on early features—for example, by simply performing global pooling to 1×1—key spatial localization information will be lost, weakening the complementary effect with deep semantics. For neuromorphic / low-power implementations, these problems are amplified into energy consumption and latency costs, affecting the system's real-time performance and stability.
[0006] Therefore, how to perform source purification, process constraint and end matching for cross-layer residual paths without significantly increasing parameters and computational load, so that low-level textures and high-level semantics can achieve selective and aligned fusion in the same scale and channel space, has become a key technical direction for improving the performance and robustness of residual networks. Summary of the Invention
[0007] This invention proposes a feature extraction method and system that integrates source-end purification and end-end alignment of residual networks. This addresses the technical problems of existing cross-layer residual networks that retain and superimpose low-level textures and noise from early layers, resulting in long-term retention of irrelevant features that interfere with subsequent high-level semantic discrimination. At the same time, features at different network stages have significant differences in spatial resolution and channel dimension, lacking targeted scale alignment and selective filtering mechanisms, leading to information mismatch and dilution of effective signals through direct fusion.
[0008] To address the aforementioned technical problems, this invention provides a feature extraction method and system that integrates source-end cleanup and end-end alignment of residual networks, comprising the following steps:
[0009] Step S1: After convolution and normalization of the input image, the first-layer feature map is obtained. Channel attention mechanism and spatial gating mechanism are applied to the first-layer feature map to obtain channel attention weights and spatial attention weights. The channel attention weights and spatial attention weights are multiplied element by element and truncated to obtain the joint mask.
[0010] Step S2: Multiply the joint mask element-wise with the first-layer feature map to obtain a cleaned feature map, and process the cleaned feature map... After convolution and batch normalization, a cross-layer carry-over representation is obtained;
[0011] Step S3: Input the cleaned feature map into the backbone of the residual network and perform layer-by-layer convolution transformation and residual superposition to obtain a deep feature map;
[0012] Step S4: Upsample the cross-layer carried representation so that its spatial resolution matches that of the deep features. Figure 1 Then, a channel attention mechanism is applied for further filtering, followed by... Convolution maps the number of channels to obtain aligned feature maps;
[0013] Step S5: Add the aligned feature map and the deep feature map element by element to obtain the fused feature map. Perform batch normalization, nonlinear activation, global average pooling and fully connected layer processing on the fused feature map in sequence, and output the classification result.
[0014] Preferably, the channel attention mechanism in step S1 employs an efficient channel attention mechanism, including:
[0015] For the first layer feature map Global average pooling is used to obtain the channel description vector. ;
[0016] For the channel description vector Apply a nuclear length of One-dimensional convolution and sigmoid activation function are used to obtain channel attention weights. ;
[0017] Among them, the core length Based on the number of channels Adaptive computation:
[0018] ;
[0019] when When it is an odd number ,when When it is even ;in This is the scaling factor. This is a bias term.
[0020] Preferably, the spatial gating mechanism in step S1 includes:
[0021] For the first layer feature map Calculate the mean plot along the channel dimension. And maximum value chart ;
[0022] The mean plot and the maximum value graph splicing along the channel dimension;
[0023] Apply to the splicing result Convolutional operations and the sigmoid activation function yield spatial attention weights. .
[0024] Preferably, the joint mask in step S1 The calculation method is as follows:
[0025] ;
[0026] in, For channel attention weights, Spatial attention weights, This indicates element-wise multiplication. The function truncates the value to Interval.
[0027] Preferably, the residual network backbone in step S3 includes at least one residual unit, and each residual unit includes a convolutional layer, a batch normalization layer, and a nonlinear activation layer.
[0028] Preferably, the backbone of the residual network is a spiking residual network, and the nonlinear activation layer is a LIF spiking neuron layer; each residual unit adopts a SEW structure, and includes, in sequence: a first Convolutional layer, first batch of normalized layers, first LIF spiking neuron layer, Dropout layer, second The system consists of convolutional layers, a second batch normalization layer, and a second LIF spiking neuron layer, with the input and output added element-wise through residual connections.
[0029] Preferably, the upsampling in step S4 uses bilinear interpolation.
[0030] Preferably, the scaling factor The range of values is The bias term The range of values is .
[0031] This invention also provides a feature extraction system that integrates source-end cleanup and end-end alignment of residual networks, comprising:
[0032] The first-layer feature filtering module performs convolution and normalization on the input image to obtain a first-layer feature map. Channel attention and spatial gating mechanisms are sequentially applied to the first-layer feature map to generate a joint mask. The joint mask is then multiplied element-wise with the first-layer feature map to obtain a cleaned feature map. Finally, the cleaned feature map is processed... After convolution and batch normalization, a cross-layer carry-over representation is obtained;
[0033] The backbone residual module is used to perform layer-by-layer convolutional transformation and residual superposition on the cleaned feature map. The backbone residual module includes at least one residual unit, and each residual unit includes a convolutional layer, a batch normalization layer and a nonlinear activation layer, and outputs a deep feature map.
[0034] A cross-layer scale alignment and attention fusion module is used to upsample, filter channel attention, and perform channel attention filtering on the cross-layer carried representation. Convolutional mapping yields an aligned feature map, and the aligned feature map is added element-wise to the deep feature map to obtain a fused feature map;
[0035] The classification module is used to process the fused feature map sequentially through batch normalization, nonlinear activation, global average pooling, and fully connected layers, and output the classification result.
[0036] Preferably, the channel attention mechanism in the first-layer feature filtering module adopts an efficient channel attention mechanism, which obtains the channel attention weights through global average pooling and one-dimensional convolution. ;
[0037] The spatial gating mechanism in the first-layer feature filtering module calculates and concatenates the mean map and the maximum map, and then... Convolution yields spatial attention weights ;
[0038] The backbone residual module is a spiking residual network, the nonlinear activation layer is a LIF spiking neuron layer, and each residual unit adopts a SEW structure, sequentially including a first... Convolutional layer, first batch of normalized layers, first LIF spiking neuron layer, Dropout layer, second Convolutional layers, second batch normalization layers, second LIF spiking neuron layers, and residual connections;
[0039] The upsampling in the cross-layer scale alignment and attention fusion module uses bilinear interpolation.
[0040] The beneficial effects of the present invention include at least the following:
[0041] First, by constructing a channel-space joint gating purification mechanism at the first layer of the network, irrelevant textures and background noise are suppressed at the source of feature propagation, effectively blocking the path of noise propagation along the residual path to the deep network, fundamentally alleviating the interference of long-term noise residue on high-level semantic discrimination.
[0042] Second, a lightweight cross-layer carrying representation design is adopted to store the key feature information after the first layer purification in a compact form, maintaining extremely low storage and computational overhead throughout the network propagation process, and providing high-quality shallow feature support for end-to-end fusion.
[0043] Third, a three-step alignment strategy is implemented in the final fusion stage, which includes bilinear interpolation upsampling, efficient channel attention filtering, and 1×1 convolutional channel mapping. This ensures that cross-layer features and deep features are accurately matched in terms of spatial scale and channel dimension, thereby achieving effective complementarity between shallow localization information and deep semantic representation.
[0044] Fourth, the overall solution has minimal parameter increments and computational overhead, and can be directly embedded into existing residual network backbone architectures such as ResNet-18, resulting in low engineering deployment costs. Attached Figure Description
[0045] Figure 1 This is a block diagram of the overall structure of the system of the present invention;
[0046] Figure 2 This is a schematic diagram of the network structure of the feature extraction method of the present invention;
[0047] Figure 3 Comparison curves of classification accuracy for three network configurations on the CIFAR-10 dataset;
[0048] Figure 4 The curves show the classification accuracy comparison of three network configurations on the CIFAR-100 dataset. Detailed Implementation
[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.
[0050] This invention proposes a feature extraction method integrating source-end purification and end-end alignment of residual networks. Its core idea is to systematically optimize the cross-layer residual path from the source to the final stage of the process. For example... Figure 1 and Figure 2 As shown, the method mainly includes the following steps:
[0051] Step S1: After convolution and normalization of the input image, the first-layer feature map is obtained. Channel attention mechanism and spatial gating mechanism are applied to the first-layer feature map to obtain channel attention weights and spatial attention weights. The channel attention weights and spatial attention weights are multiplied element by element and truncated to obtain the joint mask.
[0052] Step S2: Multiply the joint mask element-wise with the first-layer feature map to obtain the cleaned feature map. Then, process the cleaned feature map... After convolution and batch normalization, a cross-layer carry-over representation is obtained;
[0053] Step S3: Input the cleaned feature map into the backbone of the residual network and perform layer-by-layer convolution transformation and residual superposition to obtain the deep feature map;
[0054] Step S4: Upsample the cross-layer carried representation to make its spatial resolution match that of the deep features. Figure 1 Then, a channel attention mechanism is applied for further filtering, followed by... Convolution maps the number of channels to obtain aligned feature maps;
[0055] Step S5: Add the aligned feature map and the deep feature map element by element to obtain the fused feature map. Then, process the fused feature map by batch normalization, non-linear activation, global average pooling, and fully connected layer in sequence, and output the classification result.
[0056] Specifically, in this embodiment, the input image is first processed by the first convolutional layer and batch normalization to obtain the first-layer feature map. Then, a first-layer feature filter is set to simultaneously apply channel attention and spatial gating mechanisms to this feature map, generating a joint mask and completing the first-layer feature purification. Simultaneously, a carry representation for cross-layer propagation is derived. Next, the purified features are input into the backbone network composed of residual units, and deep semantic features are extracted through four stages of layer-by-layer convolutional transformation and residual superposition. Then, before entering the final fusion layer, the carry representation derived from the first layer is bilinearly interpolated and upsampled to match the spatial resolution of the deep features. Efficient channel attention is then applied for lightweight filtering, and channel number mapping is completed through 1×1 convolution. Finally, the aligned cross-layer features are added element-wise with the backbone deep features and fused. After batch normalization, non-linear activation, global average pooling, and fully connected layer processing, the final classification result is output.
[0057] I. First-layer feature purification and representation construction
[0058] The core objective of this step is to filter out irrelevant components at the source of feature propagation, prevent noise from contaminating subsequent network layers along the residual path, and generate a lightweight carry representation for end-fusion.
[0059] Suppose that the input image is processed by the first 3×3 convolutional layer and batch normalization to obtain the first layer feature map. ,in For batch size, For the number of channels, and These represent the height and width of the feature map, respectively.
[0060] The channel attention mechanism employs the efficient channel attention (ECA) design approach, modeling inter-channel dependencies through one-dimensional convolutions with adaptive kernel length. Specifically, it first performs global average pooling along the spatial dimension on the first-layer feature map to obtain the channel description vectors:
[0061] ;
[0062] in This indicates a global average pooling operation. For the first Statistics for each channel. The colon ":" in the formula represents all samples in batch dimension B; similarly, replacing the second, third, and fourth positions with ":" represents all samples in the channel dimension, feature map height dimension, and feature map width dimension, respectively. Subsequently, a kernel of length is applied to the channel description vector. One-dimensional convolution and sigmoid activation function are used to obtain channel attention weights:
[0063] ;
[0064] in This represents the Sigmoid activation function. This represents a one-dimensional (1D) convolution operation performed on a channel sequence, where the input is considered to be of length 1. The sequence. In this embodiment, the kernel length of the one-dimensional convolution is... Based on the number of channels Adaptive computation:
[0065] ;
[0066] when Take when it is odd ,when Take when it is even This is to ensure that the kernel length is odd, thus maintaining feature symmetry. This is the scaling factor, with a value range of [2,3]. This is a bias term with a value range of [1,2]. This adaptive mechanism allows channel attention to dynamically adjust the receptive field range based on the number of feature channels.
[0067] Spatial gating mechanisms aim to identify relevant spatial regions in feature maps and suppress the response to background noise. First, mean maps are calculated along the channel dimensions. And maximum value chart :
[0068] ;
[0069] ;
[0070] The mean plot reflects the average activation intensity at each spatial location, while the maximum plot captures the most salient feature responses at each location. The spatial attention weights are obtained by concatenating the two along the channel dimension and then performing a 7×7 convolution followed by a sigmoid activation function. :
[0071] ;
[0072] in This indicates that the mean plot will be displayed. And maximum value chart The pieces are stitched together along the channel dimension.
[0073] Joint mask generation and feature cleansing multiply the channel attention weights and spatial attention weights element-wise, and constrain the values within an effective range using a truncation function:
[0074] ;
[0075] in This indicates an element-wise multiplication operation. Truncate the values to the [0,1] interval. Combined mask. Simultaneously encoding channel importance and spatial saliency information, applying this mask to the first-layer feature map completes the purification process, resulting in a purified feature map. :
[0076] ;
[0077] Based on the cleaned-up features, a lightweight carry representation for end-fusion is generated through 1×1 convolution and batch normalization. :
[0078]
[0079] in The parameters are 1×1 convolution kernel parameters. This indicates batch normalization. This representation maintains a compact 64-channel form, preserving key feature information from the first layer for end-to-end fusion without significantly increasing memory overhead.
[0080] II. Residual Network Propagation
[0081] In this embodiment, the backbone network adopts a residual architecture, consisting of four stages of residual unit sequences, with corresponding channel numbers of 64, 128, 256, and 512, respectively. Each stage contains several residual units, and high-level semantic features are extracted step by step through layer-by-layer convolutional transformation and residual stacking.
[0082] Each residual unit employs a pre-activation structure design, consisting of a batch normalization layer, a nonlinear activation layer, and a convolutional layer. Let the... The input of each residual unit is Its forward propagation process can be described as follows:
[0083] ;
[0084] ;
[0085] ;
[0086] ;
[0087] ;
[0088] in Represents a non-linear activation function. The step size parameter is used to control downsampling. The residual branch... Through 1×1 convolution projection Align the shape of the input features to the shape of the main branch output to ensure the feasibility of element-wise addition.
[0089] In a preferred embodiment, the backbone of the residual network can be implemented using a spiking residual network, where each residual unit contains a Leaky Integrate-and-Fire (LIF) neuron layer as a non-linear activation layer. Specifically, the residual unit adopts a Spike-Element-Wise (SEW) structure, which sequentially includes: a first 3×3 convolutional layer, a first batch of normalization layers, a first LIF spiking neuron layer, a Dropout layer, a second 3×3 convolutional layer, a second batch of normalization layers, and a second LIF spiking neuron layer, and the input and output are added element-wise through residual connections.
[0090] The four stages are configured with 64, 128, 256, and 512 channels, and the spatial resolution is progressively downsampled through convolution operations with stride=2. After multi-stage feature extraction by the backbone network, the final output is a deep feature map. ,in and This represents the spatial dimensions of the feature map in the final layer.
[0091] III. Cross-scale alignment and attention fusion
[0092] When the backbone network enters the final residual stage, the carried representation derived from the first layer needs to be fused with the deep features. Since the two differ significantly in spatial resolution and channel dimension, direct fusion will lead to information mismatch. This step achieves precise alignment through three sub-steps: upsampling, channel attention filtering, and channel mapping.
[0093] Spatial scale alignment employs bilinear interpolation to upsample the carried representation, ensuring its spatial resolution matches that of the deep features.
[0094] ;
[0095] in, This means resampling the input feature map to the target space size using bilinear interpolation. .
[0096] Bilinear interpolation is a numerically stable sampling method that can adjust resolution while maintaining feature smoothness.
[0097] Channel attention filtering for upsampled 64-channel features A lightweight screening process is then applied again using an efficient channel attention mechanism (ECA) to suppress cross-layer channels that contribute little to the final discrimination:
[0098] ;
[0099] in, This indicates the cross-layer features after filtering. This step can dynamically adjust the importance weight of each channel according to the current task requirements, and filter out the feature components that are most discriminative for classification.
[0100] Channel mapping uses 1×1 convolution to linearly map the filtered cross-layer features from 64 channels to 512 channels to match the channel dimension of deep features:
[0101] ;
[0102] in, This represents the cross-layer feature map after channel alignment. Feature fusion adds the aligned cross-layer features to the main deep features and their fast branches element-wise on the same scale to obtain the fused feature map. :
[0103] ;
[0104] in This is the output of the current backbone convolution. For quick branch output, This is a scalar gating parameter. It can be set to a learnable parameter or a fixed constant to adjust the injection strength of cross-layer features, avoiding instability caused by excessive injection in the early stages of training. Through the above fusion strategy, the texture and edge localization information carried by shallow features and the semantic representation encoded by deep features achieve effective complementarity.
[0105] IV. Classification Output and Training Strategies
[0106] The fused features are processed sequentially through batch normalization layers, non-linear activation, global average pooling, and fully connected layers, outputting classification probabilities that match the number of classes in the dataset. Non-linear activation layers can use conventional activation functions such as ReLU and GELU; in spiking neural network implementations, LIF spiking neurons are used. Batch normalization layers stabilize the feature distribution across different batches of samples, improving the network's trainability. Global average pooling compresses spatial information into a fixed-length vector, enhancing the model's robustness to translation and deformation without introducing additional parameters. Lightweight random deactivation is then applied before the classification head to reduce co-adaptability, and finally, a fully connected layer outputs a prediction result matching the number of classes in the dataset, forming the final discrimination.
[0107] During the training phase, the cross-entropy loss function is used as a supervision signal, and label smoothing techniques can be combined to further improve generalization ability. The optimizer uses the Adam algorithm, with an initial learning rate of 0.1 and a weight decay coefficient of 0.05. A cosine annealing strategy is used for learning rate scheduling, allowing the learning rate to decay smoothly with each training epoch, which helps the model converge to a better local minimum in the later stages of training. For data augmentation, techniques such as random flipping, random pruning, and CutMix or MixUp can be combined to further improve the model's generalization performance.
[0108] V. Modular Implementation of the System
[0109] Based on the above method and process, this invention constructs a modular feature extraction system, including the following functional modules:
[0110] The first-layer feature filtering module is responsible for convolution and normalization of the input image to generate the first-layer feature map. Then, it applies channel attention mechanism and spatial gating mechanism in sequence to generate joint mask, completes the first-layer feature purification and derives cross-layer carried representation.
[0111] The backbone residual module is responsible for performing layer-by-layer convolutional transformations and residual stacking on the cleaned features. It contains a sequence of residual units in four stages. Each residual unit consists of a convolutional layer, a batch normalization layer, and a nonlinear activation layer, outputting a deep feature map. In the spiking neural network implementation, the nonlinear activation layer uses LIF spiking neuron layers.
[0112] The cross-layer scale alignment and attention fusion module is responsible for performing bilinear interpolation upsampling, efficient channel attention filtering, and 1×1 convolutional channel mapping on the carried representation to obtain the aligned feature map, and then fusing it with the deep feature map element by element.
[0113] The classification module is responsible for processing the fused feature map sequentially through batch normalization, non-linear activation, global average pooling, and fully connected layers, and outputting the final classification result.
[0114] The interfaces between modules are clearly defined, and the data flow is well-defined, facilitating independent development, testing, and flexible deployment. During the inference phase, randomization modules such as Dropout are disabled, and the statistical parameters of the batch normalization layer are fixed to ensure the determinism of the output results and the controllability of inference latency.
[0115] VI. Experimental Verification
[0116] This invention was experimentally validated on two standard image classification datasets, CIFAR-10 and CIFAR-100. CIFAR-10 contains 60,000 32×32 RGB color images, divided into 50,000 training images and 10,000 test images, covering 10 categories with a balanced distribution of samples across each category. CIFAR-100 also contains 60,000 32×32 images, but is divided into 100 fine-grained categories, with only 500 training samples per category, significantly increasing the classification difficulty.
[0117] The training hyperparameters were configured as follows: batch size 64, training epochs 300, Adam optimizer, initial learning rate 0.1, weight decay 0.05, and cosine annealing strategy for learning rate scheduling.
[0118] The experiment compared three network configurations: the base residual network (Base), the cross-layer residual spiking neural network without feature extraction (NF-CrossLayer-SNN), and the cross-layer residual spiking neural network with feature extraction proposed in this invention (F-CrossLayer-SNN). All three configurations used 6 time steps for pulse coding.
[0119] On the CIFAR-10 dataset, the Base network achieved a classification accuracy of 94.78%, the NF-CrossLayer-SNN achieved 95.47%, and the F-CrossLayer-SNN of this invention achieved the highest accuracy of 95.68%. This represents an improvement of 0.9 percentage points compared to the base network and 0.21 percentage points compared to the cross-layer residual network without feature extraction.
[0120] On the CIFAR-100 dataset, the Base network achieved a classification accuracy of 75.42%, NF-CrossLayer-SNN achieved 76.25%, while the F-CrossLayer-SNN of this invention achieved the highest accuracy of 76.61%. This represents an improvement of 1.2 percentage points compared to the base network and 0.37 percentage points compared to the cross-layer residual network without feature extraction.
[0121] like Figure 3 and Figure 4 As shown, the F-CrossLayer-SNN proposed in this invention exhibits the fastest convergence speed and the highest final accuracy on both datasets, verifying the effectiveness of the source-end purification and end-end alignment strategies in improving the performance of residual networks. Meanwhile, the number of additional parameters introduced in this invention is extremely limited, mainly derived from the 7×7 spatial convolution in the first-layer feature filter and the 1×1 channel mapping convolution in the end-end fusion layer; the overall increase in computational cost is negligible.
[0122] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described; only preferred embodiments of the present invention are illustrated. The descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. As long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.
[0123] It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of this invention. Therefore, the scope of protection of this invention should be determined by the appended claims.
Claims
1. A feature extraction method integrating source-end purification and end-end alignment of residual networks, characterized in that, Includes the following steps: Step S1: After convolution and normalization of the input image, the first-layer feature map is obtained. Channel attention mechanism and spatial gating mechanism are applied to the first-layer feature map to obtain channel attention weights and spatial attention weights. The channel attention weights and spatial attention weights are multiplied element by element and truncated to obtain the joint mask. Step S2: Multiply the joint mask element-wise with the first-layer feature map to obtain a cleaned feature map, and process the cleaned feature map... After convolution and batch normalization, a cross-layer carry-over representation is obtained; Step S3: Input the cleaned feature map into the backbone of the residual network and perform layer-by-layer convolution transformation and residual superposition to obtain a deep feature map; Step S4: Upsample the cross-layer carried representation to make its spatial resolution consistent with the deep feature map, then apply a channel attention mechanism for filtering, and then... Convolution maps the number of channels to obtain aligned feature maps; Step S5: Add the aligned feature map and the deep feature map element by element to obtain the fused feature map. Perform batch normalization, nonlinear activation, global average pooling and fully connected layer processing on the fused feature map in sequence, and output the classification result.
2. The feature extraction method for integrating source-end purification and end-end alignment of residual networks according to claim 1, characterized in that, The channel attention mechanism in step S1 employs an efficient channel attention mechanism, including: For the first layer feature map Global average pooling is used to obtain the channel description vector. ; For the channel description vector Apply a nuclear length of One-dimensional convolution and sigmoid activation function are used to obtain channel attention weights. ; Among them, the core length Based on the number of channels Adaptive computation: ； when When it is an odd number ,when When it is even ;in This is the scaling factor. This is a bias term.
3. The feature extraction method for integrating source-end purification and end-end alignment of residual networks according to claim 1, characterized in that, The spatial gating mechanism in step S1 includes: For the first layer feature map Calculate the mean plot along the channel dimension. And maximum value chart ; The mean plot and the maximum value graph splicing along the channel dimension; Apply to the splicing result Convolutional operations and the sigmoid activation function yield spatial attention weights. .
4. The feature extraction method for integrating source-end purification and end-end alignment of residual networks according to claim 1, characterized in that, Joint mask in step S1 The calculation method is as follows: ； in, For channel attention weights, Spatial attention weights, This indicates element-wise multiplication. The function truncates the value to Interval.
5. The feature extraction method for integrating source-end purification and end-end alignment of residual networks according to claim 1, characterized in that, The residual network backbone described in step S3 includes at least one residual unit, and each residual unit includes a convolutional layer, a batch normalization layer, and a nonlinear activation layer.
6. The feature extraction method for integrating source-end purification and end-end alignment of residual networks according to claim 5, characterized in that, The backbone of the residual network is a spiking residual network, and the nonlinear activation layer is a LIF spiking neuron layer; each residual unit adopts a SEW structure and includes, in sequence: a first Convolutional layer, first batch of normalized layers, first LIF spiking neuron layer, Dropout layer, second The system consists of convolutional layers, a second batch normalization layer, and a second LIF spiking neuron layer, with the input and output added element-wise through residual connections.
7. The feature extraction method for integrating source-end purification and end-end alignment of residual networks according to claim 1, characterized in that: The upsampling in step S4 uses bilinear interpolation.
8. The feature extraction method for integrating source-end purification and end-end alignment of residual networks according to claim 2, characterized in that: The scaling factor The range of values is The bias term The range of values is .
9. A feature extraction system integrating source-end purification and end-end alignment of residual networks, characterized in that, include: The first-layer feature filtering module performs convolution and normalization on the input image to obtain a first-layer feature map. Channel attention and spatial gating mechanisms are sequentially applied to the first-layer feature map to generate a joint mask. The joint mask is then multiplied element-wise with the first-layer feature map to obtain a cleaned feature map. Finally, the cleaned feature map is processed... After convolution and batch normalization, a cross-layer carry-over representation is obtained; The backbone residual module is used to perform layer-by-layer convolutional transformation and residual superposition on the cleaned feature map. The backbone residual module includes at least one residual unit, and each residual unit includes a convolutional layer, a batch normalization layer and a nonlinear activation layer, and outputs a deep feature map. A cross-layer scale alignment and attention fusion module is used to upsample, filter channel attention, and perform channel attention filtering on the cross-layer carried representation. Convolutional mapping yields an aligned feature map, and the aligned feature map is added element-wise to the deep feature map to obtain a fused feature map; The classification module is used to process the fused feature map sequentially through batch normalization, nonlinear activation, global average pooling, and fully connected layers, and output the classification result.
10. A feature extraction system integrating source-end purification and end-end alignment of residual networks according to claim 9, characterized in that: The channel attention mechanism in the first-layer feature filtering module adopts an efficient channel attention mechanism, which obtains channel attention weights through global average pooling and one-dimensional convolution. ; The spatial gating mechanism in the first-layer feature filtering module calculates and concatenates the mean map and the maximum map, and then... Convolution yields spatial attention weights ; The backbone residual module is a spiking residual network, the nonlinear activation layer is a LIF spiking neuron layer, and each residual unit adopts a SEW structure, sequentially including a first... Convolutional layer, first batch of normalized layers, first LIF spiking neuron layer, Dropout layer, second Convolutional layers, second batch normalization layers, second LIF spiking neuron layers, and residual connections; The upsampling in the cross-layer scale alignment and attention fusion module uses bilinear interpolation.