Feature fusion method based on multi-level attention guidance

Abstract

The invention discloses a feature fusion method based on multi-level attention guidance. The method specifically comprises the following steps: extracting a multi-scale feature map based on a convolutional neural network; designing a space attention module and a channel attention module, and respectively carrying out dynamic weight distribution on the space relevance and channel importance of the feature map; a cross-level attention guiding mechanism is introduced, and feature responses of different levels are fused through adaptive weights; and finally, establishing a feature fusion module, carrying out cascade and convolution optimization on the weighted multi-scale features, and generating fusion features with higher robustness. According to the method, noise interference is effectively suppressed and key feature expression is enhanced through the adaptive characteristic of an attention mechanism, and the technical problems of unstable fusion effect and detail information loss of a traditional method in a complex scene are solved. The method has the advantages that the precision and robustness of tasks such as image recognition and target detection are remarkably improved, and the method can be widely applied to the fields of automatic driving, medical image analysis and industrial quality inspection.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a feature fusion method based on multi-level attention guidance. Background Art

[0002] With the widespread application of deep learning technology in computer vision, multi-level feature fusion has become a key step in improving model performance. This is particularly true in tasks such as object detection, image segmentation, and deepfake detection. The effective integration of semantic information and detailed features at different levels directly impacts the model's discriminative capabilities. Existing feature fusion methods primarily rely on feature pyramid networks and their variants, which achieve multi-scale feature transfer through top-down or bidirectional connections. However, these methods still suffer from significant drawbacks in practical applications.

[0003] Traditional feature pyramid methods usually use simple operations such as element-by-element addition or channel splicing to fuse features from different levels. Such methods ignore the essential differences in spatial alignment and semantic consistency between shallow and deep features. For example, in deepfake detection scenarios, there is a spatial misalignment between the high-frequency artifacts contained in shallow features and the facial structure semantics encoded by deep features. Direct addition operations will cause the response intensity of key forgery clues to attenuate by more than 40%. In addition, existing methods generally adopt a fixed-weight feature fusion strategy, which cannot adaptively adjust the importance ratio of features at different levels. When faced with changes in illumination, resolution differences, or new forgery methods, the generalization performance of the model drops sharply, and the average accuracy loss in cross-dataset testing is as high as 25%.

[0004] above.

[0005] Existing technologies attempt to introduce attention mechanisms to optimize the feature fusion process, such as local application of channel attention or spatial attention. However, these methods have two fundamental flaws: first, the single-granularity attention mechanism can only capture feature correlations within the hierarchy and lacks a cross-level attention guidance mechanism, which makes it difficult for shallow detail features and deep semantic features to form effective complementarity; second, the calculation of attention weights relies on global statistics, resulting in a large amount of spatial detail information loss, making it impossible to accurately locate distributed forgery areas.

[0006] More seriously, existing multi-level fusion methods lack a dynamic feature importance assessment mechanism and are unable to distinguish discriminative features from noise interference. For example, in a public solution, the direct fusion of multi-level features introduces a large amount of background information irrelevant to forgery discrimination, resulting in a 15% increase in the model's false positive rate for samples with complex backgrounds. Furthermore, the fixed-structure feature fusion network struggles to adapt to the resolution differences between different layers, and is prone to memory overflow and increased computational latency when processing high-resolution inputs, severely restricting its deployment feasibility on edge devices.

[0007] The root cause of these shortcomings can be attributed to the lack of a cross-level spatial-channel collaborative attention guidance mechanism during the multi-level feature fusion process, as well as insufficient dynamic feature screening capabilities. This makes it difficult for the model to fully utilize the complementary information of features at different levels, and there is a serious risk of performance degradation in complex application scenarios. Summary of the Invention

[0008] In response to the problems existing in the prior art, the present invention provides a feature fusion method based on multi-level attention guidance. In response to the problems existing in the existing feature fusion methods such as insufficient noise interference suppression, rigid cross-level feature interaction, and loss of detail information in complex scenes, the present invention provides a feature fusion method based on multi-level attention guidance. Through the three technical breakthroughs of dynamic allocation of spatial and channel attention, cross-level interaction optimization, and multi-scale fusion enhancement, the adaptive enhancement and robust expression of key features are achieved. The specific process includes the following core steps:

[0009] Step 1: Extract multi-scale feature maps through pre-trained convolutional neural network (CNN), including shallow detail feature maps f s and deep semantic feature map f n-1 Bilinear interpolation is used to align the resolution of feature maps and construct a spatially consistent multi-level feature pyramid, which takes into account the expression of local details and global semantic information, providing basic feature support for subsequent fusion.

[0010] Step 2: Dynamically weight spatial attention and channel attention for multi-scale feature maps. The spatial attention module extracts the spatial attention map through global average pooling and generates the weight coefficient α in combination with the fully connected layer. s And multiply the original features element by element after being constrained by Sigmoid function to generate spatial enhanced features F s ; The channel attention module uses point convolution to compress the channel dimension and generates the channel weight map β through ReLU activation c , based on the broadcast product to filter important channels, output channel enhanced features F c .

[0011] Step 3: Construct a cross-level attention guidance module to enhance the spatial feature F s As query vector, channel enhanced feature F c As the key-value vector, the cross-level interaction weight matrix W is calculated through the multi-head attention mechanism cross , combined with Softmax normalization and dimension scaling factor Dynamic fusion generates cross-modal features F cross ,.Achieve adaptive alignment and complementary optimization of multi-level features.

[0012] Step 4: Input the cross-modal features into the feature pyramid network, perform multi-scale feature aggregation through cascaded 3×3 convolution blocks (step size 2) and global attention fusion blocks, combine fast spatial pyramid pooling operations to further optimize feature resolution and receptive field, and output fusion feature F out .

[0013] Step 5: The optimized fusion feature F out Input the downstream task network (such as a classifier or detector), map it to the target output space through the fully connected layer, and use the Sigmoid function to generate probability distribution to complete high-precision decision-making for tasks such as image recognition and target detection.

[0014] Furthermore, in step 1, the pre-trained CNN uses a backbone network such as ResNet or EfficientNet, and the shallow feature f s Focus on high-frequency details such as edges and textures, deep features f n-1 Encodes semantically abstract information. Bilinear interpolation ensures that the resolution of feature maps at different levels is aligned to the same scale (e.g., 256×256), avoiding fusion errors caused by feature misalignment and providing spatial consistency for subsequent attention modules.

[0015] Furthermore, in step 2, the spatial attention module compresses the spatial dimension through global average pooling to generate feature descriptors, and the fully connected layer learns the importance differences of different regions. For example, in the target detection task, the spatial weight can automatically focus on the boundary area of the object and suppress background interference; the channel attention module compresses the number of channels to 1 / 16 of the original through point convolution (1×1 convolution), reducing the amount of calculation, and at the same time combines ReLU activation to enhance the nonlinear expression ability, making the weight map β c Can dynamically filter task-related channels (such as skin color channels in face detection).

[0016] Furthermore, in step 3, the cross-level attention guidance module adopts a multi-head attention mechanism (such as 4 heads) to transform the spatial feature F s As the query vector, the channel feature F c As the key-value vector, the interaction weight is calculated by matrix inner product. Dimension scaling factor (d is the feature dimension) is used to alleviate the problem of excessively large inner product values in high-dimensional spaces, and Softmax normalization ensures that the weight distribution conforms to probabilistic characteristics. This design enables the model to adaptively establish correlations between spatial and channel features. For example, in medical imaging, it can strengthen the association weight between the spatial location of a lesion and its pathological characteristic channels.

[0017] Furthermore, in step 4, the convolutional blocks of the Feature Pyramid Network use a 3×3 convolution kernel and a downsampling operation with a stride of 2 to gradually compress the feature map size to 1 / 4 of its original size, while expanding the number of channels to 512 dimensions to enhance expressiveness. The global attention fusion block uses spatial pyramid pooling (ASPP) in parallel with multi-scale dilated convolutions (with dilation rates of 1 / 6 / 12 / 18) to capture contextual information from different receptive fields. Finally, multi-scale features are fused through 1×1 convolution to improve the robustness of detection for large and small objects.

[0018] Furthermore, in step five, the downstream task network is dynamically configured based on the application scenario: in autonomous driving, a YOLO detector head is used for real-time obstacle localization (≥25 FPS); in medical image analysis, a U-Net decoder is integrated to perform lesion segmentation (Dice coefficient ≥ 0.91). The probability value output by the sigmoid function is passed through a threshold τ (default 0.5) for binary classification. In actual deployment, τ can be dynamically adjusted to balance precision and recall. For example, in industrial quality inspection, setting τ = 0.7 can reduce the false positive rate.

[0019] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0020] Through the above-mentioned technological breakthroughs, the present invention significantly improves the performance of feature fusion methods in noise suppression, detail preservation, and cross-level interaction, providing a universal and efficient solution for high-precision image analysis tasks, and showing broad application prospects and industrial value in fields such as autonomous driving, medical image analysis, and industrial quality inspection. The feature fusion method based on multi-level attention guidance provided by the present invention, through the innovative combination of multi-level attention mechanism and dynamic fusion strategy, shows significant advantages in feature expression optimization, noise suppression, and cross-level interaction, which are specifically reflected in the following core effects:

[0021] The spatial attention module and the channel attention module work together to dynamically identify and enhance key regions and important channel features in an image. For example, in medical image analysis, spatial attention can precisely focus on the subtle structures at the edge of a lesion, while channel attention selects channel information that is strongly correlated with pathological features, significantly improving the expression intensity of the target area while effectively suppressing interference from background noise or irrelevant textures, enhancing the discriminability of the feature map.

[0022] The Cross-Level Attention Guidance Module uses a multi-head attention mechanism to establish a dynamic connection between shallow detail features and deep semantic features, addressing the feature misalignment problem caused by scale differences in traditional methods. In object detection tasks, this module adaptively integrates feature responses from different levels, for example, strengthening the correlation between detailed information of small-scale objects and global semantics, significantly improving the accuracy of object localization and classification in complex scenarios.

[0023] The Feature Pyramid Network achieves efficient fusion of multi-resolution features through lightweight cascaded convolutions and fast spatial pyramid pooling. This design reduces computational resource consumption while balancing the extraction of local details with global contextual information, flexibly adapting to the inspection needs of objects of varying sizes. For example, in industrial quality inspection, it can capture the fine texture of tiny defects while analyzing the compliance of the overall product structure, significantly improving the generalization and robustness of inspection tasks.

[0024] The algorithm reduces computational complexity while maintaining high accuracy through dynamic weight allocation and lightweight module design. For example, in autonomous driving scenarios, the fused features can effectively cope with interference such as lighting changes and occlusions, supporting the real-time processing requirements of on-board embedded devices. In medical image analysis, the algorithm demonstrates strong robustness to low-contrast lesions or noise interference, providing reliable support for clinical diagnosis.

[0025] Through the above-mentioned technological breakthroughs, the present invention realizes the precise extraction and efficient utilization of key information in the feature fusion process, solves the core problems of traditional methods such as unstable fusion effect and loss of details in complex scenarios, and provides a highly versatile and adaptable solution for tasks such as image recognition and target detection. It has broad application value in industrial quality inspection, autonomous driving, and medical imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is an overall architecture diagram of a feature fusion method based on multi-level attention guidance provided by an embodiment of the present invention.

[0027] Figure 2 This is a timing diagram of a cross-level attention mechanism of a feature fusion method based on multi-level attention guidance provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0029] The following takes the deep fake detection task as an example to describe the specific implementation process of the present invention in detail: First, the RGB face image to be detected is preprocessed, the image is uniformly scaled to 512×512 pixels, and the interference of illumination differences is eliminated by histogram equalization. The pixel values are then normalized so that they are distributed in the interval [-1, 1]. The pre-trained EfficientNet-B4 network is used as the backbone model, and the shallow feature map (size 128×128×64) output by the Conv3_x layer and the deep feature map (size 32×32×320) output by the Conv5_x layer are extracted respectively. The shallow features retain high-frequency detail information such as facial edges and micro-expressions, while the deep features encode the overall semantic and structural features of the face. The deep features are upsampled to a resolution of 128×128 by bilinear interpolation, forming a spatially aligned multi-scale feature pyramid with the shallow features, providing a basis for subsequent fusion.

[0030] In the spatial attention enhancement stage, global average pooling is performed on the shallow feature map to generate a spatial attention map. Subsequently, two fully connected layers are used to learn regional importance weights. A sigmoid function is used to generate a 0-1 attention coefficient, which is multiplied element-by-element with the original features to enhance areas with significant forgery traces (such as abnormal edges and unnatural textures). In the channel attention screening stage, 1×1 convolution is used on deep features to compress the channel dimension to 20 dimensions. The ReLU activation function is used to generate a channel weight map, screening channels that are strongly correlated with forged features (such as facial optical flow anomalies and discontinuous skin color distribution). Finally, feature reconstruction is completed through channel-dimensional broadcasting.

[0031] The shallow features after spatial enhancement are used as query vectors, and the deep features after channel filtering are used as key-value vectors. A 4-head attention mechanism is used to calculate the association weight matrix between shallow details and deep semantics, and the dimension scaling factor is set to the square root of the feature dimension. After normalizing the interaction weights through the Softmax function, the deep features are weighted and summed according to the weights to generate a cross-modal feature map, which retains the details and semantic consistency of the forged traces.

[0032] The cross-modal features are fed into a cascade of 3×3 convolutional blocks (stride 2), progressively downsampled to a 32×32 resolution and expanded to 512 dimensions. Batch normalization and LeakyReLU activation (α=0.2) are applied after each convolutional block. Furthermore, four sets of dilated convolutions with different dilation rates (1, 6, 12, and 18) are performed in parallel using fast spatial pyramid pooling to capture multi-scale contextual information. The multi-scale features are then concatenated and fused using 1×1 convolutions, ultimately outputting an optimized feature map with a resolution of 128×128.

[0033] The optimized features are compressed to a 512-dimensional feature vector using global average pooling. This is then mapped to a binary classification output using two fully connected layers, and a sigmoid function is used to generate a forgery probability value. A heat map is generated based on the spatial attention weights, highlighting suspected forgeries (e.g., unnatural mouth movements, unusual eye reflections). When the probability exceeds a threshold of τ = 0.65, the image is identified as a deepfake and a detection report containing positioning information is output.

[0034] This embodiment demonstrates significant advantages in deepfake detection: a spatial attention mechanism accurately locates tampering boundaries, a cross-layer fusion module effectively captures subtle forgery traces, and a multi-scale optimization design maintains high robustness in complex lighting and occlusion scenarios. This solution can be expanded to other scenarios, such as speech synthesis forgery detection and video tampering location, providing reliable technical support for verifying the authenticity of digital content.

Claims

1. A feature fusion method based on multi-level attention guidance, characterized in that: The following steps are involved: S1: Extract multi-scale feature maps through convolutional neural networks to obtain shallow detail feature maps f s And deep semantic feature map f n-1 ; S2: Perform spatial attention weighting and channel attention weighting on the multi-scale feature map to generate spatial enhanced features F s and channel enhancement feature F c ; S3: Construct a cross-level attention guidance module to perform attention on F through a double cross attention mechanism. s and F c Perform dynamic interactive fusion and output cross-modal features F cross ; S4: Input the fused cross-modal features into the feature pyramid network, and generate the optimized fused features F through cascade convolution and fast spatial pyramid pooling. out .

2. The feature fusion method based on multi-level attention guidance according to claim 1 is characterized in that The spatial attention weighting in step S2 includes: using a global average pooling layer to extract the spatial attention map M s , calculate the spatial weight coefficient α through the fully connected layer s , and press Perform feature reconstruction, where represents the element-wise product, and σ is the Sigmoid activation function.

3. The feature fusion method based on multi-level attention guidance according to claim 1 is characterized in that The channel attention weighting in step S2 includes: using a point convolution layer to extract channel features f c , combined with the ReLU activation function to generate the channel weight map β c , and press F c =f c ⊙β c Complete channel dimension feature screening, where ⊙ represents the broadcast product of the channel dimension.

4. The feature fusion method based on multi-level attention guidance according to claim 1 is characterized in that The double cross attention mechanism of step S3 is specifically as follows: the spatial enhancement feature f s As the query vector, the channel-enhanced feature f c As the key-value vector, the interaction weight matrix W is calculated through multi-head attention cross ,according to Realize feature fusion, where d is the feature dimension.

5. The feature fusion method based on multi-level attention guidance according to claim 1 is characterized in that: The feature pyramid network of step S4 includes: at least two convolution fusion blocks and one global attention fusion block, wherein the convolution kernel size is set to 3×3 and the stride is 2, and the upsampling factor is set to 2.

Images