Feature fusion module based on wavelet transform
By using a wavelet transform-based feature fusion module in the YOLOv5s model, the low-level features are downsampled and high-level features are upsampled, and feature splicing is combined with low-frequency and high-frequency components, the existing model loses detailed information and the The problem of being unable to utilize multi-scale features has been achieved to achieve stronger feature expression and detection performance.
Patent Information
- Application Number
- CN202411611193.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-12
- Publication Date
- 2025-02-18
AI Technical Summary
The existing YOLOv5s model uses PANet structure in the feature fusion part, resulting in the loss of important details of feature downsampling and upsampling operations, and the inability to fully utilize feature information from other scales, affecting the effect of detection of smaller defect targets in complex backgrounds.
The feature fusion module based on wavelet transform is adopted to downsample low-level features and upsample high-level features through wavelet transform, and feature splicing is combined with low-frequency and high-frequency components to achieve the capture and fusion of multi-scale feature information.
Effectively suppress noise, reduce information redundancy, enhance feature expression capabilities, better handle tasks with different size targets, and improve the detection performance of the model in complex contexts.
Smart Images
Figure CN119478611A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a feature fusion module, in particular to a feature fusion module based on wavelet transform. Background Art
[0002] YOLOv5 is a target detection model with a low training threshold and relatively balanced detection performance in the YOLO series of algorithms. It is currently widely used in the field of power transmission line defect detection.
[0003] The initial YOLOv5s model uses the PANet structure in the feature fusion part. Specifically, it uses strided convolution and transposed convolution to implement feature downsampling and upsampling operations, and then concats feature maps of the same size to achieve feature fusion. For insulator defect detection tasks, this method has some disadvantages:
[0004] First, the PAN network uses strided convolution and transposed convolution to implement downsampling and upsampling, which directly reduces the scale of the input feature map, resulting in the loss of important detail information, especially for small-scale defect targets, which may lead to missed detection.
[0005] Secondly, the original YOLOv5s can only fuse information of feature maps of the same scale, and cannot make full use of feature information of other scales. It is easy to ignore the global context information in the image, which will have an adverse effect on the detection of smaller defect targets in complex backgrounds. Summary of the invention
[0006] The purpose of the present invention is to provide a feature fusion module based on wavelet transform in view of the shortcomings and defects in the prior art. The feature fusion module based on wavelet transform has the function of capturing multi-scale feature information, which not only effectively suppresses noise and reduces information redundancy, but also can enhance feature expression capability.
[0007] To achieve the above-mentioned purpose, the present invention adopts the following technical scheme: a feature fusion module based on wavelet transform, which includes the following specific steps: first, down-sampling the low-level features by wavelet transform; second, up-sampling the high-level features by wavelet transform; finally, splicing the current level features with the low-level features and the high-level features to obtain the fused features.
[0008] Furthermore, the specific operation of performing wavelet transform downsampling on the low-level features is as follows: first, using wavelet transform (WT) to decompose the input features into a low-frequency wavelet component yL and a high-frequency wavelet component yH; then, high-frequency components in the horizontal (HL), vertical (LH), and diagonal (HH) directions are extracted from the high-frequency wavelet component yH; finally, the extracted high-frequency component and low-frequency wavelet component yL are concatenated into a new tensor along the channel dimension, and finally a downsampled output is obtained through a 1×1 convolution.
[0009] Furthermore, the high-frequency wavelet component contains detailed information of the image.
[0010] Furthermore, the specific operation of performing wavelet transform upsampling on high-level features is as follows: first, a 1×1 convolution operation is performed on the input features; then, a Slicing operation is used to divide the input channel into four parts: horizontal (HL), vertical (LH), diagonal (HH) and low-frequency wavelet component yL; high-frequency components in three different directions are stacked into high-frequency wavelet components; finally, an inverse wavelet transform (IWT) is used to reconstruct the high-frequency and low-frequency wavelet components to obtain the wavelet upsampling output.
[0011] It includes the following specific calculation formula: Z = Concat (f 1×1 (WTD(x 0 )),x 1 ,WTU(f 1×1 (x 2 ))); WTD(x)=Concat([HL,LH,HH],yL); WTU(x)=IWT(Stack(HL,LH,HH),yL);
[0012] yH=[HL,LH,HH],where x 0 ,x 1 ,x 2 They are three features of different scales, x 0 is the low-level feature, x 1 is the current level feature, x 2 is the high-level feature, Z is the output feature after fusion; WTD is the wavelet transform downsampling operation, WTU is the wavelet transform upsampling operation, IWT is the inverse wavelet transform; yL is the low-frequency wavelet component, yH is the high-frequency wavelet component, f 1×1 It is a convolution with a convolution kernel size of 1×1, Concat is feature concatenation, and Stack is a feature stacking operation.
[0013] Furthermore, the high-frequency wavelet component yH includes three components in different directions, namely a high-frequency horizontal component HL, a high-frequency vertical component LH and a high-frequency diagonal component HH.
[0014] After adopting the above technical solution, the beneficial effects of the present invention are: at least the following advantages:
[0015] 1. Capture of multi-scale feature information: Wavelet transform decomposes the input features into low-frequency and high-frequency components, allowing the model to process global information and detail information at the same time, retaining the multi-scale characteristics of the feature map. The low-frequency component provides the overall structure of the image, while the high-frequency component captures details and edge features, which is very useful in processing tasks with objects of different sizes.
[0016] 2. Effective noise suppression: Wavelet transform has good noise resistance. Through decomposition and reconstruction, it can filter out noise to a certain extent while retaining important image information. This is very important for improving the robustness of the model when performing target detection or segmentation tasks in complex environments.
[0017] 3. Reduce information redundancy: Compared with directly performing convolution operations on the original feature map, wavelet transform can decompose the feature map into different frequency components, reducing the redundancy between features. This decomposition and fusion method helps the model to extract useful information more effectively and improve computational efficiency.
[0018] 4. Enhanced feature expression capabilities: By performing convolution processing and reconstruction on components of different frequencies, the model can learn richer and more diverse feature expressions, thereby improving the performance of the model in various visual tasks, especially when dealing with complex backgrounds and diverse targets. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0020] Figure 1 It is a structural schematic diagram of the present invention.
[0021] Figure 2 It is a comparison chart of the YOLOv5s+WTFF model in the present invention.
[0022] Figure 3 It is a comparison chart of the initial YOLOv5s model in the present invention. DETAILED DESCRIPTION
[0023] See also Figure 1 As shown, the technical solution adopted in this specific implementation is: it includes the following specific steps:
[0024] S1, first, perform wavelet transform downsampling on the low-level features, specifically:
[0025] First, wavelet transform (WT) is used to decompose the input features into low-frequency wavelet components yL and high-frequency wavelet components yH;
[0026] Then, the high-frequency components in the horizontal (HL), vertical (LH), and diagonal (HH) directions are extracted from the high-frequency wavelet component yH; the high-frequency wavelet component contains the detailed information of the image.
[0027] Finally, the extracted high-frequency component and low-frequency wavelet component yL are concatenated into a new tensor along the channel dimension, and finally a 1×1 convolution is performed to obtain the downsampled output.
[0028] S2, secondly, perform wavelet transform upsampling on high-level features, specifically:
[0029] First, a 1×1 convolution operation is performed on the input features;
[0030] Then, the Slicing operation is used to divide the input channel into four parts: horizontal (HL), vertical (LH), diagonal (HH), and low-frequency wavelet component yL;
[0031] The high-frequency components in three different directions are stacked into high-frequency wavelet components;
[0032] Finally, the high-frequency and low-frequency wavelet components are reconstructed using inverse wavelet transform (IWT) to obtain the wavelet upsampled output.
[0033] S3, finally, concatenate the current level features with the lower level features and the higher level features to obtain the fused features.
[0034] It includes the following specific calculation formula:
[0035] Z = Concat(f 1×1 (WTD(x 0 )),x 1 ,WTU(f 1×1 (x 2 ))) (1)
[0036] WTD(x)=Concat([HL,LH,HH],yL) (2)
[0037] WTU(x)=IWT(Stack(HL,LH,HH),yL) (3)
[0038] yH=[HL,LH,HH] (4)
[0039] Among them, x 0 ,x 1 ,x 2 They are three features of different scales, x 0 is the low-level feature, x 1 is the current level feature, x 2 is the high-level feature, Z is the output feature after fusion; WTD is the wavelet transform downsampling operation, WTU is the wavelet transform upsampling operation, IWT is the inverse wavelet transform; yL is the low-frequency wavelet component, yH is the high-frequency wavelet component, f 1×1is a convolution with a convolution kernel size of 1×1, Concat is feature concatenation, and Stack is a feature stacking operation, wherein the high-frequency wavelet component yH contains three components in different directions, namely, a high-frequency horizontal component HL, a high-frequency vertical component LH, and a high-frequency diagonal component HH.
[0040] The following are the relevant experiments of the present invention
[0041] In order to verify the effectiveness of WTFF in actual detection tasks, this study conducted an ablation experiment based on the background of transmission line insulator defect detection and YOLOv5s as the benchmark model. The experimental results are as follows:
[0042]
[0043] Comparison of detection effects:
[0044] To further verify the effectiveness of WTFF, the YOLOv5s+WTFF model was used to compare the detection results with the initial YOLOv5s model. Figure 2-Figure 3 shown.
[0045] The above description is only used to illustrate the technical solution of the present invention rather than to limit it. Other modifications or equivalent substitutions made to the technical solution of the present invention by ordinary technicians in this field should be included in the scope of the claims of the present invention as long as they do not depart from the spirit and scope of the technical solution of the present invention.
Claims
1. A feature fusion module based on wavelet transform, characterized in that: It includes the following specific steps: S1, first, downsample the low-level features by wavelet transform; S2, secondly, upsampling the high-level features by wavelet transform; S3, finally, concatenate the current level features with the lower level features and the higher level features to obtain the fused features.
2. The feature fusion module based on wavelet transform according to claim 1, characterized in that: The specific operation of S1 is: 1) First, wavelet transform (WT) is used to decompose the input features into low-frequency wavelet components yL and high-frequency wavelet components yH; 2) Then, the high-frequency components in the horizontal (HL), vertical (LH), and diagonal (HH) directions are extracted from the high-frequency wavelet component yH; 3) Finally, the extracted high-frequency component and low-frequency wavelet component yL are concatenated into a new tensor along the channel dimension, and finally a 1×1 convolution is performed to obtain the downsampled output.
3. The feature fusion module based on wavelet transform according to claim 1, characterized in that: The high-frequency wavelet component contains the detail information of the image.
4. The feature fusion module based on wavelet transform according to claim 1, characterized in that: The specific operation of S2 is: 1) First, perform a 1×1 convolution operation on the input features; 2) Then, the Slicing operation is used to divide the input channel into four parts: horizontal (HL), vertical (LH), diagonal (HH) and low-frequency wavelet component yL; 3) Stack the high-frequency components in three different directions into high-frequency wavelet components; 4) Finally, the high-frequency and low-frequency wavelet components are reconstructed using the inverse wavelet transform (IWT) to obtain the wavelet upsampled output.
5. A feature fusion module based on wavelet transform, characterized in that: It includes the following specific calculation formula: Z=Concat(f 1×1 (WTD(x0)), x1,WTU(f 1×1 (x2))) (1) WTD(x)=Concat([HL,LH,HH],yL) (2) WTU(x)=IWT(Stack(HL,LH,HH),yL) (3) yH=[HL,LH,HH] (4) Among them, x0, x1, and x2 are features of three different scales, x0 is a low-level feature, x1 is a current level feature, x2 is a high-level feature, and Z is the fused output feature; WTD is a wavelet transform downsampling operation, WTU is a wavelet transform upsampling operation, and IWT is an inverse wavelet transform; yL is a low-frequency wavelet component, y H is the high frequency wavelet component, f 1×1 It is a convolution with a convolution kernel size of 1×1, Concat is feature concatenation, and Stack is a feature stacking operation.
6. The feature fusion module based on wavelet transform according to claim 5, characterized in that: The high-frequency wavelet component yH includes three components in different directions, namely a high-frequency horizontal component HL, a high-frequency vertical component LH and a high-frequency diagonal component HH.