A Visual Attention Learning Method for Ghost Image Orientation

Abstract

The present invention provides a method for learning visual attention of ghost image orientation. Existing methods for learning visual attention of orientation rely on convolutional operations for dimensional transformation and orientation information learning, with relatively high computational and parameter costs. Therefore, the present invention couples a lightweight ghost module into the method for learning visual attention of orientation, designs a dimensionality reduction ghost module to lightweight compress the feature map, and then conducts orientation information learning to save computational and parameter costs; designs a redundancy removal ghost module to reduce redundant information in the feature map, which can improve the feature quality. The present invention, as a novel visual attention mechanism, has wide applications, such as image target detection, image target recognition, image segmentation, etc.

Description

Technical Field

[0001] The present invention relates to the fields of machine vision and image / video applications, and particularly to a method for learning the visual attention of ghost directions. Background Art

[0002] The visual attention mechanism can suppress the irrelevant regions in the input feature map while highlighting the features of the key regions, effectively improving the feature learning ability of the neural network. However, using the attention mechanism often increases the model's computational cost and slows down the running speed. For example, in the direction attention learning mechanism, multiple convolutional branches are used to learn the long-range dependence in the horizontal and vertical directions, that is, the direction fusion information, and the convolutional model parameters and computational cost are huge due to the multiple uses of convolution. Summary of the Invention

[0003] The present invention provides a method for learning the visual attention of ghost directions, which uses a lightweight ghost module to efficiently fuse direction information and reduce redundant information in features, improving the feature learning ability of the model. As Figure 1 shown, the method includes:

[0004] Step 1, lightweight ghost direction information learning, designing a dimensionality reduction ghost module to achieve lightweight vertical and horizontal direction information learning.

[0005] As Figure 2 shown, the specific approach of the designed dimensionality reduction ghost module is:

[0006] Step 1.1, perform feature dimensionality reduction on the input feature map X to obtain the output feature map F1, and the calculation is as shown in formula (1):

[0007]

[0008] Wherein, represents the convolution operation, W is a convolution kernel of size I×J×1×1, where I and J = MAX(8, I / 32) respectively represent the number of channels of the input X and the number of channels of the output F1;

[0009] Step 1.2, perform a lightweight linear operation on each channel of the output feature map F1 obtained in step 1.1 to obtain F2, and its calculation is as shown in formula (2):

[0010] F2 = [Φ1(F1 1 ), Φ2(F1 2 ),..., Φ n (F1 n )], n = 1,..., J (2)

[0011] Wherein, F1 n represents the feature map on the nth channel of F1, and Φ nDenote the linear operation on F1 n Perform a linear operation;

[0012] Step 1.3: Concatenate the output feature map F1 obtained in Step 1.1 and the output feature map F2 obtained in Step 1.2 to obtain an output feature map Y with 2J channels, and the calculation is as shown in formula (3):

[0013] Y = Concat(F1, F2) (3)

[0014] where Concat represents the concatenation operation along the channel dimension.

[0015] Step 2: Lightweight ghost feature redundancy removal. Design a redundancy removal ghost module to reduce the redundancy of the input feature map and improve the feature quality.

[0016] As Figure 3 shown, the specific approach of the designed redundancy removal ghost module is as follows:

[0017] Step 2.1: Perform feature transformation on the input feature map X to obtain a feature map G1, and the calculation formula is as shown in formula (4):

[0018]

[0019] where represents the convolution operation, Ω is a convolution kernel of size , I is the number of channels of X, and the number of channels of G1 is half of X;

[0020] Step 2.2: Perform a lightweight linear operation on each channel of the output feature map G1 obtained in Step 2.1 to obtain G2, and its calculation formula is as shown in formula (5):

[0021]

[0022] where represents the feature map on the nth channel in G1, and ψ n represents the linear transformation on ;

[0023] Step 2.3: Concatenate the output feature map G1 obtained in Step 2.1 and the output feature map G2 obtained in Step 2.2 to obtain an output feature map Z, and the calculation is as shown in formula (6):

[0024] Z = Concat(G1, G2) (6)

[0025] where Concat represents the concatenation operation along the channel dimension, and the number of channels of the output Z is I, which is equal to the number of channels of X.

[0026] After adopting the above solution, the present invention couples the ghost module based on lightweight linear transformation to the existing azimuth visual attention learning method, that is, uses the dimensionality reduction ghost module to replace the convolution operations in the dimensionality transformation and the horizontal and vertical azimuth information learning in the existing azimuth visual attention, which can save computational and parameter costs, and uses the redundancy-reducing ghost module to reduce feature redundancy and improve the feature quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a structural block diagram of the ghost azimuth visual attention learning method of the present invention.

[0028] Figure 2 It is a schematic diagram of the structure of the dimensionality reduction ghost module.

[0029] Figure 3 It is a schematic diagram of the structure of the redundancy-reducing ghost module.

[0030] Figure 4 It is a schematic diagram of the backbone feature extraction network of the present invention.

[0031] Figure 5 It is a schematic diagram of the feature fusion pyramid. DETAILED DESCRIPTION OF THE INVENTION

[0032] As Figure 1 shown, the present invention provides a ghost azimuth visual attention learning method, and the specific implementation cases of the method are as follows:

[0033] Step 1, lightweight ghost azimuth information learning, design a dimensionality reduction ghost module to achieve lightweight vertical and horizontal azimuth information learning.

[0034] As Figure 2 shown, the specific approach of the designed dimensionality reduction ghost module is:

[0035] Step 1.1, perform feature dimensionality reduction on the input feature map X to obtain the output feature map F1, and the calculation is as shown in formula (1):

[0036]

[0037] Wherein, represents the convolution operation, W is a convolution kernel of size I×J×1×1, where I and J = MAX(8, I / 32) respectively represent the number of channels of the input X and the number of channels of the output F1;

[0038] Step 1.2, perform a lightweight linear operation on each channel of the output feature map F1 obtained in step 1.1 to obtain F2, and its calculation is as shown in formula (2):

[0039] F2 = [Φ1(F1 1 ), Φ2(F1 2 ),..., Φn (F1n)], n = 1, ..., J (2)

[0040] Among them, F1 n represents the feature map on the nth channel in F1, and Φ n represents performing a linear operation on F1 n ;

[0041] Step 1.3: Concatenate the output feature map F1 obtained in Step 1.1 and the output feature map F2 obtained in Step 1.2 to obtain an output feature map Y with 2J channels, and the calculation is as shown in formula (3):

[0042] Y = Concat(F1, F2) (3)

[0043] Among them, Concat represents the concatenation operation along the channel dimension.

[0044] As Figure 1 shown, the specific method for learning longitudinal and lateral orientation information is as follows:

[0045] First, perform horizontal average pooling and vertical average pooling on the input feature map X respectively. After transposing the feature map generated by the horizontal pooling kernel and performing tensor concatenation with the feature map generated by the vertical pooling kernel, send it to the dimensionality reduction ghost module to reduce the dimension. Secondly, use 1×1 convolution for horizontal and vertical information fusion. Finally, split the feature map generated in the previous step into two groups of orientation information tensors along the space, and after transforming each group of tensors to have the same number of channels as X, use them as the output horizontal and vertical orientation information.

[0046] Step 2: Lightweight ghost feature redundancy removal. Design a redundancy removal ghost module to reduce the redundancy of the input feature map and improve the feature quality.

[0047] As Figure 3 shown, the specific method for the designed redundancy removal ghost module is as follows:

[0048] Step 2.1: Perform feature transformation on the input feature map X to obtain a feature map G1, and the calculation formula is as shown in formula (4):

[0049]

[0050] Among them, represents the convolution operation, Ω is the convolution kernel of size , I is the number of channels of X, and the number of channels of G1 is half of X;

[0051] Step 2.2: Perform lightweight linear operations on each channel of the output feature map G1 obtained in Step 2.1 to obtain G2, and its calculation formula is as shown in formula (5):

[0052]

[0053] in, represents the feature map on the nth channel in G1, ψ n Express Perform linear transformation;

[0054] Step 2.3: Concatenate the output feature map G1 obtained in step 2.1 and the output feature map G2 obtained in step 2.2 to obtain the output feature map Z. The calculation is shown in formula (6):

[0055] Z=Concat(G1,G2) (6)

[0056] Among them, Concat represents the concatenation operation along the channel dimension, and the number of channels of the output Z is I, which is equal to the number of channels of X. Figure 1 As shown, the above-mentioned lightweight ghost orientation information learning step obtains the horizontal and vertical orientation information, which are multiplied by the Z position respectively to obtain the feature map of ghost orientation attention enhancement as the final output.

[0057] Furthermore, combining the ghost orientation attention learning method of the present invention with YOLO-V5 to build a target detection model as an implementation case of the present invention, specifically including the following steps:

[0058] Step S1: Figure 4 As shown, a backbone feature extraction network is constructed, and the ghost orientation visual attention learning method module of the present invention is used (see Figure 1 ) replaces the ordinary convolution layer of YOLO-V5, and the backbone feature extraction network is composed of alternating stacking of ordinary convolution and ghost orientation visual attention learning method;

[0059] Step S2: Figure 5 As shown in the figure, a feature fusion pyramid network is built to fuse features of different layers. The feature pyramid network is top-down, and the high-level features are fused with the bottom-level features through upsampling to obtain the feature map for prediction. The high-level semantic features are passed down, and the obtained feature map is input into the detection head. The feature map of the backbone feature extraction network built in step S1 is output to the feature fusion pyramid through horizontal connection to fuse features of different scales. Figure 5 SPP in the middle is the spatial pyramid pooling layer;

[0060] Step S3: Set the detection head structure, and downsample the feature map by 32, 16, and 8 times respectively to form feature maps of three scales as the detection head, forming a target detection model based on the ghost orientation visual attention learning method.

[0061] As described above, this is only an embodiment of the present invention and does not impose any limitation on the technical scope of the present invention. Therefore, any minor modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A method for learning visual attention of ghost image orientation, characterized in that: Couple a lightweight ghost module to the azimuth visual attention learning method, the method comprising: Step 1.1, lightweight ghost azimuth information learning, designing a dimensionality reduction ghost module to achieve lightweight longitudinal and lateral azimuth information learning; Step 1.2, lightweight ghost feature redundancy removal, designing a redundancy removal ghost module to reduce the redundancy of the input feature map and improve the feature quality; The construction process of the dimensionality reduction ghost module in the said step 1.1 is as follows: Step 2.

1. Perform feature dimensionality reduction on the input feature map X to obtain the output feature map , which is calculated as shown in formula (1): (1) Among them, represents a convolution operation, is a convolution kernel of size, where I and respectively represent the number of channels of the input X and the number of channels of the output . Step 2.

2. Perform a lightweight linear operation on each channel of the output feature map obtained in Step 2.1 to obtain , and its calculation is shown in Formula (2): (2) Among them, denotes the feature map on the n th channel in which means performing a linear operation on ; Step 2.3, concatenate the output feature maps obtained in Step 2.1 and the output feature maps obtained in Step 2.2 to obtain an output feature map with 2 channels J as shown in the following formula (3): Y The calculation is as follows: (3) Among them, Concat represents a concatenation operation along the channel dimension; The construction process of the redundancy removal ghost module in the said step 1.2 is as follows: Step 3.

1. Perform feature transformation on the input feature map X to obtain the feature map , and the calculation formula is as shown in formula (4): (4) Among them, represents a convolution operation, is a convolution kernel of size I is X the number of channels of The number of channels of X is half of Step 3.2: Perform a lightweight linear operation on each channel of the output feature map obtained in Step 3.1 to obtain , and its calculation formula is as shown in Formula (5): (5) Among them, represents the feature map on the n th channel in indicating a linear transformation on ; Step 3.3: Concatenate the output feature maps obtained in Step 3.1 and the output feature maps obtained in Step 3.2 to obtain the output feature map Z , which is calculated as shown in formula (6): (6) Among them, Concat represents the concatenation operation along the channel dimension, and the output Z has a channel number of I , which is equal to the channel number of X .

Images