Improved infrared small target detection method based on knowledge enhancement network

Abstract

The invention provides an improved infrared small target detection method based on a knowledge enhancement network, and belongs to the field of infrared target detection. According to the method, firstly, based on the knowledge of infrared characteristics of brightness and direction, a foreground small target in an infrared image is higher in temperature different from a surrounding scene, and is brighter visually; the contour motion trends and smooth textures of the background and the foreground in the infrared image are obviously different visually, a general saliency feature extraction scheme is improved, and saliency features of the brightness and the direction of the small infrared target are extracted; secondly, a weighting activation fusion module fusing traditional knowledge features and deep learning features is provided, a feature map is effectively weighted and activated again, and it is ensured that the model focuses on the most relevant features; the knowledge enhancement network provided by the invention can be applied to any existing target detection model due to the fact that the knowledge enhancement network is constructed to be modularized, the existing model is assisted to improve the detection capability, and the contribution significance is great on the one hand.

Description

Technical Field

[0001] The invention belongs to the field of infrared target detection, and in particular relates to an improved infrared small target detection method based on a knowledge enhancement network. Background Art

[0002] Infrared sensors are widely used in a range of fields such as surveillance, defense, and remote sensing due to their fast response and robustness, and can be used to capture fast-moving scenes. As a key task in computer vision, infrared target detection can be mainly divided into general target detection and small target detection. With the advancement of many general target detection methods, most infrared detection problems related to positioning and classification can now be solved. Therefore, single-frame infrared small target detection methods have attracted more attention. Unlike general object detection, targets in infrared images usually lack clear color and texture information. Small infrared targets are usually smaller than 30×30 pixels due to being captured from a considerable distance, such as long-distance maritime rescue missions involving individuals or ships, identifying unauthorized drones or floating objects in the air. At the same time, these objects are not only small in size, but are often surrounded by complex backgrounds, which makes infrared small target detection more challenging.

[0003] Traditional methods usually rely on hand-crafted features and heuristic-based algorithms, including filter-based methods, local contrast-based methods, and low-rank-based methods, which are difficult to generalize well in different scenarios. In recent years, with the emergence of deep learning technology, especially convolutional neural networks, it has shown strong results in various computer vision tasks including object detection. However, if the existing excellent general object detectors such as R-CNN series, YOLO series, SSD, DETR series are directly used without designing additional infrared small target feature extraction methods, due to the lack of high-level semantic information, the small target features in the deep convolutional neural network layer will be attenuated and cannot meet the performance requirements of small infrared target detection. Therefore, how to improve the detection model, extract representative features of infrared small targets, and have the ability to adapt to infrared small target detection has become a practical issue. Summary of the invention

[0004] In view of the problem that existing infrared small target detection is difficult to distinguish foreground and background, it is proposed to incorporate the brightness and direction characteristics of infrared small targets into the detection network to enhance the feature representation of small targets, making them easier to distinguish from the background. In view of the problem that small target features are lost in high-level neural networks, a multi-level feature fusion strategy combining deep learning features with traditional knowledge features of different scales is implemented. This strategy ensures the effective integration of low-level details and high-level semantic information, and can adaptively adjust the importance of different channel features, enhancing the ability of the model under different conditions.

[0005] In order to solve the above technical problems, the specific technical solution of an improved infrared small target detection method based on a knowledge enhanced network of the present invention is as follows:

[0006] An improved infrared small target detection method based on knowledge enhancement network includes the following steps:

[0007] An improved infrared small target detection method based on knowledge enhancement network is characterized by comprising the following steps:

[0008] S1: Acquire infrared small target image data and preprocess the image;

[0009] S2: Use Gaussian pyramid to extract multi-scale brightness features of the image, use Gabor filter to extract multi-scale direction features of the image, and then fuse the brightness features and direction features into multi-scale expert features;

[0010] S3: Build a multi-scale encoder-decoder network; the network includes a stacked encoder and decoder, which are stacked through convolution kernels with different pooling numbers and channel numbers. The decoder constructs deep multi-scale features by combining the outputs of convolution kernels at different stages of the encoder; the preprocessed infrared small target image data is used as the input of the encoder;

[0011] S4: Construct an adaptive weighted activation fusion module of deep learning features and traditional knowledge features to fuse the expert features and the output of the encoder by scale as the input of the decoder;

[0012] S5: Calculate the loss function, obtain the final detection model after training and testing, and apply the detection model to infrared small target detection.

[0013] The multi-scale expert features are obtained by the following steps:

[0014]

[0015] Among them, X represents the input image, I represents the brightness feature, and N represents the normalization operation. represents the normalized brightness feature map, O represents the directional feature, θ∈(0°, 45°, 90°, 135°) represents the feature map of the directional feature in these four directions, represents the normalized directional feature, and T is the expert feature after the fusion output of the saliency map.

[0016] The normalization operation N is specifically as follows:

[0017]

[0018] F represents the brightness feature map or the direction feature map, F ′The input feature map F is normalized to the range [0…M], where M is the maximum value of the normalized range, and F min and F max Respectively represent the global maximum and global minimum in the feature map F; represents the mean of the local maximum value of the feature map, n represents the number of local rectangular areas contained in the feature map F, and F N is the final normalized result.

[0019] Before merging the brightness feature I and the directional feature O into the expert feature, it is necessary to first perform coarse-grained fusion on them. The calculation method is as follows:

[0020] And c=s+1

[0021]

[0022] Among them, c and s represent the sequence numbers of adjacent feature map layers of different sizes, which are related to the number of Gaussian pyramid layers, ranging from 1 to N; It means to interpolate the coarse scale to the fine scale and then subtract the matrix elements. I(c,s) represents the multi-scale brightness feature map after the fusion of coarse and fine granularity. O(c,s,θ) represents the multi-scale directional feature map after the fusion of coarse and fine granularity in each direction.

[0023] The multi-scale encoder-decoder network uses UIUNet as the base model.

[0024] The step S4 is specifically as follows:

[0025] According to different scales, the channel attention mechanism is applied, and the importance weight of each channel is learned using the fully connected layer to fuse the expert features and the deep features. Specifically, the two feature fusion calculation methods are as follows:

[0026] R H×W×C =w1·T H×W×1 +w2·D H×W×C

[0027] w1=w C+1 [0]

[0028] w2=w C+1 [1:C+1]

[0029] Among them, the dimension of the expert feature T is [1, H, W], the encoder output feature is used as the deep feature D, the dimension is [C, H, W], the three dimensions represent the number of channels, height, and width respectively; R represents the feature after the weighted activation fusion of the deep feature and the expert feature; w C+1 The corresponding column middle dimension is [1, 1, C+1] channel importance weight array, w1 is composed of w C+1Take the value of the array subscript 0 to get the channel importance weight of the expert feature, w2 is obtained by w C+1 The array subscripts from 1 to the last value are obtained to represent the importance weight of each channel of the deep feature map.

[0030] Then, the segmentation result is generated according to the decoder of UIUNet, and the loss function is calculated. Soft-IoU Loss is used to calculate the loss. The specific calculation method is as follows:

[0031]

[0032] s represents the sigmoid operation, gt represents the true value pixel of the manually labeled target, and its value is 0 or 1. is the predicted result. ε is used to prevent division by zero.

[0033] The advantages of this method are as follows:

[0034] This method is based on the knowledge of infrared characteristics of brightness and direction: 1) In infrared images, small foreground targets are different from surrounding scenes in temperature and will be brighter visually; 2) There are obvious visual differences in the contour motion trend and smooth texture of the background and foreground in infrared images. The general significant feature extraction scheme is improved to extract the significant features of the brightness and direction of small infrared targets;

[0035] Secondly, a weighted activation fusion module that integrates traditional knowledge features and deep learning features is proposed to effectively reweight and activate feature maps to ensure that the model focuses on the most relevant features. The knowledge enhancement network proposed in the present invention can be applied to any existing target detection model due to its modular nature, assisting the existing model in improving detection capabilities, which is a significant contribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the overall model of the present invention;

[0037] Figure 2 Schematic diagram of the channel attention module structure;

[0038] Figure 3 Schematic diagram of re-weighted activation fusion of knowledge features and deep features;

[0039] Figure 4 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0040] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of an improved infrared small target detection method based on a knowledge enhancement network of the present invention in conjunction with the accompanying drawings.

[0041] The present invention is implemented by the following scheme: an improved infrared small target detection method based on knowledge enhancement network, which includes the following steps, the flow chart is as follows Figure 4 As shown:

[0042] S1: Read in image data and preprocess the image, that is, normalize the image during training, randomly crop it into pixel blocks of size 256×256, and use random flipping to enhance the data.

[0043] S2: Use Gaussian pyramid and Gabor filter to extract brightness and direction features respectively, and fuse them into a salient feature map of infrared small targets;

[0044] S3: Build a basic multi-scale encoder-decoder network to extract deep learning feature maps of images;

[0045] S4: Construct a deep learning feature and traditional knowledge feature adaptive weighted activation fusion module to generate infrared small target feature maps after feature reactivation enhancement;

[0046] S5: Obtain the predicted mask of the infrared small target and calculate the loss function; the model obtained after 400 training epochs is used as the final detection model;

[0047] S6: Test the final model on the image to obtain the test index results.

[0048] Specifically, firstly, image data is obtained, which can be obtained from a public infrared small target detection dataset, and the image is preprocessed, including normalizing the image, randomly cropping it into pixel blocks of size 256×256, and using random flipping to enhance the data.

[0049] Then, after obtaining the preprocessed infrared small target detection image data, the small targets in the infrared image usually show two characteristic attributes: the brighter area is more likely to become the detection target, and the texture with different directions from the surrounding is more likely to become the foreground small target. Based on these two salient knowledge attributes, a multi-scale salient feature extraction scheme is designed to obtain the salient feature map of the infrared small target; the specific method is as follows:

[0050] The main process of extracting the visual characteristics of infrared thermal radiation brightness, texture contours from the attributes of infrared image objects is divided into two parts: brightness and direction feature extraction and expert feature saliency map generation, such as Figure 1 The specific steps of knowledge extraction are as follows:

[0051] Step 2.1: To extract the brightness feature map I, the brightness features of the image are calculated at different scales. Specifically, five feature maps of different sizes are generated based on multi-scale Gaussian pyramid sampling. For the brightness feature I, since infrared images are usually presented in the form of grayscale images, the higher the brightness, the higher the pixel value. Therefore, the brightness pyramid can be calculated in this way.

[0052] Step 2.2: For the directional feature O, a multi-scale pyramid of texture contours is constructed using a Gabor filter; the Gabor filter is a linear filter used for edge extraction. It mainly analyzes whether an image has specific frequency content in a specific direction in a specific area. Its frequency and direction expression are similar to the human visual system, which can provide good direction selection and scale selection characteristics, and is insensitive to changes in illumination. Therefore, it is very suitable for texture direction analysis. In the spatial domain, a two-dimensional Gabor filter is a Gaussian kernel function modulated by a sinusoidal plane wave. Its impulse response is defined as a sine wave (a plane wave for a two-dimensional Gabor filter) multiplied by a Gaussian function. Due to the multiplication and convolution properties, the Fourier transform of the impulse response of the Gabor filter is the convolution of the Fourier transform of the harmonic function and the Fourier transform of the Gaussian function. The specific representation is as follows:

[0053]

[0054] x ′ =xcosθ+ysinθ

[0055] y ′ =-xsinθ+ycosθ

[0056] θ∈[0°,45°,90°,135°]

[0057] Two-dimensional Gabor kernel function It is obtained by multiplying a Gaussian envelope function and a cosine function, where represents the phase shift, which controls the phase of the sine wave; λ represents the wavelength, which determines the period of the sine wave; σ represents the standard deviation of the Gaussian function, which controls the width of the Gaussian envelope; γ is the spatial aspect ratio, which represents the scale difference between the filter in the horizontal and vertical directions (usually less than 1); θ is the direction of the filter, which controls the orientation of the filter. θ∈[0°,45°,90°,135°] determines the feature maps in four directions that can be obtained based on Gabor filtering. x and y represent the image space coordinates, x ′ ,y ′ The coordinates are rotated around (x, y). By superimposing sine or cosine waves, a selective filtering effect on a specific frequency is formed.

[0058] Step 2.3: After initially obtaining the brightness feature I and directional feature O in the above multi-scale traditional feature map, it is necessary to perform coarse-grained fusion on them to retain more feature information from different sizes. The calculation method is:

[0059] And c=s+1

[0060]

[0061] Among them, c and s represent the sequence numbers of adjacent feature map layers of different sizes, which are related to the number of multi-scale pyramid layers obtained in the previous step. The number of layers ranges from 1 to 5. The larger the sequence number, the smaller the size of the feature map. It means to interpolate the coarse scale (small-size feature map after downsampling) to the fine scale (larger-size feature map size of the previous layer) and then subtract the matrix elements. I(c,s) represents the multi-scale brightness feature map after the fusion of coarse and fine granularity, and O(c,s,θ) represents the multi-scale directional feature map after the fusion of coarse and fine granularity in each direction.

[0062] Step 2.4: In order to fuse the above two expert brightness features I(c, s) and expert direction features O(c, s, θ) to generate a saliency feature map, it is first necessary to normalize the feature maps obtained at different scales of a certain feature to obtain the saliency map corresponding to each feature. The specific formula of the normalization operation method is as follows:

[0063]

[0064] F represents the brightness feature map or the directional feature map. Compared with the brightness multi-scale feature, the texture contour multi-scale feature map has four multi-scale feature maps because it extracts sensitive features in four directions. Therefore, before the normalization operation N, it is necessary to accumulate the features in four directions according to the scale to achieve preliminary fusion. ′ The input feature map F is normalized to the range [0…M], where M is the maximum value of the normalized range, and F min and F max Respectively represent the global maximum and global minimum in the feature map F; represents the mean of the local maximum value of the feature map, which is obtained by sliding the preset local rectangular area. n represents the number of local rectangular areas contained in the feature map F. Then the entire feature map is multiplied by Get the result F after normalization operation N , the activity peak of the feature map after this normalization operation will be smoother.

[0065] Then the saliency of the brightness and texture contour feature maps are linearly fused to obtain the final saliency map of each size. The formula is as follows:

[0066]

[0067] Where X represents the input image, N represents the normalization operation, represents the normalized brightness feature map, represents the normalized directional feature, T is the expert feature after the fusion output of the salient map, that is, the salient feature map of infrared small targets. The size and number of the salient features are consistent with the results of the feature pyramid extracted by the subsequent deep learning model backbone network, namely the UIUNet model encoder, which facilitates the subsequent fusion of the feature maps of each scale of the deep feature pyramid extracted by the convolutional neural network according to the scale, so as to complete the extraction of visual feature expert knowledge.

[0068] The multi-scale encoder-decoder network uses UIUNet as the basic model. UIUNet is a U-shaped encoder-decoder architecture network. The input image is sequentially passed through the encoder and decoder to perform feature extraction. The encoder and decoder of this model are stacked through convolution kernels with different pooling numbers and channel numbers, and the deep multi-scale features are formed by combining the outputs of convolution kernels at different stages.

[0069] The construction of the deep learning feature and traditional knowledge feature adaptive weighted activation fusion module is based on the traditional knowledge feature and deep learning feature adaptive fusion strategy; specifically, the importance of different channel features is calculated based on the channel attention mechanism, and the important features in the feature map are adaptively activated to enhance the effective features of small targets in infrared images.

[0070] The attention mechanism enables the model to pay more attention to unique object features, significantly enhancing the generalization ability of the model. In addition, it also enhances the model's ability to distinguish foreground and background, as well as accurately identify small targets. This embodiment proposes a knowledge reweighted activation algorithm using the channel attention mechanism. First, the channel attention module is used to learn channel attention. Specifically, the channel attention module is as follows Figure 2 As shown in the figure, the input feature distribution is first average pooled to obtain a feature map, which can be understood as having a global receptive field, and then a fully connected neural network is used for nonlinear transformation. This process generates a one-dimensional vector with the same size as the number of channels, representing the evaluation score of each channel. Finally, the channel evaluation score is used as an importance weight and multiplied by the input feature as the output of the channel attention module. This channel attention mechanism mainly learns the correlation between channels in the feature map and selects attention for specific channels. In deep learning, the feature values ​​of different channels can be regarded as image features extracted from different angles using operators with different parameters. Channel attention focuses on determining which channel features play a more critical role in detecting objects by emphasizing the feature values ​​of these important channels.

[0071] In this embodiment, the unique expert features obtained previously are used to enhance the feature expression capability. In the specific implementation, the UIUNet model is used as the basic framework. First, Figure 1 The knowledge extraction module shown in the figure extracts multi-scale expert features T of dimension [1, H, W] from the image, where the three dimensions represent the number of channels, height, and width, respectively, and Figure 1 The encoder part of the UIUNet basic model shown extracts the deep features D corresponding to the expert feature scale, and the dimensions of the deep features correspond to [C, H, W]. Next, the channel attention mechanism is applied according to different scales, where a fully connected layer is used to learn the importance weight w of each channel. The two feature map fusion calculation methods are as follows:

[0072] R H×W×C =w1·T H×W×1 +w2·D H×W×C

[0073] w1=w C+1 [0]

[0074] w2=w C+1 [1:C+1]

[0075] Where R represents the features after weighted activation fusion of deep features and expert features, and C represents the number of channels of the deep feature map. Figure 3 The process of adaptive fusion of deep learning features and traditional knowledge features is specifically demonstrated. Taking the fusion of the first column of the image as an example, the dimension of the deep feature is [C, H, W], and the dimension of the expert feature is [1, H, W]. C+1 The corresponding column intermediate dimension is

[0076] [1, 1, C+1] channel importance weight array, w1 is composed of w C+1 Take the value of the array subscript 0 to get the channel importance weight of the expert feature, w2 is obtained by w C+1 The array subscripts from 1 to the last value are obtained to represent the importance weight of each channel of the deep feature map. R is re-weighted by multiplying the features at this scale with the learned channel importance weights, thereby generating a re-weighted activation feature map containing feature knowledge as the input of the decoder.

[0077] The traditional expert feature maps of different scales and the feature maps extracted by the deep learning model are compressed and linearly mapped to obtain the importance weight of each channel. The feature values ​​on the corresponding channels are multiplied by the weights to obtain the weighted results. The importance weighted results of the expert knowledge are broadcasted and added to the weighted feature values ​​of the deep feature map. Compared with the deep feature maps extracted from the original model, the effective integration of expert features is achieved through the knowledge re-weighted activation method through channel attention to guide the focus of attention. This method recognizes the different importance of different feature channels, and ultimately enables the model to prioritize important features under the influence of expert knowledge. It reactivates the parts of the deep feature map that are difficult to capture but are critical for target detection in specific fields, thereby focusing attention on more prominent feature channels. And due to its self-learning channel importance, it can adapt to more complex scenarios without considering the adverse effects of traditional feature guidance.

[0078] Then, the segmentation result is generated according to the feature map, and the loss function is calculated. The model obtained after 400 training epochs is used as the final detection model. Unlike the original UIUNet which uses the cross entropy function to calculate the loss, this scheme uses Soft-IoU Loss to calculate the loss and optimize the model. The specific calculation method is as follows:

[0079]

[0080] s represents the sigmoid operation. gt represents the true value pixel of the manually labeled target, which takes the value of 0 or 1. is the predicted result. ε is used to prevent division by zero.

[0081] In order to demonstrate outstanding technical results, the experiments of this scheme are conducted on three publicly available infrared small target detection datasets: IRSTD-1K, NUDT-SIRST, and NUAA-SIRST. The results in Table 1 demonstrate the superiority of this method, achieving 79.17% IoU and 9.19×10 -6 These scores surpass the current state-of-the-art results, highlighting the robustness and effectiveness of our approach in a variety of challenging situations.

[0082] Table 1 Performance comparison between various schemes and this method

[0083]

[0084]

[0085] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. An improved infrared small target detection method based on knowledge enhancement network, characterized in that: The following steps are involved: S1: Acquire infrared small target image data and preprocess the image; S2: Use Gaussian pyramid to extract multi-scale brightness features of the image, use Gabor filter to extract multi-scale direction features of the image, and then fuse the brightness features and direction features into multi-scale expert features; S3: Build a multi-scale encoder-decoder network; the network includes a stacked encoder and decoder, which are stacked through convolution kernels with different pooling numbers and channel numbers. The decoder constructs deep multi-scale features by combining the outputs of the convolution kernels at different stages of the encoder; The preprocessed infrared small target image data is used as the input of the encoder; S4: Construct an adaptive weighted activation fusion module of deep learning features and traditional knowledge features to fuse the expert features and the output of the encoder by scale as the input of the decoder; S5: Calculate the loss function, obtain the final detection model after training and testing, and apply the detection model to infrared small target detection.

2. According to claim 1, an improved infrared small target detection method based on knowledge enhancement network is characterized in that: The multi-scale expert features are obtained by the following steps: Among them, X represents the input image, I represents the brightness feature, and N represents the normalization operation. represents the normalized brightness feature map, O represents the directional feature, θ∈(0°, 45°, 90°, 135°) represents the feature map of the directional feature in these four directions, represents the normalized directional feature, and T is the expert feature after the fusion output of the saliency map.

3. The improved infrared small target detection method based on knowledge enhancement network according to claim 2 is characterized in that: The normalization operation N is specifically as follows: F represents the brightness feature map or the directional feature map. F′ is obtained by normalizing the input feature map F to the range [0…M], where M is the maximum value of the normalized range. min and F max Respectively represent the global maximum and global minimum in the feature map F; represents the mean of the local maximum value of the feature map, n represents the number of local rectangular areas contained in the feature map F, and F N is the final normalized result.

4. The improved infrared small target detection method based on knowledge enhancement network according to claim 3 is characterized in that: Before merging the brightness feature I and the directional feature O into the expert feature, it is necessary to first perform coarse-grained fusion on them. The calculation method is as follows: Among them, c and s represent the sequence numbers of adjacent feature map layers of different sizes, which are related to the number of Gaussian pyramid layers, ranging from 1 to N; It means to interpolate the coarse scale to the fine scale and then subtract the matrix elements. I(c,s) represents the multi-scale brightness feature map after the fusion of coarse and fine granularity. O(c,s,θ) represents the multi-scale directional feature map after the fusion of coarse and fine granularity in each direction.

5. The improved infrared small target detection method based on knowledge enhanced network according to claim 4 is characterized in that: The multi-scale encoder-decoder network uses UIUNet as the base model.

6. The improved infrared small target detection method based on knowledge enhancement network according to claim 5 is characterized in that: The step S4 is specifically as follows: According to different scales, the channel attention mechanism is applied, and the importance weight of each channel is learned using the fully connected layer to fuse the expert features and the deep features. Specifically, the two feature fusion calculation methods are as follows: R H×W×C =w1·T H×W×1 +w2·D H×W×C w1=w C+1 [0] w2=w C+1 [1:C+1] Among them, the dimension of the expert feature T is [1, H, W], the encoder output feature is used as the deep feature D, the dimension is [C, H, W], the three dimensions represent the number of channels, height, and width respectively; R represents the feature after the weighted activation fusion of the deep feature and the expert feature; w C +1 The corresponding column middle dimension is [1, 1, C+1] channel importance weight array, w1 is composed of w C+1 Take the value of the array subscript 0 to get the channel importance weight of the expert feature, w2 is obtained by w C+1 The array subscripts from 1 to the last value are obtained to represent the importance weight of each channel of the deep feature map.

7. The improved infrared small target detection method based on knowledge enhanced network according to claim 6 is characterized in that: According to the decoder of UIUNet, the segmentation result is generated, the loss function is calculated, and the loss is calculated using Soft-IoU Loss. The specific calculation method is as follows: s represents the sigmoid operation, gt represents the true value pixel of the manually labeled target, and its value is 0 or 1. is the predicted result. ε is used to prevent division by zero.

8. The improved infrared small target detection method based on knowledge enhanced network according to claim 7 is characterized in that: The specific formula for extracting the multi-scale directional features of an image using a Gabor filter is as follows: x′=xcosθ+ysinθ y′=-xsinθ+ycosθ θ∈[0°,45°,90°,135°] in, represents the two-dimensional Gabor kernel function, Represents phase shift; λ represents wavelength, σ represents standard deviation of Gaussian function, γ represents spatial aspect ratio, θ represents direction of filter, controls orientation of filter, θ∈[0°,45°,90°,135°] determines feature maps in four directions that can be obtained based on Gabor filtering, x, y represent image space coordinates, x′, y′ are coordinates after rotation in θ direction with (x, y) as center.

Images