An improved infrared small target detection method based on knowledge enhanced network

Abstract

The application provides an improved infrared small target detection method based on a knowledge enhanced network, and belongs to the field of infrared target detection.The method is characterized in that: first, based on the knowledge of the infrared characteristics of brightness and direction, the temperature of the foreground small target in the infrared image is higher than that of the surrounding scene, and the foreground small target is visually brighter;there are obvious differences in the visual profile motion trend and smooth texture between the background and the foreground in the infrared image, an improved general saliency feature extraction scheme is used to extract the saliency features of the brightness and direction of the infrared small target;second, a weighted activation fusion module is proposed to fuse the traditional knowledge features and the deep learning features, so that the feature maps are effectively reweighted and activated, and the model can focus on the most relevant features.The knowledge enhanced network proposed in the application has the modular property, can be applied to any existing target detection model, and can assist the existing model to improve the detection capability, which has great contribution.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of infrared target detection, and particularly relates to an improved infrared small target detection method based on a knowledge enhanced network. BACKGROUND

[0002] In a series of fields such as monitoring, national defense and remote sensing, infrared sensors are widely used due to their fast response and robustness, and can be used to capture fast-moving scenes. As a key task in computer vision, infrared target detection can be mainly divided into general target detection and small target detection. With the progress of numerous general target detection methods, most of the infrared detection problems related to positioning and classification can now be solved. Therefore, single-frame infrared small target detection methods have attracted more attention. Unlike general object detection, targets in infrared images usually lack clear color and texture information. Small infrared targets are usually smaller than 30x30 pixels due to being photographed from a considerable distance, such as remote maritime rescue missions involving individuals or ships, identifying unauthorized drones or airborne objects, and the objects are usually surrounded by complex backgrounds, which makes infrared small target detection more challenging.

[0003] Traditional methods usually rely on handcrafted features and heuristic-based algorithms, including filter-based methods, local contrast-based methods and low-rank-based methods, which are difficult to generalize well in different scenarios. In recent years, with the advent of deep learning technology, especially convolutional neural networks, strong results have been shown in various computer vision tasks, including object detection. However, if existing excellent general object detectors such as the R-CNN series, the YOLO series, SSD, and the DETR series are used directly without designing additional infrared small target feature extraction methods, the small target features in the deep convolutional neural network layers will be attenuated due to the lack of high-level semantic information, and the performance requirements of small infrared target detection cannot be met. Therefore, how to improve the detection model to extract representative features of infrared small targets and have the ability to adapt to infrared small target detection has become a problem of practical significance. SUMMARY

[0004] In view of the problem that existing infrared small target detection is difficult to distinguish foreground and background, the brightness and direction characteristics of infrared small targets are proposed to be included in the detection network to enhance the feature representation of small targets, so that they are more easily distinguished from the background. In view of the problem that small target features are lost in high-level neural networks, a multi-level feature fusion strategy combining deep learning features and different scale traditional knowledge features is realized. This strategy ensures the effective integration of low-level details and high-level semantic information, and can adaptively adjust the importance of different channel features to enhance the model's ability under different conditions.

[0005] To solve the above technical problems, the specific technical scheme of the improved infrared small target detection method based on knowledge enhanced network is as follows:

[0006] An improved infrared small target detection method based on knowledge enhanced network comprises the following steps:

[0007] An improved infrared small target detection method based on knowledge enhanced network comprises the following steps:

[0008] S1: Obtain infrared small target image data, and pre-process the image;

[0009] S2: Use a Gaussian pyramid to extract multi-scale brightness features of the image, use a Gabor filter to extract multi-scale direction features of the image, and then fuse the brightness features and the direction features into multi-scale expert features;

[0010] S3: Build a multi-scale encoder-decoder network; the network comprises stacked encoders and decoders, the encoders and the decoders are stacked through convolution kernels with different pooling numbers and channel numbers, the decoders are constituted by outputs of convolution kernels at different stages of the joint encoders to form deep multi-scale features; the pre-processed infrared small target image data is used as input of the encoders;

[0011] S4: Construct a deep learning feature and traditional knowledge feature adaptive weighted activation fusion module, fuse the expert features and the outputs of the encoders according to scales, and use the fused features as input of the decoders;

[0012] S5: Calculate a loss function, obtain a final detection model after training and testing, and apply the detection model to infrared small target detection.

[0013] The multi-scale expert features are obtained through the following steps:

[0014]

[0015] Wherein, X represents an input image, I represents a brightness feature, N represents a normalization operation, represents a normalized brightness feature map, O represents a direction feature, and θ∈(0°, 45°, 90°, 135°) represents a feature map of the direction feature in the four directions, represents a normalized direction feature, and T is the expert feature fused and output by the saliency map.

[0016] The normalization operation N is specifically as follows:

[0017]

[0018] F represents a brightness feature map or a direction feature map, F ′F is normalized to the range [0…M] by input feature map F, M is the maximum value of the normalized range, F min and F max respectively represent the global maximum and global minimum in the feature map F; represent the average of the local maximum value of the feature map, n represents the number of local rectangular regions contained in the feature map F, F N is the final normalized result.

[0019] Before fusing the brightness feature I and the direction feature O into the expert feature, it is necessary to perform coarse-grained and fine-grained fusion on them, and the calculation method is as follows:

[0020] and c = s + 1

[0021]

[0022] wherein c, s respectively represent the adjacent different size feature map layer sequence number, which is related to the number of Gaussian pyramid layers, and the number of layers is from 1 to N; representing that the coarse scale is interpolated to the fine scale and then the matrix elements are subtracted, I(c, s) represents the multi-scale brightness feature map fused by coarse and fine granularity, and O(c, s, θ) represents the multi-scale direction feature map fused by coarse and fine granularity in each direction.

[0023] The multi-scale encoder-decoder network uses UIUNet as a basic model.

[0024] The step S4 is specifically as follows:

[0025] According to different scales, the channel attention mechanism is applied, and a full connection layer is used to learn the importance weight of each channel, and the expert feature and the depth feature are fused; specifically, the two feature fusion calculation methods are as follows:

[0026] R H×W×C = w1·T H×W×1 +w2·D H×W×C

[0027] w1 = w C+1 [0]

[0028] w2 = w C+1 [1:C+1]

[0029] wherein the dimension of the expert feature T is [1, H, W], the encoder output feature is used as the depth feature D, and the dimension is [C, H, W], three dimensions respectively represent the number of channels, height and width; R represents the feature after the weighted activation fusion of the depth feature and the expert feature; w C+1 corresponding to the column, the middle dimension of which is [1, 1, C+1] channel importance weight array, w1 is obtained from w C+1The value of the array index 0 is taken to obtain a channel importance weight representing the expert feature, w2 is obtained from w C+1 The values of the array index from 1 to the last are taken to obtain the importance weight of each channel of the deep feature map.

[0030] Then, the segmentation result is generated according to the decoder of the UIUNet, the loss function is calculated, the Soft-IoU Loss is used to calculate the loss, and the specific calculation method is as follows:

[0031]

[0032] s represents the operation sigmoid, gt represents the true value pixels of the manually labeled target, and takes the value 0 or 1, is the prediction result. ε is used to prevent division by zero.

[0033] The method has the following advantages:

[0034] The method first has the knowledge of the infrared characteristics of brightness and direction: 1) The temperature of the foreground small target in the infrared image is higher than that of the surrounding scene, and it will be visually brighter; 2) The contour motion trend and smooth texture of the background and the foreground in the infrared image are visually different, the general saliency feature extraction scheme is improved, and the saliency features of the brightness and direction of the infrared small target are extracted;

[0035] Secondly, a weighted activation fusion module is proposed to fuse traditional knowledge features and deep learning features, which effectively reweights and activates the feature map to ensure that the model focuses on the most relevant features. The knowledge enhancement network proposed in the application has the modular property and can be applied to any existing target detection model to assist the existing model to improve the detection ability, which has great contribution. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 It is a schematic diagram of the overall model of the application;

[0037] Figure 2 It is a schematic diagram of the channel attention module structure;

[0038] Figure 3 It is a schematic diagram of the reweighting and activation fusion of knowledge features and deep features;

[0039] Figure 4 It is a flowchart of the method of the application. DETAILED DESCRIPTION

[0040] In order to better understand the purpose, structure and function of the application, the improved infrared small target detection method based on the knowledge enhancement network will be further described in detail below in combination with the drawings.

[0041] The application is realized by the following scheme: an improved infrared small target detection method based on a knowledge enhanced network, which comprises the following steps, and a flow chart is shown in Figure 4

[0042] S1: read in image data, and pre-process the image, that is, normalize the image during training, randomly crop the image into a pixel block with a size of 256*256, and use random flipping to enhance the data.

[0043] S2: use a Gaussian pyramid and a Gabor filter to extract brightness and direction features respectively, and fuse them into an infrared small target saliency feature map;

[0044] S3: build a basic multi-scale encoder-decoder network to extract a deep learning feature map of the image;

[0045] S4: construct a deep learning feature and traditional knowledge feature adaptive weighted activation fusion module to generate an infrared small target feature map after feature reactivation enhancement;

[0046] S5: obtain a prediction mask of the infrared small target, calculate a loss function, and use the model obtained after 400 training epochs as the final detection model;

[0047] S6: test the final model on an image to obtain a test index result.

[0048] Specifically, first, image data is obtained, which can be obtained from a public infrared small target detection dataset, and the image is pre-processed, specifically including normalizing the image, randomly cropping the image into a pixel block with a size of 256*256, and using random flipping to enhance the data.

[0049] Then, after obtaining the pre-processed infrared small target detection image data, the small targets in the infrared image usually exhibit two characteristic properties: brighter areas are more likely to be detection targets, and textures with different directions are more likely to be foreground small targets. Based on these two saliency knowledge properties, a multi-scale saliency feature extraction scheme is designed to obtain an infrared small target saliency feature map. The specific method is as follows:

[0050] The main process of extracting visual characteristic features of infrared thermal radiation brightness, texture contour from the properties of infrared image objects includes two parts: brightness and direction feature extraction and expert feature saliency map generation, as shown in the knowledge extraction part in Figure 1 The specific steps are as follows:

[0051] ​Step 2.1: To extract the intensity feature map I, the intensity features of the image are calculated at different scales; specifically, 5 feature maps with different sizes are generated based on multi-scale Gaussian pyramid sampling. For the intensity feature I, since the infrared image is usually presented in the form of a gray image, the higher the brightness, the higher the pixel value. Therefore, the intensity pyramid can be calculated in this way;

[0052] Step 2.2: For the orientation feature O, a texture contour multi-scale pyramid is constructed using Gabor filters. Gabor filters are linear filters used for edge extraction, mainly analyzing whether there is specific frequency content in a specific direction of a certain area of the image. Its frequency and direction expression is similar to the human visual system, which can provide good direction selection and scale selection characteristics, and is not sensitive to changes in lighting, so it is very suitable for texture direction analysis. In the spatial domain, a two-dimensional Gabor filter is a Gaussian kernel function modulated by a sinusoidal plane wave. Its impulse response is defined as a sinusoidal wave (for a two-dimensional Gabor filter, it is a plane wave) multiplied by a Gaussian function. Due to the convolution property of multiplication, the Fourier transform of the impulse response of the Gabor filter is the convolution of the Fourier transform of the harmonic function and the Fourier transform of the Gaussian function. The specific representation is as follows:

[0053]

[0054] x ′ = x cos θ + y sin θ

[0055] y ′ = -x sin θ + y cos θ

[0056] θ ∈ [0°, 45°, 90°, 135°]

[0057] Two-dimensional Gabor kernel function is obtained by multiplying a Gaussian envelope function and a cosine function, where represents the phase shift, which controls the phase of the sinusoidal wave; λ represents the wavelength, which determines the period of the sinusoidal wave; σ represents the standard deviation of the Gaussian function, which controls the width of the Gaussian envelope; γ is the spatial aspect ratio, which represents the scale difference of the filter in the horizontal and vertical directions (usually less than 1); θ is the direction of the filter, which controls the orientation of the filter, θ ∈ [0°, 45°, 90°, 135°] determines the four direction feature maps that can be obtained based on Gabor filtering. x, y represent the image space coordinates, and x ′ , y ′ are the coordinates after rotation with (x, y) as the center. Through the superposition of sine or cosine waves, a selective filtering effect on specific frequencies is formed.

[0058] Step 2.3: After initially obtaining the brightness feature I and directional feature O in the above multi-scale traditional feature map, it is necessary to perform coarse-grained fusion to retain more feature information from different scales. The calculation method is:

[0059] And c=s+1

[0060]

[0061] Among them, c and s represent the sequence numbers of adjacent feature map layers of different sizes, which are related to the number of multi-scale pyramid layers obtained in the previous step. The layer numbers range from 1 to 5. The larger the sequence number, the smaller the size of the feature map. This represents interpolating the coarse-scale (small-scale feature map after downsampling) to the fine-scale (larger-scale feature map of the previous layer) and then performing matrix element-wise subtraction. I(c,s) represents the multi-scale brightness feature map after the fusion of coarse and fine granularity, and O(c,s,θ) represents the multi-scale directional feature map after the fusion of coarse and fine granularity in each direction.

[0062] Step 2.4: To fuse the two expert brightness features I(c,s) and expert directional features O(c,s,θ) to generate a saliency feature map, it is first necessary to normalize the feature maps obtained at different scales of a certain feature to obtain the saliency map corresponding to each feature. The specific formula of this normalization operation is as follows:

[0063]

[0064] F represents the brightness feature map or directional feature map. Compared with the brightness multi-scale feature, the texture contour multi-scale feature map extracts sensitive features in four directions and has four multi-scale feature maps. Therefore, before the normalization operation N, the feature needs to be accumulated in four directions according to the scale to achieve preliminary fusion. ′ The input feature map F is normalized to the range [0…M], where M is the maximum value of the normalized range, and F min and F max Represent the global maximum and global minimum in the feature map F respectively; Represents the mean of the local maximum value of the feature map, which is obtained by sliding the preset local rectangular area. n represents the number of local rectangular areas contained in the feature map F. Then the entire feature map is multiplied by Get the result F after normalization operation N , the activity peak of the feature map after the normalization operation will be smoother.

[0065] Then the saliency of the brightness and texture contour feature maps are linearly fused to obtain the final saliency map of each size. The formula is as follows:

[0066]

[0067] wherein X represents the input image, N represents the normalization operation, represents the normalized brightness feature map, represents the normalized direction feature, T is the expert feature after the fusion output of the saliency map, that is, the infrared small target saliency feature map, the size and quantity of the saliency feature are consistent with the results of the feature pyramid extracted by the subsequent deep learning model backbone network, that is, the UNet model encoder, so as to facilitate the subsequent fusion of each scale feature map according to the scale and the deep feature pyramid extracted by the convolutional neural network, so as to complete the extraction of visual characteristic expert knowledge.

[0068] The multi-scale encoder-decoder network uses UNet as a basic model, and UNet is a U-shaped encoder-decoder architecture network. The input image is sequentially subjected to feature extraction by the encoder and the decoder, and the encoder and the decoder of the model are stacked by convolution kernels with different pooling numbers and channel numbers, and the outputs of the convolution kernels at different stages are combined to form deep multi-scale features.

[0069] The construction of the deep learning feature and traditional knowledge feature adaptive weighted activation fusion module is based on the adaptive fusion strategy of the traditional knowledge feature and the deep learning feature. Specifically, the importance of different channel features is calculated based on the channel attention mechanism, and the important features in the feature map are adaptively activated to enhance the effective features of the small target in the infrared image.

[0070] The attention mechanism enables the model to pay more attention to unique object features, significantly enhancing the generalization ability of the model. In addition, it also enhances the ability of the model to distinguish foreground and background, and accurately identify small targets. The embodiment proposes a knowledge re-weighting activation algorithm using a channel attention mechanism. First, the channel attention module is used to learn the channel attention. Specifically, the channel attention module is as shown in Figure 2 First, the input feature distribution is subjected to average pooling to obtain a feature map, which can be understood as having a global receptive field, and then a fully connected neural network is used for nonlinear transformation. This process generates a one-dimensional vector with the same size as the channel number, which represents the evaluation score of each channel. Finally, the channel evaluation score is used as the importance weight and multiplied by the input feature as the output of the channel attention module. This channel attention mechanism mainly learns the correlation between the channels in the feature map, and selects attention for a specific channel. In deep learning, the feature values of different channels can be regarded as image features extracted from different angles using operators with different parameters. Channel attention focuses on determining which channel features play a more critical role in detecting objects by emphasizing the feature values of these important channels.

[0071] In this embodiment, the unique expert features obtained previously are used to enhance the feature expression capability. In the specific implementation, the UIUNet model is used as the basic framework. First, Figure 1 The knowledge extraction module shown in the figure extracts multi-scale expert features T with dimensions [1, H, W] from the image, where the three dimensions represent the number of channels, height, and width, respectively, and extracts them from the image. Figure 1 The encoder part of the UIUNet basic model shown in the figure extracts deep features D corresponding to the expert feature scale, and the dimensions of the deep features are [C, H, W]. Next, a channel attention mechanism is applied according to different scales, in which a fully connected layer is used to learn the importance weight w of each channel. The two feature maps are fused as follows:

[0072] R H×W×C =w1·T H×W×1 +w2·D H×W×C

[0073] w1=w C+1 [0]

[0074] w2=w C+1 [1:C+1]

[0075] Where R represents the feature after the weighted activation fusion of deep features and expert features, and C represents the number of channels of the deep feature map. Figure 3 The process of adaptive fusion of deep learning features and traditional knowledge features is specifically demonstrated. Taking the fusion of the first column of the image as an example, the dimension of the deep feature is [C, H, W], and the dimension of the expert feature is [1, H, W]. C+1 The middle dimension of the corresponding column is

[0076] [1, 1, C+1] channel importance weight array, w1 is composed of w C+1 The value of the array subscript 0 is obtained to represent the channel importance weight of the expert feature, and w2 is obtained by w C+1 The array subscripts from 1 to the last value represent the importance weight of each channel in the deep feature map. R is reweighted by multiplying the features at this scale with the learned channel importance weights, thereby generating a reweighted activation feature map containing feature knowledge as the input of the decoder.

[0077] The traditional expert feature maps and the feature maps extracted by deep learning models of different scales are compressed in channels and linearly mapped to obtain the importance weights of each channel. The weighted results are obtained by multiplying the feature values on the corresponding channel and the weights. The importance weighted results of the expert knowledge are broadcasted to the weighted feature values of the deep feature maps. Compared with the deep feature maps extracted in the original model, the method of knowledge re-weighting activation is realized by channel attention, which realizes the effective integration of expert features to guide the attention concentration. This method recognizes the different importance of different feature channels, and finally enables the model to prioritize important features under the influence of expert knowledge. It reactivates the part of the deep feature map that is difficult to capture but essential for target detection in a specific field, thereby focusing attention on more prominent feature channels. And because of its self-learning channel importance, it can adapt to more complex scenarios without considering the adverse effects of traditional feature guidance.

[0078] Then the segmentation result is generated according to the feature map, and the loss function is calculated. The model obtained after 400 training epochs is used as the final detection model. Unlike the original UIUNet, which uses the cross-entropy function to calculate the loss, this scheme uses Soft-IoU Loss to calculate the loss and optimize the model. The specific calculation method is as follows:

[0079]

[0080] s represents the operation sigmoid. gt represents the true value pixels of the manually labeled target, which takes the value of 0 or 1. is the prediction result. ε is used to prevent division by zero.

[0081] In order to show the outstanding technical effect, the experiments of this scheme are carried out on three publicly available infrared small target detection datasets: IRSTD-1K, NUDT-SIRST and NUAA-SIRST. The results in Table 1 prove the superiority of this method, which respectively obtains 79.17% IoU and 9.19x10 -6 The scores exceed the current state-of-the-art results, highlighting the robustness and effectiveness of the method in various challenging scenarios.

[0082] Table 1 Comparison of performance of each scheme and this method

[0083]

[0084]

[0085] It is to be understood that the present application is described by way of example only, and that modifications or alterations can be made to the features and embodiments described without departing from the spirit and scope of the application. In addition, modifications can be made to the features and embodiments described to accommodate specific situations and materials without departing from the spirit and scope of the application. Accordingly, the application is not limited to the specific embodiments disclosed herein, but rather, the scope of the application includes all embodiments falling within the scope of the claims.

Claims

1. An improved infrared small target detection method based on knowledge-enhanced network, characterized in that: The following steps are involved: S1: Acquire infrared small target image data and preprocess the image; S2: Use Gaussian pyramid to extract multi-scale brightness features of the image, use Gabor filter to extract multi-scale directional features of the image, and then fuse the brightness features and directional features into multi-scale expert features; S3: Build a multi-scale encoder-decoder network; the network includes a stacked encoder and decoder, which are stacked with convolution kernels with different pooling numbers and channel numbers. The decoder constructs deep multi-scale features by combining the outputs of the convolution kernels at different stages of the encoder. The preprocessed infrared small target image data is used as the input of the encoder; S4: Construct an adaptive weighted activation fusion module for deep learning features and traditional knowledge features, and fuse the expert features and the encoder output according to the scale as the input of the decoder; S5: Calculate the loss function, obtain the final detection model after training and testing, and apply the detection model to infrared small target detection.

2. The improved infrared small target detection method based on knowledge-enhanced network according to claim 1 is characterized in that: The multi-scale expert features are obtained by the following steps: Among them, X represents the input image, I represents the brightness feature, and N represents the normalization operation. represents the normalized brightness feature map, O represents the directional feature, and θ∈(0°, 45°, 90°, 135°) represents the feature map of the directional feature in these four directions. represents the normalized directional feature, and T is the expert feature after fusion of the saliency map output.

3. The improved infrared small target detection method based on knowledge-enhanced network according to claim 2 is characterized in that: The normalization operation N is specifically as follows: F represents the brightness feature map or the direction feature map, and F′ is normalized from the input feature map F to the range [0…M], where M is the maximum value of the normalized range. min and F max Represent the global maximum and global minimum in the feature map F respectively; Represents the mean of the local maximum value of the feature map, n represents the number of local rectangular areas contained in the feature map F, F N is the final normalized result.

4. The improved infrared small target detection method based on knowledge-enhanced network according to claim 3 is characterized in that: Before fusing the brightness feature I and the directional feature O into the expert feature, they need to be fused at a coarse and fine granularity level. The calculation method is as follows: Among them, c and s represent the sequence numbers of adjacent feature map layers of different sizes, which are related to the number of Gaussian pyramid layers, ranging from 1 to N; It means that the coarse scale is interpolated to the fine scale and then the matrix elements are subtracted. I(c,s) represents the multi-scale brightness feature map after the fusion of coarse and fine granularity. O(c,s,θ) represents the multi-scale directional feature map after the fusion of coarse and fine granularity in each direction.

5. The improved infrared small target detection method based on knowledge-enhanced network according to claim 4 is characterized in that: The multi-scale encoder-decoder network uses UIUNet as the base model.

6. The improved infrared small target detection method based on knowledge-enhanced network according to claim 5 is characterized in that: The step S4 is specifically as follows: The channel attention mechanism is applied at different scales, and the fully connected layer is used to learn the importance weight of each channel to fuse the expert features and deep features. Specifically, the calculation method for the fusion of the two features is as follows: R H×W×C =w1·T H×W×1 +w2·D H×W×C w1=w C+1 [0] w2=w C+1 [1:C+1] Among them, the dimension of the expert feature T is [1, H, W], the encoder output feature is used as the depth feature D, the dimension is [C, H, W], the three dimensions represent the number of channels, height, and width respectively; R represents the feature after the weighted activation fusion of the depth feature and the expert feature; w C +1 The corresponding column middle dimension is [1, 1, C+1] channel importance weight array, w1 is composed of w C+1 The value of the array subscript 0 is obtained to represent the channel importance weight of the expert feature, and w2 is obtained by w C+1 The array subscripts from 1 to the last value are obtained to represent the importance weight of each channel of the deep feature map.

7. The improved infrared small target detection method based on knowledge-enhanced network according to claim 6 is characterized in that: According to the decoder of UIUNet, the segmentation result is generated and the loss function is calculated. Soft-IoU Loss is used to calculate the loss. The specific calculation method is as follows: s represents the sigmoid operation, gt represents the true value pixel of the manually labeled target, and its value is 0 or 1. is the prediction result, and ε is used to prevent division by zero.

8. The improved infrared small target detection method based on knowledge-enhanced network according to claim 7 is characterized in that: The specific formula for extracting the multi-scale directional features of an image using a Gabor filter is as follows: x′=xcosθ+ysinθ y′=-xsinθ+ycosθ θ∈[0°,45°,90°,135°] in, represents the two-dimensional Gabor kernel function, Represents the phase shift; λ represents the wavelength, σ represents the standard deviation of the Gaussian function, γ is the spatial aspect ratio, θ is the direction of the filter, which controls the orientation of the filter. θ∈[0°,45°,90°,135°] determines the feature maps in four directions that can be obtained based on Gabor filtering. x and y represent the image space coordinates, and x′ and y′ are the coordinates after rotating the direction of θ around (x, y).

Images