Infrared target detection method based on lightweight SSD with feature enhancement

Abstract

The application discloses an infrared target detection method based on a lightweight SSD of feature enhancement, and constructs a basic network structure for extracting features; the method comprises the following steps: inputting an original image into a preset network, performing layer-by-layer convolution through a plurality of convolution layers, and obtaining low-level feature maps and high-level feature maps of the image respectively at different convolution layers; performing feature enhancement by adding a PAC feature enhancement module designed and constructed after a low-level feature branch to process, and obtaining low-level feature maps rich in semantic information; respectively predicting the classification and the boundary box of each prediction box corresponding to each grid on the low-level feature maps rich in semantic information and the high-level feature maps; and merging the boundary boxes through a non-maximum suppression algorithm to output the final prediction result of the whole network. The application effectively improves the detection precision while improving the speed of the detection algorithm.

Description

Technical Field

[0001] This invention belongs to the field of infrared target detection, specifically relating to an infrared target detection method based on a lightweight SSD with feature enhancement. Background Technology

[0002] Object detection, as one of the fundamental problems in computer vision and digital image processing, forms the basis for subsequent processing in many other computer vision tasks, such as instance segmentation, image understanding, and object tracking. Infrared systems utilize infrared detectors to sense the infrared radiation emitted by objects to acquire information, offering advantages such as all-weather operation, good concealment, easy penetration of smoke and dust, and strong noise resistance. Currently, object detection in infrared scenarios has important applications in fields such as autonomous driving, video surveillance, and military applications.

[0003] Infrared images suffer from a lack of color and texture information, low signal-to-noise ratio and contrast, severe background noise, and low resolution. Traditional algorithms, relying on manually designed features, suffer from high false detection rates and poor robustness when identifying infrared targets. In recent years, deep learning-based two-stage algorithms have gradually improved detection accuracy; however, the increased computational load has also led to a decrease in detection speed. More and more practical applications demand higher detection speeds while maintaining a certain level of accuracy. The two-stage detection task can be decomposed into two single-stage tasks. The first stage generates candidate boxes, distinguishes between the candidate boxes and the target, and performs preliminary regression of the bounding box positions. The second stage predicts the final category of the candidate boxes and regresses the final position of the predicted boxes. Single-stage detection algorithms complete both tasks within a single stage, directly predicting the target class and bounding box of the candidate boxes. Therefore, researching single-stage infrared target detection methods is of great significance. Summary of the Invention

[0004] In view of this, the main objective of the present invention is to provide an infrared target detection method based on a feature-enhanced lightweight SSD.

[0005] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0006] An infrared target detection method based on feature-enhanced lightweight SSD includes the following steps:

[0007] Construct the basic network structure for feature extraction;

[0008] The image is input into the basic network structure and convolved layer by layer through multiple convolutional layers. Low-level feature maps and high-level feature maps of the image are obtained at different convolutional layers.

[0009] By adding a designed and constructed PAC feature enhancement module after the low-level feature branch ADWS11, feature enhancement is performed to obtain a low-level feature map with rich semantic information.

[0010] For the semantically rich low-level feature maps and high-level feature maps, predict the category and bounding box to which each predicted box belongs for each grid on the feature map;

[0011] The bounding boxes are merged using a nonmaximum suppression algorithm, and the final prediction result of the entire network is output.

[0012] Compared with existing technologies, this invention improves detection accuracy while increasing the speed of the detection algorithm. The designed feature enhancement module expands the receptive field through parallel dilated convolution and extracts and fuses features from multiple scales to obtain more comprehensive semantic information, thereby improving the detection capability for small targets. The ASL loss function uses γ + and γ - By decoupling the rate of change of the loss function for positive and negative samples, we can ensure that the features can still be learned well even when there are few positive samples. This not only balances the positive and negative samples, but also reduces the loss of simple negative samples while ensuring the contribution of positive samples, thereby improving the detection capability of small targets with a small number of positive samples. Attached Figure Description

[0013] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and, together with their descriptions, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0014] Figure 1 This is a flowchart of infrared target detection based on a feature-enhanced lightweight SSD according to an embodiment of the present invention.

[0015] Figure 2 This is a network structure diagram of the infrared target detection method based on feature-enhanced lightweight SSD according to an embodiment of the present invention.

[0016] Figure 3 This is a structural diagram of the PAC parallel dilated convolution feature enhancement module in the infrared target detection method based on feature enhancement and lightweight SSD according to an embodiment of the present invention.

[0017] Figure 4 This is a structural diagram of the multi-scale detection module of the infrared target detection method based on feature-enhanced lightweight SSD in an embodiment of the present invention.

[0018] Figure 5 The image shows the detection results of the algorithm provided in this embodiment of the invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, article, or apparatus that includes that element.

[0021] This invention provides an infrared target detection method based on a feature-enhanced lightweight SSD, such as... Figure 1 As shown, the steps are as follows:

[0022] Step 1: Construct the basic network structure for feature extraction.

[0023] Specifically, such as Figure 2 As shown, based on the lightweight MobileNet network, it is constructed by stacking multiple different asymmetric depth separable convolutional modules. The input image is downsampled through a standard 3×3 convolutional layer, and then 13 asymmetric depth separable convolutional modules (ADWS) are stacked sequentially in the forward convolution direction to form an improved AMobileNet network structure. In the asymmetric depth separable convolutional module (ADWS), the input feature map is first extracted by convolutions of size 1×3, 3×3, and 3×1, respectively. The three results are added together to obtain the output as the result of depth convolution. Then, a 1×1 pointwise convolution is performed on this depth convolution result. The pointwise convolution is used to change the number of output channels and integrate information across channels to obtain the final output.

[0024] Step 2: Input the image into the basic network structure, and convolve it layer by layer through multiple convolutional layers to obtain the low-level feature maps and high-level feature maps of the image at different convolutional layers.

[0025] Specifically, the output matrices of ADWS11 and ADWS13 from the input image are used as the low-level feature maps of the image; in the ADWS module, ADWS11 and ADWS13 are the 11th and 13th layer asymmetric depthwise convolutional modules stacked in the improved AMobileNet network structure, respectively, and their network structures are as follows. Figure 2 As shown, its outputs are 14×14×512 feature maps and 7×7×1024 feature map structures, respectively.

[0026] Four additional convolutional layers C14, C15, C16, and C17 are added directly after the deep convolutional layer ADWS13. The output of each of these four additional convolutional layers serves as a high-level feature map of the image.

[0027] Step 3: Enhance features by adding the designed and constructed PAC feature enhancement module after the low-level feature branch ADWS11, expand the receptive field, and obtain a low-level feature map rich in semantic information.

[0028] Specifically, such as Figure 3 As shown, the low-level feature map output by module ADWS11 is processed by the PAC feature enhancement module in parallel branching to obtain the output results of 4 branches, which are then spliced ​​together.

[0029] Specifically, the PAC feature enhancement module has a total of 4 branches, for the input of the previous layer:

[0030] (1) Branch 1 reduces the channel dimension by passing through a 1×1 convolutional layer, and then adds a 3×3 dilated convolution with a dilation rate of 1 after the convolutional layer to produce the output.

[0031] (2) Branch 2 reduces the channel dimension by passing through a 1×1 convolutional layer, and adds a 3×3 dilated convolution with a dilation rate of 2 after the convolutional layer to generate the output.

[0032] (3) Branch 3 reduces the channel dimension by passing through a 1×1 convolutional layer, and adds a 3×3 dilated convolution with a dilation rate of 3 after the convolutional layer to generate the output.

[0033] (4) Branch 4 first passes through a 1×1 convolutional layer to reduce the channel dimension, and then adds 3×3 dilated convolutions with dilation rates of 1, 2 and 3 respectively after the convolutional layer to generate the output.

[0034] The outputs of the above four branches are concatenated in the channel dimension, and finally concatenated in the channel dimension and added to the output image of module ADWS11 to obtain a low-level feature map rich in semantic information.

[0035] Step 4: For the semantically rich low-level feature maps and high-level feature maps, predict the classification and bounding box to which each predicted box belongs for each grid on the feature map.

[0036] Specifically, the low-level feature maps (i.e., semantically rich low-level feature maps) processed by the feature enhancement module PAC and four high-level feature maps are respectively input into the classification and regression detection network for multi-scale detection. The classification and regression detection network of this invention includes six detection branches, each corresponding to one of the six input feature maps. Each detection branch has the same structure, consisting of two 1×1 convolutional branches. One branch outputs the category corresponding to each candidate box through a softmax layer, and the other branch outputs the corresponding bounding box through a linear regression layer. For each region, the classification and regression detection network will output the location and category of six possible targets. Finally, the prior box information obtained from the six feature layers is combined using a non-maximum suppression algorithm to suppress some overlapping or erroneous prior boxes and generate the final detection result.

[0037] In this invention, for each pixel center point of the six feature layers, the ADWS11 low-level feature map enhanced by PAC is input into the classification and regression detection network, and its linear regression layer will output four prior boxes with different scales. The ADWS13 low-level feature map and the C14, C15, C16, and C17 high-level feature maps generated by four additional convolutional layers are input into the classification and regression detection network, and their linear regression layers will output six prior boxes with different scales. The position parameters of each prior box include four data points, including the center position coordinates, width, and height of the target bounding box. The softmax layer outputs preset target type probability information, where there are A target types in total, that is, the output outputs the probability of corresponding to A types of targets. The possible target types are related to the settings of the training set.

[0038] like Figure 4 As shown, for the prior boxes generated by different layers of feature maps, assuming that m layers of feature maps are used for prediction, the proportion of the prior boxes in each layer of feature maps is...

[0039]

[0040] Among them, S k S is the proportion of the prior boxes in the feature map of the k-th layer. max and S min These are the aspect ratios of the top and bottom prior boxes in the feature map of layer m, respectively. For different aspect ratios of prior boxes within the same feature map layer, the width and height of each prior box need to be calculated. If a is set... r Given five different aspect ratios ({1, 2, 3, 1 / 2, 1 / 3}), the width of each prior box is...

[0041]

[0042] Similarly, the height of each prior box is

[0043]

[0044] The center coordinates of the prior bounding box are as follows

[0045]

[0046] Among them, |f k | is the numerical value of the side length of the k-th feature map, i,j∈[0,|f k |), 0.5 is the offset setting. Generally, when the aspect ratio is 1, an additional dimension of is added. The prior box.

[0047] Each center point of the feature map generates 6 prior boxes of different sizes. The prior boxes of different sizes and aspect ratios in the feature map cover almost all the locations to be detected, which can complete the detection of all target regions of the input. By calculating the score of each prior box for each category and the background, the category to which the prior box belongs can be determined.

[0048] Since reducing the convolutional kernel size has almost no impact on detection accuracy but can reduce computational cost, a 1×1 convolutional kernel is used in the detection module, and the number of prior boxes is set to 3, 6, 6, 6, 6, 6. Assuming the input image resolution is 300×300, the object detection method using Feature Enhancement SSD outputs feature maps at six scales: 19×19, 10×10, 5×5, 3×3, 2×2, and 1×1. The first layer predicts 3 bounding boxes per grid, and the other five layers predict 6 bounding boxes per grid. Each bounding box has four parameters (the horizontal and vertical coordinates of the prior box center point, and the length and width of the prior box), predicting a total of 5 categories. Finally, the number of generated prior boxes is:

[0049] 19×19×3+(10×10+5×5+3×3+2×2+1×1)×6=1917

[0050] Step 5: Merge bounding boxes using the non-maximum suppression algorithm and output the final prediction result of the entire network.

[0051] like Figure 5 As shown, the algorithm can accurately detect targets at multiple scales, including small targets, without any missed detections. For example... Figure 5 (a) All cars at medium to long distances and Figure 5 (b) Cars and pedestrians at different distances obscuring each other, and Figure 5 (c) dense crowds and distant, relatively small cars Figure 5 (d) Cars parked to the side and a crowd in the distance. Furthermore, the target's bounding box is very accurately located and shows no deformation.

[0052] Basic network structure and parameter settings:

[0053] 1. When training the network, an improved asymmetric loss (ASL) function is used, the expression of which is:

[0054]

[0055] Where, γ + and γ - These are the focusing parameters for positive and negative samples, respectively. They decouple the rates of change of the loss function for positive and negative samples, ensuring that features can still be learned effectively even in the absence of positive samples. L + and L - This is the general cross-entropy loss for positive and negative samples. The loss for negative samples can be reduced by changing the value of m. y′ is the probability of a sample belonging to the 1st order, representing the output of the activation function, and its range is [0,1]. m The newly introduced hyperparameter is used to define the new hyperparameter, and its expression is y. m =max(y′-m,0).

[0056] Finally, based on the loss function calculation results, the weights are updated using stochastic gradient descent with momentum.

[0057] 2. The kernel size of Conv14 to Conv17 is 1×1. The size of the output feature map, stride and whether the input detection branch is used are shown in Table 1.1.

[0058] Table 1.1 Some convolutional layer parameter settings

[0059] Convolutional layer Feature map size Step length Input detection branch Conv14_1 10×10×256 1 no Conv14_2 5×5×512 2 yes Conv15_1 5×5×128 1 no Conv15_2 3×3×256 2 yes Conv16_1 3×3×128 1 no Conv16_2 2×2×256 2 yes Conv17_1 2×2×64 1 no Conv17_2 1×1×128 2 yes

[0060] 3. A lightweight SSD-based infrared target detection method with feature enhancement extracts features from six effective feature maps at different scales for detection. These six detection modules include two 1×1 convolutional branches: one branch performs target classification prediction, and the other branch performs bounding box regression. The input size, number of bounding boxes, and number of channels in the classification and regression branches for each detection module are shown in Table 1.2.

[0061] Table 1.2 Parameter settings for each detection module

[0062]

[0063]

[0064] 4. Some hyperparameters were set during the experiment as follows: the original image was scaled to 512×512, the batch size was 64, momentum was 0.9, decay was 0.0005, the initial learning rate was 0.001, the maximum number of iterations was 50,000, the learning strategy was multistep, and the learning rate was decayed by a factor of ten at 40,000 and 45,000 iterations.

[0065] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. An infrared target detection method based on feature-enhanced lightweight SSD, characterized in that, Includes the following steps: The basic network structure for feature extraction is constructed as follows: Based on the lightweight MobiLeNet network, multiple different asymmetric depth separable convolutional modules are stacked multiple times. The input image is downsampled through a standard 3×3 convolutional layer, and then 13 asymmetric depth separable convolutional modules (ADWS) are stacked sequentially in the forward direction of convolution to form an improved AMobiLeNet network structure. In the asymmetric depth separable convolutional module (ADWS), the input feature map is first extracted by convolutions of size 1×3, 3×3, and 3×1, respectively. The three results are added together as the result of depth convolution. Then, a 1×1 pointwise convolution is performed on the result of depth convolution to change the number of output channels, thereby integrating information across channels to obtain the final output. The image is input into the basic network structure and convolved layer by layer through multiple convolutional layers. Low-level feature maps and high-level feature maps of the image are obtained at different convolutional layers. The input image is processed by the output matrices of ADWS11 and ADWS13 as the low-level feature map. Four additional convolutional layers C14, C15, C16, and C17 are added after the deep convolutional layer ADWS13, with the output of each of these four additional convolutional layers serving as the high-level feature map. A designed PAC feature enhancement module is added after the low-level feature branch ADWS11 to enhance features and obtain a semantically rich low-level feature map. Specifically, the feature map output by module ADWS11 is processed in parallel branches by the PAC module. The outputs of the four branches are concatenated along the channel dimension, and finally, the concatenation is performed along the channel dimension and added to the output image of module ADWS11 to obtain a semantically rich low-level feature map. The PAC feature enhancement module has a total of four branches. For the input of the previous layer: (1) Branch 1 reduces the channel dimension by passing through a 1×1 convolutional layer, and then adds a 3×3 dilated convolution with a dilation rate of 1 after the convolutional layer to produce the output. (2) Branch 2 reduces the channel dimension by passing through a 1×1 convolutional layer, and adds a 3×3 dilated convolution with a dilation rate of 2 after the convolutional layer to generate the output. (3) Branch 3 reduces the channel dimension by passing through a 1×1 convolutional layer, and adds a 3×3 dilated convolution with a dilation rate of 3 after the convolutional layer to generate the output. (4) Branch 4 first passes through a 1×1 convolutional layer to reduce the channel dimension, and then adds 3×3 dilated convolutions with dilation rates of 1, 2 and 3 respectively after the convolutional layer to generate the output. For the semantically rich low-level feature maps and high-level feature maps, predict the category and bounding box to which each predicted box belongs for each grid on the feature map; The bounding boxes are merged using a nonmaximum suppression algorithm, and the final prediction result of the entire network is output.

2. The infrared target detection method based on feature enhancement and lightweight SSD according to claim 1, characterized in that, ADWS11 and ADWS13 are the 11th and 13th layer asymmetric depthwise convolutional modules stacked in the improved AMobi leNet network structure, respectively, and their outputs are 14×14×512 feature maps and 7×7×1024 feature map structures, respectively.

3. The infrared target detection method based on feature-enhanced lightweight SSD according to claim 1, characterized in that, The semantically rich low-level feature map and four high-level feature maps are respectively input into the classification and regression detection networks. For each pixel center point of the six feature layers, the ADWS11 low-level feature map enhanced by PAC is input into the classification and regression detection network, and its linear regression layer will output four prior boxes with different scales. The ADWS13 low-level feature map and the C14, C15, C16, and C17 high-level feature maps generated by four additional convolutional layers are input into the classification and regression detection network, and their linear regression layers will output six prior boxes with different scales. The position parameters of each prior box contain four data points, including the center position coordinates, width, and height of the target bounding box. The softmax layer outputs the preset target type probability information, where there are A target types, that is, the probability of outputting the corresponding A type of target.

4. The infrared target detection method based on feature-enhanced lightweight SSD according to claim 3, characterized in that, The classification and regression detection network contains 6 detection branches, each corresponding to 6 layers of input feature maps. Each detection branch has the same structure, consisting of two 1×1 convolutional branches. One branch outputs the category corresponding to each candidate box through a softmax layer, and the other branch outputs the corresponding bounding box through a linear regression layer. Finally, the prior box information obtained from the 6 feature layers is combined using a non-maximum suppression algorithm to suppress some overlapping or erroneous prior boxes and generate the final detection result.

5. The infrared target detection method based on feature-enhanced lightweight SSD according to claim 4, characterized in that, This method also includes training the basic network structure, which employs an improved asymmetric loss (ASL) function, as follows: Where, γ + and γ - These are the focusing parameters for positive and negative samples, respectively. They decouple the rates of change of the loss function for positive and negative samples, ensuring that features can still be learned effectively even in the absence of positive samples. L + and L - This is the general cross-entropy loss for positive and negative samples. The loss for negative samples can be reduced by changing the value of m. y′ is the output of the activation function, representing the probability that a sample belongs to the 1 group, ranging from [0,1]. m The newly introduced hyperparameter is used to define the new hyperparameter, and its expression is y. m =max(y′-m,0).

Images