Adaptive object detection method based on lightweight residual learning and deconvolution cascade

Abstract

The present invention discloses an adaptive target detection method based on a lightweight residual network and a deconvolution cascade, comprising the following steps: obtaining an image training data set and a test data set; extracting deep features of the image to be detected by combining a lightweight residual network with deep separable convolution and residual learning to obtain a deep expression of the target; using 1x1 convolution to fix the output feature map dimension of the extracted feature maps of different levels; using a deconvolution cascade structure to increase the resolution of the deep-level feature map to achieve consistency with the spatial size of the feature map of the previous layer; using semantic features to guide the candidate region generation network to adaptively generate a target candidate box that is more matched with the real target on the multi-scale feature map; and finally correcting the generated target candidate box Anchor. The present invention effectively improves the accuracy of target detection, can quickly and accurately detect targets under complex conditions, and effectively improves the real-time performance of target detection.

Description

Technical Field

[0001] The present invention relates to a target detection method, and belongs to the fields of digital image processing, deep learning, and artificial intelligence. In particular, an adaptive target detection method based on lightweight residual learning and deconvolution cascade is designed. Background Art

[0002] With the rapid development of computer vision technology, target detection technology has become a research hotspot in the field of artificial intelligence and computer vision, and has been widely used in military and civilian fields. Target detection mainly targets one or more specific targets in a video image sequence to identify and locate them. In most cases, video image acquisition devices contain rich visual content. Although they can provide more comprehensive scene information, the targets to be detected in images or videos usually have large scale variations, concentrated distribution, occlusions, and insufficient detection details. This results in the inability of current target detection algorithms to effectively extract target features and accurately locate target positions. Therefore, accurately and efficiently detecting target objects is one of the key issues in target detection tasks.

[0003] In recent years, target detection technology based on deep learning has achieved great success, and many scholars have begun to study the use of deep learning methods for target detection. The current mainstream target detection algorithms are mainly divided into two categories, one is single-stage target detection based on regression methods, and the other is two-stage target detection based on regional candidate boxes. The former is mainly represented by the YOLO series, and its detection idea is to regard the detection problem as a regression analysis problem of target position and target category information, and directly output the detection results through convolutional neural networks; the latter is mainly represented by the R-CNN series, as the name suggests. This method divides the target detection process into two main stages. The candidate region extraction module is the first part, which extracts network features through the mainstream backbone network to detect background and foreground areas. The second stage is to classify and correct the coordinates of the candidate regions to complete the accurate detection of the target. Although the former is faster, the accuracy is not satisfactory; the latter requires two convolutional network operations, which will undoubtedly lead to a higher detection accuracy of the two-stage detection network, but to a certain extent reduce the detection speed. However, with the development of convolutional neural networks, various lightweight backbone networks (such as ShuffleNet, MobileNet, etc.), convolution methods (such as deep convolution, separable convolution, point convolution, etc.) and different connection methods (such as SkipConnection, etc.) have emerged, which has continuously reduced the network complexity and computational complexity. At the same time, the continuous development of hardware equipment has also laid the foundation for the improvement of target detection speed. In addition, with the widespread application of convolutional neural networks, deconvolution has also come into people's sight. As the inverse process of convolution, deconvolution can effectively solve the problems of reduced feature map resolution and feature loss caused by deep convolution operations. It is an important means of multi-scale feature fusion.

[0004] The shortcomings of existing methods are as follows: On the one hand, traditional classic target detection algorithms are limited by manually designed manual features and selective search algorithms, resulting in low target detection accuracy, slow detection speed, and poor algorithm robustness; on the other hand, although the accuracy of target detection based on deep learning has been improved, the convolutional neural network has a large number of parameters, the algorithm structure is highly complex, and the amount of calculation is large, making it difficult to meet real-time requirements. Summary of the invention

[0005] The present invention aims to solve the above-mentioned defects and proposes an adaptive target detection algorithm based on lightweight residual learning and deconvolution cascade. Combining the advantages of residual learning and dividing the ordinary convolution operation into two parts, deep convolution layer and point convolution layer, it is used to compress network parameters and improve the computational efficiency of the network. Afterwards, a multi-scale adaptive candidate region generation network is constructed on the basis of the lightweight residual network. The high-level semantic features are weighted and integrated into the low-level feature map through the deconvolution cascade structure, which enhances the feature's ability to express the target. Multi-level and different-scale feature maps are used for target prediction. The position and shape of the candidate box are predicted according to the image features to generate sparse and arbitrarily shaped candidate boxes, thereby achieving better target detection performance.

[0006] In order to achieve the above object, the present invention proposes an adaptive target detection method based on lightweight residual learning and deconvolution cascade, comprising the following steps:

[0007] S1: Collect data through image acquisition equipment to obtain image training data set and test data set;

[0008] S2: Build a lightweight deep residual network, input the image training dataset and test dataset in S1, and perform feature extraction;

[0009] S3: Select the feature maps extracted from the last four layers of the lightweight residual network and use 1x1 convolution to fix the output feature map dimension;

[0010] S4: Construct a multi-scale adaptive candidate region generation network. Since the feature maps obtained by different layers of the network have different sizes, and the size of the feature map of the previous layer is larger than that of the feature map of the current layer, in order to fuse the feature maps of different layers, the feature maps of different sizes extracted from S3 are used to increase the resolution of the feature map using a deconvolution cascade structure to achieve consistency in the spatial size of the feature map of the previous layer. The feature maps are weighted and fused according to the channel dimension, and the candidate region generation network is used to generate predicted target boxes and category information.

[0011] S5: Based on the position and category information of the predicted target box generated by the adaptive candidate region, the multi-task loss function of the following formula is used for position correction and category regression;

[0012] L=L cls +L reg +β1L loc +β2L shape (1)

[0013] Among them, L is the total loss function of the algorithm, L cls Represents the classification loss function when classifying features, L reg Represents the regression loss function when performing position regression, L locIndicates the positioning loss when locating the target, L shape represents the shape loss of the target detection box, β1 and β2 represent the weighting coefficients of the multi-task loss function, which are 1 and 0.1 respectively.

[0014] Beneficial Effects

[0015] According to the adaptive target detection based on lightweight residual learning and deconvolution cascade according to the embodiment of the present invention, in terms of feature extraction, this method adopts a lightweight residual network for feature extraction, establishes a lightweight feature extraction network through deep separable convolution combined with the advantages of residual learning, and then performs feature fusion according to the deconvolution cascade to construct a multi-scale adaptive candidate region generation network to achieve the consistency of spatial dimensions of feature maps at different levels, and perform weighted fusion operations, and finally use semantic features to guide the network to adaptively generate target candidate boxes that better match the real target. Simulation experiments show that the present invention can effectively extract target features in video image sequences, enhance the ability of features to express targets, and can quickly and accurately identify and locate targets under conditions of occlusion, scale changes, and small targets, with high accuracy and robustness. At the same time, the lightweight network greatly reduces the amount of calculation and meets the real-time detection requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and / or additional aspects and advantages of the present invention will become more easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0017] Figure 1 is a flowchart of an adaptive target detection method based on lightweight residual learning and deconvolution cascade according to an embodiment of the present invention, and

[0018] Figure 2 is a schematic diagram of a lightweight residual network according to an embodiment of the present invention, and

[0019] Figure 3 A schematic diagram of multi-scale feature fusion of a deconvolution cascade structure according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or components having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0021] like Figure 1 As shown, the adaptive target detection method based on lightweight residual learning and deconvolution cascade according to the present invention includes the following steps:

[0022] S1: Collect data through image acquisition equipment to obtain image training data set and test data set;

[0023] S1.1: Expand the dataset by preprocessing the samples in the dataset by cropping, flipping, rotating, and scaling.

[0024] S1.2: Extract positive and negative samples from each image, annotate the positive samples to be detected, and mark the position and category of each target with a rectangular box;

[0025] S2: Build a lightweight deep residual network, input the training data set, and perform feature extraction;

[0026] It is divided into several steps:

[0027] S2.1: Input the training dataset into the lightweight residual network and perform depth-wise separable convolution on the image;

[0028] 1) Perform deep convolution on the input image, and assign a convolution kernel to each of the N channels of the input image feature F. Each convolution kernel is only responsible for performing convolution operations on the image features of the channel. The size of the convolution kernel is consistent with that of the standard convolution, with a number of N and a step size of 1, including padding operations.

[0029] 2) Perform point convolution on the feature map obtained by depth convolution in the previous step, with a convolution kernel size of 1×1 and a number of L to obtain a feature map of the specified channel dimension;

[0030] S2.2: Connect the shallow network and the deep network through skip connection, and fuse the feature information of feature maps of different levels after convolution, which is equivalent to fusing the feature information of the bottom layer into the high layer;

[0031] S3: Select the feature maps in the last four layers of the lightweight residual network and use 1x1 convolution to fix the output feature map dimension;

[0032] S4: Construct a multi-scale adaptive candidate region generation network, use the deconvolution cascade structure to increase the resolution of the deep-level feature map, achieve the consistency of the spatial size of the feature map of the previous layer, and perform weighted fusion operations on the feature maps with consistent spatial sizes according to the channel dimension, and use the candidate region generation network to generate predicted target boxes and category information;

[0033] The specific steps are as follows:

[0034] S4.1: Select the multi-level feature maps {C2, C3, C4, C5} in the lightweight residual network, corresponding to the output of the last layer of each network level;

[0035] S4.2: Use deconvolution operation on the high-level feature map P5 (obtained by 1x1 convolution of C5) to make the feature map size consistent with C4, and then weightedly fuse it with the corresponding previous feature map C4 to obtain a new feature map P4.

[0036] S4.3: Repeat the process of S4.2 until a feature map P2 with the same size as C2 is generated, which has more detailed feature information of small objects. Therefore, in addition to directly adding the same weights, this paper assigns additional weights to 6 different feature maps. The weighted fusion formula is:

[0037]

[0038] Where D(·) is the deconvolution function, α1, α2, α3, α4, α5 and α6 represent weight coefficients, which are 0.7, 0.3, 0.6, 0.4, 0.45 and 0.55 respectively. To avoid redundant feature information, the sum of the weight coefficients of each layer fusion is 1.

[0039] S4.4: Input the feature map after deconvolution cascade feature fusion into the adaptive candidate region generation network to obtain the center position and shape of the Anchor. The specific steps are as follows;

[0040] 1) The shape of the anchor generated adaptively based on the image features varies according to the position. The anchor feature adaptive branch network N is used T The features are transformed so that the features of larger anchors can encode the content of larger areas, and the features of smaller anchors can extract the content of smaller areas. This branch network uses a 3×3 deformable convolutional layer to achieve this.

[0041] f i '=N T (f i ,w i ,h i ) (3)

[0042] where f i is the feature of the ith position, w i ,h i is the corresponding Anchor shape. That is, first predict the offset from the output of the shape prediction branch, and then use deformable convolution on the original feature map to obtain f i '.

[0043] 2) Anchor center prediction branch network N L Generate a feature map F I Probability maps of the same size, Ρ(i,j|F I) indicates the probability that the target object may appear at the position (i, j) of the feature map, corresponding to the coordinates [(i+1 / 2)s, (j+1 / 2)s] in image I, where s is the step size of the feature map. N L The branch network uses a 1x1 convolutional network to obtain the confidence map of the target, and then uses the sigmoid function to convert it into a probability value. Based on the generated probability map, the location where the corresponding probability value is greater than the pre-defined threshold (the value is 0.05 in the comparison experiment) is selected to determine the area where the target may exist;

[0044] 3) After determining the possible location of the target, the shape prediction branch N S The network branch contains a 1×1 convolutional layer that produces a two-channel map containing the values ​​of dw and dh. I , the shape prediction branch will predict the best shape (w, h) for each position. Since the range of w and h may be very large, after the transformation of formula (6), the shape prediction branch will output dw and dh. These two values ​​can map w and h, where s is the step size and λ is the empirical scale factor (the value is 8 in this experiment). This nonlinear transformation mapping can map [0,1000] to [-1,1], making the shape prediction branch calculation simpler and more stable.

[0045]

[0046] S5: The position and category of the predicted target box generated by the adaptive candidate region are corrected and the category is regressed using the multi-task loss function of the following formula;

[0047] L=L cls +L reg +β1L loc +β2L shape (5)

[0048] Where L cls and L reg Represent the classification loss and regression loss in the traditional network, L shape and L loc They are the newly added anchor positioning loss and anchor shape loss respectively.

[0049] The specific steps are as follows:

[0050] S5.1: Get the real target box (x g ,y g ,w g ,h g )’s feature map (x′) g ,y′ g ,w′ g ,h′g ), the cross entropy loss function and the mean square error function are used to obtain the classification loss and regression loss respectively, and then two regions (x′) are defined in the target feature map region. g ,y′ g ,δ1w′ g ,δ1h′ g ) and (x′ g ,y′ g ,δ2w′ g ,δ2h′ g ), δ1 and δ2 are 0.2 and 0.5 respectively. (x′ g ,y′ g ,δ1w′ g ,δ1h′ g ) is the central area, (x′ g ,y′ g ,δ2w′ g ,δ2h′ g ) excluding the central area is the ignored area, and the rest is the peripheral area.

[0051] S5.2: Take the central area as a positive sample and the peripheral area as a negative sample, and use Focal Loss to train the localization branch L loc ;

[0052] S5.3: Train the shape prediction branch, using the following IOU calculation method:

[0053] vIOU(a wh ,G)= maxIOU(a wh ,G) (6)

[0054] Where IOU(·) is the definition of traditional IOU, G represents the true target box, and a wh Represents the anchor variable, enumerating 9 pairs of common anchors of different proportions and sizes as a wh , and use the maximum value as the final vIOU (a wh ,G). Determine the shape loss L of the target box anchor shape , as shown in the following formula (7), where l1 is the smooth L1 loss function, (w,h) and (w g ,h g ) represent the predicted anchor shape and the corresponding true target shape, respectively.

[0055]

[0056] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An adaptive target detection method based on lightweight residual learning and deconvolution cascade, characterized by: S1: Collect data through image acquisition equipment to obtain image training data set and test data set; S2: Build a lightweight deep residual network, input the image training dataset and test dataset in S1, and perform feature extraction; S3: Select the feature maps extracted from the last four layers of the lightweight residual network and use 1x1 convolution to fix the output feature map dimension; S4: Construct a multi-scale adaptive candidate region generation network. Since the feature maps obtained by different layers of the network have different sizes, and the size of the feature map of the previous layer is larger than that of the feature map of the current layer, in order to fuse the feature maps of different layers, the feature maps of different sizes extracted from S3 are used to increase the resolution of the feature map using a deconvolution cascade structure to achieve consistency in the spatial size of the feature map of the previous layer. The feature maps are weighted and fused according to the channel dimension, and the candidate region generation network is used to generate predicted target boxes and category information. S5: Based on the position and category information of the predicted target box generated by the adaptive candidate region, the multi-task loss function of the following formula is used for position correction and category regression; L=L cls +L reg +β1L loc +β2L shape (1) Among them, L is the total loss function of the algorithm, L cls Represents the classification loss function when classifying features, L reg Represents the regression loss function when performing position regression, L loc Indicates the positioning loss when locating the target, L shape represents the shape loss of the target detection box, β1 and β2 represent the weighting coefficients of the multi-task loss function, which are 1 and 0.1 respectively; S4 is divided into the following steps: S4.1: Select the multi-level feature maps {C2, C3, C4, C5} in the lightweight residual network, corresponding to the output of the last layer of each network level; S4.2: Use deconvolution operation on the high-level feature map P5 to make the feature map size consistent with C4, and then weightedly fuse it with the corresponding previous feature map C4 to obtain a new feature map P4; S4.3: Repeat S4.2 until a feature map P2 with the same size as C2 is generated, which has more detailed feature information of small targets; therefore, on the basis of directly adding the same weights, additional weights are assigned to 6 different feature maps, and the weighted fusion formula is: Where D(·) is the deconvolution function, α1, α2, α3, α4, α5 and α6 represent weight coefficients, which are 0.7, 0.3, 0.6, 0.4, 0.45 and 0.55 respectively. To avoid feature information redundancy, the sum of the weight coefficients of each layer fusion is 1; S4.4: Input the feature map after deconvolution cascade feature fusion into the adaptive candidate region generation network to obtain the center position and shape of the Anchor. The specific steps are as follows; 1) The shape of the anchor generated adaptively based on the image features varies according to the position. The anchor feature adaptive branch network N is used T The features are transformed so that the features of larger anchors can encode the content of larger areas, and the features of smaller anchors can extract the content of smaller areas. T This is achieved using a 3×3 deformable convolutional layer; f i '=N T (f i ,w i ,h i ) (3) where f i is the feature of the ith position, w i ,h i is the corresponding Anchor shape; that is, first predict the offset from the output of the shape prediction branch, and then use deformable convolution on the original feature map to obtain f i '; 2) Anchor center prediction branch network N L Generate a feature map F I Probability maps of the same size, Ρ(i,j|F I ) indicates the probability that the target object may appear at the position (i, j) of the feature map, corresponding to the coordinates [(i+1 / 2)s, (j+1 / 2)s] in image I, where s is the step size of the feature map; N L The branch network uses a 1x1 convolutional network to obtain the confidence map of the target, and then uses the sigmoid function to convert it into a probability value. Based on the generated probability map, the area where the target may exist is determined by selecting the location where the corresponding probability value is greater than the pre-defined threshold. 3) After determining the possible location of the target, the shape prediction branch N S The network branch contains a 1×1 convolutional layer, which produces a two-channel map containing the values ​​of dw and dh; the input feature map F I , the shape prediction branch will predict the best shape (w, h) for each position. The range of w and h may be very large, so after the transformation of formula (6), the shape prediction branch will output dw and dh. These two values ​​map w and h, where s is the step size and λ is the empirical scale factor. This nonlinear transformation mapping can map [0,1000] to [-1,1], making the shape prediction branch calculation simpler and more stable. S5 is divided into the following steps: S5.1: Get the real target box (x g ,y g ,w g ,h g )’s feature map (x′) g ,y′ g ,w′ g ,h′ g ), the cross entropy loss function and the mean square error function are used to obtain the classification loss and regression loss respectively, and then two regions (x′) are defined in the target feature map region. g ,y′ g ,δ1w′ g ,δ1h′ g ) and (x′ g ,y′ g ,δ2w′ g ,δ2h′ g ), δ1 and δ2 are 0.2 and 0.5 respectively; (x′ g ,y′ g ,δ1w′ g ,δ1h′ g ) is the central area, (x′ g ,y′ g ,δ2w′ g ,δ2h′ g ) excluding the central area is the ignored area, and the rest is the peripheral area; S5.2: Take the central area as a positive sample and the peripheral area as a negative sample, and use Focal Loss to train the localization branch L loc ; S5.3: Train the shape prediction branch, using the following IOU calculation method: vIOU(a wh ,G)= maxIOU(a wh ,G) (6) Where IOU(·) is the definition of IOU, G represents the true target box, and a wh Represents the anchor variable, enumerating 9 pairs of anchors of different proportions and sizes as a wh , and use the maximum value as the final vIOU (a wh ,G); determine the shape loss L of the target box anchor shape , as shown in the following formula (7), where l1 is the smoothL1 loss function, (w,h) and (w g ,h g ) represent the predicted anchor shape and the corresponding true target shape respectively; 2. The adaptive target detection method based on lightweight residual learning and deconvolution cascade according to claim 1, characterized in that: In S1, data is collected by an image acquisition device to obtain an image training data set and a test data set; S1.1: Preprocess the samples in the dataset by cropping, flipping, rotating, and scaling to expand the dataset; S1.2: Extract positive and negative samples from each image, annotate the positive samples to be detected, and mark the position and category of each target with a rectangular box.

3. The adaptive target detection method based on lightweight residual learning and deconvolution cascade according to claim 1, characterized in that: In S2, a lightweight deep residual network is constructed, and the training data set is input to perform feature extraction; The specific steps are as follows: S2.1: Input the training dataset into the lightweight residual network and perform depth-wise separable convolution on the image; 1) Perform deep convolution on the input image, and assign a convolution kernel to each of the N channels of the input image feature F. Each convolution kernel is only responsible for performing convolution operations on the image features of the channel. The size of the convolution kernel is consistent with that of the standard convolution, with a number of N and a step size of 1, including padding operations. 2) Perform point convolution on the feature map obtained by depth convolution in the previous step, with a convolution kernel size of 1×1 and a number of L to obtain a feature map of the specified channel dimension; S2.2: Connect the shallow network and the deep network through jump connections, and fuse the feature information of feature maps at different levels after convolution, which is equivalent to fusing the feature information of the bottom layer into the high layer.

Images