Progressive infrared peripheral scanning system real-time target detection and identification method based on DSP and GPU

Abstract

The invention provides a DSP (Digital Signal Processor) and GPU (Graphic Processing Unit)-based real-time target detection and identification method for a progressive infrared peripheral scanning system, which comprises the following steps of: acquiring an infrared peripheral scanning image of scene, target attitude angle, scale and contrast change, and segmenting the image into a plurality of overlapped square subframes; the dual-stage target detection network takes ResNet-50 as a skeleton network to carry out multi-scale feature extraction, after feature fusion, candidate target region suggestions are generated firstly, and then candidate target regions are classified and positioned; after model training is completed, a DSP end is adopted to carry out downscaling decomposition on an original circumferential scanning image of a target, then hierarchical processing is carried out, weak and small target detection processing and surface target detection processing are carried out on images of all scales respectively, and a GPU end is adopted to input image subframes output through coarse detection into a target detection network. The infrared target detection data set constructed by the method can effectively solve the generalization problem of network training; the gradual target detection and recognition algorithm is adopted, so that the target recognition time can be greatly shortened, and real-time detection and recognition can be realized by the infrared peripheral scanning system.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection of infrared circular scanning systems, and in particular relates to a real-time target detection and recognition method of a progressive infrared circular scanning system based on DSP and GPU. Background Art

[0002] Target detection and recognition technology involves two core issues: precise positioning of the target's spatial position and accurate determination of its category attributes. In the field of infrared panoramic scanning, the system achieves multi-pitch panoramic imaging through a long linear array of large-area focal plane detectors combined with a precision servo mechanism. Its horizontal coverage range is close to 360 degrees, and its imaging resolution reaches 288×50,000 to 768×40,000. While significantly expanding the detection range compared to traditional small-field-of-view infrared equipment, it also brings two technical challenges: First, the contradiction between the need for real-time processing of massive amounts of data and limited computing power. Each frame of image contains hundreds of thousands of pixels, which places a high pressure on processing. Second, complex background interference exacerbates the difficulty of target recognition. Unstructured background clutter formed by low-altitude clouds, buildings, mountains, and vegetation significantly reduces the scene adaptability of traditional algorithms.

[0003] Existing methods based on traditional image processing often employ a phased approach: first, target coarse localization is performed using edge detection methods such as threshold segmentation or the Canny operator, followed by artificial feature extraction using methods such as the Histogram of Oriented Gradients (HOG), and finally, target identification is achieved using classifiers such as support vector machines (SVM). These methods are somewhat effective in scenarios where the contrast between the target and the background is stable and the attitude angle changes are limited. However, their artificial feature design struggles to adapt to the diverse target scales, dynamic contrast changes, and complex background interference found in infrared images. In particular, in low-altitude, ground-based multi-source interference environments, the false alarm suppression capabilities of traditional algorithms decline dramatically.

[0004] In recent years, deep learning-based object detection algorithms have achieved breakthroughs in the visible light domain. However, their migration to the infrared domain faces two constraints: First, existing infrared datasets are mostly constructed for target tracking tasks. For example, the VOT-TIR dataset, while containing continuous frame sequences, suffers from limitations such as a single target type (only pedestrians or vehicles), fixed scenes, and limited variation in target appearance, resulting in insufficient generalization of trained models. Second, typical deep learning models such as Faster RCNN have large parameter counts and high computational complexity, making them difficult to meet real-time requirements when processing megapixel-level cycle scan images and unable to match the high frame rate imaging of infrared cycle scan systems. This conflict between algorithmic time consumption and the system's real-time requirements has severely restricted the engineering application of deep learning technology in infrared cycle scan systems. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems of insufficient generalization ability and poor real-time performance in the target detection and recognition of infrared circular scanning systems in the prior art, and to provide a progressive infrared circular scanning system real-time target detection and recognition method based on DSP and GPU. By constructing a diversified infrared target detection data set and a progressive detection framework, efficient real-time processing is achieved while improving target detection accuracy.

[0006] To achieve the above objectives, the technical solutions provided by the present invention are:

[0007] A real-time target detection and recognition method for a progressive infrared scanning system based on DSP and GPU is provided, comprising the following steps:

[0008] Step 1: Construct a dataset using indirect acquisition: collect infrared cyclic scan images covering multiple scenes and multiple targets with varying attitude angles, scales, and contrasts, and segment the cyclic scan images into several overlapping square image subframes.

[0009] Step 2: Construct a two-stage target detection network and feature fusion network: The target detection network uses ResNet-50 as the backbone network to extract multi-scale features. After feature fusion, the RPN sub-network is used in the first stage to generate candidate target region proposals. In the second stage, the RCNN sub-network is used to classify and locate the candidate target regions. The feature fusion network uses 1×1 convolution to unify the number of channels of the underlying feature map extracted by ResNet-50. After upsampling, it is added layer by layer with the high-level feature map to generate a multi-layer fusion feature map. The underlying feature map is downsampled to generate another layer of feature map. These feature maps are used as the input of the RPN sub-network.

[0010] Step 3: Target detection network training and optimization: The constructed target detection network is iteratively trained with the constructed dataset. After the model converges, the optimal network model parameters are obtained.

[0011] Step 4, progressive detection and recognition, includes the following sub-steps:

[0012] Step 4.1, coarse detection: Use the DSP to perform dynamic range compression and detail enhancement on the original target scan image, downscale and decompose it, and perform hierarchical processing. Perform weak target detection and area target detection on the image at each scale, and output candidate image subframes;

[0013] Step 4.2, fine detection: Use the GPU side to input the candidate image subframes output by the coarse detection into the trained target detection network to perform accurate classification and positioning.

[0014] Furthermore, a multi-task loss function is used in step 3. The first stage uses cross entropy classification loss and smoothL1 localization loss, and the second stage uses cross entropy classification loss and smoother L1 localization loss to balance the classification and localization task gradients.

[0015] Furthermore,

[0016]

[0017] Where α controls the gradient of the outer point, and β controls the changing trend of the gradient of the inner point.

[0018] Furthermore, α=2, β=2.

[0019] Furthermore, in step 3, a cross-domain transfer learning strategy is adopted, which includes the following sub-steps:

[0020] Step 3.1, weight initialization: Use the visible light dataset to pre-train the ResNet-50 skeleton network and transfer the shallow general structural features;

[0021] Step 3.2, fine-tuning training: Continue training on the constructed infrared dataset to optimize the deep network parameters.

[0022] Furthermore, in step 3.1, the MSCOCO dataset is used to pre-train the ResNet-50 skeleton network.

[0023] Furthermore, in step 1, the overlapping area of the square subframes is dynamically adjusted according to the target maximum size.

[0024] Furthermore, in step 2, the feature fusion network unifies the number of channels of the underlying feature map output by ResNet-50 through 1×1 convolution, and adds it layer by layer with the high-level features after upsampling to generate fused feature maps P2-P5. The underlying feature map is downsampled to generate P6, and P2-P6 are input into the RPN sub-network together.

[0025] Furthermore, in step 2, the RPN subnetwork sets equally spaced prior boxes on the original image, called anchors, and uses one convolution and two convolutions to obtain the positive and negative sample probabilities and bounding box position optimization of each anchor, which are divided into positive samples containing the target and negative samples not containing the target, and a certain number of positive and negative samples are screened as region proposals.

[0026] Furthermore, in step 4.1, for small targets with a pixel size of less than 20×20, background suppression filtering is performed on the downscaled image using Robinson filtering and morphological filtering operators of the corresponding scale; for surface targets with a pixel size of more than 20×20, surface target segmentation and detection are performed on the downscaled image, and finally the targets are merged at the original scale.

[0027] The advantages of the present invention are:

[0028] 1. The present invention proposes a real-time target detection and recognition method for a progressive infrared circular scanning system based on DSP and GPU. This method constructs an infrared target detection dataset that includes changes in scene, target attitude angle, scale, and contrast. This method effectively solves the generalization problem of target detection network training. The use of a progressive target detection and recognition algorithm can significantly reduce target recognition time, enabling the infrared circular scanning system, which rotates once every 0.7 seconds, to have real-time target detection and recognition capabilities.

[0029] 2. The RCNN subnetwork of the object detection network uses smoother L1 positioning loss. Compared with smooth L1, the gradient of inliers changes from a linear function to a power function. This design not only adds more nonlinearity to the network, but also enhances its feature learning ability. It also makes the gradient change trend between inliers and outliers more gentle, avoiding large gradient value changes that are detrimental to network training.

[0030] 3. In network training, a cross-domain transfer learning strategy is adopted from the visible light source domain to the infrared target domain to share shallow common features such as edges and textures, which can improve the model convergence speed and detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and / or other features and advantages of the present invention will become more readily understood through the following description with reference to the accompanying drawings, in which:

[0032] Figure 1 This is a flow chart of the real-time target detection and recognition method of the progressive infrared scanning system based on DSP and GPU of the present invention;

[0033] Figure 2 This is the process of acquiring the image subframe in the present invention;

[0034] Figure 3 This is the process of constructing the infrared target detection dataset in the present invention;

[0035] Figure 4 It is a structural diagram of the target detection network in the present invention;

[0036] Figure 5 It is a structural diagram of the feature fusion network in the present invention;

[0037] Figure 6 is a structural diagram of the region proposal network in the present invention;

[0038] Figure 7 is a gradient curve diagram of smoother L1 and smooth L1 in the present invention;

[0039] Figure 8 It is the progressive target detection and recognition method of the present invention;

[0040] Figure 9 are some images in the infrared target detection dataset constructed in the example of the present invention;

[0041] Figure 10 is the training loss function and classification accuracy of different weight initialization methods in the examples of the present invention;

[0042] Figure 11 It is the training loss function of smoother L1 and smooth L1 in the example of the present invention. DETAILED DESCRIPTION

[0043] The present invention will be described in detail below with reference to the accompanying drawings by means of exemplary embodiments of the present invention. It should be noted that the following detailed description of the present invention is only for the purpose of illustration and is not intended to limit the present invention.

[0044] The present invention provides a real-time target detection and recognition method for a progressive infrared circular scanning system based on DSP and GPU. The convolutional neural network algorithm deployed on the GPU refines the DSP coarse detection results, determines the category, eliminates false alarms, and refines the position. To ensure the operating efficiency of the algorithm, a single-frame infrared circular scanning image is divided into image subframes, and the target detection and recognition algorithm is run on each subframe. A multi-target, multi-scene, multi-pose, and multi-scale infrared target detection dataset is established, which not only overcomes the problem of insufficient training data, but also enhances the adaptability of the algorithm in different scenarios. Through a "two-step" progressive detection and recognition strategy, the infrared circular scanning system achieves real-time detection and recognition of targets in a complex low-altitude ground background.

[0045] Reference Figure 1 The present invention provides a real-time target detection and recognition method for a progressive infrared scanning system based on DSP and GPU, comprising the following steps:

[0046] Step S1, constructing a data set by indirect acquisition: collecting infrared circumscan images covering multiple scenes and multiple targets with varying attitude angles, scales, and contrasts, and dividing the circumscan images into a number of overlapping square image subframes;

[0047] Step S2: Construct a two-stage target detection network and feature fusion network: the target detection network uses ResNet-50 as the backbone network to perform multi-scale feature extraction. After feature fusion, the RPN sub-network is used in the first stage to generate candidate target region proposals. The RCNN sub-network is used in the second stage to classify and locate the candidate target regions. The feature fusion network uses 1×1 convolution to unify the number of channels of the underlying feature map extracted by ResNet-50. After upsampling, it is added layer by layer with the high-level feature map to generate a multi-layer fused feature map. The underlying feature map is downsampled to generate another layer of feature map. These feature maps serve as the input of the RPN sub-network.

[0048] Step S3, target detection network training and optimization: iteratively train the constructed target detection network with the constructed dataset, and obtain the optimal network model parameters after the model converges;

[0049] Step S4, progressive detection and identification, includes the following sub-steps:

[0050] Step S4.1, coarse detection: The DSP performs dynamic range compression and detail enhancement on the original periscopic image of the target, decomposes it into layers after downscaling, performs small target detection and area target detection on the image at each scale, and outputs candidate image subframes.

[0051] Step S4.2, fine detection: The GPU is used to input the candidate image subframes output by the coarse detection into the trained target detection network to perform accurate classification and positioning.

[0052] Regarding step S1, the single-frame image produced by the infrared circular scanning system studied in this application has a resolution exceeding one million pixels. Due to the huge amount of data, the algorithm is inefficient when directly processing the single-frame image. Therefore, when performing target detection, the single-frame image is segmented into image subframes.

[0053] Based on the existing data of this application, indirect acquisition is used when constructing the dataset. Specifically, a periscopic image is divided into several square image blocks. Figure 2 The image subframe acquisition process is shown. If segmentation is performed directly, a single target may be segmented into different image blocks, affecting subsequent detection. To address this issue, an overlap region is added between adjacent blocks during segmentation so that an target is complete in at least one image block. The size of the overlap region is selected based on the target's maximum size.

[0054] As a data-driven algorithm, CNN requires a large amount of training data to ensure generalization performance and adapt to changes in the target itself and the scene. In the field of target detection, training data refers to images containing targets and annotations describing the category and location of each target. It is usually desirable to collect data on targets in different states in as many scenes as possible. Figure 2 As shown in the figure, the background of the target in the existing data is too simple. If the algorithm is trained based on this alone, its adaptability to the scene will be relatively weak. In order to ensure that the target can be detected in infrared images of different scenes, this application constructs an infrared target detection dataset, including changes in scene, target posture angle, scale and contrast, such as Figure 3 shown.

[0055] Reference Figure 4 The target detection network used in the infrared scanning system follows the basic framework of the two-stage algorithm. The skeleton network used for feature extraction is ResNet-50. The specific structure is shown in Table 1.

[0056] Table 1 Structure of ResNet network

[0057]

[0058] It's generally believed that features extracted by shallow networks contain more positional information, while deep networks extract more semantic information. In ResNet-50, the size of the feature map is reduced by 200% with each stage. After five downsampling steps, the final feature map is only 1 / 32 of the original image, which can lead to the loss of small objects. To address this issue, a feature fusion network is developed to fuse adjacent feature maps from different stages, integrating the positional and semantic features of the object.

[0059] Reference Figure 5 The bottom feature map first undergoes 1×1 convolution to unify the number of channels, then undergoes upsampling to unify the size of the feature map, and is added to the upper feature map through lateral connections. The fused feature maps P2-P5 and the downsampled P6 of the bottom feature map are jointly input into the region proposal network, as shown in Figure 6 As shown, equally spaced prior boxes, called anchors, are set on the original image. Using one and two convolutions, the probability of positive and negative samples for each anchor is determined, and the bounding box position is optimized. These samples are classified into positive samples containing the target and negative samples excluding the target. A certain number of positive and negative samples are selected as region proposals. Region features are obtained from the corresponding feature map based on the proposed size. After the Region Align operation, the feature sizes corresponding to target regions of different sizes are unified. These features are fed into the RCNN subnetwork, which, through a fully connected layer, predicts the target's location and category.

[0060] When training the object detection network in step S3, it is preferred to define a multi-task loss function to simultaneously perform classification and localization. As a two-stage algorithm, classification and localization are performed separately in the RPN and RCNN subnetworks, and the corresponding four loss functions together constitute the final loss function. The classification loss function is cross entropy, the localization loss of the RPN network is smooth L1, and the localization loss of the RCNN subnetwork is the smoother L1 proposed in this application.

[0061] L=L RPN_cls +L RPN_loc +L RCNN_cls +L RCNN_loc (1)

[0062] Where, L RPN_cls is the classification loss of the RPN sub-network, L RPN_loc is the positioning loss of the RPN sub-network, L RCNN_cls is the classification loss of the RCNN sub-network, L RCNN_loc is the positioning loss of the RCNN sub-network.

[0063] When locating a target, the true value is used as the target and prediction is performed through a regression algorithm. The loss function is usually defined as the distance function between the predicted value and the true value:

[0064]

[0065] Where i represents the index of the bounding box involved in the regression, which will be omitted unless necessary; D represents the four dimensions of the bounding box coordinates, including the horizontal and vertical coordinates of the center point and the width and height of the bounding box; p represents the predicted value of the network; and t represents the target of the regression, that is, the true value. The detailed definitions are as follows:

[0066]

[0067] Where, ω D represents the regression parameters learned by the network (D∈{x,y,w,h}); φ(·) represents the operation process of CNN, r represents the region proposal, so φ(r) represents the region proposal feature calculated by CNN; R D represents the coordinates of the region proposal; G D Represents the coordinates of the true value box. As can be seen from the formula, the true coordinates are not directly set as the regression target, but the relative offset between the true coordinates and the current coordinates. The offset does not change with the change of the input image size, and has scale invariance.

[0068] During the training process, gradient descent is used to minimize the distance between the predicted value and the true value, i.e., the loss value. Usually, 1 is used as the threshold to divide each sample into an inlier (<1) and an outlier (>1). For the definition of the loss function, the RCNN algorithm uses the L2 loss, which is defined as follows:

[0069] L2Loss=x 2 (5)

[0070] Since the gradient of L2 loss is unbounded, special care must be taken when setting the learning rate to avoid gradient explosion at certain outliers. To improve the robustness of the loss function, the Fast RCNN algorithm proposed smooth L1, which connects the L1 loss and the L2 loss with a boundary of 1. It is defined as follows:

[0071] L1Loss=|x| (6)

[0072]

[0073] This design addresses both the non-differentiability of the L1 loss at x = 0 and the sensitivity of the L2 loss to outliers. However, during training, it is the gradient of the loss function, not the loss function itself, that contributes to the network parameter update. Therefore, when designing the loss function, more attention should be paid to its gradient behavior. Based on these observations, this application proposes smoother L1.

[0074] like Figure 7 As shown in Figure 2, compared with smooth L1, the gradient of smoother L1 for the interior point changes from a linear function to a power function.

[0075] This design adds more nonlinearity to the network, enhancing its feature learning capabilities. Furthermore, the gradient change trend between inliers and outliers becomes smoother, avoiding large gradient value changes that are detrimental to network training. In ablation experiments, it was also found that smoother L1 implicitly alleviated the imbalance between classification loss and localization loss. The gradient formula for smoother L1 is as follows:

[0076]

[0077] In the formula, α controls the gradient of the outer points, and β controls the change trend of the gradient of the inner points. A larger β value results in smoother L1 approaching the L1 loss; a smaller β value results in smoother L1 approaching smoother L1. L1 loss can be viewed as smoother L1 when α = 1 and β → +∞; smoother L1 can be viewed as smoother L1 when α = 1 and β = 1. A grid search was performed in ablation experiments to select the values of α and β. Integrating the gradient formula yields the definition of smoother L1.

[0078]

[0079] According to the present invention, a cross-domain transfer learning strategy can be adopted in step S3. Specifically, the ResNet-50 skeleton network is first pre-trained using the visible light dataset to transfer shallow common structural features. Then, training is continued on the constructed infrared dataset to optimize the deep network parameters. If training is started from scratch on a randomly initialized network using the constructed dataset, training failure is likely to occur. This hypothesis has been confirmed in experiments, so cross-domain transfer learning is utilized during training to improve model convergence speed and detection accuracy. During the target detection process, the skeleton network is used to extract image features. Some scholars have found that the features extracted by the shallow network have commonalities for different targets, usually structural features such as edges and textures. Therefore, the skeleton network weights obtained by training on a large dataset can be transferred. They are used to initialize the skeleton network used in this application, and then training is continued on the constructed dataset to fine-tune the weights. This method is equivalent to using a sufficiently large dataset to support training. The weights usually used for transfer are trained on visible light data, but this application requires target detection on infrared images. Therefore, the transfer learning performed is cross-domain, from the visible light source domain to the infrared target domain. Different source domains will produce different initialization weights, and the fine-tuning effects on the target domain will also be different. Experiments can be used to explore the optimal cross-domain transfer learning strategy.

[0080] Reference Figure 8 In order to enable the infrared scanning system to perform detection and recognition in real time, a progressive processing method based on DSP and GPU is designed, and the image processing board, display and control computer and target recognition processing system use Gigabit network port for data transmission.

[0081] The DSP-designed image processing board performs real-time coarse target detection on single-frame periscopic images. After dynamic range compression and detail enhancement, the infrared image is downscaled and decomposed. Depending on the target scale, small target detection and area target detection are performed on each image scale. Small target detection is prioritized on the original-scale image. For targets ranging from 2×2 to 10×10, Robinson filtering and morphological filtering operators of the corresponding scale are used to suppress background filtering, effectively enhancing small targets. For targets ranging from 10×10 to 20×20, the same small target processing process is performed on the downscaled image, achieving algorithm modularity. For targets larger than 20×20, area target segmentation and detection are performed on the downscaled image, and finally, targets are merged at the original scale.

[0082] Therefore, as described above, the present invention constructs an infrared target detection dataset, including changes in scenes, target attitude angles, scales, and contrast, which effectively solves the generalization problem of target detection network training; the use of a progressive target detection and recognition algorithm can significantly reduce the target recognition time, enabling the infrared periscopic system to have real-time target detection and recognition capabilities.

[0083] Next, the real-time target detection and recognition method of the progressive infrared circular scanning system based on DSP and GPU provided by the present invention is further explained with reference to examples.

[0084] In this example, the operating environment is as follows: the GPU uses an Nvidia RTX2080ti with 11GB of video memory and 4352 CUDA cores. A GPU-based object recognition system is added to the detection and recognition process, and the object detection and recognition algorithm described above is deployed.

[0085] The training data is as follows: It contains 7 scenes including roads, trees, deserts, grasslands, mountains, buildings, and cars. Each scene contains 192 images. The dataset has a total of 1344 images. Some images are as follows: Figure 9 When embedding the target into the background, each target is scaled into four sizes to simulate different distances from the detector.

[0086] When training the target detection network, a multi-task loss function is adopted. The first stage uses cross-entropy classification loss and smooth L1 localization loss, and the second stage uses cross-entropy classification loss and smoother L1 localization loss to balance the classification and localization task gradients, and extract shallow network features for cross-domain transfer learning.

[0087] In this example, the effects of different weight initialization methods on detection performance are compared, including Xavier initialization, pre-trained weights on the MSCOCO dataset and ImageNet dataset. The results are shown in Table 2.

[0088] Table 2 Comparison of different weight initialization methods

[0089]

[0090] When the network was initialized with Xavier, that is, without transfer learning, under the same conditions as the other groups of experiments, the trained network failed to detect any targets on the validation set. Figure 10 As shown in the figure, regardless of the weight initialization method used, the classification loss can be reduced to a fairly low level. However, Xavier initialization cannot reduce the localization loss. Since the classification loss is numerically much larger than the localization loss, the overall loss value still shows a downward trend, which may be caused by the small amount of training data. On the other hand, despite the significant differences between the source domain datasets (MSCOCO and ImageNet) and the constructed dataset (infrared), the effect of transfer learning is still good. Compared with ImageNet, the pre-trained weights of MSCOCO reduce the loss to a lower level and perform better in all detection indicators. Because there are more small objects in the MSCOCO dataset and the range of object scales is wider, it is believed that the pre-trained weights of MSCOCO are more suitable for network initialization.

[0091] This example compares the proposed target detection and recognition algorithm with four algorithms: SSD, RetinaNet, Faster RCNN, and Faster RCNN+FPN. The results are shown in Table 3.

[0092] Table 3 Comparative experiments of different algorithms

[0093]

[0094] As shown in the table, the proposed method achieves 82.7 mAP on the test set. Compared to the single-stage SSD and RetinaNet algorithms, the proposed two-stage detection method has significant advantages. Compared to Faster R-CNN, it achieves a 3.0 mAP improvement. When FPN is added, the advantage of 1.2 mAP is still maintained, thanks to the new smoother L1 loss function. The improvements in AP50 and AP75 indicate that the proposed algorithm can achieve higher-precision target localization.

[0095] In order to explore the optimal parameter settings in smoother L1, the ablation experiments shown in Table 4 were performed.

[0096] Table 4 Ablation experiment of smoother L1

[0097]

[0098] In smoother L1, α controls the gradient of the outer point, that is, the maximum value of the localization loss gradient, and β controls the changing trend of the inner point gradient. From another perspective, the change of α can be regarded as a rebalance between the classification loss and the localization loss, and the change of β can be regarded as a rebalance between the localization loss of the inner point and the outer point. Figure 11 As shown in Figure 3, it increases the localization loss during training, thereby alleviating the numerical imbalance between the classification loss and the localization loss.

[0099] According to experimental results, the optimal parameters of smoother L1 are α = 2, β = 2. Compared with the baseline experiment using smoother L1, the mAP of the network using smoother L1 is improved by 1.0 due to the more balanced multi-task loss function.

[0100] Experiments verified the real-time performance of the DSP- and GPU-based progressive target detection and recognition process. The system processed 48 image subframes in 2.1918 seconds, of which 0.5801 seconds were spent on data reception and format conversion, and 1.6117 seconds were spent on running the target detection algorithm and transmitting data back, resulting in a system processing speed of 21.9Hz. By adjusting the coarse detection algorithm and display control program, the infrared circular scanning system, which rotates once every 0.7 seconds, was able to achieve real-time target detection and recognition capabilities, verifying the effectiveness of the proposed method.

[0101] Finally, it should be noted that the features mentioned and / or illustrated in the above description of the exemplary embodiments of the present invention may be incorporated into one or more other embodiments in the same or similar manner, combined with features in other embodiments, or substituted for corresponding features in other implementations. The technical solutions obtained by such combination or substitution shall also be deemed to be included in the scope of protection of the present invention.

Claims

1. A real-time target detection and recognition method for a progressive infrared scanning system based on DSP and GPU, characterized by: The following steps are involved: Step 1: Construct a dataset using indirect acquisition: collect infrared cyclic scan images covering multiple scenes and multiple targets with varying attitude angles, scales, and contrasts, and segment the cyclic scan images into several overlapping square image subframes. Step 2: Construct a two-stage target detection network and feature fusion network: the target detection network uses ResNet-50 as the backbone network to perform multi-scale feature extraction. After feature fusion, the RPN sub-network is used in the first stage to generate candidate target region proposals. The RCNN sub-network is used in the second stage to classify and locate the candidate target regions. The feature fusion network unifies the number of channels of the underlying feature map extracted by ResNet-50 through 1×1 convolution, and then adds it layer by layer with the high-level feature map after upsampling to generate a multi-layer fused feature map. The underlying feature map is downsampled to generate another layer of feature map. These feature maps are used as the input of the RPN sub-network. Step 3: Target detection network training and optimization: The constructed target detection network is iteratively trained with the constructed dataset. After the model converges, the optimal network model parameters are obtained. Step 4, progressive detection and recognition, includes the following sub-steps: Step 4.1, coarse detection: Use the DSP to perform dynamic range compression and detail enhancement on the original target scan image, downscale and decompose it, and perform hierarchical processing. Perform weak target detection and area target detection on the image at each scale, and output candidate image subframes; Step 4.2, fine detection: Use the GPU side to input the candidate image subframes output by the coarse detection into the trained target detection network to perform accurate classification and positioning.

2. The real-time target detection and recognition method for a progressive infrared scanning system based on DSP and GPU according to claim 1 is characterized by: In step 3, a multi-task loss function is used. The first stage uses cross entropy classification loss and smooth L1 localization loss, and the second stage uses cross entropy classification loss and smoother L1 localization loss to balance the classification and localization task gradients.

3. The real-time target detection and recognition method for a progressive infrared scanning system based on DSP and GPU according to claim 2, characterized in that: Where α controls the gradient of the outer point, and β controls the changing trend of the gradient of the inner point.

4. The real-time target detection and recognition method for a progressive infrared scanning system based on DSP and GPU according to claim 3 is characterized by: α＝2，β＝2。 5. The real-time target detection and recognition method of a progressive infrared scanning system based on DSP and GPU according to claim 1 or 2, characterized in that: In step 3, a cross-domain transfer learning strategy is adopted, which includes the following sub-steps: Step 3.1, weight initialization: Use the visible light dataset to pre-train the ResNet-50 skeleton network and transfer the shallow general structural features; Step 3.2, fine-tuning training: Continue training on the constructed infrared dataset to optimize the deep network parameters.

6. The real-time target detection and recognition method for a progressive infrared scanning system based on DSP and GPU according to claim 5 is characterized by: In step 3.1, the MSCOCO dataset is used to pre-train the ResNet-50 skeleton network.

7. The real-time target detection and recognition method for a progressive infrared scanning system based on DSP and GPU according to claim 1 or 2, characterized in that: In step 1, the overlapping area of the square subframes is dynamically adjusted according to the target maximum size.

8. The real-time target detection and recognition method for a progressive infrared scanning system based on DSP and GPU according to claim 1 or 2, characterized in that: In step 2, the feature fusion network unifies the number of channels of the underlying feature map output by ResNet-50 through 1×1 convolution, and adds it layer by layer with the high-level features after upsampling to generate fused feature maps P2-P5. The underlying feature map is downsampled to generate P6, and P2-P6 are input into the RPN sub-network together.

9. The real-time target detection and recognition method for a progressive infrared scanning system based on DSP and GPU according to claim 1 or 2, characterized in that: In step 2, the RPN subnetwork sets equally spaced prior boxes on the original image, called anchors, and uses one convolution and two convolutions to obtain the positive and negative sample probabilities and bounding box position optimization of each anchor, which are divided into positive samples containing the target and negative samples not containing the target, and a certain number of positive and negative samples are selected as region proposals.

10. The real-time target detection and recognition method of a progressive infrared scanning system based on DSP and GPU according to claim 1 or 2, characterized in that: In step 4.1, for small targets with a pixel size of less than 20×20, background suppression filtering is performed on the downscaled image using Robinson filtering and morphological filtering operators of the corresponding scale. For surface targets with a pixel size of more than 20×20, surface target segmentation and detection are performed on the downscaled image, and finally the targets are merged at the original scale.

Images