A method for infrared monitoring and tracking of small targets

Abstract

The present invention provides a method for infrared monitoring and tracking of small targets. The method uses radar to obtain infrared video stream images of the target, performs deep learning target detection and traditional image target detection on the infrared video stream images, and then uses KCF motion target tracking technology to continuously track and detect the target and the surrounding area. The method then uses target detection guided by the tracked target, and then target-guided tracking of the target detection, cooperating and assisting each other to complete the tracking and detection of small targets. By optimizing the deep learning detection model and traditional image detection algorithm, the present invention can achieve a detection speed of 20ms / frame. Combined with KCF tracking, it can meet the requirements of front-end real-time tracking and detection of small targets.

Description

Technical Field

[0001] The invention relates to the technical field of infrared monitoring and tracking, and in particular provides a method for infrared monitoring and tracking of a small target. Background Art

[0002] In the infrared imaging scenarios in the border defense field, the target imaging is weak due to the wide-range monitoring requirements. At the same time, the rapid movement of weak targets will cause the target to disappear, deform or smear, and due to background interference and other conditions, the monitoring and tracking of weak targets at the front end becomes difficult. In recent years, the two mainstream algorithms of target tracking technology, target tracking algorithms based on correlation filters, particle filters and twin networks, have been able to achieve good results for tracking large and medium targets. However, the real-time weak target tracking and detection capabilities of front-end infrared monitoring are still limited. Weak targets have too few features and only appear as bright spots or dark spots on infrared images. The number of pixels they occupy is also small (targets of about 3*3), so detection and tracking often fail.

[0003] In recent years, the two main target tracking algorithms—target tracking algorithms based on correlation filters, particle filters, and twin networks—have achieved good results for tracking large and medium-sized targets. However, their capabilities for real-time tracking and detecting small, weak targets in front-end infrared surveillance remain limited. Because small targets have few features and appear only as bright or dark spots in infrared images, occupying a relatively small number of pixels (approximately 3x3 targets), detection and tracking often fail. Summary of the Invention

[0004] The present invention utilizes radar guidance, integrates deep learning target detection and traditional image target detection, and is based on KCF motion target tracking technology. It can continuously track and detect the target and the area around the target, guide target detection through the tracked target, and then guide tracking through the target detection target, cooperating and assisting each other to complete the tracking and detection of weak targets; and by optimizing the deep learning detection model and the traditional image detection algorithm, it can achieve a detection speed of 20ms / frame, and combined with KCF tracking, it can meet the effect of real-time tracking and detection of weak targets at the front end.

[0005] The technical task of the present invention is to provide a method for infrared monitoring and tracking of small targets in response to the above-mentioned problems.

[0006] A method for infrared monitoring and tracking of small targets uses radar guidance to acquire infrared video stream images of the target, performs deep learning target detection and traditional image target detection on the infrared video stream images, and then uses KCF motion target tracking technology to continuously track and detect the target and the area around the target. The method guides target detection through the tracked target, and then guides tracking through target detection, cooperating with each other to complete the tracking and detection of small targets.

[0007] Furthermore, the deep learning detection uses yolov7-tiny for detection, uses the entire image data of the actual scene for annotation training, generates a training model for full-image detection, and performs detection reasoning on major targets.

[0008] Furthermore, the traditional image detection adopts the image enhancement detection technology based on MPCM rapid implementation to enhance the entire image and extract the target.

[0009] Furthermore, the KCF tracking algorithm is used to update the target during the tracking detection process. The KCF tracking algorithm includes the following contents:

[0010] While tracking the current frame, two local deep learning detection models are used to perform target detection on local 128*128 and 256*256 areas centered on the target location in the previous frame. Since the chip used has two NPUs, target detection inference is performed on each NPU separately.

[0011] Furthermore, the deep learning detection model is a local target detection model obtained by training by randomly diffusing 128*128 and 256*256 local images with the target location in the actual scene as the initial point. This allows the detection of small targets, as small targets often fail to be tracked.

[0012] Furthermore, in order to speed up the inference time of the model, the method uses yolov7-tiny for detection, adopts pyramid connection with the feature layers of downsampling 4, 8, and 16, outputs the sampling header information of the feature layers of downsampling 4, 8, and 16 for weak target detection, and the inference time can reach 7ms / frame.

[0013] Typically, Yolov7-tiny detection uses feature layers of 8, 16, and 32. Compared to downsampling feature layers of 8, 16, and 32, downsampling by 4, 8, and 16 reduces the number of sampling layers, resulting in faster detection and enhanced detection of small objects. This is because the more you downsample, the more small objects are lost in the image.

[0014] Furthermore, the image enhancement detection technology adopts MPCM fast image enhancement detection technology, and simultaneously detects a local area of ​​256*256, which can detect smaller targets.

[0015] Furthermore, the application of the MPCM fast image enhancement detection technology also includes:

[0016] If the target is tracked, the actual target position of the current frame is updated by the detection box with the closest distance between the tracked target and all detected targets.

[0017] Furthermore, the application of the MPCM fast image enhancement detection technology also includes:

[0018] A confidence score is calculated by normalizing the distance with the target in the previous frame. The formula is as follows;

[0019]

[0020] Where dis represents distance, det i Represents the center point of the i-th MPCM detection target, i∈(0,n], preObj is the center point of the actual target in the previous frame;

[0021] If the target is not tracked, the target closest to the target position in the previous frame and with the highest confidence is selected as the actual target of this frame, and then the position of the tracked target in the current frame is updated, and the kcf tracking prediction of the next frame is performed.

[0022] Furthermore, the implementation of the method further includes:

[0023] If the target is not tracked or detected in the current frame, the number of frames lost is accumulated, and the position of the target in the previous frame is used as the position of the target in the current frame (small targets are easily deformed and temporarily lost);

[0024] If the cumulative number of lost frames reaches a threshold LossTh (set to 8 here), or the interactive selection ends tracking, the tracking is completed and the tracking video of the target is automatically saved, and the tracking ends.

[0025] Compared with the prior art, the infrared monitoring and tracking method for small targets of the present invention has the following outstanding beneficial effects:

[0026] By optimizing the deep learning detection model and the traditional image detection algorithm, the present invention can achieve a detection speed of 20ms / frame. Combined with KCF tracking, it can meet the effect of real-time tracking and detection of weak and small targets at the front end. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of the tracking system using the present invention;

[0028] Figure 2 It is the acceleration network structure of yolov7-tiny of the present invention. DETAILED DESCRIPTION

[0029] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0030] (1) Radar guides optoelectronic equipment toward the target;

[0031] (2) Pull the infrared video stream image and perform deep learning detection and traditional image detection on the first frame image;

[0032] (2.1) Deep learning detection uses yolov7-tiny for detection, uses the entire image data of the actual scene for annotation training, generates a training model for full-image detection, and performs detection reasoning on medium and large targets;

[0033] (2.2) Traditional image detection uses image enhancement detection technology based on MPCM to quickly implement and enhance the entire image to extract the target;

[0034] (2.3) Perform nms processing on the targets obtained in (2.1) and (2.2) to display all possible target boxes;

[0035] (2.4) Manually select the target to be tracked through the interface interaction, or click on the screen to select the target box closest to the click point for tracking;

[0036] (3) The KCF tracking algorithm is used to update the target during the tracking process;

[0037] (3.1) While tracking the current frame, two local deep learning detection models are used to detect the target in a local area of ​​128*128 and 256*256 centered on the target location in the previous frame. The chip has two NPUs, and each NPU performs target detection reasoning separately.

[0038] (3.2) The local target detection model in (2.1) is trained by randomly diffusing 128*128 and 256*256 local images from the target location in the actual scene as the initial point. This allows for the detection of small and weak targets, which often fail to be tracked. At the same time, in order to speed up the inference time of the model, yolov7-tiny is improved by only using the feature layers of downsampling 4, 8, and 16 for pyramid connection, and outputting 4, 8, and 16 downsampling head information for small and weak target detection. The network structure is shown in the figure below. Figure 1 As shown, the inference time can reach 7ms / frame;

[0039] (3.3) At the same time, MPCM fast image enhancement detection technology is applied to the local area of ​​256*256, which can also detect smaller targets. A confidence level is calculated by normalizing the distance with the target in the previous frame. The formula is as follows:

[0040]

[0041] Where dis represents distance, det iRepresents the center point of the i-th MPCM detection target, i∈(0,n], preObj is the center point of the actual target in the previous frame.

[0042] (4) All targets detected by the scheme (3) are processed by nms to obtain all target frames;

[0043] (5) If the target is tracked, the actual target position of the current frame is updated by the detection frame with the closest distance between the tracked target and all detected targets. If the target is not tracked, the target with the highest confidence and the closest distance to the target position in the previous frame is selected as the actual target of this frame, and then the position of the tracked target in the current frame is updated, and the kcf tracking prediction of the next frame is performed;

[0044] (6) The subsequent video frames are also processed in the same way as (2)(3)(4) to obtain a continuous tracking target;

[0045] (7) If the target is not tracked or detected in the current frame, the target loss frames are accumulated, and the position of the target in the previous frame is used as the position of the target in the current frame (weak targets are easily deformed and temporarily lost). If the accumulated number of lost frames reaches a threshold LossTh (set to 8 here), or when the interactive selection ends, the tracking is completed and the tracking video of the target is automatically saved, and the tracking ends.

[0046] The embodiments described above are only preferred specific implementations of the present invention. Common changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for infrared monitoring and tracking of small targets, characterized in that: The method uses radar guidance to obtain infrared video stream images of the target, performs deep learning target detection and traditional image target detection on the infrared video stream images, obtains all possible target frames after NMS processing, and then uses KCF moving target tracking technology to continuously track and detect the target and the area around the target. The tracked target is guided by target detection, and then the target detection is guided by tracking, which cooperates with each other to complete the tracking and detection of weak targets. The deep learning detection uses yolov7-tiny for detection, uses the entire image data of the actual scene for annotation training, generates a training model for full-image detection, and performs detection reasoning on major targets; The traditional image detection adopts the image enhancement detection technology based on MPCM to quickly realize the image enhancement processing and extract the target; The KCF tracking algorithm is used to update the target during the tracking detection process. The KCF tracking algorithm includes the following contents: While tracking the current frame, two local deep learning detection models are used to perform target detection on local 128*128 and 256*256 areas centered on the target location in the previous frame, and target detection inference is performed separately on each NPU.

2. A small target infrared monitoring and tracking method according to claim 1, characterized in that: The deep learning detection model is a local target detection model obtained by training by randomly diffusing 128*128 and 256*256 local images with the target location in the actual scene as the initial point.

3. A small target infrared monitoring and tracking method according to claim 2, characterized in that: When the method uses yolov7-tiny for detection, it uses the feature layers of downsampling 4, 8, and 16 for pyramid connection, and outputs the sampling head information of the feature layers of downsampling 4, 8, and 16 for weak target detection.

4. The infrared monitoring and tracking method for a small target according to claim 1, characterized in that: The image enhancement detection technology adopts MPCM fast image enhancement detection technology, and simultaneously detects a local area of ​​256*256, which can detect smaller targets.

5. A small target infrared monitoring and tracking method according to claim 4, characterized in that: The application of the MPCM fast image enhancement detection technology also includes: If the target is tracked, the actual target position of the current frame is updated by the detection box with the closest distance between the tracked target and all detected targets.

6. A small target infrared monitoring and tracking method according to claim 5, characterized in that: The application of the MPCM fast image enhancement detection technology also includes: A confidence score is calculated by normalizing the distance with the target in the previous frame. The formula is as follows; Where dis represents distance, det i Represents the center point of the i-th MPCM detection target, i∈(0,n], preOb j is the center point of the actual target in the previous frame; If the target is not tracked, the target closest to the target position in the previous frame and with the highest confidence is selected as the actual target of this frame, and then the position of the tracked target in the current frame is updated, and the kcf tracking prediction of the next frame is performed.

7. The method for infrared monitoring and tracking of a small target according to claim 1, characterized in that: The implementation of the method further includes: If the target is not tracked or detected in the current frame, the number of frames lost for the target is accumulated, and the position of the target in the previous frame is used as the position of the target in the current frame; If the cumulative number of lost frames reaches a threshold LossTh, which is set to 8, or the tracking end is interactively selected, the tracking is completed and the tracking video of the target is automatically saved, and the tracking ends.

Images