Infrared high-frame-frequency image multi-scale target detection method and system

Abstract

The invention discloses an infrared high-frame-frequency image multi-scale target detection method and system, and relates to the target tracking and detection technology, and the method comprises the steps: obtaining infrared image data; weak and small target detection and conventional target detection are respectively carried out on the acquired infrared image data, wherein the weak and small target detection and the conventional target detection are respectively suitable for target pixel specifications of different specifications; target areas detected by the weak and small target detection and the conventional target detection are combined; and performing multi-scale target de-duplication in the combined target area to obtain a detection result. The detection model is simple, the calculation amount is small, and high-frame-frequency real-time calculation can be achieved.

Description

Technical Field

[0001] The present application relates to the field of target tracking and detection, and in particular to a method and system for multi-scale target detection in infrared high frame rate images. Background Art

[0002] Wide-area infrared search and tracking systems play an important role in aerospace detection, environmental monitoring and other fields. To ensure target detection, such systems are characterized by high frame rate imaging, high-speed scanning, and large azimuth and elevation angle search and alert. Due to factors such as atmospheric attenuation, background noise interference, and the concealment of the target itself, how to accurately and in real time detect the target from a complex, high-noise background is a very challenging task. At the same time, there are many targets of inconsistent size, movement speed, and radiation characteristics in the field of view, which puts higher requirements on target detection.

[0003] Existing traditional target detection methods, such as threshold detection, edge detection, and region growing, are difficult to meet the accuracy and real-time performance of detection in multi-target and multi-scale scenarios. Image detection methods based on deep learning, such as SSD and YOLO, require a large number of data annotation samples and have high requirements for hardware computing resources. Summary of the invention

[0004] The embodiments of the present application aim to solve the problem that in high frame rate infrared application scenarios, there are many targets of inconsistent sizes, movement speeds, and radiation characteristics in multiple fields of view, and the real-time and accuracy of target detection are limited. A method and system for multi-scale target detection in infrared high frame rate images are provided to solve the above problems.

[0005] The present application provides a method for detecting multi-scale targets in infrared high frame rate images, including:

[0006] Acquire infrared image data;

[0007] Performing small target detection and conventional target detection on the acquired infrared image data, respectively, wherein the small target detection and the conventional target detection are respectively applicable to target pixel specifications of different specifications;

[0008] Merging the target areas detected by the small target detection and the conventional target detection;

[0009] Multi-scale object deduplication is performed in the merged object area to obtain the detection result.

[0010] Optionally, performing small target detection on the acquired infrared image data includes:

[0011] Guided filtering estimates the background of the acquired infrared image data;

[0012] subtracting an estimated background from the infrared image data;

[0013] The block adaptive threshold is used to detect the infrared image data with background subtracted to obtain the weak target detection result.

[0014] Optionally, performing conventional target detection on the acquired infrared image data includes:

[0015] Downsampling the acquired infrared image data multiple times to obtain infrared images of multiple scales;

[0016] For infrared images with a scale of 1 among infrared images of multiple scales, the infrared image is divided into multiple regions by block threshold detection, and the detection threshold of the histogram threshold detection is calculated independently for each region, and the corresponding detection is performed using the detection threshold;

[0017] For infrared images with scales of 2-N among infrared images of multiple scales, the DoG operator is used to extract target features, and the detection threshold of histogram threshold detection is used to perform target detection;

[0018] The image threshold detection results of the infrared image with a scale of 2-N are upsampled to restore the original image size and fused with the detection results with a scale of 1 to obtain the detection results of the conventional target detection model.

[0019] Optionally, merging the target areas detected by the small target detection and the conventional target detection includes:

[0020] Perform connected domain detection on the detection results of weak target detection and conventional target detection to merge the target area and obtain relevant feature information of the target;

[0021] The targets obtained by merging weak and small target detection are screened according to the target area and the number of points to eliminate targets larger than the corresponding preset threshold;

[0022] The targets obtained by merging conventional target detection are screened according to the target area and the number of points to eliminate targets smaller than the corresponding preset threshold;

[0023] Targets selected by weak target detection and conventional target detection are de-overlapped and merged according to their overlap rate.

[0024] Optionally, the target areas are merged, and the obtained relevant feature information includes the centroid, area, and number of included points of the target.

[0025] Optionally, after the targets are screened by weak target detection and conventional target detection, de-overlapping and merging are performed according to the target overlap rate, including:

[0026] For targets with an overlap rate greater than the threshold T, the point element area ratio of the small target detection result and the regular detection result are calculated respectively, and the sizes of the two results are compared, and the targets with a small point element area ratio are deleted.

[0027] An embodiment of the present application further proposes an infrared high frame rate image multi-scale target detection system, comprising a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the aforementioned method are implemented.

[0028] The embodiment of the present application divides the image detection task into small target detection and target detection with morphological information. The two detection paths are carried out in parallel, and the detection results are finally merged to form the final infrared image target detection method. The detection model of the present application is simple, requires little calculation, and can achieve real-time calculation with high frame rate.

[0029] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0031] Figure 1 This is a basic flow diagram of the infrared high frame rate image multi-scale target detection method according to an embodiment of the present application;

[0032] Figure 2 The following is a schematic diagram of the detection architecture of the infrared high frame rate image multi-scale target detection method according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0034] The present application embodiment provides a method for multi-scale target detection in infrared high frame rate images, such as Figure 1 As shown, the following steps are included:

[0035] In step S101 , infrared image data is acquired.

[0036] In step S102, the acquired infrared image data is subjected to small target detection and conventional target detection, respectively, wherein the small target detection and the conventional target detection are respectively applicable to target pixel specifications of different specifications. In some specific examples, the steps can be completed by using corresponding modules, for example, the small target detection module is used to detect targets with target pixel specifications of 7×7 and below, and the conventional target detection module is used to detect targets with target pixel specifications of 7×7 to 100×100.

[0037] In step S103, the target areas detected by the small target detection and the conventional target detection are merged.

[0038] In step S104, multi-scale object deduplication is performed in the merged object area to obtain a detection result.

[0039] In some embodiments, performing small target detection on the acquired infrared image data includes:

[0040] The guided filter estimates the background of the acquired infrared image data. In the specific example, the small target detection adopts the infrared small target detection technology based on the guided filter and the block adaptive threshold.

[0041] The estimated background in the infrared image data is subtracted while the signal of the target is enhanced.

[0042] The block adaptive threshold is used to detect the infrared image data with background subtracted to obtain the weak target detection result.

[0043] Conventional target detection is used for target detection in the range of 7×7 to 100×100 pixels, specifically using the large local contrast of salient targets. Existing target detection algorithms based on local contrast require the design of large templates for large targets. However, the larger the template, the greater the amount of data calculation, making it difficult to meet real-time requirements. In some embodiments, Figure 2 As shown, conventional target detection on the acquired infrared image data includes:

[0044] The acquired infrared image data is downsampled multiple times to obtain infrared images of multiple scales. For example, each frame of input data is downsampled N times to obtain infrared images of N scales. In the specific implementation, N=3, and the specific N value can be adjusted according to the actual task.

[0045] For an infrared image with a scale of 1 among the infrared images of multiple scales, the infrared image is divided into multiple regions, for example, M regions, by block threshold detection, and a detection threshold of a histogram threshold detection is independently calculated for each region, and corresponding detection is performed using the detection threshold;

[0046] For infrared images with scales of 2-N among infrared images of multiple scales, the DoG operator is used to extract target features, and the detection threshold of histogram threshold detection is used to perform target detection;

[0047] The image threshold detection results of the infrared image with a scale of 2-N are upsampled to restore the original image size and fused with the detection results with a scale of 1 to obtain the detection results of the conventional target detection model.

[0048] For example, in the case of N=3, the infrared image is downsampled in two levels to obtain infrared images of three scales (including the infrared image of N=1). The DoG operator is used to extract the target features of the sampled three scale images, and the upsampling strategy is used to fuse the feature results of the three scales. Finally, the threshold is calculated using the statistical histogram to detect the feature results and the original infrared image to obtain the detection results.

[0049] In some embodiments, merging the target areas detected by the small target detection and the conventional target detection includes:

[0050] Connected domain detection is performed on the detection results of weak target detection and conventional target detection to merge the target areas and obtain relevant feature information of the target. In some embodiments, the target areas are merged, and the relevant feature information obtained includes the center of mass, area, and number of points included in the target.

[0051] The targets obtained by merging weak and small target detection are screened according to the target area and the number of points to eliminate targets smaller than the corresponding preset thresholds, for example, targets larger than S1 and with a number of points larger than P1 are eliminated.

[0052] The targets obtained by merging conventional target detection are screened according to the target area and the number of points to eliminate targets smaller than the corresponding preset thresholds, for example, targets smaller than S2 and with a point number smaller than P2 are eliminated.

[0053] Targets selected by weak target detection and conventional target detection are de-overlapped and merged according to their overlap rate.

[0054] In some embodiments, de-overlapping and merging the targets selected by the weak target detection and the conventional target detection according to the target overlap rate includes:

[0055] For targets with an overlap rate greater than the threshold T, the point element area ratio (number of points / area) of the small target detection result and the regular detection result are calculated respectively, and the sizes of the two results are compared, and the targets with a small point element area ratio are deleted.

[0056] To ensure the accuracy of target detection, in the specific example, targets of different scales are first screened and judged to eliminate interference information; secondly, by calculating the overlapping information of targets of different scales, the overlapping targets are screened and merged to obtain the final target detection result.

[0057] The embodiment of the present application divides the image detection task into small target detection and target detection with morphological information. The two detection paths are carried out in parallel, and the detection results are finally merged to form the final infrared image target detection method. The detection model of the present application is simple, with low computational complexity, and can realize real-time calculation at a high frame rate. In addition, the method of the present application supports wide-area multi-scale and multi-target image detection.

[0058] An embodiment of the present application further proposes an infrared high frame rate image multi-scale target detection system, comprising a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the aforementioned method are implemented.

[0059] It should be noted that in the various embodiments of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0060] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0061] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0062] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A multi-scale target detection method for infrared high frame rate images, characterized in that: include: Acquire infrared image data; Performing small target detection and conventional target detection on the acquired infrared image data, respectively, wherein the small target detection and the conventional target detection are respectively applicable to target pixel specifications of different specifications; Merging the target areas detected by the small target detection and the conventional target detection; Multi-scale object deduplication is performed in the merged object area to obtain the detection result.

2. The infrared high frame rate image multi-scale target detection method according to claim 1, characterized in that: The detection of small targets on the acquired infrared image data includes: Guided filtering estimates the background of the acquired infrared image data; subtracting an estimated background from the infrared image data; The block adaptive threshold is used to detect the infrared image data with background subtracted to obtain the weak target detection result.

3. The infrared high frame rate image multi-scale target detection method according to claim 2, characterized in that: Conventional target detection on the acquired infrared image data includes: Downsampling the acquired infrared image data multiple times to obtain infrared images of multiple scales; For infrared images with a scale of 1 among infrared images of multiple scales, the infrared image is divided into multiple regions by block threshold detection, and the detection threshold of the histogram threshold detection is calculated independently for each region, and the corresponding detection is performed using the detection threshold; For infrared images with scales of 2-N among infrared images of multiple scales, the DoG operator is used to extract target features, and the detection threshold of histogram threshold detection is used to perform target detection; The image threshold detection results of the infrared image with a scale of 2-N are upsampled to restore the original image size and fused with the detection results with a scale of 1 to obtain the detection results of the conventional target detection model.

4. The infrared high frame rate image multi-scale target detection method according to claim 3, characterized in that: Merging the target areas detected by the small target detection and the conventional target detection includes: Perform connected domain detection on the detection results of weak target detection and conventional target detection to merge the target area and obtain relevant feature information of the target; The targets obtained by merging weak and small target detection are screened according to the target area and the number of points to eliminate targets larger than the corresponding preset threshold; The targets obtained by merging conventional target detection are screened according to the target area and the number of points to eliminate targets smaller than the corresponding preset threshold; Targets selected by weak target detection and conventional target detection are de-overlapped and merged according to their overlap rate.

5. The infrared high frame rate image multi-scale target detection method according to claim 4, characterized in that: The target area is merged, and the relevant feature information obtained includes the target's centroid, area, and number of points included.

6. The infrared high frame rate image multi-scale target detection method according to claim 4, characterized in that: According to the targets selected by weak target detection and conventional target detection, de-overlapping and merging are performed according to the target overlap rate, including: For targets with an overlap rate greater than the threshold T, the point element area ratio of the small target detection result and the regular detection result are calculated respectively, and the sizes of the two results are compared, and the targets with a small point element area ratio are deleted.

7. An infrared high frame rate image multi-scale target detection system, characterized in that The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images