Infrared target detection method and device in complex scene

Abstract

This invention provides an infrared target detection method and apparatus for complex scenes. The method includes: acquiring a real-time detected infrared image; the infrared image including a moving target; performing target detection on the infrared image using a classical infrared target detection algorithm and a deep learning algorithm respectively, obtaining a first target detection result and a second target detection result; fusing the first target detection result and the second target detection result to output a target location based on the infrared image. This solution combines the advantages of both algorithms, enabling accurate detection of both small infrared targets and targets with distinct shapes and textures, thereby reducing the false alarm rate of target detection.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and apparatus for infrared target detection in complex scenes. Background Technology

[0002] Infrared imaging technology is widely used in military and civilian fields. However, how to accurately detect and track small infrared targets in complex scenarios such as sky background, sea background, and sea-sky background is a bottleneck problem that urgently needs to be solved.

[0003] Currently, target detection, recognition, and tracking in infrared images are generally achieved using traditional filtering algorithms or deep learning convolutional neural networks. However, traditional filtering algorithms have weak generalization performance, while deep learning methods are less accurate when dealing with targets with indistinct texture features, both of which lead to high false alarm rates. Summary of the Invention

[0004] This invention provides an infrared target detection method and apparatus for complex scenarios, which can reduce the false alarm rate of target detection.

[0005] In a first aspect, embodiments of the present invention provide an infrared target detection method in complex scenes, comprising:

[0006] Acquire real-time detected infrared images; the infrared images include moving targets;

[0007] The infrared image is used to perform target detection using both classical infrared target detection algorithms and deep learning algorithms, and the first target detection result and the second target detection result are obtained one-to-one.

[0008] The first target detection result and the second target detection result are fused to locate the target in the infrared image output.

[0009] Preferably, the classical infrared target detection algorithm includes at least one of: a grayscale gradient algorithm, an optical flow field algorithm, and a Gaussian filtering algorithm; and / or,

[0010] The deep learning algorithm is a kernel correlation filtering algorithm.

[0011] Preferably, the first target detection result and the second target detection result are fused to locate the target in the infrared image output, including:

[0012] The number of detected targets is determined based on the first target detection result. If the number is not less than a first set number, the target with the highest confidence among the first set number of detected targets is selected as the first detection target. Otherwise, the detected target is selected as the first detection target.

[0013] Based on the second target detection result, determine whether a second detection target with a size exceeding the set value has been detected. If so, determine the required positioning target based on the overlap between the second detection target and the first detection target, and output the positioning target. If not, output the first detection target as the positioning target.

[0014] Preferably, determining the required output positioning target based on the overlap between the second detection target and the first detection target includes:

[0015] For each first detection target, the following steps are performed: determine whether the first detection target is not covered by the second detection target and whether the first detection target does not overlap with the second detection target. If so, the first detection target is identified as a positioning target; otherwise, the first detection target is identified as a non-positioning target.

[0016] The second detection target is identified as the positioning target.

[0017] Preferably, if the number of detected targets with sizes exceeding a set value is determined based on the second target detection result, then the target with the largest size is determined as the second detected target.

[0018] Preferably, the first set quantity is 2.

[0019] Preferably, it further includes:

[0020] Calculate the angular deviation of each positioning target and assign the angular deviation to the corresponding target miss distance for output.

[0021] Secondly, embodiments of the present invention also provide an infrared target detection device for complex scenes, comprising:

[0022] An acquisition unit is used to acquire real-time detected infrared images; the infrared images include moving targets;

[0023] The detection unit is used to perform target detection on the infrared image using a classical infrared target detection algorithm and a deep learning algorithm, respectively, and obtain the first target detection result and the second target detection result in a one-to-one correspondence.

[0024] The fusion unit is used to fuse the first target detection result and the second target detection result to locate the target in the infrared image output.

[0025] Thirdly, embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the method described in any embodiment of this specification.

[0026] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the methods described in any embodiment of this specification.

[0027] This invention provides an infrared target detection method and apparatus for complex scenes. Classical infrared target detection algorithms offer accuracy and real-time performance for detecting small infrared targets, while deep learning algorithms offer high accuracy for detecting targets with distinct shapes and textures. By utilizing both classical and deep learning algorithms to perform target detection on infrared images separately, and then fusing the detection results, the localized target in the infrared image can be detected. Therefore, this solution combines the advantages of both algorithms, enabling accurate detection of both small infrared targets and targets with distinct shapes and textures, thereby reducing the false alarm rate. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart of an infrared target detection method in a complex scene provided by an embodiment of the present invention;

[0030] Figure 2 This is a hardware architecture diagram of an electronic device provided in an embodiment of the present invention;

[0031] Figure 3 This is a structural diagram of an infrared target detection device for complex scenarios provided by an embodiment of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0033] Please refer to Figure 1 This invention provides a method for infrared target detection in complex scenes, the method comprising:

[0034] Step 100: Acquire the real-time detected infrared image; the target in the infrared image has at least one of the following characteristics: high-speed motion, scale change, and affine change;

[0035] Step 102: Use the classical infrared target detection algorithm and the deep learning algorithm to perform target detection on the infrared image respectively, and obtain the first target detection result and the second target detection result one by one;

[0036] Step 104: Perform target fusion on the first target detection result and the second target detection result to locate the target based on the infrared image output.

[0037] In this embodiment of the invention, classical infrared target detection algorithms are characterized by accuracy and real-time performance in detecting small infrared targets, while deep learning algorithms are characterized by high accuracy in detecting targets with distinct shapes and textures. By utilizing both classical and deep learning algorithms to perform target detection on infrared images separately, and then fusing the detection results from both, the localized targets in the infrared images are detected. Therefore, this scheme combines the advantages of both algorithms, enabling accurate detection of both small infrared targets and targets with distinct shapes and textures, thereby reducing the false alarm rate of target detection.

[0038] The following description Figure 1 The execution method for each step is shown.

[0039] First, for step 100, a real-time detected infrared image is acquired; the target in the infrared image has at least one of the following characteristics: high-speed motion, scale change, and affine change.

[0040] In this embodiment of the invention, an infrared imaging system can be used to acquire video data of moving targets in real time to obtain real-time detected infrared images. These moving targets exhibit characteristics such as high-speed motion, scale changes, and affine transformations. Detecting moving targets with only one algorithm for these characteristics results in a high false alarm rate. This embodiment considers using two different algorithms for target detection, combining the advantages of both algorithms to ensure both speed and accuracy in the target detection process.

[0041] Then, for step 102, target detection is performed on the infrared image using the classic infrared target detection algorithm and the deep learning algorithm respectively, and the first target detection result and the second target detection result are obtained one-to-one.

[0042] Classical infrared target detection algorithms offer strong real-time performance and have advantages in handling small infrared targets with indistinct texture features. Deep learning algorithms, on the other hand, can more easily identify real infrared targets with relatively clear shapes and textures, provided there is a rich training sample pool. Therefore, these two algorithms can be used separately for target detection in infrared images.

[0043] In this embodiment of the invention, considering that target detection generally has real-time requirements, two physical boards can be used to execute their respective detection algorithms to ensure the processing speed of the algorithms. These two physical boards include a classic board for executing the classic infrared target detection algorithm and a smart board for executing the deep learning algorithm.

[0044] Furthermore, after acquiring the video data, to further improve the target detection speed of each infrared image in the video data, in this embodiment of the invention, the entire system can be run on the TMS320C6678 platform. The infrared imaging system transmits the acquired video data to the classic board and the smart board in two paths at a rate of 10Hz / frame through the SRIO hardware interface on this platform. This allows the DSP chips on the classic board and the smart board to quickly receive the infrared images for target detection. The two DSPs perform calculations in parallel, thereby further improving the real-time performance of the detection.

[0045] To further improve the accuracy of target detection, this infrared imaging system can acquire infrared images at at least two resolutions, and then perform target detection on each of these images separately. For example, one resolution is 640*512, and the other is 320*256.

[0046] In one embodiment of the present invention, a complex scene may include two typical scenarios: a moving target against a complex motion background and a clearly visible target in a complex scene.

[0047] In complex motion scenarios, where the moving target is subject to cloud or ground interference, and the target moves relative to the background, the algorithm needs to handle the following situations. Classic infrared target detection algorithms used in such scenarios may include at least one of the following: grayscale gradient algorithm, optical flow field algorithm, and Gaussian filtering algorithm.

[0048] The scenario with obvious targets in complex environments is a simplified form of the previous scenario, primarily focusing on processing the infrared features of the aircraft. The deep learning algorithm used in this scenario can be the kernel correlation filter (KCF) algorithm.

[0049] After performing target detection on an infrared image using a classic infrared target detection algorithm, a first target detection result can be obtained. This first target detection result can include the coordinate position and size information of the target detection bounding box. The presence of multiple target detection bounding boxes indicates that multiple targets have been detected.

[0050] After performing target detection on an infrared image using a deep learning algorithm, a second target detection result can be obtained. This second target detection result can include the coordinate position and size information of the target detection box; if there are multiple target detection boxes, it indicates that multiple targets have been detected.

[0051] Finally, in step 104, the first target detection result and the second target detection result are fused to locate the target in the infrared image output.

[0052] When both detection algorithms can detect multiple targets, in order to improve detection speed and reduce the comparison process of multiple targets, the target fusion process for the detection results of the two targets specifically includes:

[0053] S1: Determine the number of detected targets based on the first target detection result. If the number is not less than a first set number, select the target with the highest confidence from the multiple detected targets as the first detection target. Otherwise, use the detected target as the first detection target.

[0054] Preferably, the first set quantity is 2.

[0055] For example, if the first target detection result indicates that 5 targets have been detected, then the 5 targets are sorted in descending order of confidence, and the two targets with the highest confidence are selected as the first detection targets. If the first target detection result indicates that 1 or 2 targets have been detected, then those 1 or 2 targets are directly selected as the first detection targets.

[0056] The confidence score is calculated using factors such as the number of pixels in the target, pixel grayscale values, and the degree of matching with the previous target. The higher the score, the higher the confidence score. By selecting the target with the highest confidence score as the first detection target, the authenticity of the detected target can be guaranteed to be higher.

[0057] S2: Determine whether a second detection target with a size exceeding the set value is detected based on the second target detection result. If yes, determine the required positioning target based on the overlap between the second detection target and the first detection target, and output the positioning target. If no, output the first detection target as the positioning target.

[0058] Since the intelligent board performs better in detecting targets with large areas and obvious texture features, a set value can be set, for example, 40. When the length or width of the target detected in the first target detection result exceeds this set value, the accuracy of identifying the target as a real target is higher. If no target exceeding the set value is detected, it is determined that the intelligent board has not detected a target, and the first detected target detected by the classic board is directly output as the localization target.

[0059] In one embodiment of the present invention, considering that the target area detected by the deep learning algorithm is larger, the second detection target is more likely to cover the first detection target when matching the overlap between the second and first detection targets. Therefore, in order to reduce the number of matching overlaps, if it is determined from the second target detection result that there are multiple targets with a size exceeding a set value, then the target with the largest size is determined as the second detection target. That is, the number of second detection targets is one, and it is a target with a size exceeding the set value.

[0060] After determining the first detection target and the second detection target, determining the required output positioning target based on the overlap between the second detection target and the first detection target may include:

[0061] For each first detection target, the following steps are performed: determine whether the first detection target is not covered by the second detection target and whether the first detection target does not overlap with the second detection target. If so, the first detection target is identified as a positioning target; otherwise, the first detection target is identified as a non-positioning target.

[0062] The second detection target is identified as the positioning target.

[0063] Because the area of ​​the second detected target is larger than that of the first detected target, the first detected target may be covered by the second detected target, or the outlines of the first and second detected targets may overlap. When this occurs, it indicates that both target detection algorithms have detected the same target. For the first detected target that is covered by and / or overlaps with the second detected target, since the target range detected by the intelligent board is larger and more realistic, the second detected target is output as the localization target instead of the first detected target, thus ensuring more accurate subsequent target tracking.

[0064] Regardless of whether the second detection target covers the first detection target or overlaps with the first detection target, the second detection target is identified as the positioning target.

[0065] When determining whether the first detection target is covered by the second detection target or whether they overlap, it can be determined by calculating whether the center position of the first detection target is located within the second detection target.

[0066] In this embodiment of the invention, the classic board outputs at most two first detection targets, and the smart board outputs at most one second detection target. When matching the overlap between the second detection target and the first detection target, at most two matching operations are performed to output the positioning target, thereby ensuring that the target detection meets the real-time requirements.

[0067] Furthermore, in order for the host computer to know which target detection algorithm was used to detect each positioning target, the output positioning targets can be identified. For example, the two first detection targets output by the classic board are identified by identifier 1 and identifier 2, respectively, and the second detection target output by the smart board is identified by identifier 3. After the positioning target is determined, the corresponding target detection algorithm can be determined by the identifier bits of the positioning target.

[0068] Furthermore, after detecting a target, if there are multiple targets, one target can be selected for tracking. The host computer selects the target to be tracked by issuing a target selection command. After the host computer issues the target selection command, in order to quickly track the target, one embodiment of the present invention may further include: calculating the angular offset of each target and assigning the angular offset to the miss distance of the corresponding target for output.

[0069] The calculation method for angular deflection is as follows:

[0070]

[0071]

[0072]

[0073]

[0074] Among them, Y h Y is the horizontal angular deflection. v For the vertical angle deflection, fov h For the horizontal field of view, fov v is the vertical field of view, w is the width of the target size, h is the height of the target size, and f is the 44th and 45th bytes of the image overlay information.

[0075] like Figure 2 , Figure 3As shown, this embodiment of the invention provides an infrared target detection device for complex scenarios. The device embodiment can be implemented through software, hardware, or a combination of both. From a hardware perspective, as... Figure 2 The diagram shown is a hardware architecture diagram of an electronic device for an infrared target detection device in a complex scene, provided by an embodiment of the present invention. Besides... Figure 2 In addition to the processor, memory, network interface, and non-volatile memory shown, the electronic device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing packets. Taking software implementation as an example, such as... Figure 3 As shown, a device in a logical sense is formed by the CPU of its host electronic device reading the corresponding computer program from non-volatile memory into memory and running it. This embodiment provides an infrared target detection device for complex scenes, comprising:

[0076] The acquisition unit 301 is used to acquire real-time detected infrared images; the infrared images include moving targets;

[0077] The detection unit 302 is used to perform target detection on the infrared image using a classical infrared target detection algorithm and a deep learning algorithm respectively, and obtain the first target detection result and the second target detection result in a one-to-one correspondence.

[0078] The fusion unit 303 is used to fuse the first target detection result and the second target detection result to locate the target in the infrared image output.

[0079] In one embodiment of the present invention, the classical infrared target detection algorithm includes at least one of: a grayscale gradient algorithm, an optical flow field algorithm, and a Gaussian filtering algorithm; and / or,

[0080] The deep learning algorithm is a kernel correlation filtering algorithm.

[0081] In one embodiment of the present invention, the fusion unit is specifically configured to: determine the number of detected targets based on the first target detection result; if the number is not less than a first preset number, select the target with the highest confidence from the multiple detected targets as the first detection target; otherwise, use the detected target as the first detection target; determine whether a second detection target with a size exceeding a preset value is detected based on the second target detection result; if so, determine the required output positioning target based on the overlap between the second detection target and the first detection target, and output the positioning target; otherwise, output the first detection target as the positioning target.

[0082] In one embodiment of the present invention, when the fusion unit determines the required output positioning target based on the overlap between the second detection target and the first detection target, it specifically includes:

[0083] For each first detection target, the following steps are performed: determine whether the first detection target is not covered by the second detection target and whether the first detection target does not overlap with the second detection target. If so, the first detection target is identified as a positioning target; otherwise, the first detection target is identified as a non-positioning target.

[0084] The second detection target is identified as the positioning target.

[0085] In one embodiment of the present invention, if it is determined from the second target detection result that there are multiple targets whose size exceeds a set value, then the target with the largest size is determined as the second detection target.

[0086] In one embodiment of the present invention, the first set quantity is 2.

[0087] In one embodiment of the present invention, the fusion unit is further configured to calculate the angular deviation of each positioning target and assign the angular deviation to the miss distance of the corresponding positioning target for output.

[0088] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on an infrared target detection device in a complex scene. In other embodiments of the present invention, an infrared target detection device in a complex scene may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0089] The information interaction and execution process between the modules in the above-mentioned device are based on the same concept as the method embodiment of the present invention, and the specific details can be found in the description in the method embodiment of the present invention, and will not be repeated here.

[0090] This invention also provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements an infrared target detection method in a complex scene according to any embodiment of this invention.

[0091] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program causes the processor to perform an infrared target detection method in a complex scene according to any embodiment of this invention.

[0092] Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer (or CPU or MPU) of the system or apparatus may read and execute the program code stored in the storage medium.

[0093] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0094] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0095] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0096] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion module connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion module execute some and all of the actual operations, thereby realizing the function of any of the above embodiments.

[0097] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0098] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for infrared target detection in complex scenes, characterized in that, include: Acquire real-time detected infrared images; the infrared images include moving targets; The infrared image is used to perform target detection using both classical infrared target detection algorithms and deep learning algorithms, and the first target detection result and the second target detection result are obtained one-to-one. The first target detection result and the second target detection result are fused to locate the target for the infrared image output, including: determining the number of detected targets based on the first target detection result; if the number is not less than a first set number, then selecting the first set number of targets with the highest confidence from the multiple detected targets as the first detection target; otherwise, the detected target is used as the first detection target. Based on the second target detection result, determine whether a second detection target with a size exceeding the set value has been detected. If so, determine the required positioning target based on the overlap between the second detection target and the first detection target, and output the positioning target. If not, output the first detection target as the positioning target.

2. The method according to claim 1, characterized in that, The classical infrared target detection algorithm includes at least one of the following: grayscale gradient algorithm, optical flow field algorithm, and Gaussian filtering algorithm; and / or, The deep learning algorithm is a kernel correlation filtering algorithm.

3. The method according to claim 1, characterized in that, The step of determining the required output positioning target based on the overlap between the second detection target and the first detection target includes: For each first detection target, the following steps are performed: determine whether the first detection target is not covered by the second detection target and whether the first detection target does not overlap with the second detection target. If so, the first detection target is identified as a positioning target; otherwise, the first detection target is identified as a non-positioning target. The second detection target is identified as the positioning target.

4. The method according to claim 3, characterized in that, If, based on the second target detection result, it is determined that there are multiple targets whose size exceeds the set value, then the target with the largest size is determined as the second detection target.

5. The method according to claim 1, characterized in that, The first set quantity is 2.

6. The method according to any one of claims 1-5, characterized in that, Also includes: Calculate the angular deviation of each positioning target and assign the angular deviation to the corresponding target miss distance for output.

7. An infrared target detection device for complex scenarios, characterized in that, For performing the method according to any one of claims 1-6, comprising: An acquisition unit is used to acquire real-time detected infrared images; the infrared images include moving targets; The detection unit is used to perform target detection on the infrared image using a classical infrared target detection algorithm and a deep learning algorithm respectively, and obtain the first target detection result and the second target detection result in a one-to-one correspondence. The fusion unit is used to fuse the first target detection result and the second target detection result to locate the target in the infrared image output.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program, implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-6.

Images