Double-frame difference detection algorithm of self-adaptive threshold value

Abstract

The invention introduces a threshold-adaptive dual-frame difference detection algorithm. The algorithm comprises the following specific steps: S1, carrying out alignment operation on a to-be-processed dual-frame image; s2, carrying out differential operation on the aligned double-frame image; s3, selecting a maximum value of the difference image, and preliminarily determining a target position; s4, finally determining the number N of the targets; s5, performing target detection operation by using the threshold value to obtain an optimal detection threshold value; and S6, obtaining a final detection target position by using the updated optimal detection threshold value Tmin. And S7, repeating the operations from S1 to S6 on all the to-be-processed images according to the time sequence until the task is finished. The method solves the problem that a double-frame difference detection fixed threshold algorithm does not have real-time performance, robustness and accuracy, is not only suitable for the target detection field such as infrared detection, but also can be migrated to other complex target detection fields needing high precision, high timeliness and the like, and has wide application prospects.

Description

Technical Field

[0001] The present invention relates to the technical field of moving target detection, and in particular to a dual-frame difference detection algorithm with an adaptive threshold. Background Art

[0002] In military applications, the detection of small targets at long distances, especially in complex battlefield environments, is a crucial task. Such targets usually have low contrast and weak signal characteristics and may be in a dynamically changing background. Double-frame difference is a simple and effective method for detecting small moving targets. It identifies the moving area in the video by calculating the difference between two consecutive frames of images. This method has low computational complexity and is relatively simple, and is particularly suitable for detecting targets that move slowly. Its application scenarios are relatively wide, including but not limited to: (1) Infrared detection: used to detect moving targets at long distances and in complex backgrounds; (2) Intelligent transportation systems: used for vehicle detection, counting, and traffic flow analysis, etc., to help achieve automated traffic management; (3) Security monitoring systems: Double-frame difference can be used to quickly detect intruders or abnormal activities.

[0003] However, the threshold setting in the inter-frame difference algorithm is a very critical step; the choice of threshold directly affects the accuracy and robustness of moving target detection; in practical applications, the selection of threshold needs to weigh multiple factors, including noise suppression, adaptability to illumination changes, and sensitivity to moving targets; the contradictions in threshold selection include but are not limited to: (1) noise suppression and target information retention: if the threshold is set too low, the algorithm may misjudge the noise in the image as a moving target, resulting in a large number of false detections; conversely, if the threshold is set too high, some real moving areas may be filtered out, especially when the grayscale of the target does not change much, which may lead to missed detections; (2) adaptability to illumination changes: changes in illumination conditions in the scene can affect changes in pixel values; a fixed threshold may not be able to adapt to such changes, resulting in erroneous detection results; for example, when the illumination suddenly becomes brighter or darker, using a fixed threshold may cause a large amount of background to be mistaken for a moving target; (3) environmental complexity: in complex environments (such as those with multiple textures and colors), it is more difficult to determine the appropriate threshold; different background features may produce different levels of noise, thus affecting the choice of threshold.

[0004] Currently, there is no effective adaptive threshold design algorithm. When using the dual-frame detection algorithm, users usually make qualitative estimates based on experience, then set a fixed threshold, and repeatedly try and error until the optimal threshold is found, which affects the real-time, robustness and accuracy of the application. Once the environment changes, the threshold needs to be readjusted, resulting in blindness and inaccuracy in practical use. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a dual-frame difference detection algorithm with an adaptive threshold, which solves the problems that the dual-frame difference detection fixed threshold algorithm lacks real-time performance, robustness and accuracy.

[0006] The technical solution adopted by the present invention is:

[0007] An adaptive threshold double-frame difference detection algorithm, the specific steps are:

[0008] S1: Align the two-frame images to be processed, and use image alignment to eliminate position deviation caused by jitter;

[0009] S2: Perform a differential operation on the aligned double-frame images to obtain a differential image; the specific operation is: assuming that the nth frame and the n-1th frame image in the video sequence are f n and f n-1 , the grayscale value of the corresponding pixel in the two frames is recorded as f n (x, y) and f n-1 (x, y), then the grayscale values ​​of the corresponding pixels of the two frames are subtracted and their absolute values ​​are taken to obtain the difference image d n It can be expressed as:

[0010] d n (x, y) = |f n (x, y)-f n-1 (x, y)|;

[0011] S3: Select the maximum value of the differential image, that is, the threshold T; n (x, y) is compared with the threshold T pixel by pixel. Pixels greater than or equal to the threshold T are identified as targets, otherwise they are identified as backgrounds, which can be expressed as:

[0012]

[0013] Thus, the target location is preliminarily determined;

[0014] S4: Perform connectivity analysis based on the target location determined in S3. If the regions are connected, they are considered to be the same target. Otherwise, they are considered to be multiple targets. Finally, the number of targets N is determined.

[0015] S5: Use the threshold T to perform target detection operations to determine whether the number of detected targets meets the maximum number limit detected by the optical device, that is, the maximum tolerance value M. If N is less than or equal to M, set T = T-1, and repeat the operations of steps S3 and S4 until N is greater than M. At this time, set T min =T+1,T min That is, it is the optimal detection threshold under the constraint of the maximum detection target, and it is also the most sensitive threshold;

[0016] S6: Using the updated optimal detection threshold T min , execute S3 and S4 to obtain the final detected target position, and mark the detected target on the image with a red rectangular frame for subsequent actual visualization applications;

[0017] S7: In practice, since frame images are usually continuous, the operations from S1 to S6 are repeated for all images to be processed in chronological order until the task is completed.

[0018] Due to the adoption of the above-mentioned technical solution, the present invention has the following advantages:

[0019] The present invention solves the problems of real-time, robustness and accuracy that the fixed threshold algorithm of dual-frame differential detection does not have, and obtains the optimal detection threshold under the maximum number restriction condition, that is, the most sensitive detection threshold, which is not only dynamic and adaptive, but also greatly improves the possibility of detecting the real target; the present invention is not only suitable for target detection fields such as infrared detection, but can also be migrated to other complex target detection fields requiring high precision and high timeliness, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the process of the present invention.

[0021] Figure 2 It is a scene situation table of a dataset for detecting and tracking small aircraft targets in infrared images under ground / air background in an embodiment of the present invention.

[0022] Figure 3 These are the first three consecutive frames of images of the drone target in the data set data21 in an embodiment of the present invention.

[0023] Figure 4 yes Figure 3 The first two frames of images are aligned.

[0024] Figure 5 It is the final detection result diagram of the embodiment of the present invention.

[0025] Figure 6 It is a comparison diagram of the detection thresholds of each frame of the image to be detected obtained by the algorithm of the present invention and the fixed threshold algorithm.

[0026] Figure 7 It is a comparison chart of detection probabilities of the algorithm of the present invention and the fixed threshold algorithm.

[0027] Figure 8 This is a comparison chart of the number of targets detected by the algorithm of the present invention and the fixed threshold algorithm.

[0028] Fig. 9It is a comparison chart of the detection results of the 64th frame of the test data set of the present invention. DETAILED DESCRIPTION

[0029] The present invention will be further explained below in conjunction with the accompanying drawings and embodiments, which should not be used to limit the protection scope of the present invention. The purpose of disclosing the present invention is to protect all technical improvements within the scope of the present invention.

[0030] This embodiment is executed on a computer equipped with AMD Ryzen 5 5600K CPU, 16GB RAM and RTX 3060GPU hardware functions, and MATLAB 2024a software; the data set selects "weak aircraft target detection and tracking data set of infrared images under ground / air background", such as Figure 2 As shown in the figure, the data set is provided by the ATR Key Laboratory of the School of Electronic Science of the National University of Defense Technology. It provides 21 types of frame sequence infrared image data and has relatively complete information tags. The data acquisition images are taken by an infrared device with a frequency of 100Hz. Since the higher the frequency of infrared shooting, the less obvious the temporal and spatial changes of the target between frame sequences are, the more difficult it is to detect. Therefore, the frequency is usually reduced, and the step size is 4 frames. The maximum detection tolerance of the infrared device is set to 5. In order to simulate the actual long-distance detection under complex background, the data21 data set is selected. This data set is one of the two data sets with the lowest signal-to-clutter ratio. The signal-to-clutter ratio describes the difficulty of detecting small infrared targets. Generally, the lower the signal-to-clutter ratio, the more difficult the target detection is. Figure 3 As shown, the first three consecutive frames of images, the target is very faint and can hardly be distinguished from the background.

[0031] Combined with Figure 1-9 The following is an adaptive threshold double-frame difference detection algorithm, the specific steps are:

[0032] Execute S1: Figure 3 Take the first two frames of images in the figure as an example, and perform alignment operation. The method based on feature point alignment is used to eliminate the position deviation caused by jitter by image alignment. The processed images are as follows Figure 4 As shown, Figure 4 The grayscale value of the unaligned image part in (b) is set to 0, which is shown in the figure as the right side line, the bottom side line, and the top side line are mostly black; this step also generates a mask, which is a binary image, usually used to indicate which pixels should be retained and which should be ignored. The mask is used to perform the S2 operation.

[0033] Execute S2: perform a differential operation on the aligned double-frame images, subtract the grayscale values ​​of corresponding pixels of the two frames of images, and take the absolute value thereof to obtain a differential image.

[0034] S3: Select the maximum value of the differential image and obtain a threshold value T of 76. Compare all pixel values ​​of the differential image with the T value. The area greater than or equal to the T value is preliminarily determined as the target position.

[0035] S4: Perform connectivity analysis based on the target location determined in S3. If the areas are connected, they are considered to be the same target. Otherwise, they are considered to be multiple targets. The current number of detected targets N is 1.

[0036] S5: Use the threshold T to perform target detection. The number of detected targets N = 1 is less than the maximum tolerance value M of 5. Set T = T-1 and repeat the operations of steps S3 and S4 until N is greater than 5. At this time, set T min =T+1, we get T min =45, T min It is the optimal detection threshold under the constraint of the maximum detection target and also the most sensitive threshold.

[0037] S6: Using the updated optimal detection threshold T min , execute S3 and S4 to obtain the final detected target position, and mark the detected target on the image with a red rectangular frame for subsequent actual visualization applications; Figure 5 As shown, a total of 5 targets are detected, satisfying the requirement that the maximum number of detections is less than or equal to 5, including 1 real target and 4 false targets.

[0038] S7: Repeat the operations from S1 to S6 for all images to be processed in chronological order until the task is completed.

[0039] In order to verify the effectiveness of the algorithm of the present invention, a simulation experiment is designed to compare the algorithm performance; the algorithm of the present invention is compared with the currently more commonly used fixed threshold setting method dominated by experience.

[0040] It is necessary to propose an indicator, the detection probability P, which represents the ability to detect the real target and is defined as follows:

[0041]

[0042] The frame image of whether the target is detected is judged as follows: among all the detected targets, only one target is a real target, then it is determined that the target is detected in the frame.

[0043] Since the accuracy of the simulation platform and the corresponding alignment algorithm will bring certain random errors, five groups of experiments were repeated and the average value was taken.

[0044] like Figure 6As shown in the figure, the detection threshold of each frame of the image to be detected obtained by the algorithm of the present invention, that is, the line M=5 in the figure, is compared with the three fixed threshold algorithms in the prior art, that is, T=20, T=35, and T=50 in the figure, which respectively indicate that the fixed thresholds are set to 20, 35, and 50; since the target is relatively faint, there are a large number of interferences from the highlighted target areas in the image, and there are also disturbances such as jitter and light changes. Therefore, the detection threshold of each frame of the image obtained by the algorithm has a large variation range, and almost every frame is different, and is adaptively adjusted according to changes in the environment; on the contrary, the fixed threshold remains unchanged.

[0045] like Figure 7 As shown in the figure, it is a comparison of the detection probabilities of the algorithm of the present invention and the fixed threshold algorithm; it can be seen that when the fixed threshold is set to 20, the detection probability is the highest, about 81%, followed by the algorithm designed in this paper, with a detection probability of 62%. It should be noted that the mean value of the adaptive threshold in this paper is 35; when the fixed threshold is set to 35, the detection probability drops sharply, about 48%; when the fixed threshold is set to 50, the detection probability drops sharply to only about 27%.

[0046] like Figure 8 As shown in the figure, there is a comparison of the number of detections of the algorithm of the present invention and the fixed threshold algorithm. In actual detection, due to the need for detection effectiveness, in addition to the detection probability, the number of detections is more important. In order to ensure accurate identification of the real target and rapid judgment, operators often abandon the sensitivity of the threshold, that is, the optimal threshold, and set a higher threshold based on experience.

[0047] The algorithm in this paper strictly abides by the maximum tolerance M limit, and the number of detections is always below 5. For the fixed threshold algorithm, when the threshold is set to 20, there are also multiple frames of images with more than 10 detections, which will seriously limit the timeliness in practice. What is more serious is that when the threshold is increased and set to 35 or 50, there are multiple frames of images with more than 50 detections, and even worse, about 470.

[0048] Take out the 64th frame and observe, Fig. 9 As shown in the figure, when the threshold is set to 20, there are 474 false targets, which is meaningless for actual detection. When the threshold is increased to 35 and 50, there are also more false targets, 99 and 10 respectively. On the contrary, for the algorithm proposed in this paper, the number of targets has always been stable within 5 at any time.

[0049] The parts not described in detail in this invention are prior art.

[0050] The embodiments selected herein for the purpose of disclosing the invention are currently considered to be suitable, but it should be understood that the invention is intended to include all changes and modifications of the embodiments that fall within the scope of the concept and invention.

Claims

1. An adaptive threshold double-frame difference detection algorithm, characterized in that: The specific steps are: S1: Align the two-frame images to be processed, and use image alignment to eliminate position deviation caused by jitter; S2: Perform a differential operation on the aligned double-frame images to obtain a differential image; the specific operation is: assuming that the n-th frame and the n-1-th frame image in the video sequence are f n and f n-1 , the grayscale value of the corresponding pixel in the two frames is recorded as f n (x, y) and f n-1 (x, y), then the grayscale values ​​of the corresponding pixels of the two frames are subtracted and their absolute values ​​are taken to obtain the difference image d n It can be expressed as: d n (x,y)=|f n (x,y)-f n-1 (x,y)|, S3: Select the maximum value of the differential image, that is, the threshold T; n (x, y) is compared with the threshold T pixel by pixel. Pixels greater than or equal to the threshold T are identified as targets, otherwise they are identified as backgrounds, which can be expressed as: Thus, the target location is preliminarily determined; S4: Perform connectivity analysis based on the target location determined in S3. If the regions are connected, they are considered to be the same target. Otherwise, they are considered to be multiple targets. Finally, the number of targets N is determined. S5: Use the threshold T to perform target detection operations to determine whether the number of detected targets meets the maximum number limit detected by the optical device, that is, the maximum tolerance value M. If N is less than or equal to M, set T = T-1, and repeat the operations of steps S3 and S4 until N is greater than M. At this time, set T min =T+1,T min That is, it is the optimal detection threshold under the constraint of the maximum detection target, and it is also the most sensitive threshold; S6: Using the updated optimal detection threshold T min , execute S3 and S4 to obtain the final detected target position, and mark the detected target on the image with a red rectangular frame for subsequent actual visualization applications; S7: In practice, since frame images are usually continuous, the operations from S1 to S6 are repeated for all images to be processed in chronological order until the task is completed.

Images