Air-ground infrared target tracking method based on consistency reasoning correlation filtering
Through the consistency reasoning correlation filter and dynamic threshold adaptive update strategy, a multi-time scale historical information fusion model is constructed to solve the problems of model drift and observation model degradation in infrared target tracking and realize the robust tracking of air-to-ground infrared targets.
Patent Information
- Application Number
- CN202510783571.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-12
- Publication Date
- 2025-09-23
AI Technical Summary
Existing infrared target tracking methods based on discriminative correlation filters face problems such as insufficient utilization of historical information, overfitting and model drift in air-ground scenarios, resulting in a decrease in discriminability and an inability to track targets robustly in real time.
The consistency reasoning correlation filter is combined with the forward and backward tracking consistency regularization and the filter time regularization. Through the dynamic threshold adaptive update strategy, a multi-time scale historical information fusion model is constructed to improve the discriminative power of the correlation filter and alleviate the rapid degradation of the observation model.
The real-time and robustness of infrared target tracking are improved, the problems of model drift and observation model degradation in complex scenes are solved, and robust target tracking is achieved.
Smart Images
Figure CN120689370A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of aircraft visual target tracking, and in particular to an air-to-ground infrared target tracking method based on consistency inference correlation filtering. Background Art
[0002] Air-to-ground infrared target tracking technology, with its advantages of all-weather operation, electromagnetic interference resistance, passive non-contact operation, and strong concealment, is widely used in areas such as unmanned aerial vehicle (UAV) reconnaissance, precision-guided weapon systems, border monitoring, and forest fire prevention. Infrared imaging systems generate images by capturing the difference in thermal radiation between the target and the background. Compared with visible light images, infrared images suffer from low contrast, missing texture features, and low signal-to-noise ratio. Traditional visible light target tracking algorithms often struggle to replicate the excellent performance they achieve in visible light scenarios in infrared scenarios. In air-to-ground tracking applications, robust infrared visual target tracking faces significant challenges due to factors such as the target's high maneuverability, high scale dynamics, strong interference from background thermal radiation, and changes in viewing angle.
[0003] In recent years, visual object tracking methods based on discriminative correlation filters (DCFs) have been widely used in the field of visual object tracking. Correlation filter target tracking generates dense training samples through cyclic shifting and implements efficient point-to-point operations in the frequency domain via Fourier transforms. This method strikes a good balance between rapidity, accuracy, and robustness in target tracking. Traditional DCFs train correlation filters by minimizing the least-squares error between the current frame's training samples and Gaussian-distributed training labels. This filter is then used to construct an observation model for binary classification of the target and background. Due to its low computational complexity, this type of algorithm has significant advantages for deployment on platforms with limited computing resources and high real-time requirements, such as unmanned aerial vehicles (UAVs). However, existing DCF algorithms also have some limitations: First, they underutilize historical information. Simply using a large number of historical samples for batch training not only significantly increases the computational load and slows down the algorithm, but also ignores the temporal correlation information contained in the historical filter. Second, using fixed Gaussian-distributed labels for training all historical samples can easily lead to overfitting. In the air-to-ground infrared tracking scenario, the target faces complex situations such as perspective changes and rapid movement. The above problems can easily lead to a decrease in the discrimination ability of the correlation filter and an inability to track the target robustly in real time. Summary of the Invention
[0004] The purpose of this application is to provide an air-to-ground infrared target tracking method based on consistency reasoning correlation filtering, which can improve the discriminative power of the correlation filter while achieving real-time and robust target tracking.
[0005] To achieve the above objectives, this application provides the following solutions:
[0006] This application provides an air-to-ground infrared target tracking method based on consistency reasoning correlation filtering, including:
[0007] Determine the target search area of the current frame based on the target position of the previous video frame;
[0008] Based on the target search area of the current frame, the target state of the current video frame is predicted using the consistency reasoning correlation filter to obtain the target response of the current video frame;
[0009] Determine the target position of the current video frame based on the target response of the current video frame, and use a scale-related filter to determine the target size of the current video frame based on the target position of the current video frame to obtain the target tracking result of the current video frame;
[0010] Determine the target response reliability of the current video frame based on the target response of the current video frame, and determine the dynamic threshold in combination with the target responses of historical video frames;
[0011] When the target response reliability of the current video frame is greater than the dynamic threshold, the consistency reasoning related filter is updated using forward and backward tracking consistency regularization and filter time regularization, and the step of determining the target search area of the current frame based on the target position of the previous video frame is returned.
[0012] Optionally, determining a target search area for the current frame based on a target position of a previous video frame includes:
[0013] According to the image periodicity assumption and the motion smoothness assumption, an image block is extracted from the current video frame image with the target position of the previous video frame as the center, and the extracted image block is used as the target search area of the current frame.
[0014] Optionally, the size of the image block is an integer multiple of the size of the tracking target.
[0015] Optionally, based on the target search area of the current frame, a consistency inference correlation filter is used to predict the target state of the current video frame to obtain the target response of the current video frame, including:
[0016] Extract the HOG features and grayscale features of the target search area of the current frame to obtain a feature map;
[0017] After applying the cosine window function to the feature map, the target response of the current video frame is determined using a consistency inference correlation filter.
[0018] Optionally, determining the target position of the current video frame based on the target response of the current video frame includes:
[0019] The position corresponding to the maximum target response in the current video frame is used as the target position of the current video frame.
[0020] Optionally, determining the target response reliability of the current video frame based on the target response of the current video frame includes:
[0021] determining a target response maximum value and an average peak correlation energy ratio based on the target response of the current video frame;
[0022] The target response reliability of the current video frame is determined based on the target response maximum value and the average peak correlation energy ratio.
[0023] Optionally, the target response reliability of the current video frame is expressed as:
[0024] θ t =max(R t )APCE(R t );
[0025] Where θ t is the target response reliability of the t-th video frame, max(R t ) is the maximum target response value of the t-th video frame, APCE(R t ) is the target response R of the tth video frame t The average peak correlation energy ratio.
[0026] Optionally, the dynamic threshold is expressed as:
[0027]
[0028] Where, is the dynamic threshold, i is the serial number of the video frame, N is the number of video frames before the t-th video frame, APCE(R i ) is the target response R of the i-th video frame i The average peak correlation energy ratio, max(R i ) is the maximum target response value of the i-th video frame.
[0029] Optionally, the process of updating the consistency reasoning related filter using forward and backward tracking consistency regularization and filter time regularization includes:
[0030] Construct an objective function based on forward and backward tracking consistency regularization and filter time regularization;
[0031] The consistency reasoning related filter is updated with the goal of minimizing the objective function.
[0032] Optionally, the objective function is expressed as:
[0033]
[0034] Where, is the objective function value, The target state of the d-th channel in the t-th frame, D is the channel dimension of the feature map, y is the Gaussian distribution training sample label, x t The vectorized feature map of the image region used for consistency reasoning related filter training in the t-th frame, is the vectorized feature map of the image region of the dth channel for consistency reasoning related filter training of the tth frame, w is the vectorized space regularization constraint weight, is the circular convolution operator, ⊙ is the Hadamard product operator, || ||2 is the L2-norm, || ||1 is the L1-norm, λ1 is the spatial regularization constraint coefficient, λ2 is the filter time consistency constraint coefficient, λ3 is the forward and backward tracking consistency sparse constraint coefficient, is the target state of the d-th channel in the t-1th frame, The vectorized feature map of the image region of the dth channel for consistency reasoning related filter training in the t-1th frame; h t is the vectorized consistency reasoning correlation filter, which represents the target state of the tth frame output by the consistency reasoning correlation filter.
[0035] According to the specific embodiments provided in this application, this application has the following technical effects:
[0036] This application provides an air-to-ground infrared target tracking method based on consistency reasoning correlation filtering. It adopts a joint mechanism of forward and backward tracking consistency regularization and filter time regularization to update the consistency reasoning correlation filter. It can solve the model drift problem caused by traditional single-frame optimization and improve the discriminative power of the correlation filter. In addition, during the update process of the consistency reasoning correlation filter, by adopting a dynamic threshold adaptive update strategy, the problem of the observation model being contaminated by unreliable tracking results in complex tracking scenarios is avoided, the rapid degradation of the observation model in complex tracking scenarios is alleviated, and the real-time and robustness of target tracking are improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0038] Figure 1 A flowchart of an air-to-ground infrared target tracking method based on consistency reasoning correlation filtering provided in one embodiment of the present application;
[0039] Figure 2This is a diagram illustrating an implementation framework of an air-to-ground infrared target tracking method based on consistency inference correlation filtering provided in one embodiment of the present application;
[0040] Figure 3 A schematic diagram of target tracking results of a first set of air-to-ground infrared video sequences provided in one embodiment of the present application;
[0041] Figure 4 A schematic diagram of target tracking results of a second set of air-to-ground infrared video sequences provided in one embodiment of the present application;
[0042] Figure 5 This is a schematic diagram of target tracking results for the third set of air-to-ground infrared video sequences provided in one embodiment of the present application. DETAILED DESCRIPTION
[0043] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.
[0044] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.
[0045] In an exemplary embodiment, the present application provides an air-to-ground infrared target tracking method based on consistency inference correlation filtering. The method is executed by a computer device, specifically a computer device such as a terminal or a server, or a terminal and a server. In the embodiment of the present application, the method is applied to a server as an example for explanation. Figure 1 As shown, the method includes:
[0046] Step 100: Determine a target search area for the current frame based on a target position in the previous video frame.
[0047] Step 101: Based on the target search area of the current frame, the target state of the current video frame is predicted using the consistency reasoning correlation filter to obtain the target response of the current video frame.
[0048] Step 102: determine the target position of the current video frame based on the target response of the current video frame, and use a scale-related filter to determine the target size of the current video frame based on the target position of the current video frame to obtain a target tracking result of the current video frame.
[0049] Step 103: Determine the target response reliability of the current video frame based on the target response of the current video frame, and determine a dynamic threshold in combination with the target responses of historical video frames.
[0050] Step 104 : When the target response reliability of the current video frame is greater than the dynamic threshold, the consistency reasoning related filters are updated using forward and backward tracking consistency regularization and filter time regularization, and the process returns to step 100 .
[0051] By implementing the above steps 100 to 104 , the present application can achieve real-time and robust target tracking while improving the discrimination power of the correlation filter.
[0052] In another exemplary embodiment of the present application, in order to accurately determine the target search area of the current video frame based on the target state of the previous video frame, the implementation process of the above step 100 of the present application can be described as follows: based on the image periodicity assumption and the motion smoothness assumption, an image block is extracted from the current video frame image with the target position of the previous video frame as the center, and the extracted image block is used as the target search area of the current frame. Based on this, after the image block is extracted, a cyclically shifted sample (i.e., an image selected by a cyclic manner) can be used to replace the real translation sample (i.e., an image selected by a translation manner) as a candidate sample for the current frame target.
[0053] In practical applications, to cope with the high mobility of air-to-ground infrared targets, the size of the image block (i.e., the target search area size) can be set to an integer multiple of the target size. For example, the target search area size can be set to 4 times the target size.
[0054] In another exemplary embodiment of the present application, the implementation process of the above step 101 can be described as follows:
[0055] The HOG (Histogram of Oriented Gradients) features and grayscale features of the target search area in the current frame are extracted to obtain a feature map. After applying a cosine window function to the feature map, the target response of the current video frame is determined using a consistency inference correlation filter. Based on this, in step 102, the target position is determined based on the maximum position of the target response. Once the target position is determined, the target size can be predicted using the scale correlation filter proposed by the existing DSST (Discriminative Scale Space Tracker) algorithm.
[0056] In another exemplary embodiment of the present application, in order to alleviate the rapid degradation of the observation model in complex tracking scenarios, a dynamic threshold adaptive update strategy can be used to update the observation model constructed based on the above-mentioned consistency reasoning correlation filter and scale correlation filter. Among them, the reliability of the tracking result can be determined by nonlinear fusion of the peak significance (i.e., the maximum value of the target response) and the energy concentration (i.e., the average peak correlation energy ratio) of the target response graph, and the dynamic threshold used can be determined based on the historical target response graph. Based on this, the implementation process of the above-mentioned steps 103 and 104 may include:
[0057] (1) Based on the target response of the current video frame, a target response map is obtained to determine the maximum value of the target response and the average peak-to-correlation energy (APCE).
[0058] (2) Determine the target response reliability of the current video frame based on the target response maximum value and the average peak correlation energy ratio. For example, the target response reliability of the current video frame can be expressed as:
[0059] θ t =max(R t )APCE(R t ) (1)
[0060] Where θ t is the target response reliability of the t-th video frame, max(R t ) is the maximum target response value of the t-th video frame, APCE(R t ) is the target response R of the tth video frame t The average peak correlation energy ratio.
[0061] (3) Calculate the dynamic threshold for reliability judgment using the historical target response of the previous N frames have:
[0062]
[0063] Where i is the serial number of the video frame, APCE(R i ) is the target response R of the i-th video frame i The average peak correlation energy ratio, max(R i ) is the maximum target response value of the i-th video frame.
[0064] (4) When When , it indicates that the target response of the current video frame is reliable, the tracking result is credible, and the observation model can be updated. When , it indicates that the target response of the current video frame is unreliable, the tracking result is unreliable, and the observation model is not updated.
[0065] (5) Update the correlation filter based on consistency reasoning.
[0066] For example, let is the vectorized feature map of the image region used for correlation filter training in the t-th frame, where is the vectorized feature map of the d-th channel image region, T is the vector length of each channel vectorized feature map, and D is the channel dimension of the feature map. At this time, the consistency reasoning related filter is trained by minimizing the following objective function:
[0067]
[0068] Where, is the objective function value, The target state of the d-th channel of the t-th frame, y is the Gaussian distribution training sample label, w is the vectorized space regularization constraint weight, is the circular convolution operator, ⊙ is the Hadamard product operator, ||||2 is the L2-norm, ||||1 is the L1-norm, λ1 is the spatial regularization constraint coefficient, λ2 is the filter time consistency constraint coefficient, λ3 is the forward and backward tracking consistency sparse constraint coefficient, is the target state of the d-th channel in the t-1th frame, The vectorized feature map of the image region of the dth channel for the consistency inference correlation filter training of the t-1th frame. t is the vectorized consistency reasoning correlation filter, which represents the target state of the tth frame output by the consistency reasoning correlation filter.
[0069] The h obtained by the alternating direction method of multipliers (ADMM) is used t After that, it is used to predict the target position in the next frame. The scale-dependent filter is consistent with the DSST (Discriminative Scale Space Tracker) algorithm.
[0070] Based on the above description, the implementation framework of the air-to-ground infrared target tracking method provided by this application is as follows Figure 2As shown. This application constructs a multi-time-scale historical information fusion model by designing a joint action mechanism of the forward and backward tracking consistency sparse constraint term and the filter time consistency quadratic constraint term. And by adopting the consistency reasoning related filter, through the fusion of historical information at multiple time scales, it can break through the traditional single-frame optimization paradigm and solve the model drift problem caused by traditional single-frame optimization. By adopting a dynamic threshold adaptive update strategy and judging the reliability of the tracking results by using the historical frame target response information, it can effectively alleviate the rapid degradation of the observation model in complex tracking scenarios.
[0071] In another exemplary embodiment of the present application, in order to verify the feasibility and effectiveness of the method proposed in the present application, target tracking simulation tests were performed on multiple sets of air-to-ground infrared video sequences. The experimental platform for the target tracking simulation test was configured as follows: the hardware platform used a computer configured with Intel(R) Core(TM) i5-8300 CPU@2.30H, and the software platform used MATLAB R2018a. Finally, the following results were obtained: Figure 3-Figure 5 The target tracking results shown in Figure 3-Figure 5 It can be seen from the figure that the method provided by this application can robustly track the above-mentioned air-to-ground infrared targets.
[0072] In summary, the air-to-ground infrared target tracking method based on consistency reasoning correlation filtering provided by this application, when training the consistency reasoning correlation filter, constructs a multi-scale historical information fusion model by combining the previous and next tracking consistency sparse constraints (L1-norm) with the filter time consistency quadratic constraint (L2-norm), which can solve the model drift problem caused by traditional single-frame optimization. The dynamic threshold adaptive update strategy is adopted when updating the model, which can avoid the problem of the observation model being contaminated by unreliable tracking results in complex tracking scenarios, and thus can achieve real-time and robust target tracking while improving the discriminative power of the correlation filter.
[0073] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.
[0074] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0075] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0076] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0077] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (RRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0078] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.
[0079] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0080] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.
Claims
1. A method for tracking air-to-ground infrared targets based on consistency inference correlation filtering, characterized in that: include: Determine the target search area of the current frame based on the target position of the previous video frame; Based on the target search area of the current frame, the target state of the current video frame is predicted using the consistency reasoning correlation filter to obtain the target response of the current video frame; Determine the target position of the current video frame based on the target response of the current video frame, and use a scale-related filter to determine the target size of the current video frame based on the target position of the current video frame to obtain the target tracking result of the current video frame; Determine the target response reliability of the current video frame based on the target response of the current video frame, and determine the dynamic threshold in combination with the target responses of historical video frames; When the target response reliability of the current video frame is greater than the dynamic threshold, the consistency reasoning related filter is updated using forward and backward tracking consistency regularization and filter time regularization, and the step of determining the target search area of the current frame based on the target position of the previous video frame is returned.
2. The air-to-ground infrared target tracking method based on consistency reasoning correlation filtering according to claim 1 is characterized in that: Determine the target search area of the current frame based on the target position of the previous video frame, including: According to the image periodicity assumption and the motion smoothness assumption, an image block is extracted from the current video frame image with the target position of the previous video frame as the center, and the extracted image block is used as the target search area of the current frame.
3. The air-to-ground infrared target tracking method based on consistency reasoning correlation filtering according to claim 2 is characterized in that: The size of the image block is an integer multiple of the size of the tracking target.
4. The air-to-ground infrared target tracking method based on consistency reasoning correlation filtering according to claim 1 is characterized in that: Based on the target search area of the current frame, the target state of the current video frame is predicted using the consistency inference correlation filter to obtain the target response of the current video frame, including: Extract the HOG features and grayscale features of the target search area of the current frame to obtain a feature map; After applying the cosine window function to the feature map, the target response of the current video frame is determined using a consistency inference correlation filter.
5. The air-to-ground infrared target tracking method based on consistency reasoning correlation filtering according to claim 1 is characterized in that: Determining a target position of the current video frame based on the target response of the current video frame includes: The position corresponding to the maximum target response in the current video frame is used as the target position of the current video frame.
6. The air-to-ground infrared target tracking method based on consistency reasoning correlation filtering according to claim 1 is characterized in that: Determining target response reliability of the current video frame based on the target response of the current video frame includes: determining a target response maximum value and an average peak correlation energy ratio based on the target response of the current video frame; The target response reliability of the current video frame is determined based on the target response maximum value and the average peak correlation energy ratio.
7. The air-to-ground infrared target tracking method based on consistency reasoning correlation filtering according to claim 6 is characterized in that: The target response reliability of the current video frame is expressed as: θ t =max(R t )APCE(R t ); Where θ t is the target response reliability of the t-th video frame, max(R t ) is the maximum target response value of the t-th video frame, APCE(R t ) is the target response R of the tth video frame t The average peak correlation energy ratio.
8. The air-to-ground infrared target tracking method based on consistency reasoning correlation filtering according to claim 1 is characterized in that: The dynamic threshold is expressed as: Where, is the dynamic threshold, i is the serial number of the video frame, N is the number of video frames before the t-th video frame, APCE(R i ) is the target response R of the i-th video frame i The average peak correlation energy ratio, max(R i ) is the maximum target response value of the i-th video frame.
9. The air-to-ground infrared target tracking method based on consistency reasoning correlation filtering according to claim 1 is characterized in that: The process of updating the consistency reasoning related filters using forward and backward tracking consistency regularization and filter time regularization includes: Construct an objective function based on forward and backward tracking consistency regularization and filter time regularization; The consistency reasoning related filter is updated with the goal of minimizing the objective function.
10. The air-to-ground infrared target tracking method based on consistency reasoning correlation filtering according to claim 9 is characterized in that: The objective function is expressed as: Where, is the objective function value, The target state of the d-th channel in the t-th frame, D is the channel dimension of the feature map, y is the Gaussian distribution training sample label, x t The vectorized feature map of the image region used for consistency reasoning related filter training in the t-th frame, is the vectorized feature map of the image region of the dth channel for consistency reasoning related filter training of the tth frame, w is the vectorized space regularization constraint weight, is the circular convolution operator, ⊙ is the Hadamard product operator, || ||2 is the L2-norm, || ||1 is the L1-norm, λ1 is the spatial regularization constraint coefficient, λ2 is the filter time consistency constraint coefficient, λ3 is the forward and backward tracking consistency sparse constraint coefficient, is the target state of the d-th channel in the t-1th frame, The vectorized feature map of the image region of the dth channel for consistency reasoning related filter training in the t-1th frame; h t is the vectorized consistency reasoning correlation filter, which represents the target state of the tth frame output by the consistency reasoning correlation filter.