An infrared target tracking method and system based on gradient statistics combined with local matching

Abstract

The present invention relates to the field of infrared target tracking technology, and discloses a method and system for infrared target tracking using gradient statistics combined with local matching. The tracking method comprises the following steps: S1, image acquisition and model construction; S2, image preprocessing; S3, honeycomb gradient statistics; S4, model training; S5, test sample response; S6, threshold determination; S7, direct tracking point determination; and S8, tracking point recapture determination. The present invention addresses the problems of low tracking accuracy, poor robustness, and limited applicable scenarios in the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of infrared target tracking, in particular to an infrared target tracking method and system combining gradient statistics with local matching. Background Art

[0002] Target tracking is essentially the process of finding the optimal position of a target selected in a reference image (either automatically detected and identified or manually selected) in the subsequent real-time image. Currently, commonly used infrared target tracking methods include those based on grayscale correlation matching, correlation filtering, and mean-shift. These methods have achieved good results in some specific applications. However, in the specific application of infrared detection and guidance equipment, these methods are difficult to fully meet the tracking requirements of the system due to factors such as imaging sensor noise, changes in imaging angle and distance, platform maneuvering (especially airborne or missile-borne platforms), changes in target brightness, and high-speed movement.

[0003] Since infrared imaging lacks color channel information, texture and details are relatively scarce, the distinction between target and background features is low, and tracking point drift and target tracking loss are very likely to occur. The poor robustness seriously restricts the improvement of the performance of optoelectronic detection and guidance equipment, and is a difficult problem in the field of optoelectronic image signal processing. Summary of the Invention

[0004] In order to overcome the shortcomings of the existing technology, the present invention provides an infrared target tracking method and system combining gradient statistics with local matching to solve the problems of low tracking accuracy, poor robustness, and narrow applicable scenarios in the existing technology.

[0005] The technical solution adopted by the present invention to solve the above problems is:

[0006] A method for infrared target tracking based on gradient statistics and local matching, comprising the following steps:

[0007] S1, image acquisition and model construction: collect the image of the target to be measured, build the relevant filtering model, set the initial target template information, and set the center of the initial target template information as the initial tracking point;

[0008] S2, image preprocessing: preprocessing the image of the target to be measured;

[0009] S3, honeycomb gradient statistics: select a window area with the tracking point as the center, then build a honeycomb grid in the window area, and count the gradient statistical features of each honeycomb grid;

[0010] S4, model training: training the relevant filter model;

[0011] S5, test sample response: using the trained correlation filter model, test the sample in the real-time image to obtain the test sample response value;

[0012] S6, threshold judgment: compare the maximum value of the test sample response value with the preset threshold value. If the maximum value of the test sample response value is greater than or equal to the preset threshold value, proceed to step S7; if the maximum value of the test sample response value is less than the preset threshold value, proceed to step S8;

[0013] S7, direct determination of the tracking point: the position coordinate corresponding to the maximum value of the test sample response value is used as the tracking point of the target in the current frame, and the target template information is refreshed in real time during the tracking process;

[0014] S8, tracking point recapture and determination: recapture the target using a local matching search method to determine the tracking point.

[0015] As a preferred technical solution, in step S2, median filtering is used to preprocess the image.

[0016] As a preferred technical solution, in step S3, the window size is 1.5 to 2.5 times the actual size of the target to be measured.

[0017] As a preferred technical solution, in step S3, the method for calculating the gradient statistical characteristics of the honeycomb grid is: using the maximum gradient value of all pixels in the honeycomb grid as the statistical characteristic value of each honeycomb.

[0018] As a preferred technical solution, in step S1, the correlation filter model constructed is a kernel ridge regression model, and the kernel function of the kernel ridge regression model is a composite bandwidth Gaussian radial basis kernel function.

[0019] As a preferred technical solution, in step S4 and step S5, the gradient statistical features of the honeycomb grid are used as inputs of the correlation filtering model.

[0020] As a preferred technical solution, in step S5, the specific method for obtaining the test sample response value is: select a test sample in the current frame image with the tracking point of the target in the previous frame as the center, the sample size is the same as the target template size, and the average of the regression values ​​of different bandwidth models is taken as the final response value.

[0021] As a preferred technical solution, in step S7, the specific method for refreshing the target template is: linearly accumulating and averaging the current target template and the target area centered on the tracking point to obtain a refreshed target template.

[0022] As a preferred technical solution, in step S8, the specific method of re-capturing the target using the local matching search method is: with the current tracking point as the center, the target template slides in sequence in a local window area, and the matching coefficient between the template and the corresponding image window area is calculated. If the maximum matching coefficient exceeds the set threshold, it means that the target is captured, thereby obtaining a new tracking point; otherwise, it means that the target is not found, and the memory tracking mode is entered, and the historical frame tracking data is extrapolated to obtain a new tracking point.

[0023] A gradient statistics combined with local matching infrared target tracking system, based on the above-mentioned gradient statistics combined with local matching infrared target tracking method, includes the following modules:

[0024] Image acquisition and model building module: used to acquire images of the target to be measured, build relevant filtering models, set initial target template information, and set the center of the initial target template information as the initial tracking point;

[0025] Image preprocessing module: used to preprocess the image of the target to be measured;

[0026] Honeycomb gradient statistics module: used to select a window area with the tracking point as the center, then build a honeycomb grid in the window area, and count the gradient statistical features of each honeycomb grid;

[0027] Model training module: training the relevant filter model;

[0028] Test sample response module: used to test samples in real-time images using the trained correlation filter model to obtain test sample response values;

[0029] Threshold decision module: used to compare the maximum value of the test sample response value with the preset threshold value. If the maximum value of the test sample response value is greater than or equal to the preset threshold value, the process proceeds to step S7; if the maximum value of the test sample response value is less than the preset threshold value, the process proceeds to step S8.

[0030] Tracking point direct determination module: used to take the position coordinate corresponding to the maximum value of the test sample response value as the tracking point of the current frame target, and refresh the target template information in real time during the tracking process;

[0031] Tracking point recapture determination module: used to recapture the target using the local matching search method to determine the tracking point;

[0032] The image acquisition and model construction module, the image preprocessing module, the honeycomb gradient statistics module, the model training module, the test sample response module, and the threshold judgment module are electrically connected in sequence. The threshold judgment module is electrically connected to the tracking point direct determination module and the tracking point recapture determination module respectively. The model training module is electrically connected to the tracking point direct determination module and the tracking point recapture determination module respectively.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] (1) The present invention uses honeycomb gradient statistics as the input feature of the correlation filter, which can better characterize the target invariance characteristics, more effectively distinguish the target and background, and enhance the robustness of the system target tracking; introduces the composite bandwidth kernel function into the correlation filter, so that the expression ability of the tracking model is further expanded; combines the local matching method to effectively solve the tracking problem when the target offset between adjacent frames is large;

[0035] (2) The present invention adopts honeycomb gradient statistics as the input feature of the correlation filter, introduces a composite bandwidth kernel function into the correlation filter, refreshes the target template in real time by linear weighted average, and combines technical methods such as local matching search to achieve stable tracking of infrared targets, improve the system's tracking accuracy and robustness for targets, and can be widely used in real time in various optoelectronic imaging devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a schematic diagram of the steps of a method for infrared target tracking using gradient statistics combined with local matching according to the present invention;

[0037] Figure 2 This is a schematic structural diagram of an infrared target tracking system combining gradient statistics with local matching according to the present invention;

[0038] Figure 3 A schematic diagram of a process of infrared target tracking of the present invention;

[0039] Figure 4 Schematic diagram of honeycomb gradient statistics of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be further described in detail below with reference to the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0041] Example 1

[0042] like Figures 1 to 4 As shown, for the practical application of optoelectronic imaging equipment, the technical problem to be solved by the present invention is to provide an infrared target tracking method and system with high tracking accuracy, strong robustness and wide application scenarios.

[0043] In order to achieve the above object, the specific steps of the technical solution adopted by the present invention are:

[0044] Step 1: Collect the image of the target to be measured, build the relevant filtering model, and set the initial target template information;

[0045] Step 2: preprocessing the collected image (in this embodiment, the preprocessing includes median filtering);

[0046] Step 3: Select a window area with the tracking point as the center. The window size should be 1.5 to 2.5 times the actual size of the target. Then, build a honeycomb grid in the window area and count the gradient information of all pixels in each honeycomb grid.

[0047] Step 4: Train the model;

[0048] Step 5: Use the trained model to test samples in the real-time image to obtain test sample responses;

[0049] Step 6: Threshold judgment: When the value is greater than or equal to the threshold, the tracking point is directly determined and the target template is refreshed; when the value is less than the threshold, the target is recaptured using the local matching search method to determine the tracking point.

[0050] Preferably, in step 2, five-point median filtering is used to eliminate salt and pepper noise and detector blind pixels.

[0051] Preferably, in step three, a honeycomb grid is constructed, and the maximum gradient of pixels in the honeycomb grid is used as the statistical characteristic value of each honeycomb.

[0052] Preferably, in step 4, the model adopts a kernel ridge regression model. The specific kernel function is a composite bandwidth Gaussian radial basis kernel function, where the composite bandwidth refers to a combination of Gaussian functions with different bandwidths.

[0053] Preferably, the training method of the model parameters is as follows: setting a regression value (ideal response for target tracking), inputting the gradient statistics of the target template into the model as a feature, and the parameters of the model can be calculated using a kernel matrix.

[0054] Preferably, in step five, the specific method for selecting the test sample is: selecting a test sample in the current frame image with the tracking point of the target in the previous frame as the center, and the size of the test sample is the same as that of the target template.

[0055] Preferably, the specific method for calculating the response value of the test sample to the model is: calculating the regression value of the test sample in Fourier space, and taking the average of the regression values ​​of different bandwidth models as the final response value.

[0056] Preferably, in step six, the maximum value of the test sample response value is compared with a preset threshold value.

[0057] Preferably, the specific method for directly determining the tracking point is: the position coordinate corresponding to the maximum value of the test sample response value is used as the tracking point of the current frame target.

[0058] Preferably, the target template is refreshed in real time during the tracking process.

[0059] Preferably, the specific method of recapture the target using local matching search is: with the current tracking point as the center, the target template slides in a local area, and the matching coefficient between the template and its corresponding image area is calculated. If the maximum matching coefficient exceeds the threshold, it means that the target is captured, and the image position corresponding to the maximum matching coefficient is used as the tracking point; otherwise, it means that the target is not found, and the memory tracking mode is entered, and the historical frame tracking data is used to extrapolate to obtain a new tracking point.

[0060] The present invention relates to an infrared target tracking method combining kernel correlation filtering of honeycomb gradient statistical features with local matching. Stable tracking of infrared targets is a key technology in optoelectronic systems such as precision guidance, search and tracking, surveillance and early warning.

[0061] The present invention uses honeycomb gradient statistics as the input feature of the correlation filter, which can better characterize the target invariance characteristics, more effectively distinguish the target from the background, and enhance the robustness of the system target tracking. The introduction of the composite bandwidth kernel function into the correlation filter further expands the expressive power of the tracking model. The combination of the local matching method effectively solves the tracking problem when the target offset between adjacent frames is large.

[0062] Compared with the existing technology, the present invention adopts honeycomb gradient statistics as the input feature of the correlation filter, introduces a composite bandwidth kernel function into the correlation filter, refreshes the target template in real time by linear weighted average, and combines technical methods such as local matching search to achieve stable tracking of infrared targets, improve the system's tracking accuracy and robustness for targets, and can be widely used in real time in various optoelectronic imaging devices.

[0063] Example 2

[0064] like Figures 1 to 4 As shown, as a further optimization of Example 1, this embodiment includes all the technical features of Example 1. In addition, this embodiment also includes the following technical features:

[0065] A gradient statistics combined with local matching infrared target tracking method, the specific steps are as follows:

[0066] Step 1: Capture an image and input the initial target template. This can be done directly from the imaging sensor or by processing a stored image sequence. The initial target template can be selected manually or automatically detected and identified by the system (i.e., image acquisition and model construction: acquire an image of the target, construct a relevant filter model, set the initial target template information, and set the center of the initial target template information as the initial tracking point).

[0067] Step 2: Perform median filtering on the collected image. In this specific embodiment, five-point median filtering is used to eliminate salt and pepper noise and invalid detector pixels.

[0068] Step 3: Divide the target window area into multiple honeycomb grids, with each honeycomb grid having a size of 3*3 pixels to 5*5 pixels. Calculate the gradient information of all pixels in the honeycomb grid, and use the maximum gradient as the statistical feature value of each honeycomb. In this specific embodiment, we divide the target area into m*n honeycombs, with each honeycomb grid having a size of 3*3. Figure 4 As shown in the figure, the center of the window is the current tracking point. On the one hand, choosing a honeycomb rather than a single pixel as the processing unit effectively reduces the dimension of the feature vector and the amount of subsequent computation. On the other hand, the gradient is chosen as the feature because the gradient information of the infrared target's brightness is more stable under different imaging conditions.

[0069] Step 4, model training; Commonly used ridge regression is a linear model, but the actual target tracking problem does not always satisfy the linear model. Introducing a kernel function can enhance the expressive power of the regression model, so a Gaussian radial basis kernel function is introduced into the model. The bandwidth parameter in the Gaussian function will also have an impact on the performance of the model. The fixed bandwidth limit will reduce the performance of the model, and thus a composite bandwidth is further introduced (several different bandwidths are set for the model). In order to speed up the training of the model, the model parameters can be trained in Fourier space by constructing a cyclic shift matrix. In addition, in this specific embodiment, the regression value of the model training is a two-dimensional Gaussian function.

[0070] Step 5. Calculate the response of the test sample to the model. After the model parameters are trained, during the tracking process, select an image block centered on the tracking point of the previous frame image on the current frame image as the test sample. In Fourier space, use the dot product of the kernel matrix and the model parameters, combined with the inverse Fourier transform operation, to calculate the filter response of the test sample to a single bandwidth model. Finally, the average of the filter responses of all bandwidth models is used as the final model response value.

[0071] Step 6: Threshold judgment: compare the maximum value of the test sample response value with the preset threshold value. The threshold value can be flexibly set according to changes in the tracking application scenario.

[0072] Tracking point determination: In this specific embodiment, the position coordinates corresponding to the maximum value of the test sample response value are used as the target tracking point of the current frame.

[0073] In this specific embodiment, the specific method of refreshing the target template is: linearly accumulating and averaging the current target template and the target area centered on the tracking point to obtain the refreshed target template, so that the model has a certain memory function to avoid drastic fluctuations in the model.

[0074] Target recapture based on local matching search; applying kernel correlation filtering to target tracking involves the selection of the target template area size (also the size of the training and test sample blocks). Although expanding the actual target size can overcome target shift within a certain range, an overly large template area inevitably introduces more clutter information into the training and test samples, causing tracking point offsets. This is particularly likely to result in tracking point errors if the area contains clutter from similar targets. In specific applications such as aircraft (missile) platforms and tracking high-speed moving targets, the target position between adjacent frames may deviate significantly (far greater than the target size). In this case, kernel correlation filtering tracking fails, and the target needs to be recaptured over a larger area. In this specific embodiment, the target recapture process is as follows: with the current tracking point as the center, the target template is sequentially slid within a local window area. The matching coefficient between the template and the corresponding image window area is calculated. If the maximum matching coefficient exceeds a threshold, the target is captured; otherwise, the target is not found, and the memory tracking mode is entered. If the target is captured, the image position coordinate corresponding to the maximum matching coefficient is used as the tracking point. In the memory tracking mode, the tracking point is extrapolated using the tracking data of the previous three frames.

[0075] As described above, the present invention can be preferably implemented.

[0076] All features disclosed in all embodiments in this specification, or steps in all methods or processes implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any manner.

[0077] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Based on the technical essence of the present invention and within the spirit and principles of the present invention, any simple modification, equivalent replacement and improvement of the above embodiment shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for infrared target tracking based on gradient statistics combined with local matching, characterized in that: The following steps are involved: S1, image acquisition and model construction: collect images of the target to be measured, construct a correlation filter model, set the initial target template information, and set the center of the initial target template information as the initial tracking point; the constructed correlation filter model is a kernel ridge regression model, and the kernel function of the kernel ridge regression model is a composite bandwidth Gaussian radial basis kernel function; S2, image preprocessing: preprocessing the image of the target to be measured; S3, honeycomb gradient statistics: select a window area with the tracking point as the center, then build a honeycomb grid in the window area, and count the gradient statistical features of each honeycomb grid; The calculation method of the gradient statistical characteristics of the honeycomb grid is as follows: the maximum gradient value of all pixels in the honeycomb grid is used as the statistical characteristic value of each honeycomb; S4, model training: using the gradient statistical features of the honeycomb grid as the input of the correlation filter model to train the correlation filter model; S5, test sample response: using the trained correlation filter model, test the sample in the real-time image to obtain the test sample response value; S6, threshold judgment: compare the maximum value of the test sample response value with the preset threshold value. If the maximum value of the test sample response value is greater than or equal to the preset threshold value, proceed to step S7; If the maximum value of the test sample response value is less than the preset threshold value, then proceed to step S8; S7, direct determination of the tracking point: the position coordinate corresponding to the maximum value of the test sample response value is used as the tracking point of the target in the current frame, and the target template information is refreshed in real time during the tracking process; S8, tracking point recapture determination: The target is recaptured using the local matching search method to determine the tracking point; with the current tracking point as the center, the target template slides in a local window area in sequence, and the matching coefficient between the template and the corresponding image window area is calculated. If the maximum matching coefficient exceeds the set threshold, it means that the target is captured, and a new tracking point is obtained; otherwise, it means that the target is not found, and the memory tracking mode is entered, and the historical frame tracking data is used to extrapolate to obtain a new tracking point.

2. The infrared target tracking method combining gradient statistics with local matching according to claim 1, characterized in that: In step S2, median filtering is used to preprocess the image.

3. The infrared target tracking method combining gradient statistics with local matching according to claim 1, characterized in that: In step S3, the window size is 1.5 to 2.5 times the actual size of the target to be measured.

4. The infrared target tracking method combining gradient statistics with local matching according to claim 1, characterized in that: In step S5, the specific method for obtaining the test sample response value is as follows: a test sample is selected in the current frame image with the tracking point of the target in the previous frame as the center, the sample size is the same as the target template size, and the average of the regression values ​​of different bandwidth models is taken as the final response value.

5. The infrared target tracking method combining gradient statistics with local matching according to claim 1, characterized in that: In step S7, the specific method of refreshing the target template is: linearly accumulating and averaging the current target template and the target area centered on the tracking point to obtain a refreshed target template.

6. An infrared target tracking system combining gradient statistics with local matching, characterized in that: The infrared target tracking method based on a gradient statistics combined with local matching according to any one of claims 1 to 5 comprises the following modules: Image acquisition and model building module: used to acquire images of the target to be measured, build relevant filtering models, set initial target template information, and set the center of the initial target template information as the initial tracking point; Image preprocessing module: used to preprocess the image of the target to be measured; Honeycomb gradient statistics module: used to select a window area with the tracking point as the center, then build a honeycomb grid in the window area, and count the gradient statistical features of each honeycomb grid; Model training module: training the relevant filter model; Test sample response module: used to test samples in real-time images using the trained correlation filter model to obtain test sample response values; Threshold decision module: used to compare the maximum value of the test sample response value with the preset threshold value. If the maximum value of the test sample response value is greater than or equal to the preset threshold value, the module will be transferred to the tracking point direct determination module. If the maximum value of the test sample response value is less than the preset threshold value, then go to the tracking point recapture determination module; Tracking point direct determination module: used to take the position coordinate corresponding to the maximum value of the test sample response value as the tracking point of the current frame target, and refresh the target template information in real time during the tracking process; Tracking point recapture determination module: used to recapture the target using the local matching search method to determine the tracking point; The image acquisition and model construction module, the image preprocessing module, the honeycomb gradient statistics module, the model training module, the test sample response module, and the threshold judgment module are electrically connected in sequence. The threshold judgment module is electrically connected to the tracking point direct determination module and the tracking point recapture determination module respectively. The model training module is electrically connected to the tracking point direct determination module and the tracking point recapture determination module respectively.

Images