A hyperspectral target tracking method based on four-feature fusion and weight coefficient
By employing spectral dimensionality reduction and feature fusion methods, combined with adaptive weight coefficient updates, the problems of low target feature discriminativeness and low real-time performance in hyperspectral target tracking are solved, achieving efficient target tracking in complex backgrounds.
Patent Information
- Application Number
- CN202011558301.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-25
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-12-25
AI Technical Summary
Existing hyperspectral target tracking methods suffer from poor target feature discrimination in complex backgrounds, low real-time tracking performance, susceptibility to interference from similar color backgrounds, and high computational cost, resulting in poor tracking performance.
We use hyperspectral image sequences of selected channels after spectral dimensionality reduction and hyperspectral image sequences of fused channels after spectral dimensionality reduction, combined with HOG features and depth features, to calculate weak response maps and perform weighted averaging through a kernel correlation filter tracker, and use adaptively updated weight coefficients to handle complex background interference.
It improves the speed and stability of target tracking in hyperspectral image sequences, overcomes clutter interference and target occlusion in complex backgrounds, enhances the stability of the kernel correlation filter classifier, and achieves efficient target tracking.
Smart Images

Figure CN112598708B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a hyperspectral target tracking method based on four-feature fusion and weight coefficients, and belongs to the technical field of image processing. Background Art
[0002] Target tracking in hyperspectral image sequences under complex background is an important part of the field of hyperspectral image processing technology. It has very wide practical applications in many systems such as hyperspectral anomaly detection, hyperspectral target detection and tracking systems, and hyperspectral remote sensing systems. In recent years, target tracking methods based on improved kernel correlation filtering have been widely used in the field of computer vision. The kernel correlation filtering algorithm uses the grayscale features of the base samples for tracking, but the grayscale features of the hyperspectral target are not sufficient to distinguish the complex background from the target in the background.
[0003] The document "Fully-Convolutional Siamese Networks for Object Tracking" (published at the European conference on computer vision, 2016) published by Luca Bertinetto et al. of Harvard University discloses a video target tracking method based on a fully convolutional twin network. The specific steps to implement this method are: based on the ImageNet2015 database, a neural network is trained using a pre-online learning method to solve the problem of generative similarity learning. This similarity matching function is simply evaluated during the tracking process. Then the pre-trained deep convolutional network is used as a feature encoder to improve the tracking performance. However, this method is easily interfered by similar color backgrounds, and the parameter training of the deep network is time-consuming, which greatly affects the accuracy and real-time performance of the algorithm.
[0004] The document "Tracking in aerial hyperspectral videos using deep kernelized correlation filters" (published in the journal IEEE Transactions on Geoscience and Remote Sensing, 2018) published by Burak Uzkent et al. of Stanford University discloses an airborne hyperspectral video target tracking method based on deep kernelized correlation filtering. It designs a method to expand the search area of the tracker to deal with the problem of low temporal resolution, and then uses a pre-trained neural network to improve the tracking performance. In the tracking system, a deep ROI mapping strategy is used to only forward the ROI and project the entire ROI to the ROI response map, and then estimate the target position. The shortcomings of this method are: it is necessary to build a target sample library to train the deep convolutional network, the amount of calculation is large, and the tracking effect is not good when there are many background spots, that is, when there are too many similar objects. Summary of the invention
[0005] In order to solve the existing problems of low target feature distinction and low tracking real-time performance, the present invention provides a hyperspectral target tracking method based on four-feature fusion and weight coefficient, the method comprising:
[0006] S1 loads the first frame image of the hyperspectral image sequence and preprocesses the first frame image of the hyperspectral image sequence;
[0007] S2 loads the Tth frame image in the hyperspectral image sequence as the current frame image, where T is an integer greater than or equal to 2;
[0008] S3 performs a spectral dimension reduction operation on the current frame image to obtain the current frame image after spectral dimension reduction;
[0009] S4 extracts the HOG feature of the current frame image as the first feature, and then extracts three depth features of the current frame image after spectral dimension reduction as the second to fourth features;
[0010] S5 uses the first to fourth features and the kernel correlation filter tracker to calculate four weak response maps based on the first to fourth features respectively; and uses the four weak response maps to calculate the weight coefficients of the first to fourth features respectively;
[0011] S6 uses the weight coefficients of the first to fourth features to perform a weighted average operation on the four weak response maps to obtain a strong response map, and uses the maximum value position in the strong response map as the position of the target in the current frame image;
[0012] S7 adaptively updates the weight coefficient related parameters of the first to fourth features; during the adaptive updating process, if the weight coefficient of the fourth feature exceeds a preset threshold parameter, the weight coefficient of the fourth feature is reset;
[0013] S8 determines whether the current frame image is the last frame image of the hyperspectral image sequence. If so, the tracking is completed; if not, the T+1th frame image in the hyperspectral image sequence is loaded as the current frame image, and S3 to S8 are looped until the current frame image is the last frame image of the hyperspectral image sequence, and the target tracking is completed.
[0014] Optionally, the S1 includes:
[0015] 1.1 Read the first frame of the hyperspectral image sequence;
[0016] 1.2 Use a rectangular frame to frame the target image area to be tracked in the first frame of the hyperspectral image sequence, take the center position of the target image area to be tracked as the initial position of the target to be tracked, and take the size of the search area twice that of the target image area to be tracked as the size of the search area; take the difference between the search area and the target image area to be tracked as the background image area, and the search area is 200 random image blocks around the target image area;
[0017] 1.3 Set initialization parameters.
[0018] Optionally, the S3 includes:
[0019] 3.1 Perform spectral dimension reduction operation on the current frame image according to the following formula to obtain the current frame image after spectral dimension reduction:
[0020] D rj =|R tj -R bj |
[0021] Among them, D rj R represents the average spectral response difference of all pixels in the target image area to be tracked and the background image area in the jth spectral band. tj represents the average spectral response curve of all pixels in the target image area to be tracked in the jth spectral band, R bj represents the average spectral response curve of all pixels in the background image area in the jth spectral band, and |·| represents the absolute value operation;
[0022] D rj The spectral band corresponding to the maximum value is used as the current frame image after spectral dimension reduction.
[0023] Optionally, the S4 includes:
[0024] 4.1 Extract the HOG feature of the current frame image as the first feature;
[0025] 4.2 Use the VGG-19 network to perform deep feature extraction on the current frame image after spectral dimension reduction obtained by S3, and take the fourth layer of the third convolution group in the VGG-19 network as the second feature, the fourth layer of the fourth convolution group as the third feature, and the fourth layer of the fifth convolution group as the fourth feature.
[0026] Optionally, in S5, four weak response graphs based on the first to fourth features are calculated using the first to fourth features and the kernel correlation filter tracker, including:
[0027] 5.1 Perform cyclic shift on the basis sample based on the first feature and then obtain the high-order compression matrix X1;
[0028] 5.2 Diagonalize X1 according to the following formula:
[0029]
[0030] Among them, F(·) represents the Fourier transform operation, diag(·) represents the diagonalization operation, represents the Fourier transform of vector x1, F H It means to perform Fourier transform operation and then calculate Hermitian matrix operation;
[0031] 5.3 According to the following formula, calculate the regression coefficient ω1 of the kernel correlation filter classifier based on the base sample of the first feature:
[0032]
[0033] Among them, ω represents the regression coefficient of the kernel correlation filter classifier, and the subscripts are used to distinguish the corresponding features. -1 (·) represents the inverse Fourier transform operation, λ represents the regularization parameter with a size of 0.01, y1 represents the regression value, represents the Fourier transform of y1;
[0034] 5.4 According to the following formula, ω1 is mapped to the high-dimensional feature space:
[0035]
[0036] Among them, α1 represents the parameters of the kernel correlation filter classifier based on the target image area of the first feature, Represents the mapping operation; according to the following formula, using the circulant matrix calculate
[0037]
[0038] 5.5 Calculate the weak response graph R1 based on the first feature according to the following formula:
[0039]
[0040] Wherein, z1 represents the base sample of the target image area based on the first feature;
[0041] 5.6 Repeat steps 5.1 to 5.5 to calculate weak response graphs R2 to R4 based on the second to fourth features;
[0042] 5.5 According to the formula Calculate the weight coefficients w for the first to fourth features i ;
[0043] Where i represents the feature number, ranging from 1 to 4; R pci Represents the maximum value of the weak response map of the i-th feature in the current frame image, R pai Represents the maximum value of the weak response map of the i-th feature in the historical frame image.
[0044] Optionally, the S6 includes:
[0045] 6.1 Calculate the strong response graph Q according to the following formula:
[0046]
[0047] 6.2 Take the maximum value position in Q as the position of the target in the current frame image.
[0048] Optionally, the S7 includes:
[0049] According to the following formula, the parameter R in the weight coefficient calculation formula for the first to fourth features pai To perform adaptive updates:
[0050]
[0051] Among them, μ represents the update parameter.
[0052] Optionally, if the weight coefficient of the fourth feature exceeds a preset threshold parameter, resetting the weight coefficient of the fourth feature includes:
[0053] If the weight coefficient of the fourth feature exceeds the preset threshold parameter, the weight coefficient of the fourth feature is reset according to the following formula:
[0054]
[0055] Among them, v th Indicates a preset threshold parameter.
[0056] Optionally, after S7 and before S8, the step further includes updating the base sample using the weight coefficients of the first to fourth features after adaptive update:
[0057] Update α and z according to the following formula:
[0058]
[0059]
[0060]
[0061]
[0062] Among them, α i represents the parameters of the updated kernel correlation filter classifier based on the target image region of the i-th feature, Represents the parameters of the kernel correlation filter classifier of the target image region based on the i-th feature in the current frame image, represents the parameters of the kernel correlation filter classifier of the target image region based on the i-th feature in the historical frame image, z i represents the updated test base sample of the target image region based on the i-th feature, represents the test base sample of the target image area based on the i-th feature in the current frame image, Represents the base sample of the target image area based on the i-th feature in the historical frame image.
[0063] Optionally, update parameter μ = 0.98, threshold parameter v th =0.8.
[0064] The beneficial effects of the present invention are:
[0065] The present invention overcomes the shortcomings of the prior art in that the hyperspectral video of all bands is processed simultaneously, the amount of calculation is large, and the real-time performance is poor by using the hyperspectral image sequence of the selected channel after spectral dimension reduction and the hyperspectral image sequence of the fused channel after spectral dimension reduction, so that the present invention improves the speed of target tracking in the hyperspectral image sequence under complex background. Furthermore, by adopting the response map fusion mode and updating the weight coefficient, and further using the updated weight coefficient to update the base sample, the interference of the clutter background in the complex background is effectively overcome, and the shortcomings of the prior art that the target is easily affected by the occlusion and deformation, which causes the estimation error, and causes the tracking to be offset or failed are overcome, so that the present invention enhances the stability of the kernel correlation filter classifier, and can effectively track the target in the hyperspectral image sequence under complex background. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0067] Figure 1 The present invention is a flowchart of a method for tracking a hyperspectral target based on four features and updating weight coefficients provided in one embodiment of the present invention.
[0068] Figure 2 The present invention is a flowchart of a hyperspectral target tracking method based on four features, weight coefficient update and base sample update provided in one embodiment of the present invention.
[0069] Figure 3 This is a schematic diagram of the first frame image of a hyperspectral image sequence of a selected channel after spectral dimensionality reduction in one embodiment of the present invention.
[0070] Figure 4 This is a schematic diagram of the first frame image of a hyperspectral image sequence of channels after spectral dimension reduction in one embodiment of the present invention.
[0071] Figure 5 4 are four weak response diagrams based on the first to fourth characteristics respectively in one embodiment of the present invention.
[0072] Figure 6 This is a strong response diagram in one embodiment of the present invention.
[0073] Figure 7 Schematic diagram of the target position in the 10th frame image in a hyperspectral image sequence in one embodiment of the present invention.
[0074] Figure 8 It is a schematic diagram comparing the tracking effects of various tracking methods in the 25th frame image according to an embodiment of the present invention.
[0075] Fig. 9 It is a schematic diagram comparing the tracking effects of various tracking methods in the 615th frame image in one embodiment of the present invention. DETAILED DESCRIPTION
[0076] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
[0077] Embodiment 1:
[0078] This embodiment provides a hyperspectral target tracking method based on four-feature fusion and weight coefficients, the method comprising:
[0079] S1 loads the first frame image of the hyperspectral image sequence and preprocesses the first frame image of the hyperspectral image sequence;
[0080] S2 loads the Tth frame image in the hyperspectral image sequence as the current frame image, where T is an integer greater than or equal to 2;
[0081] S3 performs a spectral dimension reduction operation on the current frame image to obtain the current frame image after spectral dimension reduction;
[0082] S4 extracts the HOG feature of the current frame image as the first feature, and then extracts three depth features of the current frame image after spectral dimension reduction as the second to fourth features;
[0083] S5 uses the first to fourth features and the kernel correlation filter tracker to calculate four weak response maps based on the first to fourth features respectively; and uses the four weak response maps to calculate the weight coefficients of the first to fourth features respectively;
[0084] S6 uses the weight coefficients of the first to fourth features to perform a weighted average operation on the four weak response maps to obtain a strong response map, and uses the maximum value position in the strong response map as the position of the target in the current frame image;
[0085] S7 adaptively updates the weight coefficient related parameters of the first to fourth features; during the adaptive updating process, if the weight coefficient of the fourth feature exceeds a preset threshold parameter, the weight coefficient of the fourth feature is reset.
[0086] S8 determines whether the current frame image is the last frame image of the hyperspectral image sequence. If so, the tracking is completed; if not, the T+1th frame image in the hyperspectral image sequence is loaded as the current frame image, and S3 to S8 are looped until the current frame image is the last frame image of the hyperspectral image sequence, and the target tracking is completed.
[0087] Embodiment 2:
[0088] This embodiment provides a hyperspectral target tracking method based on four-feature fusion and weight coefficients. Figure 1 , the method comprising:
[0089] Step 1: Load the first frame image of the hyperspectral image sequence and preprocess the first frame image of the hyperspectral image sequence;
[0090] Step 101, read the first frame image of the hyperspectral image sequence;
[0091] Specifically, the hyperspectral image sequence of the embodiment of the present invention has 16 channels, so the size of the first frame image read in is M×N×16, where M×N is the scene size.
[0092] Step 102, using a rectangular frame to frame the target image area to be tracked in the first frame image of the hyperspectral image sequence, and using the target image area to be tracked as a base sample of the first frame image, using the centroid position of the target image area to be tracked as the initial position of the target to be tracked, selecting a search area around the target image area to be tracked, and using the difference between the search area and the target image area to be tracked as the background image area;
[0093] Specifically, the search area is an area where the target is estimated to appear, which is determined by the target size and the target moving speed. In the embodiment of the present invention, the size of the target and the possible moving speed of the target are comprehensively considered, and the search area is selected to be an area twice the area of the target area to be tracked.
[0094] Step 103, setting initialization parameters.
[0095] Specifically, the weight coefficients based on the first to fourth features are initialized to 1, the update rate is initialized to 0.98, and the threshold parameter is initialized to 0.8.
[0096] Step 2: Load the Tth frame image in the hyperspectral image sequence as the current frame image, and perform spectral dimension reduction operation on the current frame image to obtain the current frame image after spectral dimension reduction;
[0097] This is achieved through the following steps:
[0098] Step 201, loading the first frame image of the hyperspectral image sequence;
[0099] Step 202, according to the following formula, a spectral dimension reduction operation based on multidimensional scaling is performed on the first frame image of the hyperspectral image sequence to obtain a hyperspectral image sequence of the selected channel after spectral dimension reduction:
[0100] D rj =|R tj -R bj |
[0101] Among them, D rj R represents the average spectral response difference of all pixels in the target image area and the background image area in the jth spectral band. tj represents the average spectral response curve of all pixels in the target image area within the jth spectral band, R bj represents the average spectral response curve of all pixels in the background image area in the jth spectral band, and |·| represents the absolute value operation.
[0102] D rj The spectral band corresponding to the maximum value is used as the hyperspectral image sequence of the selected channel after spectral dimension reduction;
[0103] Specifically, Figure 2Shown is a schematic diagram of the first frame of a hyperspectral image sequence of a selected channel after spectral dimensionality reduction.
[0104] Step 3: Extract the HOG feature of the current frame image as the first feature, and then extract the three depth features of the current frame image after spectral dimension reduction as the second to fourth features;
[0105] This is achieved through the following steps:
[0106] Step 301, extracting the HOG feature of the current frame image as the first feature;
[0107] Step 302, using the VGG-19 network to perform deep feature extraction on the current frame image after spectral dimension reduction, and using the fourth layer of the third convolution group in the VGG-19 network as the second feature, the fourth layer of the fourth convolution group as the third feature, and the fourth layer of the fifth convolution group as the fourth feature;
[0108] Step 4: Using the first to fourth features and the kernel correlation filter tracker, four weak response maps based on the first to fourth features obtained in step 3 are calculated;
[0109] This is achieved through the following steps:
[0110] Step 401, performing cyclic shift on the basis sample based on the first feature to obtain a high-order compression matrix X1;
[0111] Step 402, diagonalize X1 according to the following formula:
[0112]
[0113] Among them, F(·) represents the Fourier transform operation, diag(·) represents the diagonalization operation, represents the Fourier transform of vector x1, F H It means to perform Fourier transform operation and then Hermitian matrix operation.
[0114] Step 403, calculate the regression coefficient ω1 of the kernel correlation filter classifier based on the base sample of the first feature according to the following formula:
[0115]
[0116] Among them, ω represents the regression coefficient of the kernel correlation filter classifier, and the subscripts are used to distinguish the corresponding features. -1 (·) represents the inverse Fourier transform operation, λ represents the regularization parameter with a size of 0.01, y1 represents the regression value, represents the Fourier transform of y1; Representation vector The Hermitian matrix of .
[0117] Step 404, map ω1 to a high-dimensional feature space according to the following formula:
[0118]
[0119] Among them, α1 represents the parameters of the kernel correlation filter classifier based on the target image area of the first feature, Represents a mapping operation;
[0120] Step 405, according to the following formula, using the circulant matrix calculate
[0121]
[0122] is the Fourier transform of α1.
[0123] Step 406, calculate the weak response graph R1 based on the first feature according to the following formula:
[0124]
[0125] Wherein, z1 represents the test base sample of the target image area based on the first feature;
[0126] Step 407, repeating steps (401) to (406), and calculating weak response graphs R2 to R4 based on the second to fourth features.
[0127] Specifically, Figure 4 Shown are four weak response diagrams based on the first to fourth characteristics of the present invention.
[0128] Step 5: Using four weak response graphs based on the first to fourth features, calculate the weight coefficients of the first to fourth features respectively:
[0129] Specifically: According to the formula Calculate the weight coefficients w for the first to fourth features i ; where i represents the feature number, ranging from 1 to 4, R pci Represents the maximum value of the weak response map of the i-th feature in the current frame image, R pai Represents the maximum value of the weak response map of the i-th feature in the historical frame image.
[0130] Step 6: Use the weight coefficients of the first to fourth features to perform weighted average operation on the four weak response maps based on the first to fourth features to obtain a strong response map, and use the maximum value position in the strong response map as the position of the target in the current frame image;
[0131] This is achieved through the following steps:
[0132] Step 601, calculate the strong response graph Q according to the following formula:
[0133]
[0134] Step 602, taking the maximum value position in Q as the target position.
[0135] Specifically, Figure 5 As shown in FIG. , it is a strong response diagram of the present invention. Figure 6 , which is a schematic diagram of the target position in the 10th frame image in the hyperspectral image sequence of the present invention.
[0136] Step 7: Adaptively update the weight coefficient related parameters of the first to fourth features;
[0137] Specifically: According to the following formula, the weight coefficient related parameters of the first to fourth features are adaptively updated
[0138]
[0139] Here, μ represents an update parameter with a size of 0.98.
[0140] Specifically, if the peak value of the response map of the current frame is higher than the peak value of the response map of the historical frame, it means that the feature matching degree of the current frame is higher and more conducive to target tracking, and it is used to replace the peak value of the response map of the historical frame; if the peak value of the response map of the current frame is lower than the peak value of the response map of the historical frame, it means that the matching degree of the current frame is reduced, which is not only caused by target factors, but also may be caused by background changes. Therefore, the peak value of the response map of the historical frame needs to be updated to make it more adaptable to the background situation of the current frame.
[0141] Step 8: Reset the weight coefficient of the fourth feature. Specifically, reset the weight coefficient of the fourth feature according to the following formula:
[0142]
[0143] Among them, v th represents a threshold parameter of size 0.8.
[0144] Specifically, the fourth feature is the deepest feature among the deep features. Due to the characteristics of the deep network, it has the largest receptive field. This results in that when the target changes dramatically, if the background does not change much, it can still maintain a large response peak, which will have a great impact during fusion and cause tracking failure. Therefore, a threshold is set in the present invention to weaken the influence of the fourth feature in low matching conditions and increase the influence of the other three features.
[0145] Step 9: Update the base sample using the weight coefficients of the first to fourth features after adaptive update, including:
[0146] Update α and z according to the following formula:
[0147]
[0148]
[0149]
[0150]
[0151] Among them, α i represents the parameters of the updated kernel correlation filter classifier based on the target image region of the i-th feature, Represents the parameters of the kernel correlation filter classifier of the target image region based on the i-th feature in the current frame image, represents the parameters of the kernel correlation filter classifier of the target image region based on the i-th feature in the historical frame image, z i represents the updated test base sample of the target image region based on the i-th feature, represents the test base sample of the target image area based on the i-th feature in the current frame image, Represents the base sample of the target image area based on the i-th feature in the historical frame image.
[0152] Step 10, determining whether the current frame image is the last frame image of the hyperspectral image sequence, if so, completing the tracking; if not, loading the T+1 frame image in the hyperspectral image sequence as the current frame image, and looping steps 3 to 10 until the current frame image is the last frame image of the hyperspectral image sequence, completing the target tracking.
[0153] Embodiment three:
[0154] This embodiment provides a hyperspectral target tracking method based on four features, weight coefficient update and base sample update. Figure 2 , the method comprising:
[0155] SS1 loads the first frame image of the hyperspectral image sequence and preprocesses the first frame image of the hyperspectral image sequence;
[0156] 1.1 Read the first frame of the hyperspectral image sequence;
[0157] 1.2 Use a rectangular frame to frame the target image area to be tracked in the first frame of the hyperspectral image sequence, and use the target image area to be tracked as the base sample of the first frame, use the centroid position of the target image area to be tracked as the initial position of the target to be tracked, use a size twice that of the target image area to be tracked as the search area size, use the difference between the search area and the target image area to be tracked as the background image area, and the search area is 200 random image blocks around the target image area;
[0158] 1.3 Set initialization parameters.
[0159] SS2 loads the Tth frame image in the hyperspectral image sequence as the current frame image, where T is an integer greater than or equal to 2;
[0160] SS3 performs a spectral dimension reduction operation on the current frame image to obtain the current frame image after spectral dimension reduction;
[0161] 3.1 According to the following formula, the spectral dimension reduction operation is performed on the current frame image to obtain the current frame image after spectral dimension reduction:
[0162] D rj =|R tj -R bj |
[0163] Among them, D rj R represents the average spectral response difference of all pixels in the target image area and the background image area in the jth spectral band. tj represents the average spectral response curve of all pixels in the target image area within the jth spectral band, R bj represents the average spectral response curve of all pixels in the background image area in the jth spectral band, and |·| represents the absolute value operation.
[0164] 3.2 D rj The spectral band corresponding to the maximum value is used as the current frame image after spectral dimension reduction.
[0165] SS4 extracts the HOG feature of the current frame image as the first feature, and then extracts the three depth features of the current frame image after spectral dimension reduction as the second to fourth features;
[0166] 4.1 Extract the HOG feature of the current frame image as the first feature;
[0167] 4.2 Use the VGG-19 network to extract deep features from the current frame image after spectral dimension reduction, and use the fourth layer of the third convolution group in the VGG-19 network as the second feature, the fourth layer of the fourth convolution group as the third feature, and the fourth feature of the fifth convolution group as the fourth feature.
[0168] SS5 uses the first to fourth features and the kernel correlation filter tracker based on the basis sample update to calculate four weak response maps based on the first to fourth features;
[0169] 5.1 Perform cyclic shift on the basis sample based on the first feature and then obtain the high-order compression matrix X1;
[0170] 5.2 Diagonalize X1 according to the following formula:
[0171]
[0172] Among them, F(·) represents the Fourier transform operation, diag(·) represents the diagonalization operation, represents the Fourier transform of vector x1, H(·) represents the operation of finding the Hermitian matrix;
[0173] 5.3 According to the following formula, calculate the regression coefficient ω1 of the kernel correlation filter classifier based on the base sample of the first feature:
[0174]
[0175] Among them, ω represents the regression coefficient of the kernel correlation filter classifier, and the subscripts are used to distinguish the corresponding features. -1 (·) represents the inverse Fourier transform operation, λ represents the regularization parameter with a size of 0.01, y1 represents the regression value, represents the Fourier transform of y1;
[0176] 5.5 According to the following formula, ω1 is mapped to the high-dimensional feature space:
[0177]
[0178] Among them, α1 represents the parameters of the kernel correlation filter classifier based on the target image area of the first feature, Represents a mapping operation;
[0179] 5.6 According to the following formula, using the circulant matrix calculate
[0180]
[0181] is the Fourier transform of α1.
[0182] 5.7 Calculate the weak response graph R1 based on the first feature according to the following formula:
[0183]
[0184] Wherein, z1 represents the base sample of the target image area based on the first feature;
[0185] 5.8 Repeat steps 5.1 to 5.7 to calculate weak response graphs R2 to R4 based on the second to fourth features.
[0186] SS6 uses four weak response graphs based on the first to fourth features to calculate the weight coefficients of the first to fourth features respectively;
[0187] According to the formula Calculate the weight coefficients w for the first to fourth features i ; where i represents the feature number, which is an integer from 1 to 4; R pci Represents the maximum value of the weak response map of the i-th feature in the current frame, R pai Represents the maximum value of the weak response map of the i-th feature in the historical frame.
[0188] SS7 uses the weight coefficients of the first to fourth features to perform weighted average operation on the four weak response maps based on the first to fourth features to obtain a strong response map, and uses the maximum value position in the strong response map as the position of the target in the current frame image;
[0189] 7.1 Calculate the strong response diagram Q according to the following formula:
[0190]
[0191] 7.2 Take the maximum value position in Q as the position of the target in the current frame image.
[0192] SS8 adaptively updates the weight coefficient related parameters of the first to fourth features; resets the weight coefficient of the fourth feature;
[0193] According to the following formula, the parameter R in the weight coefficient calculation formula for the first to fourth features pai Perform adaptive updates
[0194]
[0195] Among them, μ represents the update parameter with a size of 0.98;
[0196] During the adaptive update process, if the weight coefficient of the fourth feature exceeds a predetermined threshold parameter, the weight coefficient of the fourth feature is reset. The weight coefficient of the fourth feature is reset according to the following formula:
[0197]
[0198] Among them, v th represents a threshold parameter of size 0.8.
[0199] SS9 updates the base sample using the weight coefficients of the first to fourth features;.
[0200] Update α and z according to the following formula:
[0201]
[0202]
[0203]
[0204]
[0205] Among them, α i represents the parameters of the updated kernel correlation filter classifier based on the target image region of the i-th feature, Represents the parameters of the kernel correlation filter classifier of the target image region based on the i-th feature in the current frame image, represents the parameters of the kernel correlation filter classifier of the target image region based on the i-th feature in the historical frame image, z i represents the updated test base sample of the target image region based on the i-th feature, represents the test base sample of the target image area based on the i-th feature in the current frame image, Represents the test base samples of the target image region based on the i-th feature in the historical frame image.
[0206] SS10 determines whether the current frame image is the last frame image of the hyperspectral image sequence. If so, tracking is completed; if not, the T+1th frame image in the hyperspectral image sequence is loaded as the current frame image, and S3 to S10 are looped until the current frame image is the last frame image of the hyperspectral image sequence, and target tracking is completed.
[0207] Figure 8-9 The comparison diagrams of the tracking effects of the method of the present application, the method based on convolutional features and the method based on spectral features in the 25th frame and the 615th frame images are respectively selected; Figure 8 It can be seen that in the 25th frame image, relative to the tracking frame of the method of the present application, the tracking frames of the method based on convolution features and the method based on spectral features have deviated from the tracking target. By the 615th frame image, the tracking frames of the method based on convolution features and the method based on spectral features have completely deviated from the tracking target. This is because the method of the present application adopts the response map fusion method and updates the weight coefficient, and further uses the updated weight coefficient to update the base sample, thereby effectively overcoming the interference of the clutter background in the complex background, avoiding tracking deviation, and improving the tracking accuracy.
[0208] For an introduction to methods based on convolutional features, please refer to "Fully-Convolutional Siamese Networks for Object Tracking" by Luca Bertinetto et al. from Harvard University.
[0209] For an introduction to spectral feature-based methods, please refer to the paper Tracking in aerial hyperspectral videos using deep kernelized correlation filters by Burak Uzkent et al. from Stanford University.
[0210] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.
[0211] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A hyperspectral target tracking method based on four-feature fusion and weight coefficient, characterized in that: The method comprises: S1 loads the first frame image of the hyperspectral image sequence and preprocesses the first frame image of the hyperspectral image sequence; S2 loads the Tth frame image in the hyperspectral image sequence as the current frame image, where T is an integer greater than or equal to 2; S3 performs a spectral dimension reduction operation on the current frame image to obtain the current frame image after spectral dimension reduction; S4 extracts the HOG feature of the current frame image as the first feature, and then extracts three depth features of the current frame image after spectral dimension reduction as the second to fourth features; S5 uses the first to fourth features and the kernel correlation filter tracker to calculate four weak response maps based on the first to fourth features respectively; and uses the four weak response maps to calculate the weight coefficients of the first to fourth features respectively; S6 uses the weight coefficients of the first to fourth features to perform a weighted average operation on the four weak response maps to obtain a strong response map, and uses the maximum value position in the strong response map as the position of the target in the current frame image; S7 adaptively updates the weight coefficient related parameters of the first to fourth features; during the adaptive updating process, if the weight coefficient of the fourth feature exceeds a preset threshold parameter, the weight coefficient of the fourth feature is reset; S8 determines whether the current frame image is the last frame image of the hyperspectral image sequence, and if so, completes the tracking; if not, loads the T+1th frame image in the hyperspectral image sequence as the current frame image, and loops from S3 to S8 until the current frame image is the last frame image of the hyperspectral image sequence, and completes the target tracking; The S5 includes: 5.1 Perform cyclic shift on the basis sample based on the first feature and then obtain the high-order compression matrix X1; 5.2 Diagonalize the high-order compressed matrix X1 according to the following formula: in, F(·) indicates Fourier transform operation, diag(·) indicates diagonalization operation, represents the Fourier transform of vector x1, F H It means to perform Fourier transform operation and then calculate Hermitian matrix operation; 5.3 According to the following formula, calculate the regression coefficient ω1 of the kernel correlation filter classifier based on the base sample of the first feature: in, ω represents the regression coefficient of the kernel correlation filter classifier, and the subscripts are used to distinguish the corresponding features. -1 (·) represents the inverse Fourier transform operation, λ represents the regularization parameter with a size of 0.01, y1 represents the regression value, represents the Fourier transform of y1; 5.4 According to the following formula, ω1 is mapped to the high-dimensional feature space: in, α1 represents the parameters of the kernel correlation filter classifier based on the target image region of the first feature, Represents the mapping operation; according to the following formula, using the circulant matrix calculate 5.5 Calculate the weak response graph R1 based on the first feature according to the following formula: in, z1 represents the base sample of the target image area based on the first feature; 5.6 Repeat steps 5.1 to 5.5 to calculate weak response graphs R2 to R4 based on the second to fourth features; 5.7 According to the formula Calculate the weight coefficients w for the first to fourth features i ; Where i represents the feature number, ranging from 1 to 4; R pci Represents the maximum value of the weak response map of the i-th feature in the current frame image, R pai Represents the maximum value of the weak response map of the i-th feature in the historical frame image.
2. The method according to claim 1, characterized in that S1 comprises: 1.1 Read the first frame of the hyperspectral image sequence; 1.2 Use a rectangular frame to frame the target image area to be tracked in the first frame of the hyperspectral image sequence, take the center position of the target image area to be tracked as the initial position of the target to be tracked, and take the size of the search area twice that of the target image area to be tracked as the size of the search area; take the difference between the search area and the target image area to be tracked as the background image area, and the search area is 200 random image blocks around the target image area; 1.3 Set initialization parameters.
3. The method according to claim 2, characterized in that S3 comprises: 3.1 Perform spectral dimension reduction operation on the current frame image according to the following formula to obtain the current frame image after spectral dimension reduction: D rj =|R tj -R bj | Among them, D rj R represents the average spectral response difference of all pixels in the target image area to be tracked and the background image area in the jth spectral band. tj represents the average spectral response curve of all pixels in the target image area to be tracked in the jth spectral band, R bj represents the average spectral response curve of all pixels in the background image area in the jth spectral band, and |·| represents the absolute value operation; D rj The spectral band corresponding to the maximum value is used as the current frame image after spectral dimension reduction.
4. The method according to claim 3, characterized in that said S4 comprises: 4.1 Extract the HOG feature of the current frame image as the first feature; 4.2 Use the VGG-19 network to perform deep feature extraction on the current frame image after spectral dimension reduction obtained by S3, and take the fourth layer of the third convolution group in the VGG-19 network as the second feature, the fourth layer of the fourth convolution group as the third feature, and the fourth layer of the fifth convolution group as the fourth feature.
5. The method according to claim 4, characterized in that said S6 comprises: 6.1 Calculate the strong response graph Q according to the following formula: 6.2 Take the maximum value position in Q as the position of the target in the current frame image.
6. The method according to claim 5, characterized in that The S7 includes: According to the following formula, the parameter R in the weight coefficient calculation formula for the first to fourth features pai To perform adaptive updates: Among them, μ represents the update parameter.
7. The method according to claim 6, characterized in that If the weight coefficient of the fourth feature exceeds a preset threshold parameter, resetting the weight coefficient of the fourth feature includes: If the weight coefficient of the fourth feature exceeds the preset threshold parameter, the weight coefficient of the fourth feature is reset according to the following formula: Among them, v th Indicates a preset threshold parameter.
8. The method according to claim 7, characterized in that After S7 and before S8, the method further includes updating the base sample using the weight coefficients of the first to fourth features after adaptive update: Update α and z according to the following formula: Among them, α i represents the parameters of the updated kernel correlation filter classifier based on the target image region of the i-th feature, Represents the parameters of the kernel correlation filter classifier of the target image region based on the i-th feature in the current frame image, represents the parameters of the kernel correlation filter classifier of the target image region based on the i-th feature in the historical frame image, z i represents the updated test base sample of the target image region based on the i-th feature, represents the test base sample of the target image area based on the i-th feature in the current frame image, Represents the base sample of the target image area based on the i-th feature in the historical frame image.
9. The method according to claim 8, characterized in that Update parameter μ = 0.98, threshold parameter v th =0.8.
