Hyperspectral image target tracking method based on adaptive weight and confidence

Abstract

The invention discloses a hyperspectral image target tracking method based on adaptive weight and confidence, obtains the first frame, the t frame and the adjacent frame hyperspectral images after normalization, and then reduces the image dimension by principal component analysis to obtain three single-band grayscale images Z1, Z2 and Z3 respectively. t #imgabs0#According to the dual twin network, the first frame and the adjacent frame are used as template images, and the tth frame is used as the search image. The deep features of their reduced-dimensional images are extracted respectively, and the features of the template image and the search image are simply correlated to obtain the initial response map R1 and the adjacent response map #imgabs1#. The two response maps are fused using adaptive weights w1 and w2 to obtain the fused response map #imgabs2#. The confidence determination and scale estimation are performed respectively to obtain the tracking target of the current frame hyperspectral image, and then the confidence determination is used to determine whether to let the adjacent frame hyperspectral image of the tth frame update the tth frame hyperspectral image.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a hyperspectral image target tracking method based on adaptive weight and confidence. Background Art

[0002] Currently in the field of computer vision, the use of hyperspectral video tracking is still an emerging field. Hyperspectral target tracking is widely used in hot areas such as precision guidance in the military, planetary geology and soil research in space research, and earthquake prevention and search and rescue.

[0003] However, most current target tracking research uses visible light video as data, and target tracking under visible light conditions is often difficult to distinguish because the target and background have similar colors, especially when affected by lighting changes, causing the tracked target to drift.

[0004] Hyperspectral video target tracking is based on the hyperspectral camera capturing the spectral reflectance of each object in different bands, so as to accurately locate the target. Even if the colors are similar, targets of different materials can be easily identified based on the spectral information in different bands. Therefore, hyperspectral video target tracking has gradually attracted people's attention.

[0005] There are currently two mainstream tracking frameworks, one is based on the correlation filtering framework, and the other is based on the deep learning framework of the twin network.

[0006] A typical filter-based target tracking algorithm is a context-aware correlation filter tracking algorithm, which extracts directional gradient features, incorporates context information into correlation filters, and uses the target context area as a negative sample for filter training to improve tracking accuracy. However, in a complex background, the directional gradient features are difficult to distinguish between the target and the background. When the lighting changes, the selected context area does not fully sample the background area, which is accidental in suppressing the background. Using the same suppression weight when suppressing background clutter is prone to drift.

[0007] The target tracking algorithm based on deep learning is typically based on a fully convolutional twin network. This algorithm uses a similarity comparison method to compare the similarity between the target frame and the template image, and gives the position with the greatest similarity, which is the position of the target in the target frame. The algorithm based on deep learning does not utilize spectral information, resulting in the inability of the above algorithm to track the target well when the target is very similar to the background. This method cannot change the scale of the target area when facing changes in illumination, and is prone to losing the target. Summary of the invention

[0008] In view of this, the main purpose of the present invention is to provide a hyperspectral image target tracking method based on adaptive weight and confidence.

[0009] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0010] The embodiment of the present invention provides a hyperspectral image target tracking method based on adaptive weight and confidence, the method is:

[0011] Step 1: Load the first frame of hyperspectral image, the tth frame of hyperspectral image and the adjacent frame of hyperspectral image of the tth frame in the hyperspectral image sequence, and perform grayscale normalization on them respectively to obtain the normalized first frame of hyperspectral image X1 and the normalized tth frame of hyperspectral image X2. t And the normalized hyperspectral image of the adjacent frame of the tth frame Where t represents the number of hyperspectral images in the hyperspectral image sequence, t is an integer greater than 2, a represents the number of adjacent hyperspectral images, and the value of a is t-1. t and The grayscale range of each pixel is a left-closed and right-closed interval from 0 to 1. t and The number of channels is 16;

[0012] Step 2: Use principal component analysis to analyze X1 and X t and Dimensionality reduction is performed to obtain the first frame of hyperspectral image Z1 after normalization and the tth frame of hyperspectral image Z after normalization and dimension reduction. t And the normalized and dimension-reduced hyperspectral image of the tth frame adjacent to the frame Among them, Z1, Z t and The number of channels is 1;

[0013] Step 3: Manually select the target area I1 of the first frame from Z1; t The centroid of the target area in the t-1th frame is the search area I in the tth frame. t The centroid coordinates of the t-1st frame are twice the size of the target area as the search area I of the tth frame. t Dimensions, from Z t Automatically select the search area I of the tth frame t ;exist The centroid of the target region in the t-1th frame is the target region in the adjacent frame of the tth frame. The centroid coordinates of the target area of ​​the t-1th frame are taken as the target area of ​​the adjacent frame of the tth frame. The size, from Automatically select the target area of ​​the adjacent frame of the tth frame The size of I1 is m×n pixels, where m represents the number of rows and n represents the number of columns. tThe size of is 2m×2n pixels, The size of is m×n pixels;

[0014] Step 4: Copy I1 three times to obtain the target area of ​​the first frame of the three channels Will I t Copy three times to obtain the search area of ​​the tth frame of the three channels Will Copy three times to obtain the target area of ​​the adjacent frame of the tth frame of the three channels

[0015] Step 5: Construct a dual twin network framework, which includes the first twin network and the second twin network. Set the input of the first twin network to and In the first twin network and The similarity function is Set the input of the second twin network to and In the second twin network and The similarity function is

[0016] Step 6: Extract separately through AlexNet network The deep feature E1 is extracted The deep features E t and extract The deep features

[0017] Step 7: Through E1 and E t Perform a naive correlation operation to obtain the initial response graph R1, and then With E t Perform naive correlation operations to obtain adjacent response graphs

[0018] Step 8. Determine the peak sidelobe ratio F of R1 p (R1) and Peak sidelobe ratio

[0019] Step 9: Through F p (R1) Determine the adaptive weight w of R1 i , and through and the adjacent frame confidence F c (t-1) Determine The adaptive weight w a ;

[0020] Step 10: Determine w by normalization operation i The normalized result w1 determines w a The normalized result w2;

[0021] Step 11: Through w1, R1, w2 and Determine the fusion response map

[0022] Step 12: Pass Determine the current frame reference value v t ;

[0023] Step 13: Update the adjacent frame confidence F c (t-1);

[0024] Step 14: Pass The scale estimation module realizes hyperspectral image target tracking and outputs the centroid coordinates of the hyperspectral image target and the size of the target frame in the current frame;

[0025] Step 15: Update the current frame confidence F c (t);

[0026] Step 16: Update X t ;

[0027] Step 17: read each frame of the hyperspectral image in the hyperspectral image sequence in turn, repeat steps 1 to 16, and complete the target tracking of the hyperspectral image sequence.

[0028] In the above scheme, the step 5 is specifically implemented by the following steps:

[0029] (501) Setting the convolution embedding function φ(·);

[0030] (502) Set the input of the first twin network to and

[0031] (503) Determine the first twin network and The similarity function is

[0032] Among them, b1 represents the random signal of the first twin network, b1 belongs to the real number domain, and * represents the related operation;

[0033] (504) Set the input of the second twin network to and

[0034] (505) Determine the second twin network and The similarity function is

[0035] Among them, b2 represents the random signal of the second twin network, and b2 belongs to the real number domain.

[0036] In the above scheme, the step eight is specifically implemented by the following steps:

[0037] (801) Determine the maximum value max(R1) of R1 and The maximum value of

[0038] (802) Determine the mean value mean(R1) of R1 and The average

[0039] (803) Determine the standard deviation std(R1) of R1 and Standard Deviation

[0040] (804) Determine the peak sidelobe ratio F of R1 p (R1) is

[0041] (805) OK Peak sidelobe ratio for Among them, max(·) is the maximum value operation, mean(·) is the mean operation, and std(·) is the standard deviation operation.

[0042] In the above scheme, the step nine is specifically implemented by the following steps:

[0043] (901) by F p (R1) Determine the adaptive weight w of R1 i for

[0044] (902) Passed and the adjacent frame confidence F c (t-1) Determine The adaptive weight w a for

[0045] In the above scheme, the step ten is specifically implemented by the following steps:

[0046] (1001) Determine w i The normalized result w1 is

[0047] (1002) Determine w a The normalized result w2 is

[0048] In the above scheme, the step eleven is specifically: through w1, R1, w2 and Determine the fusion response map for

[0049] In the above scheme, the step twelve is specifically implemented by the following steps:

[0050] (1201) OK Maximum

[0051] (1202) Setting coefficient μ;

[0052] (1203) Passed Determine the current frame reference value v t for

[0053] Among them, v t-1 Represents the reference value of the previous frame, v1 is 0.5, and μ is set to 0.05.

[0054] In the above scheme, the step thirteen is specifically implemented by the following steps:

[0055] (1301) Determine the fusion response map of the t-1th frame Maximum

[0056] (1302) Determine whether t is equal to 2. If t is equal to 2, F c (t-1) is initialized to 1, that is, F c (1)=1, if t is not equal to 2, execute step (1303);

[0057] (1303) Passed and v t-1 Update the adjacent frame confidence F c (t-1) is

[0058] In the above scheme, the step fifteen is specifically: and v t Update the current frame confidence F c (t) is

[0059] In the above scheme, the step sixteen is specifically implemented by the following steps:

[0060] (1601) Setting update threshold T;

[0061] (1602) by Fc (t) and T to update X t : Among them, T is set to 0.7.

[0062] Compared with the prior art, the present invention adopts a dual twin network as a framework in the feature extraction module, designs AlexNet as the backbone network for deep learning, and introduces adjacent templates based on the original template of the fully convolutional twin network; in the matching module, the two response maps are fused through adaptive weights to overcome the target loss when the illumination changes; finally, in the regression module, confidence judgment is used to update the hyperspectral image, which corrects the target drift while suppressing background clutter, thereby increasing the fault tolerance of the target tracking algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a flow chart of the present invention;

[0064] Figure 2 The first frame of the normalized basketball court hyperspectral image X1 in the embodiment of the present invention;

[0065] Figure 3 : is the normalized 70th frame of the basketball court hyperspectral image X in the embodiment of the present invention. 70 ;

[0066] Figure 4 This is a normalized hyperspectral image of the 70th frame of the basketball court adjacent to the frame in the embodiment of the present invention.

[0067] Figure 5 In the embodiment of the present invention, X1 is reduced in dimension to obtain the first frame of hyperspectral image Z1 after normalization and dimension reduction;

[0068] Figure 6 In the embodiment of the present invention, X 70 Dimensionality reduction is used to obtain the 70th frame of hyperspectral image Z after normalization and dimensionality reduction. 70 ;

[0069] Figure 7 In the embodiment of the present invention, The 70th adjacent frame hyperspectral image after normalization and dimension reduction

[0070] Figure 8 is the first frame target area I1 manually selected in Z1 in the embodiment of the present invention;

[0071] Fig. 9 In the embodiment of the present invention, Z 70 The 70th frame search area I is automatically selected 70 ;

[0072] Fig.10 In the embodiment of the present invention, The target area of ​​the 70th frame adjacent to the frame automatically selected

[0073] Fig.11 is an initial response graph R1 obtained from the first frame of the normalized basketball court hyperspectral image in an embodiment of the present invention;

[0074] Fig.12 This is an adjacent response map obtained from the 70th frame of the basketball court hyperspectral image after normalization in an embodiment of the present invention.

[0075] Fig.13 This is a fusion response image obtained from the 70th frame of the basketball court hyperspectral image after normalization in an embodiment of the present invention.

[0076] Fig.14 The tracking target of the current frame hyperspectral image obtained by normalizing the 70th frame of the basketball court hyperspectral image in the embodiment of the present invention; Specific implementation plan

[0077] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation described here is only used to solve the present invention and is not used to limit the present invention.

[0078] The embodiment of the present invention provides a hyperspectral image target tracking method based on adaptive weight and confidence, such as Figure 1 As shown, the method is implemented by the following steps:

[0079] Step 1: Load the first frame of hyperspectral image, the tth frame of hyperspectral image and the adjacent frame of hyperspectral image of the tth frame in the hyperspectral image sequence, and perform grayscale normalization on them respectively to obtain the normalized first frame of hyperspectral image X1 and the normalized tth frame of hyperspectral image X t And the normalized hyperspectral image of the adjacent frame of the tth frame

[0080] Where t represents the number of hyperspectral images in the hyperspectral image sequence, t is an integer greater than 2, a represents the number of adjacent hyperspectral images, and the value of a is t-1. 70 and The grayscale range of each pixel is a left-closed and right-closed interval from 0 to 1. 70 and The number of channels is 16;

[0081] Specifically, the first hyperspectral image, the 70th hyperspectral image, and the adjacent hyperspectral image of the 70th frame in the hyperspectral image sequence are loaded, and grayscale normalization is performed on them respectively to obtain the normalized first hyperspectral image X1 and the normalized 70th hyperspectral image X 70 And the normalized 70th adjacent frame hyperspectral image There are 318 frames in total in the loaded hyperspectral image sequence, and the resolution of the hyperspectral image is 423×211 pixels. Figure 2 In the embodiment of the present invention, the first frame of the normalized basketball court hyperspectral image X1 is Figure 3 In the embodiment of the present invention, the normalized 70th frame of the basketball court hyperspectral image X 70 , Figure 4 This is a normalized hyperspectral image of the 70th frame of the basketball court adjacent to the frame in the embodiment of the present invention. During the entire sequence, people move freely on the basketball court. The basketball is blocked by the basketball hoop and people's fingers during movement, and the basketball is also deformed in the direction of shooting.

[0082] Step 2: Use principal component analysis to transform X1, X t and Dimensionality reduction is performed to obtain the first frame of hyperspectral image Z1 after normalization and dimension reduction, and the tth frame of hyperspectral image Z after normalization and dimension reduction t And the normalized and dimension-reduced hyperspectral image of the tth frame adjacent to the frame

[0083] Among them, Z1, Z t and The number of channels is 1;

[0084] Specifically, the principal component analysis transform maps high-dimensional data to low-dimensional space through a certain mathematical model. The transformed variables will no longer be correlated in the new coordinate space. The set of uncorrelated variables at this time is the principal component. Then the principal components obtained are arranged in descending order according to the signal-to-noise ratio. The first principal component has the most information, and the information stored in the remaining principal components will decrease as the sequence number increases. In this way, a small number of principal components can represent a large amount of original data; most of the image information obtained by principal component analysis can be stored in a small number of principal components Z1 with a small sequence number, and X 70 Most of the image information obtained can be stored in a small number of principal components Z 70 within, will Most of the image information can be stored in a small number of principal components In this paper, not only the data dimension is reduced, but also the information of the original image is kept as much as possible. However, in the field of single target tracking, most of the useful information is on the tracked target, while principal component analysis simply retains the useful information of the original image. Figure 5 In the embodiment of the present invention, X1 is reduced in dimension to obtain the first frame of hyperspectral image Z1 after normalization and dimension reduction, Figure 6 In the embodiment of the present invention, X 70 Dimensionality reduction is used to obtain the 70th frame of hyperspectral image Z after normalization and dimensionality reduction. 70 , Figure 7 In the embodiment of the present invention, The 70th adjacent frame hyperspectral image after normalization and dimension reduction

[0085] Step 3: Manually select the target area I1 of the first frame from Z1; t The centroid of the target area in the t-1th frame is the search area I in the tth frame. t The centroid coordinates of the t-1st frame are twice the size of the target area as the search area I of the tth frame. t Dimensions, from Z t Automatically select the search area I of the tth frame t ;from The centroid of the target region in the t-1th frame is the target region in the adjacent frame of the tth frame. The centroid coordinates of the target area of ​​the t-1th frame are taken as the target area of ​​the adjacent frame of the tth frame. The size, from Automatically select the target area of ​​the adjacent frame of the tth frame

[0086] The size of I1 is m×n pixels, where m represents the number of rows and n represents the number of columns. t The size of is 2m×2n pixels, The size of is m×n pixels;

[0087] Specifically, the target position centroid coordinates of the first frame single-band image Z1 after dimensionality reduction in the hyperspectral image sequence are (64, 59). A 12×12 pixel image block is manually selected on Z1 with the first frame target area centroid as the target area I1 of the first frame hyperspectral image after dimensionality reduction. 69 The target position centroid coordinates are (60, 54), at Z 70 The 69th frame Z 69 The centroid of the target area is the 70th frame Z 70 Target Area I 70 The centroid coordinates of the Z 70A 24×24 pixel image block is automatically selected as the search area I of the 70th frame of the hyperspectral image after dimensionality reduction 70 , in Z 70 The 69th frame Z 69 The centroid of the target area is the 70th frame Z 70 Adjacent frame target area The centroid coordinates of the Z 70 A 12×12 pixel image block is automatically selected as the adjacent frame target area of ​​the 70th frame of the hyperspectral image after dimensionality reduction. Figure 8 is the first frame target area I1 manually selected in Z1 in the embodiment of the present invention, Fig. 9 In the embodiment of the present invention, Z 70 The 70th frame search area I is automatically selected 70 , Fig.10 In the embodiment of the present invention, The target area of ​​the 70th frame adjacent to the frame automatically selected

[0088] Step 4: Copy I1 three times to obtain the target area of ​​the first frame of the three channels Will I t Copy three times to obtain the search area of ​​the tth frame of the three channels Will Copy three times to obtain the target area of ​​the adjacent frame of the tth frame of the three channels

[0089] Specifically, The size is 12×12×3, 12×12 The spatial resolution of The number of channels, The size is 24×24×3, 24×24 is The spatial resolution of The number of channels, The size is 12×12×3, 12×12 The spatial resolution of The number of channels.

[0090] Step 5: Construct a dual twin network framework, which includes the first twin network and the second twin network. Set the input of the first twin network to and In the first twin network and The similarity function is Set the input of the second twin network to and In the second twin network and The similarity function is

[0091] Specifically, (501) sets the convolution embedding function φ(·);

[0092] Specifically, the convolution embedding function φ(·) is a convolution layer containing multiple convolution kernels, which is used to extract the features of the image.

[0093] (502) Set the input of the first twin network to and

[0094] Specifically, the twin network is a prior art. The twin network framework used in the present invention is used to obtain the similarity scores of two images. The input of the first twin network is The image resolution is 12×12 pixels, and the input of the first Siamese network is The image resolution is 24×24 pixels.

[0095] (503) Determine the first twin network and The similarity function is f1(I1,I t )

[0096]

[0097] Among them, b1 represents the random signal of the first twin network, b1 belongs to the real number domain, and * represents the related operation;

[0098] Specifically, To extract the input image The feature of E1 in the present invention is a depth feature of size 6×6×128, where 6×6 represents the spatial size and 128 represents the number of channels. To extract the input image The characteristics of the present invention are 70 The size of the deep feature is 22×22×128, where 22×22 represents the spatial size, 128 is the number of channels, the size of b1 is 10, and the output is a similarity score map between two images, The image resolution is 17×17.

[0099] (504) Set the input of the second twin network to and

[0100] Specifically, the twin network is a prior art. The twin network framework used in the present invention is used to obtain the similarity scores of the two images. The input of the second twin network is The image resolution is 12×12 pixels, and the input of the second Siamese network is The image resolution is 24×24 pixels.

[0101] (505) Determine the second twin network and The similarity function is

[0102]

[0103] Among them, b2 represents the random signal of the second twin network, and b2 belongs to the real number domain.

[0104] Specifically, To extract the input image The present invention The size of the deep features is 6×6×128, where 6×6 represents the spatial size and 128 is the number of channels. To extract the input image The characteristics of the present invention are 70 The size of the deep feature is 22×22×128, where 22×22 represents the spatial size, 128 is the number of channels, the size of b1 is 10, and the output is a similarity score map between two images, The image resolution is 17×17.

[0105] Step 6: Extract separately through AlexNet network The deep feature E1 is extracted The deep features E t and extract The deep features

[0106] Specifically, the AlexNet network is a prior art. The convolutional layer of AlexNet used in the present invention is used to extract deep features. The size of E1 in the present invention is 6×6×128 deep features, where 6×6 represents the spatial size and 128 represents the number of channels. 70 The size of the deep feature is 22×22×128, where 22×22 represents the spatial size and 128 is the number of channels. The size of the deep features is 6×6×128, where 6×6 represents the spatial size and 128 is the number of channels.

[0107] Step 7: Through E1 and E t Perform a naive correlation operation to obtain the initial response graph R1, and then With Et Perform naive correlation operations to obtain adjacent response graphs

[0108] Specifically, the specific method for determining the naive correlation in the present invention is to multiply the size of E1 of 6×6×128 by 1 in the size of E2 of 22×22×128. 70 Slide up, each time you slide, the corresponding elements are multiplied and added to get a grayscale value. 70 After all the sliding is completed, a complete initial response graph R1 can be obtained. In the present invention, the image resolution of R1 is 17×17×1, and the size of 6×6×128 With a step size of 1, the size of E is 22×22×128. 70 Slide up, each time you slide, the corresponding elements are multiplied and added to get a grayscale value. In the deep feature E 70 When all sliding is completed, a complete adjacent response graph can be obtained. In the present invention The image resolution is 17×17×1, Fig.11 is the initial response graph R1 obtained from the first frame of the normalized basketball court hyperspectral image in the embodiment of the present invention, Fig.12 This is an adjacent response map obtained from the 70th frame of the basketball court hyperspectral image after normalization in an embodiment of the present invention.

[0109] Step 8. Determine the peak sidelobe ratio F of R1 p (R1), then confirm Peak sidelobe ratio

[0110] (801) Determine the maximum value of R1 max(R1), and then determine The maximum value of

[0111] Specifically, in the embodiment of the present invention, the value of max(R1) of the first frame of the basketball court hyperspectral image is 0.92, and the value of max(R2) of the 70th frame of the basketball court hyperspectral image is 0.92. The value of is 0.89.

[0112] (802) Determine the mean value mean(R1) of R1, and then determine The average

[0113] Specifically, in the embodiment of the present invention, the value of mean(R1) of the first frame of the basketball court hyperspectral image is 0.39, and the value of mean(R2) of the 70th frame of the basketball court hyperspectral image is 0.39. The value of is 0.42.

[0114] (803) Determine the standard deviation std(R1) of R1, and then determine Standard Deviation

[0115] Specifically, in the embodiment of the present invention, the value of std(R1) of the first frame of the basketball court hyperspectral image is 1.63, and the value of std(R2) of the 70th frame of the basketball court hyperspectral image is 1.63. The value of is 0.94.

[0116] (804) Determine the peak sidelobe ratio F of R1 p (R1) is

[0117]

[0118] Specifically, in the embodiment of the present invention, the peak sidelobe ratio F of the first frame of the basketball court hyperspectral image is p The value of (R1) is 0.325.

[0119] (805) OK Peak sidelobe ratio for

[0120]

[0121] Specifically, in the embodiment of the present invention, the peak sidelobe ratio of the 70th frame of the basketball court hyperspectral image is The value of is 0.5.

[0122] Among them, max(·) is the maximum value operation, mean(·) is the mean operation, and std(·) is the standard deviation operation.

[0123] Step 9: Through F p (R1) Determine the adaptive weight w of R1 i , and then through and the adjacent frame confidence F c (t-1) Determine The adaptive weight w a ;

[0124] (901) by F p (R1) Determine the adaptive weight w of R1 i for

[0125]

[0126] Specifically, in the embodiment of the present invention, the adaptive weight w of the first frame of the basketball court hyperspectral image is i The value of is 0.165.

[0127] (902) Passed and the adjacent frame confidence F c (t-1) Determine The adaptive weight w a for

[0128]

[0129] Specifically, in the embodiment of the present invention, the adjacent frame confidence F of the 70th frame of the basketball court hyperspectral image is c The value of (69) is 0.748, and the adaptive weight w of the 70th frame of the basketball court hyperspectral image 69 The value of is 0.374.

[0130] Step 10: Determine w by normalization operation i The normalized result w1 determines w a The normalized result w2;

[0131] (1001) Determine w i The normalized result w1 is

[0132]

[0133] Specifically, in the embodiment of the present invention, the first frame of the basketball court hyperspectral image w i The normalized result w1 has a value of 0.306.

[0134] (1002) Determine w a The normalized result w2 is

[0135]

[0136] Specifically, in the embodiment of the present invention, the 70th frame of the basketball court hyperspectral image w a The normalized result w2 has a value of 0.694.

[0137] Step 11: Through w1, R1, w2 and Determine the fusion response map

[0138] Through w1, R1, w2 and Determine the fusion response map for

[0139]

[0140] Specifically, in the embodiment of the present invention, the fusion response map of the 70th frame of the basketball court hyperspectral image is In the present invention The image resolution is 17×17×1, Fig.13This is a fusion response image obtained from the 70th frame of the basketball court hyperspectral image after normalization in an embodiment of the present invention.

[0141] Step 12: Pass Determine the current frame reference value v t ;

[0142] (1201) OK Maximum

[0143] Specifically, in the embodiment of the present invention, the 70th frame of the basketball court hyperspectral image The value of is 0.88.

[0144] (1202) Setting coefficient μ;

[0145] Specifically, μ is 0.05;

[0146] (1203) Passed Determine the current frame reference value v t for

[0147]

[0148] Among them, v t-1 Indicates the reference value of the previous frame, v1 is 0.5, μ is a number greater than 0 and less than 1. The larger μ is, the smaller v t The slower it changes, the smaller μ is, and v t As the change is faster, μ is set to 0.05 in the present invention.

[0149] Specifically, in the embodiment of the present invention, when the 70th frame of the basketball court hyperspectral image The value is greater than the reference value v of the 69th frame 69 When The value of is assigned to the 70th frame reference value v 70 , the 70th frame reference value v 70 The value is 0.88. When the 70th frame of the basketball court hyperspectral image The value is less than the reference value v of the 69th frame 69 When The value of is assigned to the 70th frame reference value v 70 , the reference value v of the 69th frame 69 The value is 0.9, the reference value v of the 70th frame 70 The value of is 0.899.

[0150] Step 13: Update the adjacent frame confidence F c (t-1);

[0151] (1301) Determine the fusion response map of the t-1th frame Maximum

[0152] Specifically, in the embodiment of the present invention, the fusion response map of the 69th frame is Maximum The value of is 0.87.

[0153] (1302) Determine whether t is equal to 2. If t is equal to 2, F c (t-1) is initialized to 1, that is, F c (1)=1, if t is not equal to 2, execute step (1303);

[0154] Specifically, in the embodiment of the present invention, if t is equal to 70 and t is not equal to 2, step (1303) is executed.

[0155] (1303) Passed and v t-1 Update the adjacent frame confidence F c (t-1) is

[0156]

[0157] Specifically, in the embodiment of the present invention, the 69th frame reference value v 69 The value is 0.9, the fusion response map of the 69th frame Maximum The value of F is 0.87, and the confidence of adjacent frames is F c The value of (69) is updated to 0.97.

[0158] Step 14: Pass The scale estimation module realizes hyperspectral image target tracking and outputs the centroid coordinates of the hyperspectral image target and the size of the target frame in the current frame;

[0159] Specifically, the fusion response map of the 70th frame of the basketball court hyperspectral image The target is input into the scale estimation module to obtain the target of the 70th frame of the basketball court hyperspectral image. The centroid coordinates of the target are (62, 55). The size of the target box is 15×15, where 15 represents the number of rows and 15 represents the number of columns.

[0160] Step 15: Update the current frame confidence F c (t);

[0161] pass and v t Update the current frame confidence F c (t) is

[0162]

[0163] Specifically, assuming that the 70th frame reference value v70 The value is 0.88, the 70th frame of the basketball court hyperspectral image The value of is 0.899, and the confidence F of the 70th frame of the basketball court hyperspectral image is updated. c The value of (70) is 1.0216.

[0164] Step 16: Update X t ;

[0165] (1601) Setting update threshold T;

[0166] Specifically, T is 0.7;

[0167] (1602) by F c (t) and T to update X t ;

[0168]

[0169] The smaller the value of T, the greater the t The more frequent the update, the larger the value of T, and X t The slower the update, T is a number greater than 0 and less than 2. Since too frequent updates will cause target tracking drift, and too slow updates will reduce the effect of the proposed model, T is set to 0.7 in the present invention.

[0170] Specifically, the confidence F of the 70th frame of the basketball court hyperspectral image is c The value of (70) is 1.0216. When T is set to 0.7, the confidence F of the 70th frame of the basketball court hyperspectral image c (70) is greater than T, which means that the target of the normalized 70th frame hyperspectral image may have drifted, and it is necessary to update the normalized 69th frame hyperspectral image back to the 70th frame hyperspectral image to overcome the target drift phenomenon. When T is set to 1.2, the confidence F of the 70th frame basketball court hyperspectral image c (70) is less than T, which means that the target of the 70th normalized hyperspectral image frame has not drifted, and there is no need to change the 70th normalized hyperspectral image frame.

[0171] Step 17: read each frame of the hyperspectral image in the hyperspectral image sequence in turn, repeat steps 1 to 16, and complete the target tracking of the hyperspectral image sequence.

[0172] Specifically, each frame of the 318-frame ball hyperspectral image sequence is loaded in turn, and steps 1 to 16 are repeated to obtain the centroid coordinates and target frame size of the target in 70 frames of hyperspectral images, and the final hyperspectral image sequence target tracking is completed. Fig.14 It is the tracking target of the current frame hyperspectral image obtained by normalizing the 70th frame hyperspectral image of the basketball court in the embodiment of the present invention.

[0173] The present invention proposes a double twin neural network HVT target tracking algorithm based on the Alexnet framework. The target in the image can be enhanced through dimensionality reduction through principal component analysis. The image after dimensionality reduction is input into the double twin network framework. The two twin networks are used to adopt adaptive scale fusion for the output response graph, which can automatically correct the rapid movement of the target in the image due to some uncontrollable factors, thereby increasing the fault tolerance of the target tracking algorithm.

[0174] Finally, the present invention proposes a new optimization algorithm structure module - confidence judgment, which can make decisions based on the credibility of the fused response graph and decide whether to update the target in the adjacent frame image. The confidence judgment module can better track the target in a complex environment. The algorithm has good robustness in dealing with target tracking.

[0175] The above descriptions are merely preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention.

Claims

1. A hyperspectral image target tracking method based on adaptive weight and confidence, characterized in that: The method is: Step 1: Load the first frame of hyperspectral image, the tth frame of hyperspectral image and the adjacent frame of hyperspectral image of the tth frame in the hyperspectral image sequence, and perform grayscale normalization on them respectively to obtain the normalized first frame of hyperspectral image X1 and the normalized tth frame of hyperspectral image X2. t And the normalized hyperspectral image of the adjacent frame of the tth frame Where t represents the number of hyperspectral images in the hyperspectral image sequence, t is an integer greater than 2, a represents the number of adjacent hyperspectral images, and the value of a is t-1. t and The grayscale range of each pixel is a left-closed and right-closed interval from 0 to 1. t and The number of channels is 16; Step 2: Use principal component analysis to analyze X1 and X t and Dimensionality reduction is performed to obtain the first frame of hyperspectral image Z1 after normalization and the tth frame of hyperspectral image Z after normalization and dimension reduction. t And the normalized and dimension-reduced hyperspectral image of the tth frame adjacent to the frame Among them, Z1, Z t and The number of channels is 1; Step 3: Manually select the target area I1 of the first frame from Z1; t The centroid of the target area in the t-1th frame is the search area I in the tth frame. t The centroid coordinates of the t-1st frame are twice the size of the target area as the search area I of the tth frame. t Dimensions, from Z t Automatically select the search area I of the tth frame t ;exist The centroid of the target region in the t-1th frame is the target region in the adjacent frame of the tth frame. The centroid coordinates of the target area of ​​the t-1th frame are taken as the target area of ​​the adjacent frame of the tth frame. The size, from Automatically select the target area of ​​the adjacent frame of the tth frame The size of I1 is m×n pixels, where m represents the number of rows and n represents the number of columns. t The size of is 2m×2n pixels, The size of is m×n pixels; Step 4: Copy I1 three times to obtain the target area of ​​the first frame of the three channels Will I t Copy three times to obtain the search area of ​​the tth frame of the three channels Will Copy three times to obtain the target area of ​​the adjacent frame of the tth frame of the three channels Step 5: Construct a dual twin network framework, which includes the first twin network and the second twin network. Set the input of the first twin network to and In the first twin network and The similarity function is Set the input of the second twin network to and In the second twin network and The similarity function is Step 6: Extract separately through AlexNet network The deep feature E1 is extracted The deep features E t and extract The deep features Step 7: Through E1 and E t Perform a naive correlation operation to obtain the initial response graph R1, and then With E t Perform naive correlation operations to obtain adjacent response graphs Step 8. Determine the peak sidelobe ratio F of R1 p (R1) and Peak sidelobe ratio Step 9: Through F p (R1) Determine the adaptive weight w of R1 i , and through and the adjacent frame confidence F c (t-1) Determine The adaptive weight w a ; Step 10: Determine w by normalization operation i The normalized result w1 determines w a The normalized result w2; Step 11: Through w1, R1, w2 and Determine the fusion response map Step 12: Pass Determine the current frame reference value v t ; Step 13: Update the adjacent frame confidence F c (t-1); Step 14: Pass The scale estimation module realizes hyperspectral image target tracking and outputs the centroid coordinates of the hyperspectral image target and the size of the target frame in the current frame; Step 15: Update the current frame confidence F c (t); Step 16: Update X t ; Step 17: read each frame of the hyperspectral image in the hyperspectral image sequence in turn, repeat steps 1 to 16, and complete the target tracking of the hyperspectral image sequence; The step thirteen is specifically implemented by the following steps: (1301) Determine the fusion response map of the t-1th frame Maximum (1302) Determine whether t is equal to 2. If t is equal to 2, F c (t-1) is initialized to 1, that is, F c (1)=1, if t is not equal to 2, execute step (1303); (1303) Passed and v t-1 Update the adjacent frame confidence F c (t-1) is The step fifteen is specifically: and v t Update the current frame confidence F c (t) is The step sixteen is specifically implemented by the following steps: (1601) Setting update threshold T; (1602) by F c (t) and T to update X t : Among them, T is set to 0.

7.

2. The hyperspectral image target tracking method based on adaptive weight and confidence according to claim 1, characterized in that: The step 5 is specifically implemented by the following steps: (501) Setting the convolution embedding function φ(·); (502) Set the input of the first twin network to and (503) Determine the first twin network and The similarity function is Among them, b1 represents the random signal of the first twin network, b1 belongs to the real number domain, and * represents the related operation; (504) Set the input of the second twin network to and (505) Determine the second twin network and The similarity function is Among them, b2 represents the random signal of the second twin network, and b2 belongs to the real number domain.

3. The hyperspectral image target tracking method based on adaptive weight and confidence according to claim 1 or 2, characterized in that: The step eight is specifically implemented by the following steps: (801) Determine the maximum value max(R1) of R1 and The maximum value of (802) Determine the mean value mean(R1) of R1 and The average (803) Determine the standard deviation std(R1) of R1 and Standard Deviation (804) Determine the peak sidelobe ratio F of R1 p (R1) is (805) OK Peak sidelobe ratio for Among them, max(·) is the maximum value operation, mean(·) is the mean operation, and std(·) is the standard deviation operation.

4. The hyperspectral image target tracking method based on adaptive weight and confidence according to claim 3, characterized in that: The step nine is specifically implemented by the following steps: (901) by F p (R1) Determine the adaptive weight w of R1 i for (902) Passed and the adjacent frame confidence F c (t-1) Determine The adaptive weight w a for 5. The hyperspectral image target tracking method based on adaptive weight and confidence according to claim 4, characterized in that: The step ten is specifically implemented by the following steps: (1001) Determine w i The normalized result w1 is (1002) Determine w a The normalized result w2 is 6. The hyperspectral image target tracking method based on adaptive weight and confidence according to claim 5, characterized in that: The step eleven is specifically as follows: through w1, R1, w2 and Determine the fusion response map for 7. The hyperspectral image target tracking method based on adaptive weight and confidence according to claim 6, characterized in that: The step twelve is specifically implemented by the following steps: (1201) OK Maximum (1202) Setting coefficient μ; (1203) Passed Determine the current frame reference value v t for Among them, v t-1 Represents the reference value of the previous frame, v1 is 0.5, and μ is set to 0.05.

Images