Hyperspectral target tracking band selection method based on signal definition and information correlation

Abstract

The invention discloses a hyperspectral target tracking band selection method based on signal definition and information correlation, and the method comprises the steps: evaluating the definition of a band image through employing a Laplacian algorithm, and carrying out the processing of an initial band image through employing Gaussian smoothing; the one-dimensional entropy and the two-dimensional entropy are used for evaluating the average information amount contained in the wave band image; mutual information is used for measuring the correlation degree between the two wave bands, and the wave band most similar to the target is found; using a difference module to evaluate a difference between a target and a background in the band image; a comprehensive evaluation index is constructed to evaluate the wave band quality, three most valuable wave bands are selected and input into a pre-training network to put forward depth features, and a correlation filtering tracker is used to complete a tracking task. According to the invention, an existing network model trained on a color image is migrated to hyperspectral target tracking, and the purposes of channel number matching and band redundancy reduction are achieved.

Description

Technical Field

[0001] The invention belongs to the field of hyperspectral image target tracking, and in particular relates to a hyperspectral target tracking band selection method based on signal clarity and information correlation. Background Art

[0002] In recent years, object tracking algorithms for color videos have made significant progress, but they are still prone to drift when dealing with complex scenes such as similar or cluttered backgrounds. To address this problem, researchers have tried to use a variety of visual cues such as color intensity, color name, texture, and depth features to improve the tracking effect of the algorithm. However, these methods have a key physical limitation, that is, color images can only record the color intensity information of the red, green, and blue channels, which is difficult to fully reflect the real physical reflectance characteristics of the target, which has become the main bottleneck for achieving robust target tracking.

[0003] Compared with traditional color images, hyperspectral images have significant advantages in target tracking. Hyperspectral images can capture dozens or even hundreds of continuous spectral channels from visible light to near-infrared bands, and record the fine spectral characteristics of the target in different spectral bands. Such rich spectral information can not only more comprehensively reflect the physical reflection characteristics of the target, but also effectively distinguish the target from similar objects in the background, and reduce background interference in complex scenes. In addition, hyperspectral data shows strong robustness in dealing with challenges such as lighting changes, occlusion, and background clutter, making it possible to achieve more stable and accurate target tracking in complex environments.

[0004] Although hyperspectral target tracking has broad application prospects, it still faces many challenges. First, the data dimension of hyperspectral images is high and the utilization rate of effective spectral information is low. While improving the spectral resolution, hyperspectral remote sensing technology also introduces a large amount of information redundancy, which increases the difficulty of further image processing. This is mainly because the wavelength difference between adjacent bands in hyperspectral images is extremely small, usually only at the nanometer level, resulting in a high degree of correlation between bands. Therefore, how to effectively remove redundant data and reduce data dimensions while ensuring that important information is not lost has become an important problem to be solved in the field of hyperspectral image processing. Finally, since the acquisition of hyperspectral videos requires specialized hyperspectral video cameras, the number of training samples is limited, which makes it very difficult to train a good tracker. The only existing public dataset contains only about 13,000 annotated hyperspectral training samples, which is small in scale and a typical small sample dataset.

[0005] Benefiting from the Fourier transform method, the correlation filtering tracking algorithm significantly improves the tracking accuracy while ensuring the tracking speed. However, a major drawback of the classic correlation filtering-based tracking algorithm is that the tracker relies on low-level handcrafted features, which cannot effectively express the semantic information of the target. Many networks have been well-trained on color image datasets, and the VGG network trained on the RGB image dataset can be used for feature extraction in hyperspectral target tracking. This not only solves the problem of scarce datasets but also enhances the semantic expression ability of features. However, the number of channels in hyperspectral images is much larger than 3, and hyperspectral data cannot be directly input into the network model trained on color images.

[0006] In this context, how to reduce the redundancy of hyperspectral bands and achieve channel number matching has become a key research direction in the field of hyperspectral target tracking. Summary of the Invention

[0007] The purpose of the present invention is to provide a hyperspectral target tracking band selection method based on signal clarity and information correlation, which focuses on comprehensively evaluating the quality of hyperspectral channels from two perspectives: within the band and between bands, and designs an efficient band selection strategy. Through this method, several most valuable bands are selected from the hyperspectral video, which not only achieves the purpose of channel number matching but also effectively avoids the redundancy problem caused by existing hyperspectral trackers overemphasizing the spectral information of all bands.

[0008] The technical solution to achieve the purpose of the present invention is: a hyperspectral target tracking band selection method based on signal clarity and information correlation, including the following steps:

[0009] Step 1, extract each band from the hyperspectral image to obtain a grayscale image. After calculating the grayscale image of a single channel with a Laplacian 3×3 convolution kernel, a response map is obtained, and the variance of this response map is calculated to evaluate the clarity of each band.

[0010] Step 2, use information entropy to measure the uncertainty and the amount of information contained in a random variable. Among them, the amount of information contained in the aggregation feature of the gray level distribution in the image is evaluated by one-dimensional entropy, and the spatial feature of the gray level distribution of the image is evaluated by two-dimensional entropy.

[0011] Step 3, use mutual information to evaluate the correlation between the target region and the band image.

[0012] Step 4, use a difference module to evaluate the difference between the target and the background in the band image; calculate the spectral difference curve by comparing the spectral curves of the target and its background, and after calculation, select the band with the largest value as the low-dimensional data.

[0013] Step 5: Use Laplacian, entropy analysis, mutual information, and target-background difference modules to quantify intra-band and inter-band characteristics, construct a comprehensive evaluation index, select the most valuable bands, use the selected bands as the input of the pre-trained network to extract deep features, and use filters to track the target.

[0014] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the above-mentioned hyperspectral target tracking band selection method based on signal clarity and information correlation.

[0015] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned hyperspectral target tracking band selection method based on signal clarity and information correlation.

[0016] A computer program product includes a computer program, which, when executed by a processor, implements the above-mentioned hyperspectral target tracking band selection method based on signal clarity and information correlation.

[0017] Compared with the prior art, the present invention has the following remarkable advantages: (1) Different from some existing hyperspectral video trackers that synthesize hyperspectral images into pseudo-color images as the input of RGB-based trackers, the band selection strategy of the present invention retains the hyperspectral semantic features at the visual level by evaluating the clarity, information content, correlation with the target, and target-background difference of the input hyperspectral bands; (2) The present invention effectively combines supervised and unsupervised methods. Laplacian and entropy analysis are not restricted by target variables and belong to unsupervised methods, while mutual information and target-background difference strategies use target variable information to select bands and belong to supervised methods. The present invention effectively combines the advantages of these two types of methods; (3) The quality of the bands is evaluated from multiple dimensions, different weight coefficients are applied to different modules, and a comprehensive index is constructed to select the most valuable bands. Description of the Drawings

[0018] Figure 1 It is a flowchart of the hyperspectral target tracking band selection method based on signal clarity and information correlation of the present invention. Detailed Embodiments

[0019] The present invention proposes a hyperspectral target tracking band selection method based on signal clarity and information correlation. The method includes: using the Laplace algorithm to evaluate the clarity of the band image. Considering that the Laplace operator can highlight the details of the image but is also sensitive to noise, Gaussian smoothing is used to process the initial band image; considering that bands with high information entropy are more beneficial for target tracking, one-dimensional entropy and two-dimensional entropy are used to evaluate the amount of average information contained in the band image; considering that bands with high correlation with the target can maximize the capture of the unique spectral features of the target, making the target more obvious and distinguishable in a complex background, mutual information is used to measure the degree of association between two bands to find the band most similar to the target; considering that highlighting the difference between the target and the background in a hyperspectral image helps to represent and retain the semantic features between video channels, a difference module is used to evaluate the difference between the target and the background in the band image; a comprehensive evaluation index is constructed to evaluate the band quality, and the three most valuable bands are selected and input into the pre-trained network to extract deep features, and a correlation filter tracker is used to complete the tracking task. The present invention transfers the existing network model trained on color images to hyperspectral target tracking, achieving the purpose of matching the number of channels and reducing band redundancy.

[0020] To facilitate those skilled in the art to better understand the solution of the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the described embodiments are only some examples of the present invention, not all possible embodiments.

[0021] As Figure 1 shown, a hyperspectral target tracking band selection method based on signal clarity and information correlation of the present invention includes the following steps:

[0022] Step 1, first extract each band from the hyperspectral image to obtain a grayscale image. Then, after calculating the single-channel grayscale image with a 3×3 Laplace convolution kernel, a response map will be obtained. Finally, calculate the variance of this response map to evaluate the clarity of each band, including the following steps:

[0023] Step 1.1, use Gaussian smoothing to process the k-th band image. The formula for Gaussian smoothing is as follows:

[0024]

[0025] Among them, B k (x,y) is the value of the two-dimensional Gaussian function, representing the weight at the position (x,y); σ is the standard deviation of the Gaussian distribution.

[0026] Step 1.2: Convolve the Laplace operator with the gray values of each pixel of the image to obtain a gradient matrix. The pixel of the k-th band image is denoted as B k (x, y), and the gradient matrix of the k-th band is expressed as:

[0027]

[0028] where, represents the convolution operation, B k represents the band image, and L represents the convolution kernel, as shown below:

[0029]

[0030] Step 1.3: Take the sum of the squares of the gradients of each pixel as the evaluation function. The calculation of the Laplace index of the k-th band is shown in the following formula:

[0031]

[0032] Step 2: Use information entropy to measure the uncertainty and the amount of information of a random variable. Among them, the amount of information contained in the aggregation characteristics of the gray distribution in the image is evaluated by one-dimensional entropy, and the spatial characteristics of the image gray distribution are evaluated by two-dimensional entropy, including the following steps:

[0033] Step 2.1: Let P i represent the proportion of pixels with gray value i in the k-th band image. Then, the unary gray entropy of the gray image is defined as:

[0034]

[0035] Step 2.2: The two-dimensional entropy of the image can reflect the spatial characteristics of the image gray distribution; the two-dimensional entropy in the k-th band image B k is denoted as Z k :

[0036]

[0037] where, P i,j is the joint probability density of the pixel pair (i, j) in the image, representing the probability that the pixel values i and j appear simultaneously. M and N are the value ranges of the image gray values and the size of the image, respectively. In this embodiment, M = N = 256, that is, the range of gray values is from 0 to 255.

[0038] Step 2.3: The entropy value of the k-th band is obtained by adding the one-dimensional entropy and the two-dimensional entropy:

[0039] E k = S k + Z k

[0040] Step 3. Use mutual information to evaluate the correlation between the target region and the band image. Under the given target conditions, for specific tasks or feature extraction, it can minimize the uncertainty of the target to the greatest extent, including the following steps:

[0041] Step 3.1. Given two random variables X and Y, the mutual information between them can be expressed as:

[0042]

[0043] where p(x, y) is the joint probability distribution of X and Y, that is, the probability that X = x and Y = y occur simultaneously; p(x) and p(y) are the marginal probability distributions of X and Y respectively, that is, the probability that X = x or Y = y occurs;

[0044] When X and Y are discrete variables, the mutual information between them is:

[0045]

[0046] Step 3.2. From the concept of information entropy, the relationship between mutual information and information entropy is as follows:

[0047] I(X; Y) = H(X) - H(X|Y)

[0048] = H(Y) - H(Y|X)

[0049] = H(X) + H(Y) - H(XY)

[0050] where H(X) and H(Y) are the information entropies of X and Y respectively, H(X|Y) and H(Y|X) are conditional entropies, and H(XY) is the joint information entropy of X and Y. The formulas are as follows:

[0051]

[0052] For discrete variables, the following calculations are as follows:

[0053]

[0054] Step 3.3. Mark a target label map according to the position information of the ground truth. The spatial size of the label map is the same as that of the band image, and its values are composed of 0 and 1. Set the values in the area enclosed by the ground truth to 1, and the values in other areas to 0. With the target label map, calculate the mutual information between it and the band image to obtain the mutual information M on the k-th band k .

[0055] Step 4: Use a difference module to evaluate the difference between the target and the background in the band image. Calculate the spectral difference curve by comparing the spectral curves of the target and its background. After calculation, select the band with the largest value as the low-dimensional data, including the following steps:

[0056] Step 4.1: Use the mean to quantify the spatial and spectral variations of the target region T k and the background region G k . Separate the target region and the background region from the band image according to the ground truth, and then calculate the mean difference between the two regions.

[0057] D k = |Mean(T k ) - Mean(G k )|

[0058] where Mean(T k ) and Mean(G k ) represent the means of the target region and the background region, respectively.

[0059] Step 4.2: After calculation, select the band with the largest value as the low-dimensional data.

[0060] Step 5: Comprehensively use modules such as Laplacian, entropy analysis, mutual information, and target-background difference to quantify the intra-band and inter-band characteristics, construct a comprehensive evaluation index to select the most valuable bands, use the selected bands as the input of the pre-trained network to extract deep features, and use a correlation filter to track the target, including the following steps:

[0061] Step 5.1: Comprehensively use modules such as Laplacian, entropy analysis, mutual information, and target-background difference to quantify the intra-band and inter-band characteristics, and construct a comprehensive evaluation index (denoted as A k ) to select the most valuable bands. The evaluation index A k of the band image B k is defined as follows:

[0062] A k = αF k + βE k + γM k + δD k

[0063] where α, β, γ, and δ are the weight coefficients of different modules, respectively.

[0064] Step 5.2: Select the three most valuable bands as the input of the pre-trained network, that is, the first three channels with the largest A k value as the candidate channels, denoted as F.

[0065] F = {Bmax1 , B max2 , B max3}

[0066] Step 5.3: Use the pre-trained network VGGNet to extract deep features and use a correlation filter tracker to complete the tracking task.

[0067] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hyperspectral target tracking band selection method based on signal clarity and information correlation, characterized in that: The following steps are involved: Step 1: Take each band from the hyperspectral image to obtain a grayscale image. After the grayscale image of a single channel is calculated by the Laplace 3×3 convolution kernel, a response map is obtained. The variance of the response map is calculated to evaluate the clarity of each band. Step 2, use information entropy to measure the uncertainty of random variables and the amount of information contained, where the amount of information contained in the aggregation characteristics of the grayscale distribution in the image is evaluated using one-dimensional entropy, and the spatial characteristics of the image grayscale distribution are evaluated using two-dimensional entropy; Step 3, using mutual information to evaluate the correlation between the target area and the band image; Step 4, using a difference module to evaluate the difference between the target and the background in the band image; comparing the spectral curves of the target and its background to calculate the spectral difference curve, after calculation, select the band with the largest value as the low-dimensional data; Step 5: Use Laplace, entropy analysis, mutual information and target-background difference modules to quantify intra-band and inter-band characteristics, construct a comprehensive evaluation index, select the most valuable bands, use the selected bands as the input of the pre-trained network to extract deep features, and use filters to track the target.

2. The band selection method based on signal clarity and information correlation according to claim 1, characterized in that: Step 1 contains the following steps: Step 1.1, use Gaussian smoothing to process the k-th band image. The formula of Gaussian smoothing is as follows: Among them, B k (x, y) is the value of the two-dimensional Gaussian function, which represents the weight at the position (x, y); σ is the standard deviation of the Gaussian distribution; Step 1.2: Use the Laplace operator to convolve with the grayscale value of each pixel in the image to obtain the gradient matrix. The image pixel of the kth band is recorded as B k (x, y), the gradient matrix of the kth band is expressed as: in, represents the convolution operation, B k Represents the band image, and L represents the convolution kernel, as shown below: Step 1.3, take the square sum of the gradients of each pixel as the evaluation function, and the calculation of the Laplace index of the kth band is as follows:

3. The band selection method based on signal clarity and information correlation according to claim 2, characterized in that: Step 2 contains the following steps: Step 2.1, let P i It represents the proportion of pixels with gray value i in the k-th band image, and the unary gray entropy of the gray image is defined as: Step 2.2, the two-dimensional entropy of the image can reflect the spatial characteristics of the image grayscale distribution; The k-th band image B k The two-dimensional entropy in is represented by Z k : Among them, P i,j is the joint probability density of the pixel pair (i, j) in the image, indicating the probability that pixel values ​​i and j appear at the same time; M and N are the range of image grayscale values ​​and the size of the image, respectively; Step 2.3, entropy value E of the kth band k It is obtained by adding one-dimensional entropy and two-dimensional entropy: E k =S k +Z k 。 4. The band selection method based on signal clarity and information correlation according to claim 3, characterized in that: M=N=256.

5. The band selection method based on signal clarity and information correlation according to claim 3, characterized in that: Step 3 contains the following steps: Step 3.1, given two random variables X and Y, the mutual information between them can be expressed as: Among them, p(x,y) is the joint probability distribution of X and Y, that is, the probability of X=x and Y=y occurring at the same time; p(x) and p(y) are the marginal probability distributions of X and Y respectively, that is, the probability of X=x or Y=y occurring; When X and Y are discrete variables, the mutual information between them is: Step 3.2, from the concept of information entropy, we can know that the relationship between mutual information and information entropy is as follows: I(X;Y)=H(X)-H(X|Y) =H(Y)-H(Y|X) =H(X)+H(Y)-H(XY) Where H(X) and H(Y) are the information entropy of X and Y respectively, H(X|Y) and H(Y|X) are the conditional entropy, and H(XY) is the joint information entropy of X and Y. The formula is as follows: For discrete variables, the calculation is as follows: Step 3.3, mark a target label map according to the location information of the ground truth. The spatial size of the label map is the same as the band image, and its value is composed of 0 and 1; set the value of the area framed by the ground truth to 1, and the value of other areas to 0; calculate the mutual information between the target label map and the band image, and obtain the mutual information M on the kth band k .

6. The band selection method based on signal clarity and information correlation according to claim 5, characterized in that: Step 4 includes the following steps: Step 4.1: Use the mean to quantify the target area T k and background area G k The spatial and spectral changes of the image are analyzed; the target area and the background area are separated from the band image according to the groundtruth, and then the mean difference between the two areas is calculated: D k =|Mean(T k )-Mean(G k )| Mean(T k ) and Mean(G k ) represent the mean values ​​of the target area and the background area respectively; Step 4.2, after calculation, select the band with the largest value as the low-dimensional data.

7. The band selection method based on signal clarity and information correlation according to claim 6, characterized in that: Step 5 includes the following steps: Step 5.1, use Laplace, entropy analysis, mutual information and target-background difference modules to quantify intra-band and inter-band characteristics and construct a comprehensive evaluation index A k To select the most valuable band, band image B k Evaluation indicator A k The definition is as follows: A k =αF k +βE k +γM k +δD k Among them, α, β, γ and δ are the weight coefficients of different modules respectively; Step 5.2, select the three most valuable bands as the input of the pre-trained network, that is, the band with the largest A k The first three channels of value B max1 ,B max2 ,B max3 As a candidate channel, denoted as F: F={B max1 ,B max2 ,B max3 } In step 5.3, use the pre-trained network VGGNet to extract deep features and use the correlation filter tracker to complete the tracking task.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images