Hyperspectral target tracking band selection method based on signal clarity and information correlation

Abstract

The application discloses a hyperspectral target tracking band selection method based on signal definition and information correlation, and the method comprises the following steps: using a Laplace algorithm to evaluate the definition of a band image, and using Gaussian smoothing to process the initial band image; using one-dimensional entropy and two-dimensional entropy to evaluate how much average information is contained in the band image; using mutual information to measure the correlation degree between two bands, and finding the band most similar to the target; using a difference module to evaluate the difference between the target and the background in the band image; constructing a comprehensive evaluation index to evaluate the band quality, selecting three most valuable bands to input into a pre-training network to propose deep features, and using a correlation filter tracker to complete a tracking task. The application migrates the existing network model trained on a color image to the hyperspectral target tracking, realizes channel number matching, and reduces the purpose of band redundancy.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of hyperspectral image target tracking, and particularly relates to a hyperspectral target tracking band selection method based on signal articulation and information correlation. BACKGROUND

[0002] In recent years, significant progress has been made in color video target tracking algorithms, but when dealing with complex scenes such as similar backgrounds or cluttered backgrounds, the tracking algorithm is still prone to drift. To address this problem, researchers have tried to use various visual cues such as color intensity, color name, texture, and depth features to improve the tracking effect of the algorithm. However, these methods have a key physical limitation, that is, color images can only record the color intensity information of the red, green, and blue three channels, and it is difficult to fully reflect the true physical reflection characteristics of the target, which becomes the main bottleneck of realizing robust target tracking.

[0003] Compared with traditional color images, hyperspectral images have significant advantages in target tracking. Hyperspectral images can capture dozens or even hundreds of continuous spectral channels from visible light to near-infrared wavelengths, recording the fine spectral characteristics of the target at different spectral wavelengths. Such rich spectral information not only can more comprehensively reflect the physical reflection characteristics of the target, but also can effectively distinguish similar objects in the target and background, reducing the background interference in complex scenes. In addition, hyperspectral data has strong robustness in dealing with challenges such as light changes, occlusions, and background clutter, making it possible to achieve more stable and accurate target tracking in complex environments.

[0004] Although hyperspectral target tracking has broad application prospects, it still faces many challenges. First, the data dimension of hyperspectral images is high and the utilization rate of effective spectral information is low. Hyperspectral remote sensing technology improves the spectral resolution while introducing a large amount of information redundancy, making it more difficult for further image processing. This is mainly because the wavelength difference between adjacent bands in hyperspectral images is very small, usually only in nanometers, resulting in a high degree of correlation between bands. Therefore, how to effectively remove redundant data and reduce data dimension while ensuring that important information is not lost has become an important problem that needs to be solved in the field of hyperspectral image processing. Finally, due to the need for special hyperspectral video camera equipment for the collection of hyperspectral video, the number of training samples is limited, making it difficult to train a good tracker. The only existing public dataset contains only about 13,000 annotated hyperspectral training samples, which is small in size and belongs to a typical small sample dataset.

[0005] Benefiting from the Fourier transform method, the correlation filter tracking algorithm significantly improves the accuracy of tracking while ensuring the tracking speed. However, the classical correlation filter based tracking algorithm has a big defect that the tracker relies on low-level hand-crafted features, and the hand-crafted features cannot effectively express the semantic information of the target. Many networks have been well trained in color image data sets, and the VGG network trained in the RGB image data set can be used for feature extraction of hyperspectral target tracking, so as to solve the problem of insufficient data set and improve the semantic expression ability of the features. However, the number of channels of the hyperspectral image is much larger than 3, and the hyperspectral data cannot be directly input into the network model trained in the color image.

[0006] Under this background, how to reduce the redundancy of the hyperspectral band and realize the matching of the number of channels has become a key research direction in the field of hyperspectral target tracking. SUMMARY

[0007] The purpose of the present application is to provide a hyperspectral target tracking band selection method based on signal clarity and information correlation, which focuses on comprehensively evaluating the quality of the hyperspectral channel from the internal and inter-band angles, and designs an efficient band selection strategy. Through this method, a number of bands with the most value are selected from the hyperspectral video, not only achieving the purpose of matching the number of channels, but also effectively avoiding the redundancy problem caused by the excessive attention to the spectral information of all bands by the existing hyperspectral tracker.

[0008] The technical solution for achieving the purpose of the present application is: a hyperspectral target tracking band selection method based on signal clarity and information correlation, comprising the following steps:

[0009] Step 1: Take out each band from the hyperspectral image to obtain a gray-scale image, and calculate a response map after the single-channel gray-scale image is subjected to a Laplace 3*3 convolution kernel, calculate the variance of the response map, and evaluate the clarity of each band;

[0010] Step 2: Use information entropy to measure the uncertainty and information amount of a random variable, wherein the information amount contained in the aggregation feature of the gray-scale distribution in the image is evaluated by one-dimensional entropy, and the spatial feature of the image gray-scale distribution is evaluated by two-dimensional entropy;

[0011] Step 3: Use mutual information to evaluate the correlation between the target area and the band image;

[0012] Step 4: Use the difference module to evaluate the difference between the target and the background in the band image; calculate the spectral difference curve by comparing the spectral curves of the target and the background, and select the band with the maximum value after calculation as the low-dimensional data;

[0013] Step 5, using Laplace, entropy analysis, mutual information and object-background difference module to quantify the band and inter-band characteristics, build a comprehensive evaluation index, select the most valuable band, select the band as the input of the pre-training network to extract deep features, and use the filter to track the target.

[0014] An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the above-mentioned hyperspectral target tracking band selection method based on signal clarity and information correlation.

[0015] A computer readable storage medium having a computer program stored thereon, wherein the program is executable by a processor to implement the above-mentioned hyperspectral target tracking band selection method based on signal clarity and information correlation.

[0016] A computer program product comprising a computer program, wherein the computer program is executable by a processor to implement the above-mentioned hyperspectral target tracking band selection method based on signal clarity and information correlation.

[0017] Compared with the prior art, the present application has the following advantages: (1) Unlike some existing hyperspectral video trackers that combine hyperspectral images into pseudo-color images as input for RGB-based trackers, the band selection strategy of the present application preserves the visual hierarchical hyperspectral semantic features by evaluating the clarity, information content, correlation with the target, and target-background difference of the input hyperspectral bands; (2) The present application effectively combines supervised and unsupervised methods; Laplace and entropy analysis are not limited by target variables and belong to unsupervised methods, while mutual information and target-background difference strategies use target variable information to select bands and belong to supervised methods, and the present application effectively combines the advantages of these two types of methods; (3) The quality of the bands is evaluated from multiple dimensions, different weight coefficients are applied to different modules, and a comprehensive index is constructed to select the most valuable band. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a flowchart of the hyperspectral target tracking band selection method based on signal clarity and information correlation of the present application. DETAILED DESCRIPTION

[0019] This invention proposes a hyperspectral target tracking band selection method based on signal sharpness and information correlation. The method includes: using the Laplacian algorithm to evaluate the sharpness of the band image; considering that the Laplacian operator can highlight image details but is also sensitive to noise, Gaussian smoothing is used to process the initial band image; considering that bands with high information entropy are more beneficial for target tracking, one-dimensional entropy and two-dimensional entropy are used to evaluate the average amount of information contained in the band image; considering that bands with high correlation to the target can capture the unique spectral features of the target to the maximum extent, making the target more obvious and identifiable in complex backgrounds, mutual information is used to measure the degree of correlation between two bands to find the band most similar to the target; considering that highlighting the difference between the target and the background in a hyperspectral image helps to represent and preserve the semantic features between video channels, a difference module is used to evaluate the difference between the target and the background in the band image; a comprehensive evaluation index is constructed to evaluate the band quality; three of the most valuable bands are selected and input into a pre-trained network to extract deep features; and a correlation filter tracker is used to complete the tracking task. This invention transfers existing network models trained on color images to hyperspectral target tracking, achieving the goal of matching the number of channels and reducing band redundancy.

[0020] To facilitate a better understanding of the present invention by those skilled in the art, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the described embodiments are only a part of the examples of the present invention and not all possible embodiments.

[0021] like Figure 1 As shown, the present invention provides a method for selecting hyperspectral target tracking bands based on signal sharpness and information correlation, comprising the following steps:

[0022] Step 1: First, extract each band from the hyperspectral image to obtain a grayscale image. Then, process the single-channel grayscale image using a 3×3 Laplacian convolution kernel to obtain a response map. Finally, calculate the variance of this response map to evaluate the sharpness of each band. This includes the following steps:

[0023] Step 1.1: Use Gaussian smoothing to process the k-th band image. The formula for Gaussian smoothing is as follows:

[0024]

[0025] Among them, B k (x,y) is the value of a two-dimensional Gaussian function, representing the weight at position (x,y); σ is the standard deviation of the Gaussian distribution.

[0026] Step 1.2, the Laplace operator is convolved with the gray value of each pixel point of the image to obtain a gradient matrix, and the kth band image pixel is denoted as B k (x, y), and the gradient matrix of the kth band is denoted as:

[0027]

[0028] Wherein, represents the convolution operation, B k represents the band image, and L represents the convolution kernel, as follows:

[0029]

[0030] Step 1.3, taking the square sum of the gradient of each pixel point as the evaluation function, the calculation of the Laplace index of the kth band is as follows:

[0031]

[0032] Step 2, the information entropy is used to measure the uncertainty and the amount of information contained in the random variable, wherein the information amount contained in the aggregation characteristics of the gray scale distribution in the image is evaluated by one-dimensional entropy, and the spatial characteristics of the image gray scale distribution are evaluated by two-dimensional entropy, including the following steps:

[0033] Step 2.1, let P i represent the proportion of pixels with gray value i in the kth band image, then the one-dimensional gray entropy of the gray image is defined as:

[0034]

[0035] Step 2.2, the two-dimensional entropy of the image can reflect the spatial characteristics of the image gray scale distribution; the two-dimensional entropy of the kth band image B k is denoted as Z k :

[0036]

[0037] Wherein, P i,j is the joint probability density of the pixel pair (i, j) in the image, which represents the probability of the pixel value i and j appearing at the same time. M and N are the value range of the image gray value and the size of the image respectively, and in this embodiment, M = N = 256, that is, the range of the gray value is from 0 to 255.

[0038] Step 2.3, the entropy value of the kth band is obtained by adding the one-dimensional entropy and the two-dimensional entropy:

[0039] E k = S k + Z k

[0040] Step 3, use mutual information to evaluate the correlation of the target region and the waveband image, under the given target condition, for specific tasks or feature extraction, the uncertainty of the target can be reduced to the maximum, including the following steps:

[0041] Step 3.1, given two random variables X and Y, the mutual information between them can be expressed as:

[0042]

[0043] Where p(x, y) is the joint probability distribution of X and Y, that is, the probability of X=x and Y=y occurring at the same time; p(x) and p(y) are the marginal probability distributions of X and Y, that is, the probability of X=x or Y=y occurring;

[0044] When X and Y are discrete variables, the mutual information between them is:

[0045]

[0046] Step 3.2, according to the concept of information entropy, the relationship between mutual information and information entropy is as follows:

[0047] I(X; Y) = H(X) - H(X|Y)

[0048] = H(Y) - H(Y|X)

[0049] = H(X) + H(Y) - H(XY)

[0050] Where H(X) and H(Y) are the information entropy of X and Y, H(X|Y) and H(Y|X) are the conditional entropy, and H(XY) is the joint information entropy of X and Y, whose formula is as follows:

[0051]

[0052] For discrete variables, the following calculation is made:

[0053]

[0054] Step 3.3, according to the position information of the ground truth, a target label map is marked, which has the same spatial size as the waveband image and its value is composed of 0 and 1. The value in the region enclosed by the ground truth is set to 1, and the value in other regions is set to 0. With the target label map, the mutual information between it and the waveband image is calculated to obtain the mutual information M k .

[0055] Step 4, using a difference module to evaluate the difference between the target and the background in the band image. The spectral difference curve is calculated by comparing the spectral curve of the target and its background. After calculation, the band with the maximum value is selected as the low-dimensional data, including the following steps:

[0056] Step 4.1, using the mean to quantify the spatial and spectral changes of the target region T k and the background region G k . According to the ground truth, separate the target region and the background region from the band image, and then calculate the mean difference of the two regions.

[0057] D k = |Mean(T k )-Mean(G k )|

[0058] Where Mean(T k ) and Mean(G k ) represent the mean of the target region and the background region, respectively.

[0059] Step 4.2, after calculation, select the band with the maximum value as the low-dimensional data.

[0060] Step 5, comprehensively use the Laplace, entropy analysis, mutual information and target-background difference modules to quantify the intra-band and inter-band characteristics, construct a comprehensive evaluation index to select the most valuable band, and use the selected band as the input of the pre-trained network to extract deep features, and use the relevant filter to track the target, including the following steps:

[0061] Step 5.1, comprehensively use the Laplace, entropy analysis, mutual information and target-background difference modules to quantify the intra-band and inter-band characteristics, and construct a comprehensive evaluation index (denoted as A k ) to select the most valuable band. The evaluation index A k of the band image B k is defined as follows:

[0062] A k = αF k + βE k + γM k + δD k

[0063] Where α, β, γ and δ are the weight coefficients of different modules.

[0064] Step 5.2, select the three most valuable bands as the input of the pre-trained network, that is, the first three channels with the maximum A k value as the candidate channels, denoted as F.

[0065] F = {Bmax1 max2 max3}

[0066] Step 5.3, using a pre-trained network VGGNet to extract deep features, using a correlation filter tracker to complete the tracking task.

[0067] The above examples are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing examples, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.​​

Claims

1. A hyperspectral target tracking band selection method based on signal intelligibility and information relevance, characterized in that, The method comprises the following steps: Step 1, taking out each waveband from the hyperspectral image to obtain a gray image, and obtaining a response map after the single-channel gray image is calculated by using a Laplace 3*3 convolution kernel, and calculating the variance of the response map to evaluate the definition of each waveband; Step 2, using information entropy to measure the uncertainty and the amount of information contained by random variables, wherein the amount of information contained by the aggregation features of the gray distribution in the image is evaluated by one-dimensional entropy, and the spatial features of the gray distribution in the image are evaluated by two-dimensional entropy; Step 3, using mutual information to evaluate the correlation between the target region and the waveband image; Step 4, using a difference module to evaluate the difference between the target and the background in the waveband image; calculating a spectral difference curve by comparing the spectral curves of the target and the background, and selecting the waveband with the maximum value as low-dimensional data after calculation; Step 5, using the Laplace, entropy analysis, mutual information and target-background difference modules to quantify the intra-waveband and inter-waveband characteristics, constructing a comprehensive evaluation index, and selecting the most valuable waveband, taking the selected waveband as the input of the pre-training network to extract deep features, and using a filter to track the target. 2.The hyperspectral target tracking band selection method based on signal intelligibility and information relevance according to claim 1, wherein, Step 1 comprises the following steps: Step 1.1, using Gaussian smoothing to process the kth waveband image, and the formula of Gaussian smoothing is as follows: where B k (x,y) is the value of the two-dimensional Gaussian function, representing the weight at position (x,y); σ is the standard deviation of the Gaussian distribution; Step 1.2, the Laplace operator is convolved with the gray value of each pixel point of the image to obtain a gradient matrix, and the image pixel of the kth band is denoted as B k (x, y), and the gradient matrix of the kth band is denoted as: wherein represents a convolution operation, B k denotes a waveband image, L denotes a convolution kernel, as follows: Step 1.3, taking the square sum of the gradient of each pixel point as an evaluation function, and the calculation of the Laplace index of the kth waveband is as follows: 3.The hyperspectral target tracking band selection method based on signal intelligibility and information relevance according to claim 2, characterized in that, Step 2 comprises the following steps: Step 2.1, let P i Let P(k, i) denote the proportion of pixels with gray value i in the k-th band image. The monochrome gray entropy of a gray image is defined as: Step 2.2, the two-dimensional entropy of the image can reflect the spatial features of the gray distribution of the image; The two-dimensional entropy in the kth band image B k is denoted as Z k : Among them, P i,j It is the joint probability density of pixel pairs (i,j) in the image, representing the probability that pixel values ​​i and j appear simultaneously; M and N are the range of gray values ​​and the size of the image, respectively. Step 2.3, the entropy value E of the kth waveband k The one-dimensional entropy and the two-dimensional entropy are added together to obtain: E k = S k + Z k .

4. The hyperspectral target tracking band selection method based on signal intelligibility and information relevance according to claim 3, characterized in that, M=N=256.

5. The hyperspectral target tracking band selection method based on signal intelligibility and information relevance according to claim 3, characterized in that, Step 3 comprises the following steps: Step 3.1, given two random variables X and Y, the mutual information between them is represented as: Wherein, p(x,y) is the joint probability distribution of X and Y, that is, the probability that X=x and Y=y occur at the same time; p(x) and p(y) are the marginal probability distributions of X and Y, that is, the probability that X=x or Y=y occurs; When X and Y are discrete variables, the mutual information between them is: Step 3.2, according to the concept of information entropy, the relationship between mutual information and information entropy is as follows: I(X;Y)=H(X)-H(X|Y)=H(Y)-H(Y|X)=H(X)+H(Y)-H(XY) Wherein H(X) and H(Y) are the information entropy of X and Y respectively, H(X|Y) and H(Y|X) are conditional entropy, and H(XY) is the joint information entropy of X and Y, and its formula is as follows: For discrete variables, the following calculation is performed: Step 3.3, label a target label map according to the position information of the ground truth, the spatial size of the label map is the same as the band image, and the value is composed of 0 and 1; the value in the region boxed by the ground truth is set to 1, and the value in other regions is set to 0; calculate the mutual information of the target label map and the band image to obtain the mutual information M on the kth band k .

6. The hyperspectral target tracking band selection method based on signal intelligibility and information relevance according to claim 5, characterized in that, Step 4 comprises the following steps: Step 4.1, using mean values to quantify target region T k and background region G k spatial and spectral variations; from the band images, separate the target region and background region according to groundtruth, then calculate the mean difference of the two regions: D k = |Mean(T k )-Mean(G k )| where Mean(T k ) and Mean(G k ) represent the mean values of the target region and the background region, respectively. Step 4.2, after calculation, the waveband with the maximum value is selected as low-dimensional data.

7. The hyperspectral target tracking band selection method based on signal intelligibility and information relevance according to claim 6, characterized in that, Step 5 comprises the following steps: Step 5.1, Quantify the inter-band and intra-band characteristics using Laplacian, entropy analysis, mutual information and object-background difference modules to construct a comprehensive evaluation index A k to select the most valuable bands, band image B k evaluation index A k is defined as follows: A k = aF k + βE k + γM k + δD k Wherein, α, β, γ and δ are weight coefficients of different modules; Step 5.2, select the three most valuable bands as input to the pre-trained network, i.e. the first three channels B k with the largest A max1 ,B max2 ,B max3 as candidate channels, denoted as F: F = {B max1 ,B max2 ,B max3} Step 5.3, using the pre-training network VGGNet to extract deep features, and using a related filter tracker to complete the tracking task.

8. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the method in any one of claims 1-7.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the method in any one of claims 1-7.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1-7.

Images