A hyperspectral image classification method based on functional representation under multiple constraints
Through the functional representation and superpixel segmentation method under multi-constraint conditions, combined with functional principal component analysis and support vector machine, the problems of insufficient utilization of spectral curve features and lack of training samples in hyperspectral image classification are solved, and high-precision Spectral and spatial information fusion classification.
Patent Information
- Application Number
- CN202210427847.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-22
- Publication Date
- 2024-11-01
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing hyperspectral image classification methods are difficult to fully utilize the continuity and correlation characteristics of the spectral curve when processing high-dimensional complex data, and ignore the shape and texture information of the land objects, resulting in limited classification accuracy.
Using a functional representation method based on multi-constraint conditions, the hyperspectral image is fitted into a spectral curve fitting function through the B-spline function, and superpixel segmentation and spatial constraints are introduced to generate hyperspectral image functionalization under multi-constraint conditions. It is expressed that the functional principal component analysis and support vector machine are classified.
It improves the classification accuracy of hyperspectral images, effectively utilizes spectral and spatial information, reduces incorrectly classified pixel points, and enhances the recognition ability of complex scenes, especially in low training samples.
Smart Images

Figure BDA0003610464400000021 
Figure BDA0003610464400000031 
Figure BDA0003610464400000043
Abstract
Description
Technical Field
[0001] The present invention relates to the field of remote sensing image information recognition and extraction, and in particular to a hyperspectral image classification method based on functional representation under multiple constraints. Background Art
[0002] In recent years, the development of sensors has enabled the use of hyperspectral imaging systems to obtain rich spectral information of ground objects. Hyperspectral datasets provide valuable information for studying land use and land cover, and are therefore widely used in vegetation and water resources research, food safety and agriculture, and military affairs. However, the inherent characteristics of hyperspectral images, such as high dimensionality, redundancy, and nonlinear data space, have brought great challenges to hyperspectral image processing.
[0003] In order to solve the Hughes phenomenon in hyperspectral image classification, researchers have introduced many spectral dimensionality reduction methods to remove redundant information. According to the processing method of hyperspectral data, there are two main types: 1) Convert spectral data into vector form for dimensionality reduction. Principal component analysis (PCA) and its extended kernel principal component analysis (KPCA) are the most representative methods. In addition, manifold learning and representation-based dimensionality reduction methods have also been applied to hyperspectral images. 2) Tensor dimensionality reduction method. The structure of hyperspectral data conforms to the tensor form, so many methods based on tensor representation have been generated. However, the spectral curve reflects the inherent law of land cover. The above two methods regard spectral data as discrete vectors and fail to fully explore its continuity and correlation characteristics. Functional data analysis (FDA) uses the massive features that lead to the "dimensionality disaster" as the information source for function fitting. At the same time, through the feature extraction methods unique to functional data, such as functional principal component analysis (FPCA), a more intuitive explanation of the data is obtained, which provides a new idea for processing high-dimensional and complex data. Hyperspectral remote sensing uses dozens or even hundreds of continuous and narrow spectral channels to continuously image objects, providing the necessary data conditions for fitting pixel spectral curves. Since spectral curves are usually not analytical functions, they cannot be further processed and analyzed using methods based on function analysis. The spectral vectors collected by the imaging spectrometer in the spectral range can be regarded as high-density sampling of the spectral curve corresponding to the pixel, and the analytic function model of the spectral curve can be established. Li et al. first expressed the spectral vector as a function, and used FPCA to extract the functional features of the fitted spectral curve. Finally, the support vector machine (SVM) was used to classify the hyperspectral image with the extracted principal components of each order as features, which improved the classification accuracy of the hyperspectral image. Wei et al. used the kernel extreme learning machine (KELM) to classify the hyperspectral image with the principal components extracted by FPCA as features, and the classification accuracy was further improved. The above methods no longer regard hyperspectral data as discrete vectors, but use wavelength-spectral measurement information instead. However, considering the "same object, different spectrum" and "same spectrum, different objects" phenomena in hyperspectral images, classification using only spectral information has certain limitations. The combination of spatial and spectral information greatly improves the classifier's ability to recognize complex scenes and effectively reduces the number of misclassified pixels in the classification map obtained by classification using only spectral information.Majdar et al. used the watershed superpixel segmentation map to regularize the classification map generated by the FPCA+SVM algorithm, and considered spatial information for the first time in the hyperspectral image FDA. Lv et al. integrated the spatial regular neighborhood as spatial prior information into the functional data analysis, and the image classification accuracy was greatly improved. Ye et al. extended the linear discriminant analysis (LDA) to the functional field and used the spatial regular neighborhood information to propose a functional data discriminant analysis (FDDA) method. In the above studies, although spatial information was incorporated into the hyperspectral image FDA, local regular neighborhood windows were used to extract pixel spatial information, ignoring the local homogeneous region characteristics of the pixel point, failing to fully utilize the rich shape and texture information of the ground objects, and failing to effectively extract edge features. Summary of the invention
[0004] In view of the deficiencies of the prior art, the present invention proposes a hyperspectral image classification method based on functional representation under multiple constraints.
[0005] A hyperspectral image classification method based on functional representation under multiple constraints specifically includes the following steps:
[0006] Step 1: Functionalize the hyperspectral image under multiple constraints;
[0007] Step 1.1: Given a hyperspectral image y = {y i ,i=1,…,N}, where i is the pixel index, N is the total number of pixels, and y i =(y ij ,j=1,…,M) T is the spectral measurement vector of pixel i, T is the transpose, j is the band index, M is the total number of bands, y ij is the spectral measurement of pixel i in band j; since hyperspectral remote sensing uses continuous, narrow bandwidth spectral channel imaging, y i Usually expressed as a high-dimensional vector;
[0008] Step 1.2: Hyperspectral image processing based on high-dimensional features has problems such as information redundancy, dimensionality disaster, and difficulty in modeling high-order statistical features. The FDA method is used to interpret the hyperspectral image and express the spectral curve as a spectral curve fitting function. Due to the high spectral resolution of the hyperspectral image, y i As a high-density sampling of the reflectance spectrum curve of the pixel i area, it becomes the information source for constructing the spectrum curve fitting function; assuming that the smooth function x i (t) is the spectral curve fitting function of pixel i, which is expressed as the weighted sum of a set of independent basis functions:
[0009]
[0010] Among them, k is the basis function index, K is the number of basis functions, φ k (t) is the basis function, c ik is the weight coefficient corresponding to the basis function; due to the advantages of B-spline function in approximating non-periodic function data and parameter estimation, B-spline function is selected as the basis function of spectral curve fitting function; let the band center frequency corresponding to band j in the given hyperspectral image y be t j , according to the spectral frequency t of the hyperspectral image 1 <t 2 <… <t M , the p-th k-th B-spline basis function N is defined by the deBoor-Cox recursive formula k,p The general form of (t) is as follows:
[0011]
[0012] Among them, t k is the corresponding spectral frequency; the number of basis functions K determines the function fitting effect of the original data. Generally, the number of hyperspectral bands M and the number of B-spline basis functions p are used to determine the number of basis functions, that is, K = Mp–1;
[0013] Considering the data fitting effect and computational complexity, p=3 degree B-spline is usually selected for fitting:
[0014] Formula (1) can be rewritten as a linear combination of a set of B-spline systems:
[0015] x i (t) = B(t) T c i (3)
[0016] Where B(t)=(N 1,3 (t),N 2,3 (t),…,N K,3 (t)) T ;c i =(c i1 ,c i2 ,…,c iK ) T ;
[0017] Use the simple least squares fitting method to solve the coefficient c in equation (3): i , the sum of squares of the error terms can be expressed as:
[0018] SSE i =||y i -x i|| 2 =(y i -Bc i ) T (y i -Bc i ) (4)
[0019] Where B = (B (t 1 ),B(t 2 ),…,B(t M )) T is the B-spline basis matrix of order M×K; i =(x i (t 1 ),x i (t 2 ),…,x i (t M )) T is the function value of the spectrum fitting function;
[0020] Step 1.3: In order to obtain the optimal spectral curve fitting function and better express the characteristics of hyperspectral data, a new hyperspectral image functional representation framework is constructed, and the error term sum of squares model is expanded to:
[0021] SSE i =SSE W +SSE λ +SSE η (5)
[0022] Among them, SSE W is the weighted model of formula (4), which is used to measure the fitting effect of the function curve and is defined as:
[0023] SSE W =(y i -Bc i ) T W(y i -Bc i ) (6)
[0024] Among them, W is the weight matrix, which is calculated by the mean square residual. Since the variance-covariance matrix of the residuals cannot be fully estimated in general, the covariance between the residuals is set to 0, then W is the diagonal weight matrix, SSE λ is a rough penalty term, which is used to measure the smoothness of the fitting function itself and is defined as:
[0025] SSE λ =λ∫[D (m) x i (t)] 2 dt=λc i T [∫D(m) B(t)D (m) B(t) T dt]c i =λc i T Rc i (7)
[0026] Among them, R is the inner product matrix of the m-order derivative of the B-spline basis system, that is, R = ∫D (m) B(t)D (m) B(t) T dt, integral term [D (m) x i (t)] 2 Represents x i The m-th derivative of (t), usually m=2, is used to calculate the curvature, λ(≥0) is the smoothing parameter, which is determined by the generalized cross validation method; SSE η is a spatial coherence term, which is used to introduce spatial relationships in the function fitting process and is defined as:
[0027]
[0028] in, is the average function representation of the superpixel neighborhood where pixel i is located, for The vector representation of the integral term The purpose of is to enhance the spatial consistency between pixels, so that the spectral curve fitting function of a pixel is as similar as possible to the spectral curve fitting functions of other pixels in its superpixel neighborhood, and η is a coefficient that controls the degree of spatial constraint;
[0029] In order to obtain the average function representation of the neighborhood of any pixel i It is necessary to accurately determine the neighborhood range of pixel i; entropy rate superpixel segmentation, ERS, is used to generate superpixel segmentation maps; the ERS algorithm is a graph-based clustering method, which first maps the image to a graph G = (V, E), where V is the vertex set of pixels and E is the edge weight set of the similarity of adjacent pixels; then, according to the given number of superpixels L, the objective function is optimized, that is, by selecting a subset A of E, a graph G = (V, A) containing L connected subgraphs is obtained; its objective function consists of the entropy rate H(A) of random walks on the graph and the balance term B(A) on the cluster distribution, that is:
[0030]
[0031] Among them, α≥0 is the weight coefficient used to adjust the balance between the two items; st is the abbreviation of subject to; in order to reduce the computational complexity, PCA is first used to obtain the first principal component of the hyperspectral image, and then ERS is applied to it to obtain the superpixel segmentation result S={S l ,l=1,…,L}, where l is the superpixel region index and L is the number of superpixels; let S l is the superpixel where pixel i is located, and S is defined l The average function of is expressed as:
[0032]
[0033] Where U is S l The number of pixels included; x i '(t) is S l The initial spectral curve fitting function of each pixel in, at this time i=1,…,U;
[0034] Step 1.4: Substitute equations (6)–(8) into equation (5) to construct the optimal error term sum of squares model:
[0035]
[0036] Then the optimal functional representation coefficient c i * By minimizing formula (11), we can obtain:
[0037]
[0038] By i Find the partial derivative and get c i * ,Right now:
[0039]
[0040] Since the spectral curve fitting function x of each pixel i '(t) is a priori unknown, and the average function of each superpixel is expressed as It has not been explicitly expressed yet; set the spatial constraint parameter η in equation (13) to 0 and solve it to obtain the initial coefficient c i , that is, c i =(B T WB+λR) -1 B T Wy i , using c i Calculate x i '(t) is calculated by formula (11) to obtain Then each pixel in the hyperspectral image is optimally represented as:
[0041]
[0042] After the hyperspectral image is functionalized under multiple constraints, the representation of hyperspectral image pixel i is represented by vector y i Convert to function representation
[0043] Step 2: Perform functional principal component analysis on the hyperspectral image function representation under multiple constraints to generate a principal component set;
[0044] After obtaining the functional representation of the hyperspectral image under multiple constraints, FPCA is used to generate the principal component of the functional data. FPCA is an extension of the traditional PCA in the field of functional data processing. It regards the function as a variable and converts the task of finding the eigenvalues and eigenvectors of the covariance matrix into the characteristic analysis problem of solving the eigenfunction ξ(t) of the covariance function V(s, t). Assuming that D (≤K) principal component components are used for classification, the dth principal component f of pixel i is generated. id as follows:
[0045]
[0046] Where d≤D,ξ d (t) is the normalized eigenfunction corresponding to the dth principal component, which is orthogonal to the eigenfunctions corresponding to the remaining D-1 principal components; ξ d (t) is expressed by the same set of B-spline systems:
[0047]
[0048] Where b is the coefficient vector; the discrete eigenvalue equation Vξ=ρξ is converted into a functional eigenvalue equation:
[0049] ∫V(s,t)ξ d (t)dt=ρξ d (t) (17)
[0050] Among them, ρ is the eigenvalue; according to formula (3), we have Expand the covariance function V(s,t) as:
[0051]
[0052] Formula (17) can be rewritten as:
[0053] N -1 c i *T c i * Hb=ρb (19)
[0054] Where H = ∫B(t)B(t) T dt is the inner product of the B-spline basis function; after obtaining b, the eigenfunction ξ can be obtained through equation (16): d (t), then ξ d Substitute (t) into formula (15) to calculate ξ d (t) and x i The inner product of (t) can get the principal component f id ; After feature extraction using FPCA, the functional representation of pixel i under multiple constraints The corresponding principal component is f i , f i =(f id ,d=1,…,D), then all pixel principal components constitute an N×D-dimensional principal component set f={f i , i=1,…,N}, at this time, the classification of the hyperspectral image y is converted into the classification of the principal component set f;
[0055] Step 3: Use support vector machine to classify the principal component set;
[0056] First, remove the background point information in the hyperspectral image y corresponding to the principal component set f to obtain the sample set, and then randomly select training samples from each category of the sample set according to a certain proportion, and the rest are used as test samples; suppose the training sample set f train ={(f z’ ,v z’ ),z'=1,…,Z}, where f z’ is the sample vector; v z’ ∈{+1,-1} is the category number; z' is the training sample index; Z is the total number of training samples; for the V-category classification problem of hyperspectral images, the one-against-one, OAO criterion is used to transform it into a set of two-category classification problems, that is, a SVM classifier is designed between any two types of training samples, and a total of V(V-1) / 2 classifiers are trained; the problem of finding the optimal classification hyperplane for SVM two-category classification is transformed into a minimization constraint programming problem:
[0057]
[0058]
[0059] Among them, C is the preset penalty factor, the weight vector ω and the bias b are the optimal hyperplane ω T ·f z’ +b=0 parameters, ξ z’ is the relaxation factor; by introducing the Lagrangian function to solve the dual problem of equation (20), the Lagrangian multiplier α can be obtainedz’ * , and then use α z’ * Find the optimal solution of parameters ω and b * and b * , construct the classification discriminant function according to the optimal solution parameters:
[0060]
[0061] Among them, sgn() is the sign function, K(f i ,f j ) is the kernel function that satisfies Mercer's theorem, which is defined by the Gaussian kernel function as:
[0062]
[0063] Where σ is the width parameter of the kernel function. For any test sample, V(V-1) / 2 SVM classifiers are used for classification, and the final classification result is the label with the most votes among the V(V-1) / 2 predicted labels.
[0064] Beneficial technical effects of the present invention:
[0065] The superpixel segmentation method is an over-segmentation in image segmentation. As a method to effectively extract spatial information and reveal structural features, it plays an important role in the spectral-spatial information fusion method. In order to reveal the intrinsic essential characteristics of hyperspectral data, make full use of band-related information, and take into account spatial structural information, the present invention proposes a multi-constrained functional representation algorithm. The algorithm expresses the spectral vector as a function form through a linear combination of B-spline basis functions. The accuracy, smoothness and superpixel neighborhood information of the fitting curve are considered in the functional representation; then FPCA is used to obtain a few principal components that can most effectively explain the data; finally, it is combined with the SVM classifier to classify the extracted features. Through experiments with several comparative algorithms on Indian Pines, University of Pavia and Salinas hyperspectral datasets, it is verified that the algorithm of the present invention can effectively utilize the spectral and spatial information of hyperspectral images to achieve fine classification of objects. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 Effect of the number of superpixels on the overall accuracy (%) of Indian Pines (a), University of Pavia (b) and Salinas (c) under different training sample ratios according to an embodiment of the present invention; wherein the numbers 1-3 in Figures a1, a2 and a3 represent the cases when the training sample ratio is 1%, 5% and 10%, respectively;
[0067] Figure 2Sensitivity analysis of the spatial constraint parameter η of the algorithm MC-FDA+SVM in the embodiment of the present invention;
[0068] Figure 3 Fitting curve function (solid line) and spectral vector (dashed line) of 12 sample points of Indian Pines data set in the embodiment of the present invention, wherein Figures (a) to (c) represent classes 1, 8 and 9 respectively, wherein the numbers 1-4 in Figures a1, a2, a3 and a4 represent 4 randomly selected sample points in each class respectively;
[0069] Figure 4 Classification diagram of the Indian Pines dataset according to an embodiment of the present invention;
[0070] Figure 5 Classification diagram of the University of Pavia data set in an embodiment of the present invention;
[0071] Figure 6 Classification diagram of the Salinas dataset according to an embodiment of the present invention. DETAILED DESCRIPTION
[0072] The present invention will be further described below in conjunction with the accompanying drawings and embodiments;
[0073] A hyperspectral image classification method based on functional representation under multiple constraints specifically includes the following steps:
[0074] Step 1: Functionalize the hyperspectral image under multiple constraints;
[0075] Step 1.1: Given a hyperspectral image y = {y i ,i=1,…,N}, where i is the pixel index, N is the total number of pixels, and y i =(y ij ,j=1,…,M) T is the spectral measurement vector of pixel i, j is the band index, M is the total number of bands, y ij is the spectral measurement of pixel i in band j; since hyperspectral remote sensing uses continuous, narrow bandwidth spectral channel imaging, y i Usually expressed as a high-dimensional vector;
[0076] Step 1.2: Hyperspectral image processing based on high-dimensional features has problems such as information redundancy, dimensionality disaster, and difficulty in modeling high-order statistical features. The FDA method is used to interpret the hyperspectral image and express the spectral curve as a spectral curve fitting function. Due to the high spectral resolution of the hyperspectral image, y i As a high-density sampling of the reflectance spectrum curve of the pixel i area, it becomes the information source for constructing the spectrum curve fitting function; assuming that the smooth function x i(t) is the spectral curve fitting function of pixel i, which is expressed as the weighted sum of a set of independent basis functions:
[0077]
[0078] Among them, k is the basis function index, K is the number of basis functions, φ k (t) is the basis function, c ik is the weight coefficient corresponding to the basis function; due to the advantages of B-spline function in approximating non-periodic function data and parameter estimation, B-spline function is selected as the basis function of spectral curve fitting function; let the band center frequency corresponding to band j in the given hyperspectral image y be t j , according to the spectral frequency t of the hyperspectral image 1 <t 2 <… <t M , the p-th k-th B-spline basis function N is defined by the deBoor-Cox recursive formula k,p The general form of (t) is as follows:
[0079]
[0080] Among them, t k is the corresponding spectral frequency; the number of basis functions K determines the function fitting effect of the original data. Generally, the number of hyperspectral bands M and the number of B-spline basis functions p are used to determine the number of basis functions, that is, K = Mp–1;
[0081] Considering the data fitting effect and computational complexity, p=3 degree B-spline is usually selected for fitting:
[0082] Formula (1) can be rewritten as a linear combination of a set of B-spline systems:
[0083] x i (t) = B(t) T c i (3)
[0084] Where B(t)=(N 1,3 (t),N 2,3 (t),…,N K,3 (t)) T ;c i =(c i1 ,c i2 ,…,c iK ) T ;
[0085] Use the simple least squares fitting method to solve the coefficient c in equation (3): i , the sum of squares of the error terms can be expressed as:
[0086] SSEi =||y i -x i || 2 =(y i -Bc i ) T (y i -Bc i ) (4)
[0087] Where B = (B (t 1 ),B(t 2 ),…,B(t M )) T is the B-spline basis matrix of order M×K; i =(x i (t 1 ),x i (t 2 ),…,x i (t M )) T is the function value of the spectrum fitting function;
[0088] Step 1.3: In order to obtain the optimal spectral curve fitting function and better express the characteristics of hyperspectral data, it is also necessary to consider: (1) The noise contained in hyperspectral data generally does not satisfy the standard Gaussian distribution model of independent and identical distribution, and there are non-stationary errors; (2) The least squares fitting model is difficult to control the smoothness of the fitting curve, and the hyperspectral data contains some spectral detail information with weak resolution that brings roughness to the spectral curve; (3) The above FDA only uses spectral information but does not consider spatial information. For more accurate classification, spatial neighborhood constraints should be introduced, but the traditional regular spatial neighborhood cannot effectively reveal the spatial structure information of hyperspectral images compared to superpixel neighborhoods. In view of this, a new functional representation framework for hyperspectral images is constructed, and the error term sum of squares model is expanded to:
[0089] SSE i =SSE W +SSE λ +SSE η (5)
[0090] Among them, SSE W is the weighted model of formula (4), which is used to measure the fitting effect of the function curve and is defined as:
[0091] SSE W =(y i -Bc i ) T W(y i -Bc i ) (6)
[0092] Among them, W is the weight matrix, which is calculated by the mean square residual. Since the variance-covariance matrix of the residuals cannot be fully estimated in general, the covariance between the residuals is set to 0, then W is the diagonal weight matrix, SSE λ is a rough penalty term, which is used to measure the smoothness of the fitting function itself and is defined as:
[0093] SSE λ =λ∫[D (m) x i (t)] 2 dt=λc i T [∫D (m) B(t)D (m) B(t) T dt]c i =λc i T Rc i (7)
[0094] Among them, R is the inner product matrix of the m-order derivative of the B-spline basis system, that is, R = ∫D (m) B(t)D (m) B(t) T dt, integral term [D (m) x i (t)] 2 Represents x i The m-th order derivative of (t), usually set m = 2 to calculate the curvature, λ (≥ 0) is the smoothing parameter, determined by generalized cross validation. SSE η is a spatial coherence term, which is used to introduce spatial relationships in the function fitting process and is defined as:
[0095]
[0096] in, is the average function representation of the superpixel neighborhood where pixel i is located, for The vector representation of the integral term The purpose of is to enhance the spatial consistency between pixels, so that the spectral curve fitting function of a pixel is as similar as possible to the spectral curve fitting functions of other pixels in its superpixel neighborhood, and η is a coefficient that controls the degree of spatial constraint;
[0097] In order to obtain the average function representation of the neighborhood of any pixel i It is necessary to accurately determine the neighborhood range of pixel i; entropy rate superpixel segmentation, ERS, is used to generate superpixel segmentation maps; the ERS algorithm is a graph-based clustering method, which first maps the image to a graph G = (V, E), where V is the vertex set of pixels and E is the edge weight set of the similarity of adjacent pixels; then, according to the given number of superpixels L, the objective function is optimized, that is, by selecting a subset A of E, a graph G = (V, A) containing L connected subgraphs is obtained; its objective function consists of the entropy rate H(A) of random walks on the graph and the balance term B(A) on the cluster distribution, that is:
[0098]
[0099] Among them, α≥0 is the weight coefficient used to adjust the balance between the two items; compared with simple linear iterative clustering (SLIC), the ERS algorithm does not excessively pursue the size similarity of the segmented area, can distinguish the boundaries at a lower cost, and can better reflect the spatial regularity of the objects in the scene; in order to reduce the computational complexity, PCA is first used to obtain the first principal component of the hyperspectral image, and then ERS is applied to it to obtain the superpixel segmentation result S = {S l ,l=1,…,L}, where l is the superpixel region index and L is the number of superpixels; let S l is the superpixel where pixel i is located, and S is defined l The average function of is expressed as:
[0100]
[0101] Where U is S l The number of pixels included; x i '(t) is S l The initial spectral curve fitting function of each pixel in, at this time i=1,…,U;
[0102] Step 1.4: Substitute equations (6)–(8) into equation (5) to construct the optimal error term sum of squares model:
[0103]
[0104] Then the optimal functional representation coefficient c i * It can be obtained by minimizing formula (11), that is:
[0105]
[0106] By i Find the partial derivative and get c i* ,Right now:
[0107]
[0108] Since the spectral curve fitting function x of each pixel i '(t) is a priori unknown, and the average function of each superpixel is expressed as It has not been explicitly expressed yet; set the spatial constraint parameter η in equation (13) to 0 and solve it to obtain the initial coefficient c i , that is, c i =(B T WB+λR) -1 B T Wy i , using c i Calculate x i '(t) can be calculated by formula (11) Then each pixel in the hyperspectral image can be optimally represented as:
[0109]
[0110] After the hyperspectral image is functionalized under multiple constraints, the representation of hyperspectral image pixel i is represented by vector y i Convert to function representation
[0111]
[0112] Step 2: Perform functional principal component analysis on the hyperspectral image function representation under multiple constraints;
[0113] After obtaining the functional representation of the hyperspectral image under multiple constraints, FPCA can be used to generate the principal component of the functional data. FPCA is an extension of the traditional PCA in the field of functional data processing. It regards the function as a variable and converts the task of finding the eigenvalues and eigenvectors of the covariance matrix into the characteristic analysis problem of solving the eigenfunction ξ(t) of the covariance function V(s,t). Assuming that D (≤K) principal component components are used for classification, the dth principal component f of pixel i is generated. id as follows:
[0114]
[0115] Where d≤D,ξ d (t) is the normalized eigenfunction corresponding to the dth principal component, which is orthogonal to the eigenfunctions corresponding to the remaining D-1 principal components; ξ d (t) is expressed by the same set of B-spline systems:
[0116]
[0117] Where b is the coefficient vector. Convert the discrete eigenvalue equation Vξ=ρξ into a functional eigenvalue equation:
[0118] ∫V(s,t)ξ d (t)dt=ρξ d (s) (17)
[0119] Among them, ρ is the eigenvalue. According to formula (3), we have Expand the covariance function V(s,t) as:
[0120]
[0121] Formula (17) can be rewritten as:
[0122] N -1 c i *T c i * Hb=ρb (19)
[0123] Where H = ∫B(t)B(t) T dt is the inner product of the B-spline basis function; after obtaining b, the eigenfunction ξ can be obtained through equation (16): d (t), then ξ d Substitute (t) into formula (15) to calculate ξ d (t) and x i The inner product of (t) can get the principal component f id ; After feature extraction using FPCA, the functional representation of pixel i under multiple constraints The corresponding principal component is f i , f i =(f id ,d=1,…,D), then all pixel principal components constitute an N×D-dimensional principal component set f={f i , i=1,…,N}, at this time, the classification of the hyperspectral image y is converted into the classification of the principal component set f;
[0124] Step 3: Use support vector machine to classify the principal component set;
[0125] First, remove the background point information in the hyperspectral image y corresponding to the principal component set f to obtain the sample set, and then randomly select training samples from each category of the sample set according to a certain proportion, and the rest are used as test samples; suppose the training sample set f train ={(f z’ ,v z’ ),z'=1,…,Z}, where f z’is the sample vector; v z’ ∈{+1,-1} is the category number; z' is the training sample index; Z is the total number of training samples; for the V-category classification problem of hyperspectral images, the one-against-one, OAO criterion is used to transform it into a set of two-category classification problems, that is, a SVM classifier is designed between any two types of training samples, and a total of V(V-1) / 2 classifiers are trained; the problem of finding the optimal classification hyperplane for SVM two-category classification can be transformed into a minimization constraint programming problem:
[0126]
[0127]
[0128] Among them, C is the preset penalty factor, the weight vector ω and the bias b are the optimal hyperplane ω T ·f z’ +b=0 parameters, ξ z’ is the relaxation factor; by introducing the Lagrangian function to solve the dual problem of equation (20), the Lagrangian multiplier α can be obtained z’ * , and then use α z’ * Find the optimal solution of parameters ω and b * and b * , the classification discriminant function can be constructed according to the optimal solution parameters:
[0129]
[0130] Among them, sgn() is the sign function, K(f i ,f j ) is the kernel function that satisfies Mercer's theorem, which is defined by the Gaussian kernel function as:
[0131]
[0132] Where σ is the width parameter of the kernel function. For any test sample, V(V-1) / 2 SVM classifiers are used for classification, and the final classification result is the label with the most votes among the V(V-1) / 2 predicted labels.
[0133] Experimental data and parameter settings:
[0134] The experiment selected widely used real hyperspectral datasets: Indian Pines, University of Pavia and Salinas to verify the advantages of the proposed MC-FDA+SVM method on the MATLAB R2017b platform. The water absorption bands were removed from the three sets of hyperspectral image datasets, and the experiments were carried out using effective bands. The basic information can be seen in Table 1.
[0135] Table 1 Standard datasets for hyperspectral image classification used in the experiment
[0136] Tab.1 The standard dataset of hyperspectral image classification used in the experiment
[0137]
[0138] Since the superpixel neighborhood information in MC-FDA+SVM largely determines the functional representation of the spectral measure, and there is no fixed standard for determining the number of superpixels, we first explore the sensitivity of the number of superpixels under different sample conditions. Figure 1 The relationship between the number of superpixels and the overall classification accuracy of the three datasets at 1%, 5%, and 10% training samples is shown respectively. The number of superpixels is set to {20, 50, 75, 100, 150, 200, 300}. In order to highlight the trend of accuracy changes, the University of Pavia dataset adds an experiment with 10 superpixels.
[0139] As attached Figure 1 As shown in the figure, as the number of superpixels increases, the overall classification accuracy almost always continues to rise first, reaches the optimal classification accuracy, and then gradually decreases. This is because when the number of superpixels is too small, the superpixel will contain information from other categories, resulting in mixed superpixels, which affects the classification accuracy; when the number of superpixels is too large, each superpixel is insufficient to provide sufficient neighborhood spatial information for constraint, resulting in a decrease in classification accuracy.
[0140] After determining the optimal number of superpixels, a sensitivity analysis of the spatial constraint parameter η was performed when the number of superpixels in the three datasets was optimal, namely 150, 20, and 50, respectively, under 10% training samples. The η values of the Indian Pines and Salinas datasets were all set to {1, 3, 5, 7, 9, 15, 20, 30}, and the η value of the University of Pavia dataset was set to {1, 3, 5, 7, 9, 15, 20}. Figure 2As shown in the figure, the overall classification accuracy of the University of Pavia hyperspectral image increases rapidly with the increase of η and then shows a downward trend early. The distribution of objects in the Indian Pines dataset is relatively irregular, and a higher spatial information importance needs to be imposed to achieve the optimal classification accuracy. However, excessively increasing the spatial constraint parameter will correspondingly reduce the weight of spectral information, resulting in reduced classification accuracy. The spatial resolution of the Salinas dataset is much higher than that of the Indian Pines dataset, and a slight spatial constraint can achieve excellent classification accuracy without relying too much on the choice of η.
[0141] In order to verify the effectiveness of the superpixel algorithm, after functionalizing the hyperspectral dataset to obtain the spectral curve fitting function of each pixel, the present invention selected three representative ground object categories from the 16 ground object categories in the Indian Pines dataset: category 1 with irregular boundaries, category 8 with relatively regular boundaries, and category 9 with a small number of samples (see Appendix 1). Figure 4 (b)), and randomly select 4 sample points from each class, and compare the polyline drawn by the spectral measurement vector with the fitting curve, as shown in the attached figure. Figure 3 As shown. Applying a rough penalty term smoothes the areas with excessive amplitude changes in the original spectral vector, that is, it eliminates some spectral details with weak resolution. Applying a superpixel constraint minimizes the distance between each pixel and other pixel functions in its superpixel. Due to the powerful edge extraction capability of ERS, the ground object categories are well distinguished. It can be seen that there are some differences in the inter-class spectral measurement curves of randomly selected sample points of classes 8 and 9, but the function fitting is basically similar. Therefore, these sample points can be correctly classified, while the fourth sample point a4 of class 1 may not be accurately classified because it is not divided into the correct superpixel area. Nevertheless, from the effect of function fitting, it can be seen that the result of superpixel segmentation is reliable. Therefore, applying a superpixel constraint increases the stability of the fitting curve expression and the distinguishability of the features extracted by subsequent dimensionality reduction.
[0142] Experimental results and analysis:
[0143] The MC-FDA+SVM algorithm is compared and analyzed by combining the traditional PCA and SVM (PCA+SVM), replacing the traditional PCA from the functional perspective and then using SVM (FPCA+SVM), and using the local neighborhood information for spatial constraint function data analysis method (SFDA+SVM) based on FPCA. The range of the smoothing parameter λ of FPCA+SVM, SFDA+SVM and MC-FDA+SVM is set to 10. -8 , 10 -7 ,…,1. For the SVM classifier, set the penalty term c and Gaussian kernel parameter g to a range of 2 -10 , 2 -9, …, 2 10 . The cross-validation method was used to obtain the optimal selection of the above parameters for each algorithm. The settings of the remaining parameters of SFDA+SVM are the same as those in the original paper, that is, the spatial constraint parameter η is set to 1, the number of iterations is set to 2, and the spatial neighborhood window is 5×5. In addition, in order to reduce the computational complexity, each algorithm uses 30 principal components to approximately optimize the classification performance. In order to test the case of finite samples, the present invention randomly selects γ training samples from each class of the ground truth images of the three data sets used (γ=1%, 3%, 5%, 7%, 10%), and the rest are used as test samples. The overall accuracy (OA[%]), average accuracy (AA[%]) and Kappa coefficient are used to comprehensively evaluate each method. In order to avoid the influence of bias and sample selection, each experiment was repeated ten times to obtain the average value.
[0144] Table 2 shows the classification accuracy of the Indian Pines dataset under the condition of small sample size (γ = 1%), including overall accuracy (OA [%]), average accuracy (AA [%]), Kappa coefficient and various types of accuracy [%]. It can be seen that when γ = 1%, the classification results of the comparison algorithms are greatly affected and cannot obtain satisfactory classification accuracy, while the MC-FDA + SVM algorithm achieves the best in terms of overall accuracy, average accuracy and Kappa coefficient. It can also achieve the best classification effect for almost all land object categories, especially when there is only one training sample (classes 1, 7, 9, 16), the algorithm shows an absolute advantage over the comparison algorithm. This shows that the proposed MC-FDA + SVM algorithm can adapt well to the small sample situation. For the categories that the comparison algorithm cannot classify well (such as class 6, 10, 12, 15), MC-FDA + SVM also achieves good classification accuracy, which shows the stability of the proposed algorithm. Tables 3–5 list the overall accuracy of all methods under different proportions of training samples for the Indian Pines, University of Pavia, and Salinas datasets. It can be seen that the overall accuracy of all methods increases with the increase of training samples, and the MC-FDA + SVM algorithm also shows a significant classification accuracy advantage in the case of low training samples (3%) compared with the other comparison algorithms in the case of sufficient samples (10%), indicating that the superpixel spatial neighborhood information can make up for the lack of training samples to a certain extent. For the Indian Pines dataset, the overall accuracy of FPCA + SVM is better than that of PCA + SVM, indicating that when processing complex datasets with low spatial resolution, the function representation of spectral vectors can better reflect the spectral characteristics. For the University of Pavia and Salinas datasets with high spatial resolution and easy to distinguish categories, the overall accuracy of FPCA + SVM is slightly lower than that of PCA + SVM, indicating that when smoothing the spectral curve, some spectral details that are helpful for classification will be eliminated. By comparing the FPCA+SVM, SFDA+SVM, and MC-FDA+SVM algorithms, we can conclude that the superiority of the method combining spatial and spectral information compared to classification using only spectral information, and that superpixels have stronger neighborhood constraints and spatial structure representation than traditional neighborhood spaces. In addition, since the objects in the University of Pavia and Salinas datasets are evenly distributed, the comparison algorithms can achieve good classification accuracy when there are enough sample points, and the algorithm of the present invention is still ahead of the comparison algorithm.
[0145] Table 2 Classification results of Indian Pines dataset under γ=1%
[0146] Tab.2 Classification results of Indian Pines data set underγ=1%
[0147]
[0148] Table 3 Overall classification accuracy of the Indian Pines dataset under different γ
[0149] Tab.3 Overall classification accuracy of Indian Pines data set underdifferentγ
[0150]
[0151]
[0152] Table 4 Overall classification accuracy of the University of Pavia dataset under different γ
[0153] Tab.4 Overall classification accuracy of University of Pavia data setunder differentγ
[0154]
[0155] Table 5 Overall classification accuracy of the Salinas dataset under different γ
[0156] Tab.5 Overall classification accuracy of Salinas data set underdifferentγ
[0157]
[0158] The classification diagrams generated by various methods for the Indian Pines, University of Pavia, and Salinas datasets with 10% training samples are shown in the attached figure. Figure 4 , Attachment Figure 5 and attached Figure 6 As shown in Figure 2. From a visual point of view, MC-FDA+SVM achieved the best classification effect, especially the classification diagram of the Salinas dataset is almost the same as the standard classification diagram. Figure 1For the Indian Pines dataset with low spatial resolution, the PCA+SVM and FPCA+SVM methods only use spectral information, and the boundaries are severely damaged. However, MC-FDA+SVM can distinguish the boundaries of objects well while greatly reducing the number of misclassified pixels, which intuitively proves the superiority of the proposed algorithm.
[0159] By combining FDA and superpixel segmentation methods, this paper proposes a functional representation method for hyperspectral images under multi-constraint conditions, which takes into account non-stationary errors, fitting curve smoothness and pixel neighborhoods. It can make full use of spectral and spatial constraint information to improve the classification accuracy of hyperspectral images. On the one hand, the use of B-spline basis functions to express spectral information solves the "dimensionality curse" problem in hyperspectral image processing; on the other hand, the functional representation framework under multi-constraint conditions effectively utilizes superpixel neighborhood information, alleviates the problem of lack of training samples, and improves the classifier's ability to distinguish features extracted from complex scenes. The fitting results show that compared with the original spectral vector, the fitting curve can express spectral information more stably. The experimental results compared with other homologous technologies based on SVM show that MC-FDA+SVM has higher classification accuracy. Experiments on three real hyperspectral data sets have demonstrated the effectiveness and superiority of the proposed algorithm.
Claims
1. A hyperspectral image classification method based on functional representation under multiple constraints. It is characterized in that The specific steps include: Step 1: Functionalize the hyperspectral image under multiple constraints; Step 1.1: Given a hyperspectral image y = {y i ,i=1,…,N}, where i is the pixel index, N is the total number of pixels, and y i =(y ij ,j=1,…,M) T is the spectral measurement vector of pixel i, T is the transpose, j is the band index, M is the total number of bands, y ij is the spectral measurement of pixel i in band j; since hyperspectral remote sensing uses continuous, narrow bandwidth spectral channel imaging, y i Expressed as a high-dimensional vector; Step 1.2: Hyperspectral image processing based on high-dimensional features has information redundancy, dimensionality disaster, and difficulty in modeling high-order statistical features. The FDA method is used to interpret the hyperspectral image and express the spectral curve as a spectral curve fitting function. Due to the high spectral resolution of the hyperspectral image, y i As a high-density sampling of the reflectance spectrum curve of the pixel i area, it becomes the information source for constructing the spectrum curve fitting function; assuming that the smooth function x i (t) is the spectral curve fitting function of pixel i, which is expressed as the weighted sum of a set of independent basis functions: Among them, k is the basis function index, K is the number of basis functions, φ k (t) is the basis function, c ik is the weight coefficient corresponding to the basis function; due to the advantages of B-spline function in approximating non-periodic function data and parameter estimation, B-spline function is selected as the basis function of spectral curve fitting function; let the band center frequency corresponding to band j in the given hyperspectral image y be t j , according to the spectral frequency t of the hyperspectral image 1 <t 2 <… <t M , the p-th k-th B-spline basis function N is defined by the deBoor-Cox recursive formula k,p The general form of (t) is as follows: Among them, t k is the corresponding spectral frequency; the number of basis functions K determines the function fitting effect of the original data, and the number of basis functions is determined by the number of hyperspectral bands M and the number of B-spline basis functions p, that is, K = Mp-1; Considering the data fitting effect and computational complexity, p=3 degree B-spline is selected for fitting: Formula (1) can be rewritten as a linear combination of B-spline systems: x i (t)=B(t) T c i (3) where B(t) = (N 1,3 (t), N 2,3 (t), …, N K,3 (t)) T ; c i = (c i1 , c i2 , …, c iK ) T ; Use the simple least squares fitting method to solve the coefficient c in equation (3): i , and the sum of squares of the error terms is expressed as: SSE i =||y i -x i || 2 =(y i -Bc i ) T (y i -Bc i ) (4) Where B = (B (t 1 ),B(t 2 ),…,B(t M )) T is the M×K order B-spline basis matrix; x i =(x i (t 1 ),x i (t 2 ),…,x i (t M )) T is the function value of the spectrum fitting function; Step 1.3: In order to obtain the optimal spectral curve fitting function and better express the characteristics of hyperspectral data, a new hyperspectral image functional representation framework is constructed, and the error term sum of squares model is expanded to: SSE i =SSE W +SSE λ +SSE η (5) Among them, SSE W is the weighted model of formula (4), which is used to measure the fitting effect of the function curve and is defined as: SSE W =(y i -Bc i ) T W(y i -Bc i ) (6) Among them, W is the weight matrix, which is calculated by the mean square residual. Since the variance-covariance matrix of the residuals cannot be fully estimated in general, the covariance between the residuals is set to 0, then W is the diagonal weight matrix, SSE λ is a rough penalty term, which is used to measure the smoothness of the fitting function itself and is defined as: SSE λ =λ∫[D (m) x i (t)] 2 dt=λc i T [∫D (m) B(t)D (m) B(t) T dt]c i =λc i T Rc i (7) Among them, R is the inner product matrix of the m-order derivative of the B-spline basis system, that is, R = ∫D (m) B(t)D (m) B(t) T dt, integral term [D (m) x i (t)] 2 Represents x i The m-th order derivative of (t), with m = 2, is used to calculate the curvature, and λ ≥ 0 is the smoothing parameter, which is determined by the generalized cross validation method; SSE η is a spatial coherence term, which is used to introduce spatial relationships in the function fitting process and is defined as: in, is the average function representation of the superpixel neighborhood where pixel i is located, for The vector representation of the integral term The purpose of is to enhance the spatial consistency between pixels, so that the spectral curve fitting function of a pixel is as similar as possible to the spectral curve fitting functions of other pixels in its superpixel neighborhood, and η is a coefficient that controls the degree of spatial constraint; In order to obtain the average function representation of the neighborhood of any pixel i It is necessary to accurately determine the neighborhood range of pixel i; entropy rate superpixel segmentation, ERS, is used to generate superpixel segmentation maps; the ERS algorithm is a graph-based clustering method, which first maps the image to a graph G = (V, E), where V is the vertex set of pixels and E is the edge weight set of the similarity of adjacent pixels; then, according to the given number of superpixels L, the objective function is optimized, that is, by selecting a subset A of E, a graph G = (V, A) containing L connected subgraphs is obtained; its objective function consists of the entropy rate H(A) of random walks on the graph and the balance term B(A) on the cluster distribution, that is: Among them, α≥0 is the weight coefficient used to adjust the balance between the two items; st is the abbreviation of subject to; in order to reduce the computational complexity, PCA is first used to obtain the first principal component of the hyperspectral image, and then ERS is applied to it to obtain the superpixel segmentation result S={S l ,l=1,…,L}, where l is the superpixel region index and L is the number of superpixels; let S l is the superpixel where pixel i is located, and S is defined l The average function of is expressed as: Where U is S l The number of pixels included; x i '(t) is S l The initial spectral curve fitting function of each pixel in, at this time i=1,…,U; Step 1.4: Substitute equations (6)-(8) into equation (5) to construct the optimal error term sum of squares model: Then the optimal functionalization coefficient c i * By minimizing formula (11), we can obtain: By i Find the partial derivative and get c i * ,Right now: Since the spectral curve fitting function x of each pixel i '(t) is a priori unknown, and the average function of each superpixel is expressed as It has not been explicitly expressed yet; set the spatial constraint parameter η in equation (13) to 0 and solve it to obtain the initial coefficient c i , that is, c i =(B T WB+λR) -1 B T Wy i , using c i Calculate x i '(t) is calculated by formula (11) to obtain Then each pixel in the hyperspectral image is optimally represented as: After the hyperspectral image is functionalized under multiple constraints, the representation of hyperspectral image pixel i is represented by vector y i Convert to function representation Step 2: Perform functional principal component analysis on the hyperspectral image function representation under multiple constraints to generate a principal component set; at this time, the classification of the hyperspectral image is converted into the classification of the principal component set; Step 3: Use support vector machine to classify the principal component set.
2. The hyperspectral image classification method based on functional representation under multiple constraints according to claim 1, It is characterized in that Step 2 is as follows: After obtaining the functional representation of the hyperspectral image under multiple constraints, FPCA is used to generate the principal component of the functional data. FPCA is an extension of the traditional PCA in the field of functional data processing. It regards the function as a variable and converts the task of finding the eigenvalues and eigenvectors of the covariance matrix into the characteristic analysis problem of solving the eigenfunction ξ(t) of the covariance function V(s, t). Assuming that D≤K principal component components are used for classification, the dth principal component f of pixel i is generated. id as follows: Where d≤D,ξ d (t) is the normalized eigenfunction corresponding to the dth principal component, which is orthogonal to the eigenfunctions corresponding to the remaining D-1 principal components; ξ d (t) is expressed by the same set of B-spline systems: Where b is the coefficient vector; the discrete eigenvalue equation Vξ=ρξ is converted into a functional eigenvalue equation: ∫V(s,t)ξ d (t)dt=ρξ d (t) (17) Among them, ρ is the eigenvalue; according to formula (3), we have Expand the covariance function V(s,t) as: Formula (17) can be rewritten as: N -1 c i *T c i * Hb=ρb (19) Where H = ∫B(t)B(t) T dt is the inner product of the B-spline basis function; after obtaining b, the eigenfunction ξ is obtained through equation (16): d (t), then ξ d Substitute (t) into equation (15) to calculate ξ d (t) and x i (t) is the inner product, that is, the principal component f id ; After feature extraction using FPCA, the functional representation of pixel i under multiple constraints The corresponding principal component is f i , f i =(f id ,d=1,…,D), then all pixel principal components constitute an N×D-dimensional principal component set f={f i , i=1,…,N}, at this time, the classification of the hyperspectral image y is converted into the classification of the principal component set f.
3. The hyperspectral image classification method based on functional representation under multiple constraints according to claim 1, It is characterized in that Step 3 is as follows: First, remove the background point information in the hyperspectral image y corresponding to the principal component set f to obtain the sample set, and then randomly select training samples from each category of the sample set according to a certain proportion, and the rest are used as test samples; suppose the training sample set f train ={(f z’ ,v z '),z'=1,…,Z}, where f z ' is the sample vector; v z '∈{+1,-1} is the category number; z' is the training sample index; Z is the total number of training samples; for the V-category classification problem of hyperspectral images, the one-against-one, OAO criterion is used to transform it into a set of two-category classification problems, that is, a SVM classifier is designed between any two types of training samples, and a total of V(V-1) / 2 classifiers are trained; the problem of finding the optimal classification hyperplane for SVM two-category classification is transformed into a minimization constraint programming problem: Among them, C is the preset penalty factor, the weight vector ω and the bias b are the optimal hyperplane ω T ·f z’ +b=0 parameters, ξ z’ is the relaxation factor; by introducing the Lagrangian function to solve the dual problem of equation (20), we can get the Lagrangian multiplier α z’ * , and then use α z’ * Find the optimal solution of parameters ω and b * and b * , construct the classification discriminant function according to the optimal solution parameters: Among them, sgn() is the sign function, K(f i ,f j ) is the kernel function that satisfies Mercer's theorem, which is defined by the Gaussian kernel function as: Where σ is the width parameter of the kernel function. For any test sample, V(V-1) / 2 SVM classifiers are used for classification, and the final classification result is the label with the most votes among the V(V-1) / 2 predicted labels.
