Hyperspectral image multi-level feature representation method based on tensor auto-encoder network

Abstract

The invention discloses a hyperspectral image multi-level feature representation method based on a tensor auto-encoder network, and the method comprises the steps: firstly introducing a deep auto-encoder network, and constructing a hyperspectral image feature representation frame based on the deep auto-encoder network; and secondly, performing tensorization processing on the deep auto-encoder network, and establishing a hyperspectral image multi-level feature representation model based on the tensorization auto-encoder network, namely stacking output of each layer of the deep auto-encoder network to obtain a tensor form, and constructing and introducing a tensor decomposition item to realize reverse transmission of the auto-encoder network. According to the method, the depth auto-encoder network is subjected to tensorization processing, the multi-level feature representation capacity is improved, output of all layers of the depth auto-encoder network can be combined in a unified mode, the shallow layer output characteristics and the deep layer output characteristics are covered at the same time, and therefore a reliable judgment basis is provided for tasks such as target classification.

Description

Technical Field

[0001] The present invention belongs to the field of hyperspectral image processing, and in particular relates to a multi-level feature representation method for hyperspectral images based on a tensor autoencoder network. Background Art

[0002] Hyperspectral image feature representation is an important technology in the field of hyperspectral image processing. It aims to extract and express features from hyperspectral image data that can describe the image content and distinguish different objects or targets. These features usually combine spectral information and spatial information, and can fully and accurately reflect the characteristics of hyperspectral images, providing strong data support for subsequent classification, recognition, detection and other applications. In addition, through feature representation, the originally complex and redundant hyperspectral data can be made concise and efficient, thereby improving the efficiency and accuracy of data processing. At the same time, a good feature representation method can also enhance the distinction between samples of different categories and improve the accuracy and robustness of the classification algorithm.

[0003] Deep learning is considered to be the most advanced and effective means of hyperspectral image feature representation. It can mine and utilize the deep features hidden behind hyperspectral image data. Compared with traditional machine learning methods, feature extraction methods based on deep learning can obtain more complex structural information and have stronger robustness and feature perception capabilities. It should be noted that most of the existing deep learning methods use the deepest output features in the deep network as feature representation, but lack multi-level feature representation capabilities and ignore the role of shallow features in the deep network. If the multi-level feature representation capabilities of deep learning can be improved, shallow output and deep output can be fully combined and utilized to construct a representation network that can output multi-level features, this will be very valuable and meaningful for analysis tasks related to hyperspectral images. Summary of the invention

[0004] The technical problem to be solved by the present invention is to design a multi-level feature representation method that can simultaneously utilize the shallow and deep features of hyperspectral images, thereby providing a reliable basis for classification and other related tasks.

[0005] In order to achieve the above object, the present invention adopts the following technical scheme: a multi-level feature representation method of hyperspectral images based on a tensor autoencoder network, comprising the following steps:

[0006] Step 1: Transform the hyperspectral image Expand into a matrix representation X∈R Q×L , where Q = S × P represents the spatial resolution of the hyperspectral image, and L represents the spectral resolution of the hyperspectral image;

[0007] Step 2: Input the hyperspectral image X into the autoencoder network to encode the hyperspectral image X, that is:

[0008]

[0009] Among them, λ represents the regularization parameter, V j , U j , X j and Y j They represent the encoding matrix, decoding matrix, input quantity, and output features of the jth layer in the network respectively; J represents the total number of layers of the autoencoder network;

[0010] X j =Y j-1 (j≥2) and X 1 =X;

[0011] Step 3: Perform tensor quantization on equation (1) and stack all network layer output features Y j ∈R Q×L′ (1≤j≤J), get the tensor form Formula (1) is transformed into:

[0012]

[0013] Among them, α is a coupling parameter used to control the importance of tensor decomposition terms; Represents tensor product; Cat(·) represents the output feature Y for stacking all layers j The operator, express The tensor core of Tensor decomposition is performed to obtain, where Q″<<Q, K″<<L′, J″<<J; In addition, the factor matrix W 1 ∈R Q×Q″ , W 2 ∈R L′×L″ , W 3 ∈R J ×J″ ;

[0014] Since the factor matrix W 1 , W 2 and W 3 They are The expanded matrix of the potential principal components in different modes is the factor matrix W 3 The corresponding potential principal component constraint is the identity matrix, so equation (2) is written as:

[0015]

[0016] in, Represented in the factor matrix W 1 ′ and W 2 ′ under the tensor core;

[0017] because It is different from Y j Along the connection in the direction of the network layer, the multi-level feature representation model of the hyperspectral image (3) is rewritten as follows:

[0018]

[0019] Among them, F j Represents Y j The eigenmatrix form of ;

[0020] Step 4: Solve the multi-level feature representation model (4) of the hyperspectral image to obtain the output Y of each layer j The eigenvalue matrix F j The optimal solution and the factor matrix W′ along the spectral direction 1 ;

[0021] Step 5, the final multi-level feature representation F of the hyperspectral image is:

[0022]

[0023] It can be seen from formula (20) that the final feature representation is the output Y of each layer j The intrinsic matrix form of the proposed method is multiplied and accumulated with its factor matrix along the spectral direction. Compared with most traditional feature extraction methods based on deep learning, the features extracted by the present invention contain both shallow and deep features of the hyperspectral image data and have richer information.

[0024] Furthermore, in step 4, the multi-level feature representation model (4) of the hyperspectral image is solved to obtain the output Y of each layer. j The eigenvalue matrix F j The optimal solution and the factor matrix W′ along the spectral direction 1 , the specific steps are as follows:

[0025] First, rewrite the constraint-containing formula (4) into the following unconstrained form:

[0026]

[0027] Among them, γ is the coupling parameter;

[0028] Introduce auxiliary variable A j =Y j , formula (5) is rewritten as:

[0029]

[0030] Among them, β is the coupling parameter;

[0031] The alternating update optimization strategy is used to solve equation (6):

[0032] 1) Update V j : Fixed U j , Y j 、F j , W′ 1 , W′ 2 and A j , then the variable V j Update it with:

[0033]

[0034] The closed-form solution of formula (7) is expressed as:

[0035]

[0036] 2) Update U j : Fixed V j , Y j 、F j , W′ 1 , W′ 2 and A j , then the variable U j Update it with:

[0037]

[0038] The solution of formula (9) is expressed as:

[0039] U j =(X j Y j T )(Y j Y j T ) -1 (8)

[0040] 3) Update Y j : Fixed V j , U j 、F j , W′ 1 , W′ 2 and A j , then the variable Y j Update it with:

[0041]

[0042] The solution of formula (10) is expressed as:

[0043]

[0044] Where I represents the identity matrix;

[0045] 4) Update F j : Fixed V j , U j , Y j , W′ 1 , W′ 2 and A j , then the variable F j Update it with:

[0046]

[0047] The solution of formula (12) is expressed as:

[0048]

[0049] 5) Update W′ 1 : Fixed V j , U j , Y j 、F j , W′ 2 and A j , then the variable W′ 1 Update it with:

[0050]

[0051] The solution of formula (14) is expressed as:

[0052]

[0053] 6) Update W′ 2 : Fixed V j , U j , Y j 、F j , W′ 1 and A j , then the variable W′ 2 Update it with:

[0054]

[0055] The solution of formula (16) is expressed as:

[0056]

[0057] 7) Update A j : Fixed V j , U j , Y j 、F j , W′ 1 and W′ 2 , then variable Aj Update it with:

[0058]

[0059] The solution of formula (18) is expressed as:

[0060]

[0061] Among them, soft(x,y)=sign(x)·max(|x|-y,0) is the soft threshold operation function; here, sign(x) is the sign function, and max(|x|-y,0) is the maximum value function, which is used to compare the size of |x|-y and 0;

[0062] After iterative updates from 1) to 7), the optimal V j , U j , Y j 、F j , W′ 1 , W′ 2 and A j .

[0063] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0064] The multi-level feature representation method of hyperspectral images based on tensorized autoencoder networks proposed in this invention stacks the outputs of each layer of the deep autoencoder network to obtain a tensor form, and constructs and introduces tensor decomposition terms to achieve reverse transmission of the autoencoder network. Compared with existing deep learning methods, this method has significant multi-level feature representation capabilities, can unify the outputs of each layer of the deep autoencoder network, and covers both shallow and deep output characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a flow chart of the steps of the present invention; DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0067] The present invention constructs a representation network that can output multi-level features, and can simultaneously use shallow outputs and deep outputs as discriminative features to perform related tasks, such as classification. It is not difficult to find that tensors can unify the outputs of each layer of the deep network. Therefore, tensorization of the existing deep network is expected to achieve multi-level feature representation of hyperspectral images. In this embodiment, a method for multi-level feature representation of hyperspectral images based on a tensorized autoencoder network is proposed. The outputs of each layer of the deep autoencoder network are stacked to obtain a tensor form, and tensor decomposition terms are constructed and introduced to achieve reverse transmission of the autoencoder network. Compared with existing deep learning methods, this method has significant multi-level feature representation capabilities, can unify the outputs of each layer of the deep autoencoder network, and covers both shallow output characteristics and deep output characteristics. The overall flow chart of the present invention is as follows: Figure 1 As shown, the main steps involved are as follows:

[0068] Step 1: Input hyperspectral image Its two-dimensional matrix form can be expressed as X∈R Q×L , where Q = S × P represents the spatial resolution of the hyperspectral image, and L represents the spectral resolution of the hyperspectral image.

[0069] Step 2: Introduce an autoencoder network to encode the hyperspectral image X, that is:

[0070]

[0071] Among them, λ represents the regularization parameter, V j , U j , X j and Y j They represent the encoding matrix, decoding matrix, input quantity and output features of the jth layer in the network respectively. Here X j =Y j-1 (j≥2) and X 1 =X.

[0072] Step 3: Perform tensor quantization on equation (1), that is, stack all network layer output features Y j ∈R Q×L′ (1≤j≤J), obtain the corresponding tensor form In this way, formula (1) can be transformed into:

[0073]

[0074] Among them, α is a coupling parameter used to control the importance of the tensor decomposition terms. Represents tensor product. Cat(·) represents the output feature Y for stacking all layers jIt is worth noting that in order to ensure that the output features Y of different layers j Can be stacked smoothly, set each Y j The dimensions of are the same. In formula (2), express The tensor core is obtained by Tensor decomposition is performed to obtain, where Q″<<Q, L″<<L′, J″<<J. In addition, the factor matrix W 1 ∈R Q×Q″ , W 2 ∈R L′×L″ , W 3 ∈R J×J″ As can be seen from formula (2), the output features of all network layers are stacked into a tensor and subjected to tensor decomposition operations. At the same time, the tensor and its decomposition operations also affect the autoencoder network, thereby realizing multi-level feature representation of hyperspectral images.

[0075] Considering that formula (2) contains both matrix operations and tensor operations, it is difficult to solve it directly. Therefore, it is necessary to transform formula (2) into a form containing only matrix operations. Specifically: Since the factor matrix W 1 , W 2 and W 3 They are The expanded matrix of the potential principal components in different modes is the factor matrix W 3 The corresponding potential principal component constraint is the identity matrix, so equation (2) can be written as:

[0076]

[0077] in, Represented in the factor matrix W′ 1 and W′ 2 The tensor cores under . It is different from Y j Along the connection in the network layer direction, equation (3) can be rewritten as follows:

[0078]

[0079] Among them, F j Represents Y j It can be seen that formula (4) only contains matrix operation forms.

[0080] Step 4: Design an alternating update optimization strategy to solve the multi-level feature representation model (4) of the hyperspectral image based on the tensor autoencoder network in step 3. Specifically, first rewrite the constraint-containing equation (4) into the following unconstrained form:

[0081]

[0082] Among them, γ is the coupling parameter. Introduce auxiliary variable A j =Y j , formula (5) can be rewritten as:

[0083]

[0084] Where β is the coupling parameter. In formula (6), the objective function is non-convex and contains multiple variables. However, for any single variable (fixing other variables), the objective function is convex. In summary, the specific process of the alternating update optimization strategy is shown as follows:

[0085] 1) Update V j : Fixed U j , Y j 、F j , W′ 1 , W′ 2 and A j , then the variable V j It can be updated by:

[0086]

[0087] The closed-form solution of formula (7) can be expressed as:

[0088]

[0089] 2) Update U j : Fixed V j , Y j 、F j , W′ 1 , W′ 2 and A j , then the variable U j It can be updated by:

[0090]

[0091] The solution of formula (9) can be expressed as:

[0092] U j =(X j Y j T )(Y j Y j T ) -1 (10)

[0093] 3) Update Y j : Fixed V j , U j 、F j, W′ 1 , W′ 2 and A j , then the variable Y j It can be updated by:

[0094]

[0095] The solution of formula (10) can be expressed as:

[0096]

[0097] Where I represents the identity matrix.

[0098] 4) Update F j : Fixed V j , U j , Y j , W′ 1 , W′ 2 and A j , then the variable F j It can be updated by:

[0099]

[0100] The solution of formula (12) can be expressed as:

[0101]

[0102] 5) Update W′ 1 : Fixed V j , U j , Y j 、F j , W′ 2 and A j , then the variable W′ 1 It can be updated by:

[0103]

[0104] The solution of formula (14) can be expressed as:

[0105]

[0106] 6) Update W′ 2 : Fixed V j , U j , Y j 、F j , W′ 1 and A j , then the variable W′ 2 It can be updated by:

[0107]

[0108] The solution of formula (16) can be expressed as:

[0109]

[0110] 7) Update A j : Fixed V j , U j , Y j 、F j , W′ 1 and W′ 2 , then variable A j It can be updated by:

[0111]

[0112] The solution of formula (18) can be expressed as:

[0113]

[0114] Among them, soft(x,y)=sign(x)·max(|x|-y,0) is the soft threshold operation function. Here, sign(x) is the sign function, and max(|x|-y,0) is the maximum value function, which is used to compare the size of |x|-y and 0.

[0115] After iterative updates from 1) to 7), the optimal V j , U j , Y j 、F j , W′ 1 , W′ 2 and A j .

[0116] Step 5: When the multi-level feature representation model (4) of the hyperspectral image based on the tensorized autoencoder network proposed in step 3 is optimized by the alternating update optimization strategy in step 4, the optimal solution F is obtained. j and W′ 1 After that, the final multi-level feature representation F of the hyperspectral image can be obtained, namely:

[0117]

[0118] It can be seen from formula (20) that the final feature representation is the output Y of each layer j The intrinsic matrix form of and its factor matrix along the spectral direction are multiplied and accumulated. Compared with most traditional feature extraction methods based on deep learning, the features extracted by the present invention contain both shallow and deep features of hyperspectral image data, and have richer information.

[0119] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A multi-level feature representation method for hyperspectral images, characterized in that: The steps include: Step 1: Transform the hyperspectral image Expand into a matrix representation X∈R Q×L , where Q = S × P represents the spatial resolution of the hyperspectral image, and L represents the spectral resolution of the hyperspectral image; Step 2: Input the hyperspectral image X into the autoencoder network to encode the hyperspectral image X, that is: Among them, λ represents the regularization parameter, V j , U j , X j and Y j They represent the encoding matrix, decoding matrix, input quantity, and output features of the jth layer in the network respectively; J represents the total number of layers of the autoencoder network; X j =Y j-1 (j≥2) and X1=X; Step 3: Perform tensor quantization on equation (1) and stack all network layer output features Y j ∈R Q×L′ (1≤j≤J), get the tensor form Formula (1) is transformed into: Among them, α is a coupling parameter used to control the importance of tensor decomposition terms; Represents tensor product; Cat(·) represents the output feature Y for stacking all layers j The operator, express The tensor core of Tensor decomposition is performed to obtain, where Q″<<Q, L″<<L′, J″<<J; in addition, the factor matrix W1∈R Q×Q″ , W2∈R L′×L″ , W3∈R J×J″ ; Since the factor matrices W1, W2 and W3 are The extended matrix of the potential principal components in different modes constrains the potential principal components corresponding to the factor matrix W3 to be the unit matrix. Therefore, formula (2) is written as: s.t.Y j =V j X j ,j=1,2,…,J, in, represents the tensor kernel under the factor matrices W1′ and W2′; because It is different from Y j Along the connection in the direction of the network layer, the multi-level feature representation model of the hyperspectral image (3) is rewritten as follows: Among them, F j Represents Y j The eigenmatrix form of ; Step 4: Solve the multi-level feature representation model (4) of the hyperspectral image to obtain the eigenvalue matrix F of each layer output Y j The optimal solution and the factor matrix W′1 along the spectral direction; Step 5, the final multi-level feature representation F of the hyperspectral image is:

2. According to claim 1, a method for multi-level feature representation of hyperspectral images is characterized in that: In step 4, the multi-level feature representation model (4) of the hyperspectral image is solved to obtain the output Y of each layer j The eigenvalue matrix F j The optimal solution and the factor matrix W′1 along the spectral direction, the specific steps are as follows: First, rewrite the constraint-containing formula (4) into the following unconstrained form: Among them, γ is the coupling parameter; Introduce auxiliary variable A j =Y j , formula (5) is rewritten as: Among them, β is the coupling parameter; The alternating update optimization strategy is used to solve equation (6): 1) Update V j : Fixed U j , Y j 、F j , W′1, W′2 and A j , then the variable V j Update it with: The closed-form solution of formula (7) is expressed as: 2) Update U j : Fixed V j , Y j 、F j , W′1, W′2 and A j , then the variable U j Update it with: The solution of formula (9) is expressed as: 3) Update Y j : Fixed V j , U j 、F j , W′1, W′2 and A j , then the variable Y j Update it with: The solution of formula (10) is expressed as: Where I represents the identity matrix; 4) Update F j : Fixed V j , U j , Y j , W′1, W′2 and A j , then the variable F j Update it with: The solution of formula (12) is expressed as: 5) Update W′1: fix V j , U j , Y j 、F j , W′2 and A j , then the variable W′1 is updated by the following formula: The solution of formula (14) is expressed as: 6) Update W′2: fix V j , U j , Y j 、F j , W′1 and A j , then the variable W′2 is updated by the following formula: The solution of formula (16) is expressed as: 7) Update A j : Fixed V j , U j , Y j 、F j , W′1 and W′2, then variable A j Update it with: The solution of formula (18) is expressed as: Among them, soft(x,y)=sign(x)·max(|x|-y,0) is the soft threshold operation function; sign(x) is the sign function, and max(|x|-y,0) is the maximum value function used to compare the size of |x|-y and 0; After iterative updates from 1) to 7), the optimal V j , U j , Y j 、F j , W′1, W′2 and A j .

Images