Progressive hyperspectral image compression method based on context model
Through the progressive hyperspectral image compression method based on the context model, the context feature fusion and distortion reduction module are used to solve the problems of high bit count and poor image quality in hyperspectral image compression, and efficient compression and high-quality image reproduction are achieved structure.
Patent Information
- Application Number
- CN202411199302.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-29
AI Technical Summary
The existing hyperspectral image compression method is difficult to achieve efficient compression while ensuring image quality. Especially in multi-band hyperspectral image processing, the context information of the image is not fully utilized, resulting in unsatisfactory compression effect.
Using a progressive hyperspectral image compression method based on context model, image preprocessing, potential tensor extraction, and slice processing are carried out by constructing encoder, decoder, hyperencoder, slice module, context feature fusion model and reduced distortion module. and entropy encoding, combining context feature fusion and reduced distortion modules, optimize bit information and image reconstruction.
It effectively reduces the number of bits during hyperspectral image compression, improves the quality of image reconstruction, ensures efficient compression and transmission, and reduces noise interference.
Smart Images

Figure CN119011862A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of hyperspectral image processing and relates to a progressive hyperspectral image compression method based on a context model. Background Art
[0002] In the process of hyperspectral image processing, due to the huge amount of data and the limitation of transmission bandwidth, efficient compression and transmission become an important challenge. Hyperspectral images contain rich spectral information, and each pixel has data of multiple bands, which makes it difficult for traditional image compression methods to cope with the characteristics of hyperspectral images. In addition, hyperspectral images are also affected by noise and signal interference during storage and transmission, which further increases the difficulty of data compression and transmission. Therefore, how to achieve efficient hyperspectral image compression while ensuring image quality is a problem that needs to be solved urgently.
[0003] Although existing hyperspectral image compression methods, such as those based on transform coding, predictive coding, and low-rank matrix decomposition, have improved compression efficiency to a certain extent, these methods often have obvious limitations. Transform coding methods are insufficient in processing high-frequency information, while predictive coding methods rely on the accuracy of the prediction model. Low-rank matrix decomposition methods have high computational complexity and are difficult to meet the needs of real-time applications. Especially when facing multi-band hyperspectral images, these methods fail to make full use of the contextual information of the image, resulting in unsatisfactory compression effects. Summary of the invention
[0004] The purpose of the present invention is to provide a progressive hyperspectral image compression method based on a context model, which solves the problem of high bit number and poor reconstructed image quality when hyperspectral images are compressed.
[0005] The technical solution adopted by the present invention is a progressive hyperspectral image compression method based on a context model, comprising the following steps:
[0006] Step 1: Select a public dataset of hyperspectral images for preprocessing and then divide it into a training set and a validation set;
[0007] Step 2, construct an encoder, a decoder, a super encoder, a super decoder, a slicing module, a context feature fusion model and a distortion reduction module;
[0008] Step 3, input the preprocessed image data into the encoder to obtain a latent tensor, input the latent tensor into the super encoder, the super encoder performs nonlinear feature extraction on the latent tensor to obtain a super latent tensor, and then inputs it into the super decoder after quantization to obtain the entropy parameter mean M and standard deviation ∑;
[0009] Step 4: Input the potential tensor into the slicing module. The slicing module divides the decentralized and quantized potential tensor into multiple slices of the same shape and arranges them according to the priority of importance to obtain the slice tensor T l , calculate the probability tensor P of each slice based on the entropy parameter mean, standard deviation and slice tensor l ;
[0010] Step 5: The approximate tensor reconstructed from the encoded slice, the mean and standard deviation of the entropy parameters, and the probability tensor are used as three context information. The updated probability tensor is obtained through the context feature fusion model. The slice tensor is entropy encoded and decoded using the updated probability tensor, and then the slice is reorganized.
[0011] In step 6, the slice tensor after the slice reorganization is input into the distortion reduction module to obtain a more refined latent tensor, and then the result is output through the decoder to reconstruct the hyperspectral image.
[0012] In step 1, the preprocessing includes spectral channel normalization and invalid band screening of the selected multiple hyperspectral image public data sets, and then cropping the other data sets according to the minimum number of bands in the multiple data sets, and dividing the preprocessed hyperspectral image public data sets into training set and validation set in a ratio of 7:3.
[0013] In step 2, the encoder consists of a residual block with a downsampling function, a normal convolutional layer and an attention module for image to potential tensor conversion. The decoder consists of a residual block with an upsampling function and an attention module for image reconstruction. The super encoder consists of a convolutional layer and a LeakyReLU activation function. The super decoder consists of a convolutional layer, a sub-pixel convolutional layer and a LeakyReLU activation function. The context feature fusion model consists of a residual block, a convolutional layer, an activation function and a normalization function. Each context feature is processed by a residual block and then concatenated. Then, the output result is obtained by processing the convolutional layer and the activation function. The output result is normalized to obtain an updated probability tensor. The distortion reduction module consists of a residual block.
[0014] The specific process of step 3 is as follows:
[0015] Step 3.1, normalize the preprocessed image data and normalize the pixel values to the interval [0, 1];
[0016] Step 3.2, input the normalized image data into the encoder, extract the image features in the normalized image data for the first time through the residual block with downsampling function, and perform the first downsampling, then perform the second image feature extraction, and perform the second downsampling, and enhance the feature representation through the attention block, and finally use a common convolution layer for the last downsampling, and enhance the feature representation through the attention block to obtain the potential tensor;
[0017] Step 3.3, input the latent tensor into the super encoder, perform preliminary processing on the latent tensor through a convolutional layer conv3x3 to obtain the first feature map, then apply the LeakyReLU activation function to perform nonlinear feature extraction on the feature map, and then process the feature map through the second convolutional layer conv3x3, and apply the LeakyReLU activation function to perform nonlinear feature extraction on the feature map again to obtain the super latent tensor;
[0018] In step 3.4, the super latent tensor is quantized and then input into the super decoder. The super decoder recovers Gaussian distribution parameters from the high-dimensional features for decoding the compressed features. The features are gradually upsampled through the convolution layer and the sub-pixel convolution layer to generate Gaussian distribution parameters for decoding, namely the entropy parameter mean M and standard deviation ∑.
[0019] In step 3.4, the quantized super latent tensor is input into the super decoder to generate Gaussian distribution parameters for decoding. The specific process is as follows: first, the input feature is preliminarily processed through a convolutional layer conv3x3 to obtain the first feature map, and then the LeakyReLU activation function is applied to the feature map for nonlinear feature extraction; then the feature map is upsampled through a sub-pixel convolutional layer subpel_conv3x3 to double the resolution of the feature map, and the LeakyReLU activation function is applied again for nonlinear feature extraction; next, the feature map is processed through a convolutional layer and subjected to the LeakyReLU activation function for nonlinear feature extraction; the feature map is upsampled again through a sub-pixel convolutional layer to double the resolution again, and subjected to the LeakyReLU activation function for nonlinear feature extraction, and finally the result is output through a convolutional layer.
[0020] Step 4: The specific process is as follows:
[0021] Step 4.1: Input the potential tensor into the slicing module. The slicing module divides the decentralized and quantized potential tensor into L slices of the same shape and size, and arranges them according to the priority of importance to obtain the slice tensor T l , l=1,...,L, measured according to the rate-distortion priority, T 1 is the most important slice, T L It is the least important slice;
[0022] Step 4.2, calculate the probability tensor P of each slice based on the entropy parameter mean, standard deviation and slice tensor l :
[0023]
[0024] Among them, P l ∈R 3C×H×W ,, R is the potential tensor, C, H and W represent the number of channels, height and width of the image respectively, 3C is because each slice is calculated in the form of a triple, each triple is 0 (3) , 1 (3) , 2 (3) .
[0025] Step 5: The specific process is as follows:
[0026] Step 5.1: Construct the context required by the context feature fusion model, using the encoded slice T 1:l-1 Reconstructed approximate tensor is the first context, connected with the entropy parameters mean and standard deviation as the second context, and the probability tensor P l As a third context;
[0027] Step 5.2, P l The features of the other two contexts are first extracted through multi-layer residual blocks, and then fused through residual blocks and convolutional layers, which triples the number of tensor channels. Then, they are split into an additional term △P∈R according to the channel method. 3C×H×W and a scaling term S∈R C×H×W ;
[0028] Step 5.3, convert the scaling term S into the conversion term D by the following formula:
[0029] D=s l +(s h -s l )×sigmoid(S)
[0030] Among them, (s l ,s h ) represents the constraint range of each element, that is, each element is within (s l ,s h ) l represents the minimum constraint value, s h represents the highest constraint value;
[0031] Step 5.4, transform the probability tensor P l Add to the additional term △P, sum the two, and then modulate the sum using the conversion term D to get the updated probability tensor Last pair Perform entropy coding and decoding, and finally reassemble the slices.
[0032] In step 5, the context feature fusion model is trained using the training set. The loss function used is cross entropy. The module is trained to learn to modify the probability tensor P with the additional term △P l The input probability in is then flattened or sharpened using the transformation term D to minimize the bitstream length. Finally, the context feature fusion model is verified using the validation set.
[0033] Step 6 includes taking the entropy parameters mean M and standard deviation ∑, and the latent tensor after entropy decoding and slicing and reorganizing As the input data of the distortion reduction module, it then passes through four layers of residual blocks to obtain a single channel output, and then performs channel cascading, regresses the residual △Y and generates the sum to obtain a more refined potential tensor
[0034]
[0035] The more refined potential tensor is input into the decoder. The decoder performs feature enhancement on the input more refined potential tensor through an attention block, and then performs the first image feature extraction and the first upsampling through the residual block with upsampling function, and then performs the second image feature extraction and the second upsampling, and finally performs the third image feature extraction and the third upsampling to restore the spatial resolution of the feature map, and finally generates an output image that matches the input image.
[0036] In step 6, the training set is used to train the distortion reduction module, and the loss function used is:
[0037]
[0038] Among them, L CE represents the cross entropy loss, Y represents the latent tensor, represents the Frobenius norm, which is used to limit the size of internal elements, and λ is a hyperparameter, that is, the loss function balance coefficient, which is used to control the trade-off between the cross entropy loss and the Frobenius norm regularization term;
[0039] The distortion reduction module is trained with the above loss function, using the Adam optimizer, with a batch size of 8 and a learning rate of 10. -4 ,The module is trained to use two contexts to reduce the error between the input tensor and the ,reconstructed tensor, so as to reduce the distortion of the hyperspectral image.,After the training is completed, the module is verified using the ,validation set to measure the output image quality by the bit rate per pixel, peak ,signal-to-noise ratio and the maximum similarity index of the structure.
[0040] The beneficial effect of the present invention is that, by constructing a slicing module, a context feature fusion model and a distortion reduction module, the input data is sliced and sorted and encoded according to the degree of importance, and then the context feature fusion model is used to reduce the bit information used in encoding, thereby reducing the bit information when the hyperspectral image is compressed. Finally, the reconstructed potential tensor is refined by the distortion reduction module, thereby ensuring the high quality of the reconstructed hyperspectral image. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 It is a flow chart of a progressive hyperspectral image compression method based on a context model of the present invention;
[0042] Figure 2 It is a schematic diagram of a context feature fusion model in a progressive hyperspectral image compression method based on a context model of the present invention;
[0043] Figure 3 It is a schematic diagram of a distortion reduction module in the context model-based progressive hyperspectral image compression method of the present invention. DETAILED DESCRIPTION
[0044] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0045] Example 1
[0046] A progressive hyperspectral image compression method based on context model, referring to Figure 1 , including the following steps:
[0047] Step 1: Select a public dataset of hyperspectral images for preprocessing and then divide it into a training set and a validation set;
[0048] Step 2, construct an encoder, a decoder, a super encoder, a super decoder, a slicing module, a context feature fusion model and a distortion reduction module;
[0049] Step 3, input the preprocessed image data into the encoder to obtain a latent tensor, input the latent tensor into the super encoder, the super encoder performs nonlinear feature extraction on the latent tensor to obtain a super latent tensor, and then inputs it into the super decoder after quantization to obtain the entropy parameter mean M and standard deviation ∑;
[0050] Step 4: Input the potential tensor into the slicing module. The slicing module divides the decentralized and quantized potential tensor into multiple slices of the same shape and arranges them according to the priority of importance to obtain the slice tensor T l , calculate the probability tensor P of each slice based on the entropy parameter mean, standard deviation and slice tensor l ;
[0051] Step 5: The approximate tensor reconstructed from the encoded slice, the mean and standard deviation of the entropy parameters, and the probability tensor are used as three context information. The updated probability tensor is obtained through the context feature fusion model. The slice tensor is entropy encoded and decoded using the updated probability tensor, and then the slice is reorganized.
[0052] In step 6, the slice tensor after the slice reorganization is input into the distortion reduction module to obtain a more refined latent tensor, and then the result is output through the decoder to reconstruct the hyperspectral image.
[0053] Example 2
[0054] A context model-based progressive hyperspectral image compression method comprises the following steps:
[0055] Step 1: Select the public datasets of Chikusei, Washington DC, Pavia Centre and CAVE hyperspectral images for preprocessing. First, perform spectral channel normalization and invalid band screening. Then, crop the other datasets according to the minimum number of bands in multiple datasets. Then, divide the preprocessed hyperspectral image public datasets into training set and validation set in a ratio of 7:3.
[0056] Step 2, construct an encoder, a decoder, a super encoder, a super decoder, a slicing module, a context feature fusion model and a distortion reduction module;
[0057] The encoder consists of a residual block with downsampling function, a normal convolutional layer and an attention module for image to potential tensor conversion. The decoder consists of a residual block with upsampling function and an attention module for image reconstruction. The super encoder consists of a convolutional layer and a LeakyReLU activation function. The super decoder consists of a convolutional layer, a sub-pixel convolutional layer and a LeakyReLU activation function for calculating the mean and standard deviation of the entropy parameter. The slicing module is used to divide the decentralized and quantized potential tensor into several small slices of the same shape and arrange them according to the priority of importance, and then perform entropy coding according to the priority. The context feature fusion model consists of a residual block, a convolutional layer, an activation function and a normalization function. Each context feature is spliced after being processed by the residual block, and then the output result is obtained by convolution and activation function. The output result is normalized to obtain the updated probability tensor. The distortion reduction module consists of multiple residual blocks. The potential tensor at the time of reconstruction is connected with the mean and standard deviation of the entropy parameter by channel and output through the residual block to obtain a more refined potential tensor.
[0058] Step 3, input the preprocessed image data into the encoder, extract the image features in the normalized image data for the first time through the residual block with downsampling function, and perform the first downsampling, then perform the second image feature extraction, and perform the second downsampling, and enhance the feature representation through the attention block, and finally use a common convolution layer for the last downsampling, and enhance the feature representation through the attention block to obtain the potential tensor, and input the potential tensor into the super encoder, and the super encoder performs preliminary processing on the potential tensor through a convolution layer conv3x3 to obtain the first feature map, and then applies the LeakyReLU activation function to perform nonlinear feature extraction on the feature map, and then processes the feature map through the second convolution layer conv3x3, and again applies the LeakyReLU activation function to perform nonlinear feature extraction on the feature map to obtain the super potential tensor, which is quantized and passed to the super decoder to obtain the entropy parameter mean M and standard deviation ∑;
[0059] Step 4: Input the potential tensor into the slicing module. The slicing module divides the decentralized and quantized potential tensor into multiple slices of the same shape and arranges them according to the priority of importance to obtain the slice tensor T l , calculate the probability tensor P of each slice based on the entropy parameter mean, standard deviation and slice tensor l ;
[0060] Step 5: The approximate tensor reconstructed from the encoded slice, the mean and standard deviation of the entropy parameters, and the probability tensor are used as three context information. The updated probability tensor is obtained through the context feature fusion model. The slice tensor is entropy encoded and decoded using the updated probability tensor, and then the slice is reorganized.
[0061] Step 6: Take the entropy parameter mean M and standard deviation ∑, and the potential tensor after entropy decoding and slicing reorganization As the input data of the distortion reduction module, it then passes through four layers of residual blocks to obtain a single channel output, and then performs channel cascading, regresses the residual △Y and generates the sum to obtain a more refined potential tensor The decoder then generates reconstructed data and outputs a reconstructed hyperspectral image.
[0062] Example 3
[0063] A context model-based progressive hyperspectral image compression method comprises the following steps:
[0064] Step 1: Select the public datasets of Chikusei, Washington DC, Pavia Centre and CAVE hyperspectral images for preprocessing. First, perform spectral channel normalization and invalid band screening. Normalization ensures that the spectral channel value range is consistent and reduces the deviation during model training. The invalid bands are screened out to ensure the validity of the data. Then, the other three datasets are cropped according to the minimum number of bands in these four datasets. Before training, these hyperspectral images are randomly cropped into non-overlapping 256×256 batches. These batches are divided into training sets and validation sets in a ratio of 7:3 to ensure the generalization ability of the model.
[0065] Step 2, construct an encoder, a decoder, a super encoder, a super decoder, a slicing module, a context feature fusion model and a distortion reduction module;
[0066] The encoder consists of a residual block with downsampling function, a normal convolutional layer and an attention module for image to potential tensor conversion. The decoder consists of a residual block with upsampling function and an attention module for image reconstruction. The super encoder consists of a convolutional layer and a LeakyReLU activation function. The super decoder consists of a convolutional layer, a sub-pixel convolutional layer and a LeakyReLU activation function for calculating the mean and standard deviation of the entropy parameter. The slicing module is used to divide the decentralized and quantized potential tensor into several small slices of the same shape and arrange them according to the priority of importance, and then perform entropy coding according to the priority. The context feature fusion model consists of a residual block, a convolutional layer, an activation function and a normalization function. Each context feature is spliced after being processed by the residual block, and then the output result is obtained by convolution and activation function. The output result is normalized to obtain the updated probability tensor. The distortion reduction module consists of multiple residual blocks. The potential tensor at the time of reconstruction is connected with the mean and standard deviation of the entropy parameter by channel and output through the residual block to obtain a more refined potential tensor.
[0067] Step 3, input the preprocessed image data into the encoder to obtain a latent tensor, input the latent tensor into the super encoder, the super encoder performs nonlinear feature extraction on the latent tensor to obtain a super latent tensor, and then inputs it into the super decoder after quantization to obtain the entropy parameter mean M and standard deviation ∑;
[0068] Step 3: The specific process is as follows:
[0069] Step 3.1, normalize the preprocessed image data and normalize the pixel values to the interval [0, 1];
[0070] Step 3.2, input the normalized image data into the encoder, extract the image features in the normalized image data for the first time through the residual block with a step size of 2 and a downsampling function, and perform the first downsampling, then perform the second image feature extraction, and perform the second downsampling, and enhance the feature representation by adding an attention block, and finally use a common convolution layer for the last downsampling, and enhance the feature representation by adding an attention block to obtain the potential tensor;
[0071] Step 3.3, input the latent tensor into the super encoder, perform preliminary processing on the latent tensor through a convolutional layer conv3x3 to obtain the first feature map, then apply the LeakyReLU activation function to perform nonlinear feature extraction on the feature map, and then process the feature map through the second convolutional layer conv3x3, and apply the LeakyReLU activation function to perform nonlinear feature extraction on the feature map again to obtain the super latent tensor;
[0072] Step 3.4, quantize the super latent tensor and then input it into the super decoder. The super decoder recovers the Gaussian distribution parameters from the high-dimensional features for decoding the compressed features. The features are gradually upsampled through the convolution layer and the sub-pixel convolution layer to generate the Gaussian distribution parameters for decoding, that is, the entropy parameter mean M and standard deviation ∑. The specific process is as follows: first, the input features are preliminarily processed through a convolution layer conv3x3 to obtain the first feature map, and then the LeakyReLU activation function is applied to the feature map for nonlinear feature extraction; then the feature map is upsampled through a sub-pixel convolution layer subpel_conv3x3 to double the resolution of the feature map, and the LeakyReLU activation function is applied again for nonlinear feature extraction. Next, the feature map is processed through a convolution layer and subjected to the LeakyReLU activation function for nonlinear feature extraction. The feature map is upsampled again through a sub-pixel convolution layer to double the resolution again, and subjected to the LeakyReLU activation function for nonlinear feature extraction. Finally, the result is output through a convolution layer.
[0073] Step 4: Input the potential tensor into the slicing module. The slicing module divides the decentralized and quantized potential tensor into multiple slices of the same shape and arranges them according to the priority of importance to obtain the slice tensor T l , calculate the probability tensor P of each slice based on the entropy parameter mean, standard deviation and slice tensor l ;
[0074] Step 4: The specific process is as follows:
[0075] Step 4.1: Input the potential tensor into the slicing module. The slicing module divides the decentralized and quantized potential tensor into L slices of the same shape and size. Each slice has the same size as the template tensor and is arranged according to the priority of importance to obtain the slice tensor T. l , l=1,...,L, measured according to the rate-distortion priority, T 1 is the most important slice, T L It is the least important slice;
[0076] Step 4.2, calculate the probability tensor P of each slice based on the entropy parameter mean, standard deviation and slice tensor l :
[0077]
[0078] Among them, P l ∈R 3C×H×W , R is the potential tensor, C, H and W represent the number of channels, height and width of the image respectively, 3C is because each slice is calculated in the form of a triple, each triple is 0 (3) , 1 (3) , 2 (3)
[0079] Step 5, see Figure 2 , the approximate tensor reconstructed from the encoded slice, the mean and standard deviation of the entropy parameters, and the probability tensor are used as three context information, and the updated probability tensor is obtained through the context feature fusion model. The slice tensor uses the updated probability tensor for entropy encoding and entropy decoding to form a 1 to T L The bit stream is then sliced and reassembled;
[0080] Step 5: The specific process is as follows:
[0081] Step 5.1: Construct the context required by the context feature fusion model, using the encoded slice T 1:l-1 Reconstructed approximate tensor is the first context, connected with the entropy parameters mean and standard deviation as the second context, and the probability tensor P l As a third context;
[0082] Step 5.2, First Context After three layers of input and output, both the residual block has a single channel; the second context information M and ∑ pass through four residual blocks in turn. In the first residual block, the number of channels is adjusted to 2C, and convolution and nonlinear activation processing are performed in two convolutional layers with a channel number of 2C. The feature map of the 2C channel is output through the residual connection. The second residual block repeats this process and continues to maintain the output of the 2C channel. In the third residual block, the input 2C channel feature map remains 2C channel after passing through the first convolutional layer, but is compressed to C channel in the second convolutional layer. The number of output feature map channels is reduced to C. The two convolutional layers of the fourth residual block both maintain C channels, and the final C channel feature map is output through the residual connection; the third context P l The processing of is similar to that of the second context, except that 2C is replaced by 3C. The whole process achieves a balance between information extraction and computational complexity by first increasing the number of channels to enrich feature expression and then gradually compressing feature dimensions.
[0083] Step 5.3, the third context P l The features of the two contexts are first extracted by multi-layer residual blocks, and then connected step by step. Then, they are passed through 6 layers of residual blocks with 3C input and output channels, 3 convolutional layers and LeakyReLU activation function. The convolution kernel in the residual block is 3×3, the step size and padding are 1, and the convolution kernel in the convolution layer is 1×1, the step size is 1, and the padding is 0. Then, the fused feature tensor is obtained by one convolution. The result obtained at this time has the same l Same spatial resolution, but three times more channels.
[0084] Step 5.4: Split into an additional term △P∈R according to the channel method 3C×H×W and a scaling term S∈R C×H×W , the scaling term S is converted into the transformation term D by the following formula:
[0085] D=s l +(s h -s l )×sigmoid(S)
[0086] Among them, (s l ,s h ) represents the constraint range of each element, that is, each element is within (s l ,s h ) l represents the minimum constraint value, s h represents the highest constraint value;
[0087] Step 5.5, transform the probability tensor P lAdd to the additional term △P, sum the two, then use the conversion term D to modulate the sum, and use the normalization function to get the updated probability tensor Last pair Perform entropy coding and decoding, and finally reassemble the slices.
[0088] In step 5, the context feature fusion model is trained using the training set. The loss function used is cross entropy. The module is trained to learn to modify the probability tensor P with the additional term △P l The input probability in is then flattened or sharpened using the transformation term D to minimize the bitstream length. Finally, the context feature fusion model is verified using the validation set.
[0089] Step 6, see Figure 3 , with entropy parameter mean M and standard deviation ∑, and the potential tensor after entropy decoding and slicing reorganization As the input data of the distortion reduction module, it then passes through four layers of residual blocks to obtain a single channel output, and then performs channel cascading, regresses the residual △Y and generates the sum to obtain a more refined potential tensor
[0090]
[0091] The more refined potential tensor is input into the decoder. The decoder performs feature enhancement on the input more refined potential tensor through an attention block, and then performs the first image feature extraction and the first upsampling through the residual block with upsampling function, and then performs the second image feature extraction and the second upsampling, and finally performs the third image feature extraction and the third upsampling to restore the spatial resolution of the feature map, and finally generates an output image that matches the input image.
[0092] In step 6, the training set is used to train the distortion reduction module, and the loss function used is:
[0093]
[0094] Among them, L CE represents the cross entropy loss, Y represents the latent tensor, represents the Frobenius norm, which is used to limit the size of internal elements, and λ is a hyperparameter, that is, the loss function balance coefficient, which is used to control the trade-off between the cross entropy loss and the Frobenius norm regularization term;
[0095] The distortion reduction module is trained with the above loss function, using the Adam optimizer with a batch size of 8 and a learning rate of 10. -4,The module is trained to use two contexts to reduce the error between the input tensor and the ,reconstructed tensor, so as to reduce the distortion of the hyperspectral image.,After the training is completed, the module is verified using the ,validation set to measure the output image quality by the bit rate per pixel, peak ,signal-to-noise ratio and the maximum similarity index of the structure.
Claims
1. A progressive hyperspectral image compression method based on a context model, characterized in that: The following steps are involved: Step 1: Select a public dataset of hyperspectral images for preprocessing and then divide it into a training set and a validation set; Step 2, construct an encoder, a decoder, a super encoder, a super decoder, a slicing module, a context feature fusion model and a distortion reduction module; Step 3, input the preprocessed image data into the encoder to obtain a latent tensor, input the latent tensor into the super encoder, the super encoder performs nonlinear feature extraction on the latent tensor to obtain a super latent tensor, and then inputs it into the super decoder after quantization to obtain the entropy parameter mean M and standard deviation ∑; Step 4: Input the potential tensor into the slicing module. The slicing module divides the decentralized and quantized potential tensor into multiple slices of the same shape and arranges them according to the priority of importance to obtain the slice tensor T l , calculate the probability tensor P of each slice based on the entropy parameter mean, standard deviation and slice tensor l ; Step 5: The approximate tensor reconstructed from the encoded slice, the mean and standard deviation of the entropy parameters, and the probability tensor are used as three context information. The updated probability tensor is obtained through the context feature fusion model. The slice tensor is entropy encoded and decoded using the updated probability tensor, and then the slice is reorganized. In step 6, the slice tensor after the slice reorganization is input into the distortion reduction module to obtain a more refined latent tensor, and then the result is output through the decoder to reconstruct the hyperspectral image.
2. The context model-based progressive hyperspectral image compression method according to claim 1, characterized in that: In the step 1, the preprocessing includes normalizing the spectral channels and screening invalid bands of the selected multiple public hyperspectral image data sets, and then trimming the other data sets according to the minimum number of bands in the multiple data sets, and dividing the preprocessed public hyperspectral image data sets into a training set and a validation set in a ratio of 7:
3.
3. The context model-based progressive hyperspectral image compression method according to claim 2, characterized in that: In the step 2, the encoder is composed of a residual block with a downsampling function, a normal convolutional layer and an attention module, which are used for converting images to potential tensors. The decoder is composed of a residual block with an upsampling function and an attention module, which are used for image reconstruction. The super encoder is composed of a convolutional layer and a LeakyReLU activation function. The super decoder is composed of a convolutional layer, a sub-pixel convolutional layer and a LeakyReLU activation function. The context feature fusion model is composed of a residual block, a convolutional layer, an activation function and a normalization function. Each context feature is spliced after being processed by the residual block, and then processed by the convolution and activation function to obtain an output result. The output result is subjected to a normalization function to obtain an updated probability tensor. The distortion reduction module is composed of a residual block.
4. The context model-based progressive hyperspectral image compression method according to claim 3, characterized in that: The specific process of step 3 is as follows: Step 3.1, normalize the preprocessed image data and normalize the pixel values to the interval [0, 1]; Step 3.2, input the normalized image data into the encoder, extract the image features in the normalized image data for the first time through the residual block with downsampling function, and perform the first downsampling, then perform the second image feature extraction, and perform the second downsampling, and enhance the feature representation through the attention block, and finally use a common convolution layer for the last downsampling, and enhance the feature representation through the attention block to obtain the potential tensor; Step 3.3, input the latent tensor into the super encoder, perform preliminary processing on the latent tensor through a convolutional layer conv3x3 to obtain the first feature map, then apply the LeakyReLU activation function to perform nonlinear feature extraction on the feature map, and then process the feature map through the second convolutional layer conv3x3, and apply the LeakyReLU activation function to perform nonlinear feature extraction on the feature map again to obtain the super latent tensor; In step 3.4, the super latent tensor is quantized and then input into the super decoder. The super decoder recovers Gaussian distribution parameters from the high-dimensional features for decoding the compressed features. The features are gradually upsampled through the convolution layer and the sub-pixel convolution layer to generate Gaussian distribution parameters for decoding, namely the entropy parameter mean M and standard deviation ∑.
5. The context model-based progressive hyperspectral image compression method according to claim 4, characterized in that: In the step 3.4, the quantized super latent tensor is input into the super decoder to generate Gaussian distribution parameters for decoding. The specific process is: first, the input feature is preliminarily processed through a convolution layer conv3x3 to obtain the first feature map, and then the LeakyReLU activation function is applied to the feature map for nonlinear feature extraction; then the feature map is upsampled through a sub-pixel convolution layer subpel_conv3x3 to double the resolution of the feature map, and the LeakyReLU activation function is applied again for nonlinear feature extraction; next, the feature map is processed through a convolution layer and subjected to the LeakyReLU activation function for nonlinear feature extraction; the feature map is upsampled again through a sub-pixel convolution layer to double the resolution again, and subjected to the LeakyReLU activation function for nonlinear feature extraction, and finally the result is output through a convolution layer.
6. The context model-based progressive hyperspectral image compression method according to claim 5, characterized in that: The specific process of step 4 is as follows: Step 4.1: Input the potential tensor into the slicing module. The slicing module divides the decentralized and quantized potential tensor into L slices of the same shape and size, and arranges them according to the priority of importance to obtain the slice tensor T l , l=1,...,L, according to the rate-distortion priority as the standard, T1 is the most important slice, T L It is the least important slice; Step 4.2, calculate the probability tensor P of each slice based on the entropy parameter mean, standard deviation and slice tensor l : Among them, P l ∈R 3C×H×W ,, R is the potential tensor, C, H and W represent the number of channels, height and width of the image respectively, 3C is because each slice is calculated in the form of a triple, each triple is 0 (3) , 1 (3) , 2 (3) .
7. The context model-based progressive hyperspectral image compression method according to claim 6, characterized in that: The specific process of step 5 is as follows: Step 5.1: Construct the context required for the context feature fusion model, using the encoded slice T1:l -1 Reconstructed approximate tensor is the first context, connected with the entropy parameters mean and standard deviation as the second context, and the probability tensor P l As a third context; Step 5.2, P l The features of the other two contexts are first extracted through multi-layer residual blocks, and then fused through residual blocks and convolutional layers, which triples the number of tensor channels. Then, they are split into an additional term △P∈R according to the channel method. 3C ×H×W and a scaling term S∈R C×H×W ; Step 5.3, convert the scaling term S into the conversion term D by the following formula: D = +(-sl) × sigmoid(S) Among them, (s l ,s h ) represents the constraint range of each element, that is, each element is within (s l ,s h ) l represents the minimum constraint value, s h represents the highest constraint value; Step 5.4, transform the probability tensor P l Add to the additional term △P, sum the two, and then modulate the sum using the conversion term D to get the updated probability tensor Last pair Perform entropy coding and decoding, and finally reassemble the slices.
8. The context model-based progressive hyperspectral image compression method according to claim 7, characterized in that: In step 5, the context feature fusion model is trained using the training set, and the loss function used is the cross entropy. The module is trained to learn to modify the probability tensor P with the additional term △P l The input probability in is then flattened or sharpened using the transformation term D to minimize the bitstream length. Finally, the context feature fusion model is verified using the validation set.
9. The context model-based progressive hyperspectral image compression method according to claim 8, characterized in that: The step 6 includes taking the entropy parameter mean M and standard deviation ∑, and the potential tensor after entropy decoding and slicing reorganization As the input data of the distortion reduction module, it then passes through four layers of residual blocks to obtain a single channel output, and then performs channel cascading, regresses the residual △Y and generates the sum to obtain a more refined potential tensor The more refined potential tensor is input into the decoder. The decoder performs feature enhancement on the input more refined potential tensor through an attention block, and then performs the first image feature extraction and the first upsampling through the residual block with upsampling function, and then performs the second image feature extraction and the second upsampling, and finally performs the third image feature extraction and the third upsampling to restore the spatial resolution of the feature map, and finally generates an output image that matches the input image.
10. The context model-based progressive hyperspectral image compression method according to claim 9, characterized in that: In step 6, the distortion reduction module is trained using the training set, and the loss function used is: Among them, L CE represents the cross entropy loss, Y represents the latent tensor, represents the Frobenius norm, which is used to limit the size of internal elements, and λ is a hyperparameter, that is, the loss function balance coefficient, which is used to control the trade-off between the cross entropy loss and the Frobenius norm regularization term; The distortion reduction module is trained with the above loss function, using the Adam optimizer, with a batch size of 8 and a learning rate of 10. -4 ,The module is trained to use two contexts to reduce the error between the input tensor and the ,reconstructed tensor, so as to reduce the distortion of the hyperspectral image.,After the training is completed, the module is verified using the ,validation set to measure the output image quality by the bit rate per pixel, peak ,signal-to-noise ratio and the maximum similarity index of the structure.
Citation Information
Patent Citations
Hyperspectral remote sensing image classification method based on self-attention context network
CN112287978A
Image compression method based on multi-scale space and context information fusion
CN114792347A
Hyperspectral image classification method and system based on context attention characteristics
CN116543203A
Hyperspectral remote sensing image classification method based on self-attention context network
US20230260279A1
Attention based context modelling for image and video compression
US20240244274A1
Cited By
End-to-end image compression method based on mixed attention and SwinV2 entropy model
CN119946271A
Collection core identification method and device based on feature decoupling and residual connection
CN121170477A
Hyperspectral image compression network and compression method based on multi-scale spectrum and spatial feature enhancement
CN121711491A
Spectral gradient guided hyperspectral image compression and reconstruction method and system
CN122156331A
