A super-resolution enhancement method and device for smart city monitoring based on hyperspectral
By using denoising convolutional neural networks and super-resolution enhancement models in hyperspectral imaging technology, the problem of insufficient spatial resolution in smart city monitoring is solved, and efficient image enhancement and detail improvement is achieved. It is suitable for Various application scenarios.
Patent Information
- Application Number
- CN202411139454.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-20
AI Technical Summary
Hyperspectral imaging technology is limited by spatial resolution in smart city surveillance. The existing super-resolution method has complex calculations and limited effects, making it difficult to meet the needs of specific scenarios.
Using image denoising method based on denoising convolutional neural network (DnCNN) and complex super-resolution enhancement models, including initial encoder, initial decoder, super-resolution encoder, adaptive coding factor adjustment block, spatial enhancement attention Force module and channel enhancement attention module, which generates high-resolution images by training total loss and iterating model parameters by Adam optimizer.
It significantly improves the resolution and clarity of hyperspectral images, improves the accuracy and efficiency of smart city monitoring equipment, is suitable for a variety of scenarios, and is promoted to other fields such as remote sensing image processing and medical image analysis.
Smart Images

Figure CN118657662B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image enhancement technology, and in particular to a hyperspectral-based smart city monitoring super-resolution enhancement method and device. Background Art
[0002] With the rapid development of artificial intelligence, hyperspectral imaging has achieved remarkable performance, especially in the construction of smart city cameras, this technology has shown great potential. Hyperspectral imaging technology provides rich spectral information about objects and scenes by acquiring image data at each band of the electromagnetic spectrum. This technology has important application value in many fields such as environmental monitoring, urban planning, agriculture, and forest management. In the construction and management of smart cities, hyperspectral imaging technology can provide key data such as detailed urban environmental monitoring, pollution detection, and land use classification. However, in practical applications, hyperspectral images are often limited by spatial resolution. Hyperspectral imaging devices usually have a trade-off between spectral resolution and spatial resolution, that is, high spectral resolution is often accompanied by lower spatial resolution. This limitation limits the application effect of hyperspectral data in fine-grained urban monitoring. In order to overcome this limitation, super-resolution technology came into being. Super-resolution technology aims to restore or generate high-resolution images from low-resolution images through algorithms and data processing methods. Through super-resolution enhancement technology, the spatial resolution of hyperspectral images can be improved, thereby improving their application performance and accuracy in smart city monitoring. Current super-resolution methods mainly include interpolation, reconstruction, and learning-based deep learning methods. Traditional interpolation methods (such as bilinear interpolation, bicubic interpolation, etc.) are relatively simple, but have limited effects; reconstruction methods (such as sparse representation, dictionary learning, etc.) can improve image quality to a certain extent, but cannot meet the needs of some specific scenarios; deep learning-based methods (such as convolutional neural networks, generative adversarial networks, etc.) show more significant effects, but are also accompanied by challenges in computational complexity and model training.
[0003] In the context of smart city monitoring, the super-resolution enhancement method using hyperspectral images has important practical significance. By improving the image resolution, various elements in the city (such as roads, buildings, green spaces, etc.) can be more accurately identified and monitored, providing more detailed and precise data support. This not only helps urban planning and management, but also improves the efficiency and accuracy of environmental monitoring and emergency response.
[0004] Therefore, in response to the above problems, a hyperspectral-based smart city monitoring super-resolution enhancement method and device are proposed to solve the above problems. Summary of the invention
[0005] In view of the deficiencies in the prior art, the present invention develops a hyperspectral-based smart city monitoring super-resolution enhancement method and device, which can significantly improve the resolution and clarity of smart city monitoring equipment, thereby achieving more accurate and efficient monitoring and analysis.
[0006] The technical solution of the present invention to solve the technical problem is:
[0007] A hyperspectral-based super-resolution enhancement method for smart city monitoring comprises the following steps:
[0008] S1. Image acquisition: The image denoising method based on the denoising convolutional neural network DnCNN is used to process the images in the publicly available PaviaCenter dataset to obtain hyperspectral images. ;
[0009] S2. Create a super-resolution enhancement model: The super-resolution enhancement model is composed of an initial encoder, an initial decoder, a super-resolution encoder, an adaptive coding factor adjustment block, a spatial enhancement attention module, a channel enhancement attention module and a super-resolution decoder. Then the hyperspectral image is Input to the super-resolution enhancement model to obtain the super-resolution reconstructed image ;
[0010] S3. Train super-resolution enhancement model: Reconstruct image based on super-resolution and real images Calculate the total training loss, then use the Adam optimizer to iterate the parameters in the super-resolution enhancement model to obtain the trained super-resolution enhancement model. ;
[0011] S4. Output the final result: The image to be enhanced is preprocessed and then input into the trained super-resolution enhancement model The final super-resolution reconstructed image is obtained .
[0012] In a specific implementation, the initial encoder is composed of a convolution block and an average pooling layer in sequence. The convolution block is composed of three downsampling convolution layers, a batch normalization layer, and a sigmoid function. The convolution kernel of each downsampling convolution layer is 3×3, the step size is 1, and the padding is 0.
[0013] Hyperspectral images Input into the initial encoder to obtain the initial encoding features ;
[0014] The initial decoder consists of the first convolutional block, the fully connected block, the second convolutional block, and the average pooling layer in sequence;
[0015] The first convolutional block consists of a downsampling convolutional layer with a convolution kernel of 3×3, a stride of 1, and padding of 0, a batch normalization layer, and a ReLU activation function;
[0016] The fully connected block consists of the first fully connected layer and the second fully connected layer. The input dimension of the first fully connected layer is 802816 and the output dimension is 200704. The input dimension of the second fully connected layer is 200704 and the output dimension is 50176.
[0017] The second convolution block consists of a view function, a convolution layer with a kernel of 1×1, a stride of 1, and padding of 0, batch normalization, and a ReLU activation function. The output dimension of the view function is the same as that of the hyperspectral image. same;
[0018] The initial encoding features Input into the initial decoder to obtain the initial reconstructed image .
[0019] In a specific implementation, the super-resolution encoder is composed of a hyperspectral convolution block, a reconstruction convolution block, a feature fusion layer, a fusion convolution block, and a maximum pooling layer in sequence;
[0020] The hyperspectral convolution consists of the first convolution layer with a convolution kernel of 3×3, a step size of 2, and a padding of 0, a batch normalization layer, a second convolution layer with a convolution kernel of 5×5, a step size of 3, and a padding of 0, and a ReLU activation function. The feature fusion layer input into the hyperspectral convolution obtains the hyperspectral features ;
[0021] The reconstruction convolution block consists of the third convolution layer with a convolution kernel of 1×1, a stride of 1, and a padding of 0, a batch normalization layer, a fourth convolution layer with a convolution kernel of 3×3, a stride of 1, and a padding of 0, and a ReLU activation function. Input into the reconstruction convolution block to obtain the initial reconstruction features ;
[0022] The feature fusion layer is composed of the concat function, batch normalization layer and ReLU activation function in sequence, which combines the hyperspectral features and initial reconstruction features Input into the feature fusion layer to obtain the fusion feature ;
[0023] The fusion convolution block is composed of the fifth convolution layer with a convolution kernel of 1×1, a step size of 1, and a padding of 0, the first batch of normalization layers, the sixth convolution layer with a convolution kernel of 3×3, a step size of 2, and a padding of 0, the second batch of normalization layers, the seventh convolution layer with a convolution kernel of 5×5, a step size of 2, and a padding of 1, the regression calculation formula, and the ReLU activation function. The fusion feature Input to the fifth convolutional layer and the first batch of normalization layers in sequence and then output features , the feature Input to the sixth convolutional layer and the second batch of normalization layers in sequence and then output features , the feature Input to the seventh convolutional layer and output features , then the feature , and Input into the regression calculation formula to get the features , the regression calculation formula is as follows:
[0024] ,
[0025] Finally, the features The super-resolution coding features are obtained by sequentially inputting the ReLU activation function and the maximum pooling layer of the fused convolution block. .
[0026] In a specific implementation, the adaptive coding factor adjustment block is composed of a first normalization layer, a ReLU activation function, a first regulator, a second normalization layer, and a second regulator in sequence;
[0027] The initial reconstructed image Input the first normalization layer and ReLU activation function to get the first adaptive feature , the initial encoding features and super-resolution encoding features Enter into the formula to get the first adjustment factor , the first adaptive feature Multiply by the first adjustment factor , and obtain the first adaptive adjustment feature , the initial encoding features Multiply by the first adjustment factor Get the initial coding adjustment factor , the super-resolution encoding feature Multiply by the first adjustment factor Get the super-resolution coding adjustment factor , the first adaptive adjustment feature Input to the second normalization layer and the second regulator, and adjust the initial encoding factor , super-resolution coding adjustment factor Input into the second regulator to obtain the second adaptive regulation characteristic , the second adaptive adjustment feature With the initial reconstructed image Add together to get the adaptive coding feature ;
[0028] Among them, the calculation formula of the first regulator is:
[0029] ,
[0030] The calculation formula of the second regulator is:
[0031] ,
[0032] in, Represents the initial encoding features The average value of Represents super-resolution encoding features The standard deviation of Represents the initial encoding features The standard deviation of represents the standard deviation calculation function, Represents the first adaptive adjustment feature The average value of represents the first regulator, Represents the second regulator.
[0033] In a specific implementation, the spatial enhanced attention module is composed of the formula The convolution layer consists of a convolution kernel of 1×1, a step size of 1, and a padding of 0, a batch normalization layer, and a ReLU activation function; the super-resolution encoding features and adaptive coding features Add input to formula In the above example, we obtain the spatial enhanced attention feature , the spatially enhanced attention feature Input to the convolution layer and batch normalization layer to obtain spatial features ;
[0034] formula as follows:
[0035] ,
[0036] in, Representation formula The output spatial enhanced attention features, represents transpose;
[0037] The channel-enhanced attention module is composed of the formula , a convolution layer with a convolution kernel of 3×3, a stride of 2, and a padding of 0, a batch normalization layer, and a ReLU activation function; the spatial features and adaptive coding features Add input to formula In c, we get the channel-enhanced attention feature , the channel enhances the attention feature Input to the convolution layer and batch normalization layer to obtain channel features ;
[0038] formula as follows:
[0039] ,
[0040] in, Representation formula The output channel enhances the attention feature, Indicates transpose.
[0041] In a specific implementation, the super-resolution decoder is composed of a first convolution block, a fully connected block, a second convolution block, an average pooling layer, and a save function in sequence;
[0042] The first convolutional block consists of an upsampling convolutional layer with a convolution kernel of 1×1, a stride of 1, and padding of 0, a batch normalization layer, and a Sigmoid activation function;
[0043] The fully connected block consists of 3 fully connected layers. The input dimension of the first fully connected layer is 3211264 and the output dimension is 2097152. The input dimension of the second fully connected layer is 2097152 and the output dimension is 1048576. The input dimension of the third fully connected layer is 1048576 and the output dimension is 262144.
[0044] The second convolutional block consists of a view function with a dimension of 512×512, a convolutional layer with a convolution kernel of 1×1, a stride of 1, and a padding of 0, a batch normalization layer, and a ReLU activation function;
[0045] Channel Features Input into the super-resolution decoder to obtain the super-resolution reconstructed image .
[0046] In a specific implementation, the total training loss The calculation formula is as follows:
[0047] ,
[0048] in, represents the reconstruction loss, represents the cross entropy loss, Indicates a real picture.
[0049] The present invention also proposes a computing device, comprising a memory configured to store computer-executable instructions; and a processor configured to execute a hyperspectral-based smart city monitoring super-resolution enhancement method when the computer-executable instructions are executed by the processor.
[0050] The effects provided in the content of the invention are only the effects of the embodiments, rather than all the effects of the invention. The above technical solution has the following advantages or beneficial effects:
[0051] The present invention proposes a hyperspectral-based smart city monitoring super-resolution enhancement method and device. By combining attention enhancement and adaptive coding technology, the present invention can improve the efficiency and accuracy of face recognition of urban cameras, and enhance the clarity and generation efficiency of blurred images in smart city monitoring equipment. The specific advantages are as follows:
[0052] (1) High-precision image reconstruction: By combining the DnCNN image denoising method and a complex super-resolution enhancement model, the present invention can effectively remove noise and significantly improve the resolution and detail of hyperspectral images through the attention mechanism, so that the reconstructed image R and the final image R̃ have higher clarity and accuracy;
[0053] (2) Adaptive enhancement mechanism: The adaptive coding factor adjustment block, spatial enhancement attention module, and channel enhancement attention module in the super-resolution enhancement model can be adaptively adjusted according to the characteristics of the input image, thereby improving the generalization ability and robustness of the model and adapting to the processing requirements of hyperspectral images under different types and conditions;
[0054] (3) Efficient training and optimization: The Adam optimizer is used to iteratively update the model parameters, which can quickly and effectively converge to the optimal parameter combination, improving the training efficiency and effect of the model. At the same time, the optimization strategy based on the loss function ensures the quality and consistency of the model output image;
[0055] (4) Wide application: This method is not only suitable for hyperspectral image processing in smart city monitoring, but can also be extended to other fields that require high-resolution image reconstruction and enhancement, such as remote sensing image processing, medical image analysis, and precision agriculture monitoring.
[0056] (5) Enhanced smart city monitoring capabilities: By improving the resolution and clarity of monitoring images, the present invention can significantly improve the effectiveness and accuracy of smart city monitoring equipment, which will help to better carry out tasks such as urban management, traffic monitoring and environmental protection. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.
[0058] Figure 1 This is a model structure diagram of the present invention.
[0059] Figure 2 This is an experimental comparison result diagram for the present invention. DETAILED DESCRIPTION
[0060] The following description further describes the specific details of the present invention for a full understanding of the present invention. The terms used in the description of the present invention are only used to illustrate the advantages and features of the present invention and are not intended to limit the present invention.
[0061] Example 1
[0062] like Figure 1 As shown, a hyperspectral-based smart city monitoring super-resolution enhancement method comprises the following steps:
[0063] S1. Image acquisition: The image denoising method based on the denoising convolutional neural network DnCNN is used to process the images in the publicly available PaviaCenter dataset to obtain hyperspectral images. ;
[0064] S2. Create a super-resolution enhancement model: The super-resolution enhancement model is composed of an initial encoder, an initial decoder, a super-resolution encoder, an adaptive coding factor adjustment block, a spatial enhancement attention module, a channel enhancement attention module and a super-resolution decoder. Then the hyperspectral image is Input to the super-resolution enhancement model to obtain the super-resolution reconstructed image ;
[0065] S3. Training super-resolution enhancement model: Reconstructing images based on super-resolution and real images Calculate the total training loss, then use the Adam optimizer to iterate the parameters in the super-resolution enhancement model to obtain the trained super-resolution enhancement model. ;
[0066] S4. Output the final result: The image to be enhanced is preprocessed and then input into the trained super-resolution enhancement model The final super-resolution reconstructed image is obtained .
[0067] In a specific implementation, the initial encoder is composed of a convolution block and an average pooling layer in sequence. The convolution block is composed of three downsampling convolution layers, a batch normalization layer, and a sigmoid function. The convolution kernel of each downsampling convolution layer is 3×3, the step size is 1, and the padding is 0.
[0068] Hyperspectral images Input into the initial encoder to obtain the initial encoding features ;
[0069] The initial decoder consists of the first convolutional block, the fully connected block, the second convolutional block, and the average pooling layer in sequence;
[0070] The first convolutional block consists of a downsampling convolutional layer with a convolution kernel of 3×3, a stride of 1, and padding of 0, a batch normalization layer, and a ReLU activation function;
[0071] The fully connected block consists of the first fully connected layer and the second fully connected layer. The input dimension of the first fully connected layer is 802816 and the output dimension is 200704. The input dimension of the second fully connected layer is 200704 and the output dimension is 50176.
[0072] The second convolution block consists of a view function, a convolution layer with a kernel of 1×1, a stride of 1, and padding of 0, batch normalization, and a ReLU activation function. The output dimension of the view function is the same as that of the hyperspectral image. same;
[0073] The initial encoding features Input into the initial decoder to obtain the initial reconstructed image .
[0074] In a specific implementation, the super-resolution encoder is composed of a hyperspectral convolution block, a reconstruction convolution block, a feature fusion layer, a fusion convolution block, and a maximum pooling layer in sequence;
[0075] The hyperspectral convolution consists of the first convolution layer with a convolution kernel of 3×3, a step size of 2, and a padding of 0, a batch normalization layer, a second convolution layer with a convolution kernel of 5×5, a step size of 3, and a padding of 0, and a ReLU activation function. The feature fusion layer input into the hyperspectral convolution obtains the hyperspectral features ;
[0076] The reconstruction convolution block consists of the third convolution layer with a convolution kernel of 1×1, a stride of 1, and a padding of 0, a batch normalization layer, a fourth convolution layer with a convolution kernel of 3×3, a stride of 1, and a padding of 0, and a ReLU activation function. Input into the reconstruction convolution block to obtain the initial reconstruction features ;
[0077] The feature fusion layer is composed of the concat function, batch normalization layer and ReLU activation function in sequence, which combines the hyperspectral features and initial reconstruction features Input into the feature fusion layer to obtain the fusion feature ;
[0078] The fusion convolution block is composed of the fifth convolution layer with a convolution kernel of 1×1, a step size of 1, and a padding of 0, the first batch of normalization layers, the sixth convolution layer with a convolution kernel of 3×3, a step size of 2, and a padding of 0, the second batch of normalization layers, the seventh convolution layer with a convolution kernel of 5×5, a step size of 2, and a padding of 1, the regression calculation formula, and the ReLU activation function. The fusion feature Input to the fifth convolutional layer and the first batch of normalization layers in sequence and then output features , the feature Input to the sixth convolutional layer and the second batch of normalization layers in sequence and then output features , the feature Input to the seventh convolutional layer and output features , then the feature , and Input into the regression calculation formula to get the features , the regression calculation formula is as follows:
[0079] ,
[0080] Finally, the features The super-resolution coding features are obtained by sequentially inputting the ReLU activation function and the maximum pooling layer of the fused convolution block. .
[0081] In a specific implementation, the adaptive coding factor adjustment block is composed of a first normalization layer, a ReLU activation function, a first regulator, a second normalization layer, and a second regulator in sequence;
[0082] The initial reconstructed image Input the first normalization layer and ReLU activation function to get the first adaptive feature , the initial encoding features and super-resolution encoding features Enter into the formula to get the first adjustment factor , the first adaptive feature Multiply by the first adjustment factor , and obtain the first adaptive adjustment feature , the initial encoding features Multiply by the first adjustment factor Get the initial coding adjustment factor , the super-resolution encoding feature Multiply by the first adjustment factor Get the super-resolution coding adjustment factor , the first adaptive adjustment feature Input to the second normalization layer and the second regulator, and adjust the initial encoding factor , super-resolution coding adjustment factor Input into the second regulator to obtain the second adaptive regulation characteristic , the second adaptive adjustment feature With the initial reconstructed image Add together to get the adaptive coding feature ;
[0083] Among them, the calculation formula of the first regulator is:
[0084] ,
[0085] The calculation formula for the second regulator is:
[0086] ,
[0087] in, Represents the initial encoding features The average value of Represents super-resolution encoding features The standard deviation of Represents the initial encoding features The standard deviation of represents the standard deviation calculation function, Represents the first adaptive adjustment feature The average value of represents the first regulator, Represents the second regulator.
[0088] In a specific implementation, the spatial enhanced attention module is composed of the formula The convolution layer consists of a convolution kernel of 1×1, a step size of 1, and a padding of 0, a batch normalization layer, and a ReLU activation function; the super-resolution encoding features and adaptive coding features Add input to formula In the above example, we obtain the spatial enhanced attention feature , the spatially enhanced attention feature Input to the convolution layer and batch normalization layer to obtain spatial features ;
[0089] formula as follows:
[0090] ,
[0091] in, Representation formula The output spatial enhanced attention features, represents transpose;
[0092] The channel-enhanced attention module is composed of the formula , a convolution layer with a convolution kernel of 3×3, a stride of 2, and a padding of 0, a batch normalization layer, and a ReLU activation function; the spatial features and adaptive coding features Add input to formula In c, we get the channel-enhanced attention feature , the channel enhances the attention feature Input to the convolution layer and batch normalization layer to obtain channel features ;
[0093] formula as follows:
[0094] ,
[0095] in, Representation formula The output channel enhances the attention feature, Indicates transpose.
[0096] In a specific implementation, the super-resolution decoder is composed of a first convolution block, a fully connected block, a second convolution block, an average pooling layer, and a save function in sequence;
[0097] The first convolutional block consists of an upsampling convolutional layer with a convolution kernel of 1×1, a stride of 1, and padding of 0, a batch normalization layer, and a Sigmoid activation function;
[0098] The fully connected block consists of 3 fully connected layers. The input dimension of the first fully connected layer is 3211264 and the output dimension is 2097152. The input dimension of the second fully connected layer is 2097152 and the output dimension is 1048576. The input dimension of the third fully connected layer is 1048576 and the output dimension is 262144.
[0099] The second convolutional block consists of a view function with a dimension of 512×512, a convolutional layer with a convolution kernel of 1×1, a stride of 1, and a padding of 0, a batch normalization layer, and a ReLU activation function;
[0100] Channel Features Input into the super-resolution decoder to obtain the super-resolution reconstructed image .
[0101] In a specific implementation, the total training loss The calculation formula is as follows:
[0102] ,
[0103] in, represents the reconstruction loss, represents the cross entropy loss, Indicates a real picture.
[0104] Example 2
[0105] A computing device includes a memory configured to store computer-executable instructions; and a processor configured to execute a hyperspectral-based super-resolution enhancement method for smart city monitoring when the computer-executable instructions are executed by the processor.
[0106] Example 3
[0107] In the embodiment of the present invention, the Pavia Center dataset is used for experiments, and the effectiveness of the method is verified by comparative experiments. In the experiment, we compared the method with three other commonly used super-resolution enhancement methods, namely: bicubic interpolation Bicubic, super-resolution convolutional neural network SRCNN and deep super-resolution network VDSR. The experimental results are shown in Table 1.
[0108] Table 1 Comparison results of the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) between the method in the present invention and the existing method
[0109]
[0110] Table 1 shows the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) indicators of different methods on the Pavia Center dataset. It can be seen that this method is superior to other comparison methods in both PSNR and SSIM indicators, indicating that this method has obvious advantages in improving image resolution and maintaining image structure. Through experimental verification, the hyperspectral-based smart city monitoring super-resolution enhancement method of the present invention has significant advantages in improving image resolution and quality, and can effectively improve the monitoring effect and accuracy of smart city monitoring equipment.
[0111] Example 4
[0112] In the embodiment of the present invention, in order to better observe the effect of image enhancement and verify the applicability of this method to complex weather, the original data of urban haze weather images collected from the Internet are used for experiments, and the effectiveness of this method is verified by comparative experiments. In the experiment, this method is compared with the deep super-resolution network VDSR. The experimental results are as follows: Figure 2 As shown, Figure 2In the figure, (a) is a city image in haze weather, (b) is a city image in clear weather, (c) is the result of enhancing image (a) with the VDSR method, and (d) is the result of enhancing image (a) with this method. It can be seen that (c) is not good in color control. Although it has achieved a certain dehazing effect, the color contrast is too large, and the resolution of details such as tree branches is not improved much, which is quite different from (b). (d) shows that this method can basically restore the state of clear weather, and the control of image color is better than that of the VDSR method. The effect presented is very close to that of (b). It can be seen that the effect of processing (a) with this method and the deep super-resolution network VDSR is compared with image (b). It can be intuitively seen that the image enhanced by this method is more delicate in details and the tree branches are clearer, which shows that this method has achieved the effect of super-resolution enhancement.
[0113] Although the above describes the specific implementation mode of the invention in conjunction with the drawings, it is not intended to limit the scope of protection of the invention. Based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present invention.
Claims
1. A hyperspectral-based super-resolution enhancement method for smart city monitoring, characterized in that: The steps include: S1. Image acquisition: The image denoising method based on the denoising convolutional neural network DnCNN is used to process the images in the publicly available PaviaCenter dataset to obtain hyperspectral images. ; S2. Create a super-resolution enhancement model: The super-resolution enhancement model is composed of an initial encoder, an initial decoder, a super-resolution encoder, an adaptive coding factor adjustment block, a spatial enhancement attention module, a channel enhancement attention module and a super-resolution decoder. Then the hyperspectral image is Input to the super-resolution enhancement model to obtain the super-resolution reconstructed image ; Step S2 is specifically as follows: The initial encoder consists of a convolution block and an average pooling layer. The convolution block consists of three downsampling convolution layers, a batch normalization layer, and a sigmoid function. The convolution kernel of each downsampling convolution layer is 3×3, the stride is 1, and the padding is 0. Hyperspectral images Input into the initial encoder to obtain the initial encoding features ; The initial decoder consists of the first convolutional block, the fully connected block, the second convolutional block, and the average pooling layer in sequence; The first convolutional block consists of a downsampling convolutional layer with a convolution kernel of 3×3, a stride of 1, and padding of 0, a batch normalization layer, and a ReLU activation function; The fully connected block consists of the first fully connected layer and the second fully connected layer. The input dimension of the first fully connected layer is 802816 and the output dimension is 200704. The input dimension of the second fully connected layer is 200704 and the output dimension is 50176. The second convolution block consists of a view function, a convolution layer with a kernel of 1×1, a stride of 1, and padding of 0, batch normalization, and a ReLU activation function. The output dimension of the view function is the same as that of the hyperspectral image. same; The initial encoding features Input into the initial decoder to obtain the initial reconstructed image ; The super-resolution encoder consists of a hyperspectral convolution block, a reconstruction convolution block, a feature fusion layer, a fusion convolution block, and a maximum pooling layer in sequence; The hyperspectral convolution consists of the first convolution layer with a convolution kernel of 3×3, a step size of 2, and a padding of 0, a batch normalization layer, a second convolution layer with a convolution kernel of 5×5, a step size of 3, and a padding of 0, and a ReLU activation function. The feature fusion layer input into the hyperspectral convolution obtains the hyperspectral features ; The reconstruction convolution block consists of the third convolution layer with a convolution kernel of 1×1, a stride of 1, and a padding of 0, a batch normalization layer, a fourth convolution layer with a convolution kernel of 3×3, a stride of 1, and a padding of 0, and a ReLU activation function. Input into the reconstruction convolution block to obtain the initial reconstruction features ; The feature fusion layer is composed of the concat function, batch normalization layer and ReLU activation function in sequence, which combines the hyperspectral features and initial reconstruction features Input into the feature fusion layer to obtain the fusion feature ; The fusion convolution block is composed of the fifth convolution layer with a convolution kernel of 1×1, a step size of 1, and a padding of 0, the first batch of normalization layers, the sixth convolution layer with a convolution kernel of 3×3, a step size of 2, and a padding of 0, the second batch of normalization layers, the seventh convolution layer with a convolution kernel of 5×5, a step size of 2, and a padding of 1, the regression calculation formula, and the ReLU activation function. The fusion feature Input to the fifth convolutional layer and the first batch of normalization layers in sequence and then output features , the feature Input to the sixth convolutional layer and the second batch of normalization layers in sequence and then output features , the feature Input to the seventh convolutional layer and output features , then the feature , and Input into the regression calculation formula to get the features , the regression calculation formula is as follows: , Finally, the features It is input into the ReLU activation function and the maximum pooling layer of the fusion convolution block in turn to obtain the super-resolution coding features. ; S3. Training super-resolution enhancement model: Reconstructing images based on super-resolution and real images Calculate the total training loss, then use the Adam optimizer to iterate the parameters in the super-resolution enhancement model to obtain the trained super-resolution enhancement model. ; S4. Output the final result: The image to be enhanced is preprocessed and then input into the trained super-resolution enhancement model The final super-resolution reconstructed image is obtained .
2. The hyperspectral-based super-resolution enhancement method for smart city monitoring according to claim 1 is characterized by: The adaptive coding factor adjustment block is composed of a first normalization layer, a ReLU activation function, a first regulator, a second normalization layer, and a second regulator in sequence; The initial reconstructed image Input the first normalization layer and ReLU activation function to get the first adaptive feature , the initial encoding features and super-resolution encoding features Enter into the formula to get the first adjustment factor , the first adaptive feature Multiply by the first adjustment factor , and obtain the first adaptive adjustment feature , the initial encoding features Multiply by the first adjustment factor Get the initial coding adjustment factor , the super-resolution encoding feature Multiply by the first adjustment factor Get the super-resolution coding adjustment factor , the first adaptive adjustment feature Input to the second normalization layer and the second regulator, and adjust the initial encoding factor , super-resolution coding adjustment factor Input into the second regulator to obtain the second adaptive regulation characteristic , the second adaptive adjustment feature With the initial reconstructed image Add together to get the adaptive coding feature ; Among them, the calculation formula of the first regulator is: , The calculation formula for the second regulator is: , in, Represents the initial encoding features The average value of Represents super-resolution encoding features The standard deviation of Represents the initial encoding features The standard deviation of represents the standard deviation calculation function, Represents the first adaptive adjustment feature The average value of represents the first regulator, Represents the second regulator.
3. The hyperspectral-based super-resolution enhancement method for smart city monitoring according to claim 2 is characterized by: The spatial enhanced attention module is composed of the formula The convolution layer consists of a convolution kernel of 1×1, a step size of 1, and a padding of 0, a batch normalization layer, and a ReLU activation function; the super-resolution encoding features and adaptive coding features Add input to formula In the above example, we obtain the spatial enhanced attention feature , the spatially enhanced attention feature Input to the convolution layer and batch normalization layer to obtain spatial features ; formula as follows: , in, Representation formula The output spatial enhanced attention features, represents transpose; The channel-enhanced attention module is composed of the formula , a convolution layer with a convolution kernel of 3×3, a stride of 2, and a padding of 0, a batch normalization layer, and a ReLU activation function; the spatial features and adaptive coding features Add input to formula In c, we get the channel-enhanced attention feature , the channel enhances the attention feature Input to the convolution layer and batch normalization layer to obtain channel features ; formula as follows: , in, Representation formula The output channel enhances the attention feature, Indicates transpose.
4. The method for super-resolution enhancement of smart city monitoring based on hyperspectral according to claim 3 is characterized by: The super-resolution decoder consists of the first convolutional block, the fully connected block, the second convolutional block, the average pooling layer, and the save function in sequence; The first convolutional block consists of an upsampling convolutional layer with a convolution kernel of 1×1, a stride of 1, and padding of 0, a batch normalization layer, and a Sigmoid activation function; The fully connected block consists of 3 fully connected layers. The input dimension of the first fully connected layer is 3211264 and the output dimension is 2097152. The input dimension of the second fully connected layer is 2097152 and the output dimension is 1048576. The input dimension of the third fully connected layer is 1048576 and the output dimension is 262144. The second convolutional block consists of a view function with a dimension of 512×512, a convolutional layer with a convolution kernel of 1×1, a stride of 1, and a padding of 0, a batch normalization layer, and a ReLU activation function; Channel Features Input into the super-resolution decoder to obtain the super-resolution reconstructed image .
5. According to the hyperspectral-based super-resolution enhancement method for smart city monitoring according to claim 4, it is characterized in that training Total loss The calculation formula is as follows: , in, represents the reconstruction loss, represents the cross entropy loss, Indicates a real picture.
6. A computing device, characterized in that: It includes a memory configured to store computer-executable instructions; and a processor configured to execute a hyperspectral-based smart city monitoring super-resolution enhancement method as described in any one of claims 1 to 5 when the computer-executable instructions are executed by the processor.
Citation Information
Patent Citations
Hyperspectral single-image super-resolution method based on inter-band attention mechanism adversarial learning
CN111161141A
Hyper-spectral and panchromatic image fusion method for extracting multi-scale spatial-spectral characteristics based on AE
CN112634137A
Image super-resolution method based on recursive attention mechanism
CN114022355A
Indoor old person image super-resolution reconstruction method and system based on self-attention
CN115131214A
Change attention-based variational self-encoding image super-resolution method and system
CN115984117A
