Method for enhancing gas segmentation precision of hyperspectral image and related device

Abstract

The invention provides a method for enhancing the gas segmentation precision of a hyperspectral image and a related device, and the method comprises the following specific steps: inputting the hyperspectral image into a hyperspectral image gas segmentation model, and outputting a hyperspectral image segmentation result; wherein the hyperspectral image gas segmentation model takes a U-Net network as a basic model, an up-sampling part of the U-Net network is inserted into a gas de-camouflage module and a gas interpolation module based on adaptive point sampling, and the gas de-camouflage module is used for fusing local features and global features of a down-sampling feature map to generate a fused feature map; the gas interpolation module based on adaptive point sampling is used for adjusting the position of a sampling point in real time according to local pixel features of the image to obtain a feature map with spatial consistency and key details, so that the purpose of improving the precision of a hyperspectral image segmentation task can be achieved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more specifically to a method and related device for enhancing the gas segmentation accuracy of hyperspectral images. Background Art

[0002] The hyperspectral image segmentation technology has broad application prospects in the fields of environmental monitoring and greenhouse gas emission detection. However, due to its unique high-dimensional characteristics and complex spatial variability, traditional segmentation methods face significant challenges when dealing with such images. Hyperspectral images not only contain rich spectral information but also exhibit extremely high resolution in the spatial dimension. This characteristic enables us to capture subtle spectral changes in the environment, thereby achieving precise monitoring of various environmental factors such as gas emissions, vegetation health, and water pollution. However, precisely this high dimensionality and fine spatial resolution pose higher requirements for existing image segmentation technologies. Traditional image segmentation algorithms may perform well when dealing with low-dimensional images, but when faced with hyperspectral images, due to the sharp increase in computational complexity and feature extraction difficulty, it is often difficult to achieve the desired segmentation effect. Hardware improvements, such as enhancing the resolution of image acquisition devices, although can improve the image quality at the source, the high cost and long R & D cycle limit the wide application of this approach.

[0003] In view of this, researchers have begun to explore optimizing the segmentation effect of hyperspectral images through technological innovations at the software level. Among them, the upsampling technology, as an effective method to improve the spatial resolution and quality of images, has gradually become a research hotspot. Traditional interpolation methods, such as bilinear interpolation and bicubic interpolation, although can improve the resolution of images to a certain extent, since they estimate pixel values based on fixed mathematical formulas, they often cannot accurately capture the detailed information in the images, especially when dealing with hyperspectral images with complex spectral characteristics and spatial variations, the effect is particularly limited. In recent years, deep learning technologies based on convolutional neural networks have made significant progress in the field of hyperspectral image segmentation due to their powerful feature extraction and pattern recognition capabilities. By training deep learning models, the spectral and spatial features in the images can be automatically learned and extracted to achieve precise segmentation of different ground objects or gas emission regions. However, existing deep learning models still face challenges when dealing with hyperspectral images, especially when dealing with targets such as gas emissions with complex spatial and spectral characteristics, the generalization ability and segmentation accuracy of the models still need to be improved.

[0004] To overcome these challenges, researchers have started to introduce dynamic upsampling techniques into the hyperspectral image segmentation task. Dynamic upsampling techniques, such as pixel shuffle and deformable convolutional networks, can adaptively adjust the sampling strategy according to the different content of the image, thereby improving the computational efficiency while retaining more detailed information. The pixel shuffle technique realizes an efficient upsampling method by rearranging the channels of the feature map. It can significantly improve the resolution of the image without increasing additional computational complexity. The deformable convolutional network, on the other hand, enables the convolutional kernel to perform flexible sampling at different spatial positions by introducing learnable offsets, thus better capturing the detailed features and shape changes in the image. These dynamic upsampling techniques provide new ideas for hyperspectral image segmentation. However, in practical applications, how to effectively combine the spectral and spatial characteristics of gases, as well as global and local features, remains an urgent problem to be solved. Summary of the Invention

[0005] To alleviate the problems of insufficient global correlation modeling of features and inadequate detail processing in existing hyperspectral image upsampling techniques, the present invention proposes a method and related device for enhancing the gas segmentation accuracy of hyperspectral images. The gas de-masking module of the hyperspectral image gas segmentation model is used to fuse local features and global features, dynamically distinguish gas regions and background regions to generate a fused feature map; subsequently, the gas interpolation module based on adaptive point sampling of the hyperspectral image gas segmentation model adjusts the position of the sampling points in real time according to the local pixel features of the image, ensuring that the upsampled image retains key details while maintaining spatial consistency, so as to improve the accuracy of the hyperspectral image segmentation task.

[0006] To achieve the above object, the present invention provides the following technical solutions: A method for enhancing the gas segmentation accuracy of hyperspectral images, the specific steps are as follows:

[0007] Input the hyperspectral image into the hyperspectral image gas segmentation model to output the hyperspectral image segmentation result;

[0008] Among them, the hyperspectral image gas segmentation model is based on the U-Net network. The gas de-masking module and the gas interpolation module based on adaptive point sampling are inserted into the upsampling part of the U-Net network. The gas de-masking module is used to fuse the local features and global features of the downsampled feature map to generate a fused feature map; the gas interpolation module based on adaptive point sampling is used to adjust the position of the sampling points in real time according to the local pixel features of the image to obtain a feature map with spatial consistency and key details.

[0009] Further, the gas de - camouflage module includes a mask generation module and a feature enhancement module. The mask generation module is used to divide the feature map generated by downsampling into a feature map with a gas channel and a feature map with a background channel, and calculate probabilities to obtain a gas mask and a background mask. The feature enhancement module is used to obtain a global feature map by using the global information of the gas and the background and the binary feature maps of the gas and the background masks, and add the global feature map and the given feature map to obtain an enhanced feature map.

[0010] Further, the mask generation module includes a 1×1 convolution and a SoftMax function. Among them, a 1×1 convolution operation is performed on the feature map to obtain a feature map with a gas channel and a feature map with a background channel, and then the SoftMax function is used for normalization processing to calculate the probability that each pixel belongs to the gas or the background, obtaining a gas mask and a background mask.

[0011] Further, in the feature enhancement module, the gas mask and the background mask are respectively multiplied element - by - element with the input feature map to obtain local feature maps. Global average pooling is performed on the local feature maps to obtain gas global information and background global information respectively. At the same time, a predefined threshold is used to binarize the gas and background masks to obtain binary feature maps. The sum of the result of multiplying the gas binary feature map with the gas global information and the result of multiplying the background binary feature map with the background global information is used to generate a global feature map.

[0012] Further, the gas interpolation module based on adaptive point sampling includes a standard grid generation module and a pixel offset map generation module. The standard grid generation module is used to generate a standard grid on the feature map. The pixel offset map generation module is used to perform a linear projection on the enhanced feature map to obtain a scaling factor, and then normalize the scaling factor through a Sigmoid function to obtain an offset map recording the displacement value of each pixel. The offset map is combined with the standard grid to obtain a set of sampling points for upsampling.

[0013] Further, the range of the scaling factor is [0, 0.5]. The bilinear interpolation method is used for upsampling, and the feature map is upsampled using the set of sampling points.

[0014] The present invention also provides a system for enhancing the gas segmentation accuracy of hyperspectral images, which implements the steps of the method for enhancing the gas segmentation accuracy of hyperspectral images. The system includes:

[0015] An image acquisition module, which is used to acquire hyperspectral images;

[0016] An image segmentation module, which is used to input the hyperspectral image into the hyperspectral image gas segmentation model and output the hyperspectral image segmentation result;

[0017] Among them, the hyperspectral image gas segmentation model is based on the U-Net network. A gas de-camouflage module and a gas interpolation module based on adaptive point sampling are inserted into the upsampling part of the U-Net network. The gas de-camouflage module is used to fuse the local features and global features of the downsampled feature map to generate a fused feature map; the gas interpolation module based on adaptive point sampling is used to adjust the position of the sampling points in real time according to the local pixel features of the image to obtain a feature map with spatial consistency and key details.

[0018] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above method for enhancing the gas segmentation accuracy of hyperspectral images are implemented.

[0019] The present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the above method for enhancing the gas segmentation accuracy of hyperspectral images are implemented.

[0020] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the above method for enhancing the gas segmentation accuracy of hyperspectral images are implemented.

[0021] Compared with the prior art, the present invention has at least the following beneficial effects:

[0022] The present invention proposes a method for enhancing the gas segmentation accuracy of hyperspectral images. By inserting a gas de-camouflage module and a gas interpolation module based on adaptive point sampling into the hyperspectral image gas segmentation model, the gas segmentation accuracy is effectively improved. The introduction of these two modules enables the model to better process the local and global features of hyperspectral images, thereby improving the accuracy of the segmentation results.

[0023] Furthermore, the gas de-camouflage module realizes the accurate distinction between gas and background regions through a mask generation module and a feature enhancement module. The mask generation module can generate accurate gas masks and background masks, while the feature enhancement module uses these masks and global information to enhance the feature map, making the model more robust in the segmentation task; the mask generation module can efficiently calculate the probability that each pixel belongs to gas or background, thereby generating accurate gas masks and background masks. This design simplifies the calculation process while ensuring the accuracy and efficiency of mask generation; the feature enhancement module realizes the effective extraction and utilization of the global information of gas and background. This design enables the model to better understand the overall structure of the image and retain key details during the segmentation process.

[0024] Furthermore, the gas interpolation module based on adaptive point sampling realizes the dynamic adjustment of sampling points through the standard grid generation module and the pixel offset map generation module. This design makes the upsampling process more flexible and accurate, capable of retaining more boundary details and reducing the occurrence of distortion. Limiting the range of the scaling factor within [0, 0.5] ensures a moderate adjustment amplitude of the sampling points, avoiding the distortion problem caused by excessive adjustment. At the same time, using the bilinear interpolation method for upsampling further improves the accuracy and smoothness of the upsampling result.

[0025] The system of the present invention integrates an image acquisition module and an image segmentation module, realizing the full-process automation from image acquisition to the output of the segmentation result. This design improves the practicability and convenience of the system, enabling users to more conveniently perform the gas segmentation task of hyperspectral images.

[0026] The device of the present invention realizes a method for enhancing the gas segmentation accuracy of hyperspectral images through the collaborative work of a memory and a processor. This design enables the terminal device to have powerful image processing capabilities, meeting the needs of users for the gas segmentation task of hyperspectral images.

[0027] The computer-readable storage medium of the present invention stores a computer program for realizing the method for enhancing the gas segmentation accuracy of hyperspectral images. This design enables users to conveniently obtain and use this method, while ensuring the reliability and security of the program.

[0028] The computer program product of the present invention includes a computer program for realizing the method for enhancing the gas segmentation accuracy of hyperspectral images. This design enables users to quickly obtain the gas segmentation result of hyperspectral images by installing and running this program, improving work efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] By reading the following detailed description of the non-limiting embodiments with reference to the accompanying drawings, other features, objectives, and advantages of the present invention will become more apparent:

[0030] Figure 1 It is the overall framework diagram of a method for enhancing the gas segmentation accuracy of hyperspectral images according to the present invention;

[0031] Figure 2 It is the flowchart of a method for enhancing the gas segmentation accuracy of hyperspectral images according to the present invention embedded in the UNet architecture. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0033] The present invention provides a method for enhancing the gas segmentation accuracy of hyperspectral images, as Figure 1As shown in the figure, a hyperspectral image gas segmentation network is constructed. This network is based on the U-Net network. A gas de-camouflage module and a gas interpolation module based on adaptive point sampling are inserted into the upsampling part of the U-Net network. The gas de-camouflage module is used to fuse local features and global features, dynamically distinguish gas regions and background regions, and generate a fused feature map. Subsequently, the adaptive point sampling module adjusts the position of the sampling points in real time according to the local pixel features of the image, ensuring that the upsampled image retains key details while maintaining spatial consistency, so as to improve the accuracy of the hyperspectral image segmentation task. The specific steps of the above method are as follows:

[0034] Step 1: Construct a hyperspectral image dataset and divide it into three parts: training, validation, and testing. Data augmentation is performed by randomly scaling the image (between 0.5 and 2.0), followed by random cropping and flipping. In addition, to ensure that the input image is square, padding is applied to the image edges;

[0035] The training dataset consists of 996 high-quality non-degraded remote sensing images in the Industrial Smoke Plume Data Set; the validation dataset consists of 215 high-quality non-degraded remote sensing images in the Industrial Smoke Plume Data Set; the testing dataset consists of 215 high-quality non-degraded remote sensing images in the Industrial Smoke Plume Data Set;

[0036] Step 2: Construct a hyperspectral image gas segmentation network. As Figure 2 shown, select the U-Net network as the basic model. The U-Net network includes a downsampling part and an upsampling part. Among them, the upsampling part uses a Gas Sensing Upsampling (GasUpper) module. The GasUpper module includes a gas de-camouflage module and a gas interpolation module based on adaptive point sampling. Specifically:

[0037] Step 2.1 Downsampling part (encoder part)

[0038] The dataset images obtained in Step 1 are sent to the encoder part of the U-Net network after data augmentation. For each encoding stage, features are extracted through two 3×3 convolutional kernels, and then downsampling is performed through a 2×2 max-pooling layer to generate feature maps of different scales. The four encoding layers output feature maps of different resolutions, representing information from low-level details to high-level semantics.

[0039] Step 2.2 The gas de-camouflage module includes a mask generation module and a feature enhancement module, where:

[0040] Step 2.2.1 The mask generation module includes a 1×1 convolution and a SoftMax function, which are used to generate masks for gas and background in the feature map respectively, as follows:

[0041] Input the given feature map \(X\in\mathbb{R}\) H×W×C into the gas de - camouflage module. The mask generation module of the gas de - camouflage module performs a 1×1 convolution operation on the extracted feature map \(X\), converting the input feature map into a feature map with two channels, representing gas and background. Then, the SoftMax function is used to normalize the result, calculating the probability that each pixel belongs to gas or background, and obtaining the gas mask \(M\) G and the background mask \(M\) B , which is expressed by formula (1):

[0042] \(M = \text{Softmax}(f(X)),\ (1)\)

[0043] Step 2.2.2 In the feature enhancement module, the generated gas mask and background mask are multiplied element - by - element with the input feature map respectively to extract local detail features, obtaining the local feature map \(X\) L , as shown in formula (2). The local feature map \(X\) L can capture the boundary information between the gas and background regions. Perform global average pooling (GAP) on the local feature map \(X\) L to extract the gas global information \(V\) G and the background global information \(V\) B ; at the same time, use the predefined threshold \(threshold\) to binarize the gas and background masks to obtain the binarized feature maps; add the product of the gas binarized feature map and the gas global information \(V\) G and the product of the background binarized feature map and the background global information \(V\) B to generate the global feature map \(X\) G , as shown in formula (3).

[0044] \(X\) L =X\odot M G +X\odot M B ,\ (2)

[0045] \(X\) G =(V G \cdot(M G >threshold)+V B \cdot(M B >threshold)).\ (3)

[0046] Then, add the global feature map \(X\) G and the given feature map to obtain the enhanced feature map \(X\) E .

[0047] Step 2.3, the gas interpolation module based on adaptive point sampling includes a standard grid generation module and a pixel offset map generation module, where:

[0048] Step 2.3.1 The standard grid generation module is used to generate a standard grid G on the feature map X;

[0049] In Step 2.3.2, in the pixel offset map generation module, the enhanced feature map X E and the original feature map first pass through a skip connection and a convolutional layer to obtain X D , as seen in formula (4), and then a scaling factor is generated through linear projection, and the scaling factor is normalized through the Sigmoid function to limit its range between [0, 0.5], generating an offset map O recording the displacement value of each pixel, as shown in formula (5);

[0050] X D = F(concat(X, X E ))), (4)

[0051] O = Sigmoid(Linear(X D )) × 0.5, (5)

[0052] Step 2.3.3 Combine the offset map O with the standard grid G to obtain a sampling point set S = O + G for upsampling. Set the upsampling scale to 2 and use the bilinear interpolation method for upsampling to restore the spatial resolution of the feature map to the resolution of the corresponding layer in the encoder. Use the sampling point set S to upsample the feature map X to generate an image of the target size. The upsampled feature map can accurately reflect the details of the gas region, especially in the region with blurred boundaries.

[0053] Step 2.4, splice the upsampled feature map with the feature map of the corresponding level in the encoder (i.e., skip connection), and combine the feature map in the decoder with the feature map of the corresponding resolution in the encoder to ensure that the details of the original image are not lost during the upsampling process.

[0054] Step 3, obtain a training set and train the hyperspectral image gas segmentation network. During the training process, calculate the segmentation loss at each stage of the decoder using the cross-entropy loss function and accumulate these losses to calculate the total loss. The calculation formula is:

[0055]

[0056] where N represents the number of decoder layers, P i is the prediction result of the i-th layer, and G is the true segmentation label.

[0057] Set hyperparameters: Set the total number of training iterations to 80,000, decay the learning rate by a weight of 0.0005 at the 8000th iteration, and configure a momentum of 0.9. Set the batch size to 4, set the initial learning rate to 0.001, use the SGD optimizer to update the network parameters, save the model weights, and train the model using contrastive learning loss, aggregation loss, and iterative refinement loss during the training phase to obtain a trained hyperspectral image gas segmentation network;

[0058] Step 4, input the test set images into the trained hyperspectral image gas segmentation network, output the hyperspectral image segmentation results, and complete the hyperspectral image gas segmentation task.

[0059] In summary, through the gas de - camouflage module, the present invention effectively improves the ability to distinguish gas from the background region. The local feature map captures edge details, while the global feature helps to understand the structure of the overall image. By combining these two, the model can better handle the boundary details of the gas and maintain global consistency. In addition, the module adaptively processes the features of different regions by generating segmentation masks for gas and background, which is particularly important for complex gas edges and can enhance the detail retention ability of the model in the segmentation task; the present invention dynamically adjusts the sampling points through the gas interpolation module based on adaptive point sampling to ensure that more boundary details are retained during the up - sampling process, and the high - resolution feature map generated by this module can more accurately reflect the shape and position of the gas, reducing the distortion during the up - sampling process, and can greatly improve the accuracy and clarity of gas segmentation.

[0060] The following further describes the effect of the present invention through simulation experiments.

[0061] 1. Simulation experiment conditions:

[0062] The hardware platform for the simulation experiment of the present invention: The processor is an Intel(R) Core(TM) i7 - 11700X CPU, the main frequency is 2.5GHz, the memory is 32.0GB, and the graphics card is a 3090Ti.

[0063] The software platform: Windows 10 operating system, Pytorch 1.12.0.

[0064] In the simulation experiment of the present invention, 1211 images in the Industrial Smoke Plume Data Set are used as the training - validation set of the model, and 215 images are used as the test set. The size of the images in this data set is 120×120 pixels, and the image format is.tif. The 215 test set images are enhanced to obtain a test sample set.

[0065] 2. Simulation content and its result analysis:

[0066] The simulation experiment of the present invention uses the U-Net network and the Twins model, replaces their upsampling methods with the present invention and four existing technologies (Nearest method, Bilinear method, Deconv method, Pixel Shuffle method), and performs gas segmentation on the input test sample set respectively, and finally compares the effectiveness of the present invention.

[0067] To verify the simulation experiment effect of the present invention, we only focus on the gas region, and use four evaluation indexes, namely Acc (Accuracy), IoU (Intersection over Union), F1-score and Precision, for the present invention and four existing technologies on the test samples, and combine them with FLOPs (Floating Point Operations Per Second) and Params (Parameters) to observe the experiment effect: Acc reflects the proportion of correctly classified pixels in the total number, IoU measures the overlapping degree between the predicted true value and the ground truth, Precision reflects the accuracy of the prediction result, while F1-score balances the precision and recall of the model, and comprehensively evaluates the overall performance of the model in positive and negative sample classification. The higher these four indexes are, the better the detection and classification effects are. At the same time, FLOPs and Params are used to measure the computational complexity and parameter scale of the model. Lower FLOPs and Params indicate that the model has better computational efficiency and storage performance while maintaining high accuracy.

[0068] Four methods in the present invention and its existing advanced methods are respectively used in the simulation experiment:

[0069] The Nearest and Bilinear interpolation methods of the existing technology refer to an early mathematical technology, which has been widely used in various fields, such as image processing, etc. The Nearest method refers to achieving the interpolation effect by copying the value of the nearest pixel point, while the Bilinear method refers to estimating the value of a new pixel by the weighted average of four adjacent pixels in a two-dimensional plane.

[0070] The Deconv method of the prior art refers to a method for reconstructing the features of a convolutional network by backpropagation, which was adopted in the paper published by Zeiler et al. (Zeiler M D, Krishnan D, Taylor G W, et al., “Deconvolutional networks,” IEEE Computer Society Conference on computer vision and pattern recognition. 2010:2528-2535.).

[0071] The Pixel Shuffle method of the prior art refers to an efficient sub-pixel convolution method adopted in the paper published by Shi et al. (Shi W, Caballero J, Huszár F, et al. “Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network,” Proceedings of the IEEE conference on computer vision and pattern recognition. 2016:1874-1883.).

[0072] The comparison results between the present invention and the average values of all objective evaluation indexes in the above four prior art methods are shown in Table 1 as follows:

[0073] Table 1 Objective evaluation value evaluation table of the present invention and comparative methods

[0074]

[0075] As can be seen from Table 1, the detection effects of the two indexes IoU and Acc of the present invention on the Industrial Smoke Plume Data Set are better than those of the existing methods.

[0076] The following is the device embodiment of the present invention, which can be used to execute the method embodiment of the present invention. For the details not disclosed in the device embodiment, please refer to the method embodiment of the present invention.

[0077] In another embodiment of the present invention, a system for enhancing the gas segmentation accuracy of hyperspectral images is further provided, which can implement the steps of the method for enhancing the gas segmentation accuracy of hyperspectral images. The system for enhancing the gas segmentation accuracy of hyperspectral images includes an image acquisition module and an image segmentation module. Specifically:

[0078] The image acquisition module is used to acquire hyperspectral images;

[0079] The image segmentation module is used to input the hyperspectral image into the hyperspectral image gas segmentation model and output the hyperspectral image segmentation result;

[0080] Among them, the hyperspectral image gas segmentation model is based on the U-Net network. The upsampling part of the U-Net network inserts a gas de-camouflage module and a gas interpolation module based on adaptive point sampling. The gas de-camouflage module is used to fuse the local features and global features of the downsampled feature map to generate a fused feature map; the gas interpolation module based on adaptive point sampling is used to adjust the position of the sampling points in real time according to the local pixel features of the image to obtain a feature map with spatial consistency and key details.

[0081] In another embodiment of the present invention, a terminal device is further provided. The terminal device includes a processor and a memory. The memory is used to store a computer program. The computer program includes program instructions. The processor is used to execute the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can implement the operations of the steps of a method for enhancing the gas segmentation accuracy of hyperspectral images.

[0082] In another embodiment of the present invention, the present invention further provides a storage medium, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in a terminal device, used to store programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and, of course, the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, and the operating system of the terminal is stored in this storage space. Moreover, one or more instructions suitable for being loaded and executed by the processor are stored in this storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. One or more instructions stored in the computer-readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the method for enhancing the gas segmentation accuracy of hyperspectral images in the above embodiments.

[0083] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0084] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0085] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions in Figure 1 one flow or multiple flows and / or blocksFigure 1 The functions specified in one or more boxes.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for implementing the steps of the functions specified in one Figure 1 process or more processes and / or boxes Figure 1 or more boxes.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: the specific implementation manners of the present invention can still be modified or equivalently replaced, and any modification or equivalent replacement that does not depart from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A method for enhancing the accuracy of gas segmentation in hyperspectral images, characterized in that: The specific steps are as follows: Input the hyperspectral image into the hyperspectral image gas segmentation model and output the hyperspectral image segmentation result; Among them, the hyperspectral image gas segmentation model uses the U-Net network as the basic model. The upsampling part of the U-Net network is inserted with a gas de-camouflage module and a gas interpolation module based on adaptive point sampling. The gas de-camouflage module is used to fuse the local features of the downsampled feature map with the global features to generate a fused feature map; the gas interpolation module based on adaptive point sampling is used to adjust the position of the sampling points in real time according to the local pixel features of the image to obtain a feature map with spatial consistency and key details.

2. The method for enhancing the gas segmentation accuracy of hyperspectral images according to claim 1, characterized in that: The gas de-camouflage module includes a mask generation module and a feature enhancement module. The mask generation module is used to divide the feature map generated by downsampling into a feature map with a gas channel and a feature map with a background channel, and calculate the probability to obtain a gas mask and a background mask; The feature enhancement module is used to obtain a global feature map by using the global information of the gas and background and the binary feature map of the gas and background mask, and to add the global feature map and the given feature map to obtain an enhanced feature map.

3. The method for enhancing the gas segmentation accuracy of hyperspectral images according to claim 2, characterized in that: The mask generation module includes a 1×1 convolution and a SoftMax function, wherein a 1×1 convolution operation is performed on the feature map to obtain a feature map with a gas channel and a feature map with a background channel, and then the SoftMax function is used for normalization to calculate the probability of each pixel belonging to the gas or the background to obtain the gas mask and the background mask.

4. The method for enhancing the gas segmentation accuracy of hyperspectral images according to claim 3, characterized in that: In the feature enhancement module, the gas mask and the background mask are respectively multiplied element-by-element with the input feature map to obtain a local feature map, and the local feature map is subjected to global average pooling to obtain the gas global information and the background global information respectively; at the same time, the gas and background masks are binarized using a predefined threshold to obtain a binary feature map; the multiplication result of the gas binary feature map with the gas global information and the multiplication result of the background binary feature map with the background global information are added to generate a global feature map.

5. The method for enhancing the gas segmentation accuracy of hyperspectral images according to claim 1, characterized in that: The gas interpolation module based on adaptive point sampling includes a standard grid generation module and a pixel offset map generation module. The standard grid generation module is used to generate a standard grid on the feature map; the pixel offset map generation module is used to perform linear projection on the enhanced feature map to obtain a scaling factor, and then normalize the scaling factor through a Sigmoid function to obtain an offset map that records the displacement value of each pixel; the offset map is combined with the standard grid to obtain a sampling point set for upsampling.

6. The method for enhancing the gas segmentation accuracy of hyperspectral images according to claim 5, characterized in that: The scaling factor range is [0, 0.5]; bilinear interpolation method is used for upsampling, and the feature map is upsampled using the sampling point set.

7. A system for enhancing the accuracy of gas segmentation in hyperspectral images, characterized in that: The steps of implementing a method for enhancing the gas segmentation accuracy of a hyperspectral image according to any one of claims 1 to 6, the system comprising: An image acquisition module, used for acquiring hyperspectral images; An image segmentation module is used to input the hyperspectral image into the hyperspectral image gas segmentation model and output the hyperspectral image segmentation result; Among them, the hyperspectral image gas segmentation model uses the U-Net network as the basic model. The upsampling part of the U-Net network is inserted with a gas de-camouflage module and a gas interpolation module based on adaptive point sampling. The gas de-camouflage module is used to fuse the local features of the downsampled feature map with the global features to generate a fused feature map; the gas interpolation module based on adaptive point sampling is used to adjust the position of the sampling points in real time according to the local pixel features of the image to obtain a feature map with spatial consistency and key details.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of a method for enhancing the gas segmentation accuracy of a hyperspectral image are implemented as claimed in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of a method for enhancing the gas segmentation accuracy of a hyperspectral image are implemented as claimed in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of a method for enhancing the gas segmentation accuracy of a hyperspectral image are implemented as claimed in any one of claims 1 to 6.

Images