A hyperspectral image generation method and related apparatus

Abstract

The application discloses a hyperspectral image generation method and related device, and relates to the technical field of image processing. The method comprises the following steps: performing pixel-level semantic segmentation on an RGB image to obtain a semantic category map; the semantic category map comprises a plurality of pixel points and corresponding ground object categories; performing matching and filling of pure endmember spectra on each pixel point according to the semantic category map and an endmember spectral library to obtain an initial hyperspectral data cube; the endmember spectral library comprises pure endmember spectra corresponding to a plurality of ground object categories; a boundary region of a target pixel point in the semantic category map is determined, and spectral mixing processing is performed on the pure endmember spectra in the boundary region of the target pixel point to obtain mixed pixel spectra of the target pixel point; and the mixed pixel spectra of the target pixel point are used to update the initial hyperspectral data cube to obtain a final hyperspectral image. The application can improve the accuracy and reliability of the reconstructed hyperspectral image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a hyperspectral image generation method and related device. BACKGROUND

[0002] As an advanced optical detection method, hyperspectral imaging technology can simultaneously obtain the radiation information of a target scene in hundreds of continuous and narrow spectral bands, thereby forming a data cube containing rich spectral characteristics. Compared with the traditional RGB imaging technology which can only record the information of three wide-band red, green and blue, the hyperspectral image (HSI) can reveal the unique spectral response characteristics of the ground object. This fine spectral resolution capability makes it have great application potential in many fields. For example, in the field of earth observation, it is used to realize fine classification and identification of ground objects; in the field of security monitoring, it helps to detect and identify specific small targets from complex backgrounds; in the field of agriculture, it can accurately evaluate the growth state and health condition of vegetation; in the field of resource exploration and environmental monitoring, it also provides strong technical support for identifying specific material components or pollutant distribution.

[0003] However, the professional hyperspectral imaging equipment relied on to realize the above-mentioned application potential usually faces real challenges such as high cost, large system volume, complicated data acquisition process and complex subsequent data processing. These factors jointly restrict the popularization and flexible application of hyperspectral technology in high-time-efficiency requirement, large-scale coverage or resource-limited scenarios. In contrast, the RGB imaging technology relies on mature and inexpensive imaging devices, and the equipment is portable, the data source is extremely rich, and the acquisition cost is extremely low. Therefore, how to use the widely available RGB image to reconstruct or predict the corresponding hyperspectral image, that is, to realize the conversion from RGB to hyperspectral image, has become an important research direction in the field of computer vision and remote sensing image processing. This research direction aims to break through the bottleneck of hyperspectral data acquisition and significantly reduce the application threshold.

[0004] Currently, the existing technical methods for the conversion task from RGB images to hyperspectral images can be mainly divided into two categories and face corresponding challenges. One method is based on a shallow learning model or a sparse coding theory, trying to establish a mapping relationship between RGB three-channel values and a high-dimensional spectral vector. However, this kind of method is often insufficient in modeling ability for the spectral nonlinear mixing effect and spatial heterogeneity commonly existing in real complex scenes, resulting in limitations in spectral accuracy and visual authenticity of the reconstructed hyperspectral images. Another method uses deep learning technology, especially the powerful feature extraction capability of a convolutional neural network (CNN), to learn a complex mapping function from RGB images to hyperspectral images in an end-to-end manner. Although this kind of method has improved the overall reconstruction quality compared with the shallow method, the model learning process may not fully incorporate and utilize prior knowledge about the categories of ground objects in the scene. This lack of utilization of physical prior knowledge may lead to a decrease in the spectral reconstruction accuracy of the model when facing specific categories of targets, especially small-sized targets or targets in complex background interference, and sometimes the reconstructed results will deviate from the real physical spectral characteristics of the ground objects. In addition, deep learning methods generally rely on large-scale, accurately registered RGB-HSI data pairs as training samples, and obtaining such high-quality paired data itself is a difficult task. Therefore, exploring new ways to more effectively integrate image content understanding and spectral physical property constraints to improve the accuracy, reliability and practicality of the reconstructed hyperspectral images is crucial for promoting the technology to a wider range of applications. SUMMARY

[0005] The purpose of the present application is to provide a hyperspectral image generation method and related device, which can improve the accuracy and reliability of the reconstructed hyperspectral image.

[0006] To achieve the above-mentioned purpose, the present application provides the following solutions:

[0007] In a first aspect, the present application provides a hyperspectral image generation method, comprising:

[0008] performing pixel-level semantic segmentation on the obtained RGB image to obtain a semantic category map; the semantic category map comprises a plurality of pixel points and corresponding ground object categories;

[0009] performing pure endmember spectrum matching and filling on each pixel point according to the semantic category map and a pre-constructed endmember spectrum library to obtain an initial hyperspectral data cube; the endmember spectrum library comprises pure endmember spectra corresponding to a plurality of ground object categories respectively;

[0010] A boundary region of the target pixel point in the semantic category map is determined, and a pure endmember spectrum in the boundary region of the target pixel point is subjected to spectral mixing processing to obtain a mixed pixel spectrum of the target pixel point. Then, the mixed pixel spectrum of the target pixel point is used to update an initial hyperspectral data cube to obtain a final hyperspectral image. The target pixel point is a pixel point different from a ground object category of any adjacent pixel point of the target pixel point.

[0011] Optionally, pixel-level semantic segmentation is performed on the obtained RGB image to obtain a semantic category map, specifically including:

[0012] The pixel-level semantic segmentation is performed on the obtained RGB image by using a full convolution network model to obtain a semantic category map. The full convolution network model includes an encoder module, a decoder module and a pixel-level classifier connected in sequence.

[0013] The encoder module is constructed based on a deep residual network. The deep residual network is a ResNet-34 network.

[0014] The decoder module includes a plurality of up-sampling fusion sub-modules connected in sequence. Each up-sampling fusion sub-module includes an up-sampling unit and a feature fusion unit connected in sequence.

[0015] The pixel-level classifier includes a convolution layer and a Sigmoid activation function.

[0016] Optionally, the encoder module is configured to perform multi-scale feature extraction on the obtained RGB image to obtain a deep feature map set. The deep feature map set includes a plurality of deep feature maps of different scales arranged in sequence.

[0017] The current up-sampling fusion sub-module in the decoder module is configured to first perform an up-sampling operation on a previous feature fusion result by using a transposed convolution layer to obtain a current up-sampling result, then match a deep feature map of the same scale as the current up-sampling result from the deep feature map set, and perform a feature fusion operation on the current up-sampling result and the deep feature map of the same scale by using a skip connection mechanism to obtain a current feature fusion result. The current up-sampling fusion sub-module is any up-sampling fusion sub-module. When the current up-sampling fusion sub-module is the first up-sampling fusion sub-module, the previous feature fusion result is the last deep feature map. When the current up-sampling fusion sub-module is the last up-sampling fusion sub-module, the current feature fusion result is a high-resolution feature map.

[0018] The pixel-level classifier is configured to perform a convolution operation on the high-resolution feature map to obtain a category score map of each pixel point, and convert the category score map of each pixel point into a category probability map of each pixel point by using a Sigmoid activation function. Then, the semantic category map is obtained based on the category probability map of each pixel point.

[0019] Optionally, based on the semantic category map and a pre-built endmember spectral library, the pure endmember spectra of each pixel are matched and filled to obtain an initial hyperspectral data cube, specifically including:

[0020] Initialize a zero-value hyperspectral data cube;

[0021] Based on the land cover category of each pixel in the semantic category map, the corresponding clean endmember spectrum is matched from the pre-built endmember spectrum library, and the corresponding clean endmember spectrum is filled into the corresponding spatial position in the zero-value hyperspectral data cube to obtain the initial hyperspectral data cube; the endmember spectrum library is built based on the USGS spectral database.

[0022] Optionally, the pure endmember spectra within the boundary region of the target pixel are subjected to spectral mixing processing to obtain the mixed pixel spectra of the target pixel, specifically including:

[0023] Based on the land cover category corresponding to the target pixel in the semantic category map, an initial binary abundance map of the target pixel is obtained;

[0024] A spatial convolution operation is performed on the initial binary abundance map of the target pixel using a two-dimensional Gaussian kernel to obtain a spatially regularized continuous abundance map of the target pixel.

[0025] The continuous abundance map of the target pixels is normalized to obtain the normalized abundance coefficients of the target pixels.

[0026] Based on the normalized abundance coefficient of the target pixel and the pure endmember spectrum in the boundary region of the target pixel, the mixed pixel spectrum of the target pixel is synthesized by a linear spectral mixing model.

[0027] Optionally, based on the land cover category corresponding to the target pixel in the semantic category map, an initial binary abundance map of the target pixel is obtained, specifically including:

[0028] According to the formula Obtain the initial binary abundance map of the target pixel;

[0029] in, For pixels The initial binary abundance plot; The Kronecker function; For pixels Corresponding land feature categories; This refers to the category of land features.

[0030] Optionally, the linear spectral mixing model is expressed as:

[0031] ;

[0032] in, For pixels Mixed pixel spectra; For pixels Normalized abundance coefficients; For the first The first land cover category corresponding to the first A pure endmember spectrum; This represents the total number of pure endmember spectra.

[0033] Optionally, after obtaining the final hyperspectral image, the method for generating the hyperspectral image further includes:

[0034] Additive Gaussian white noise is added to the final hyperspectral image to obtain the true hyperspectral image.

[0035] In a second aspect, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for generating a hyperspectral image as described above.

[0036] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for generating a hyperspectral image as described above.

[0037] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method for generating a hyperspectral image as described above.

[0038] According to the specific embodiments provided in this application, this application has the following technical effects:

[0039] This application provides a method and related apparatus for generating hyperspectral images. By performing pixel-level semantic segmentation on RGB images, it solves the problem of inaccurate land cover category identification in hyperspectral image generation, achieving accurate land cover category extraction and providing reliable basic data for subsequent spectral matching and mixing. By performing endmember spectral matching through a pre-constructed endmember spectral library and performing spectral mixing processing on clean endmember spectra within boundary regions, it solves the problem of unrealistic spectral representation in boundary regions of hyperspectral data, achieving a more detailed and realistic representation of spectral information and improving the accuracy and detail restoration capability of the final hyperspectral image. Through the combination of the above steps, this application can reconstruct hyperspectral images with higher accuracy and richer information from conventional RGB images, providing reliable technical support for remote sensing image analysis and applications. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 An application environment diagram for a hyperspectral image generation method provided in Embodiment 1 of this application;

[0042] Figure 2 This is a schematic flowchart of a method for generating a hyperspectral image provided in Embodiment 1 of this application;

[0043] Figure 3 The original RGB image and semantic category map provided in Embodiment 1 of this application;

[0044] Figure 4 This is a continuous abundance map after small target space regularization provided in Embodiment 1 of this application;

[0045] Figure 5 This is the final hyperspectral image provided in Embodiment 1 of this application;

[0046] Figure 6 This is a schematic diagram of the structure of a computer device provided in Embodiment 3 of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] Example 1

[0050] The hyperspectral image generation method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send the acquired RGB image to server 104. Server 104 performs pixel-level semantic segmentation on the RGB image to obtain a semantic category map; based on the semantic category map and the endmember spectral library, it matches and fills the clean endmember spectra of each pixel to obtain an initial hyperspectral data cube; it determines the boundary region of the target pixel in the semantic category map and performs spectral mixing processing on the clean endmember spectra within the boundary region of the target pixel to obtain the mixed pixel spectrum of the target pixel; it updates the initial hyperspectral data cube using the mixed pixel spectrum of the target pixel to obtain the final hyperspectral image. Server 104 can then feed back the obtained final hyperspectral image to terminal 102.

[0051] The terminal 102 can be, but is not limited to, various desktop computers, laptops, and IoT devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers, or it can be a cloud server.

[0052] In one exemplary embodiment, such as Figure 2 As shown, a method for generating hyperspectral images is provided. This method is executed by a computer device, specifically a terminal or server, or both. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 201 to 203. Wherein:

[0053] Step 201: Perform pixel-level semantic segmentation on the acquired RGB image to obtain a semantic category map; the semantic category map includes multiple pixels and their corresponding land cover categories, such as... Figure 3 As shown.

[0054] Step 202: Based on the semantic category map and the pre-built endmember spectral library, perform pure endmember spectrum matching and filling for each pixel to obtain an initial hyperspectral data cube; the endmember spectral library includes pure endmember spectra corresponding to various land cover categories.

[0055] Step 203: Determine the boundary region of the target pixel in the semantic category map, and perform spectral mixing processing on the pure endmember spectra within the boundary region of the target pixel to obtain the mixed pixel spectrum of the target pixel. Then, use the mixed pixel spectrum of the target pixel to update the initial hyperspectral data cube to obtain the final hyperspectral image. The target pixel is a pixel whose land cover category is different from any of its adjacent pixels.

[0056] By implementing steps 201 to 203 above, pixel-level semantic segmentation of the RGB image solves the problem of inaccurate land cover category identification in hyperspectral image generation, achieving accurate land cover category extraction and providing reliable basic data for subsequent spectral matching and mixing. Endmember spectral matching is performed using a pre-constructed endmember spectral library, and spectral mixing is applied to the clean endmember spectra within the boundary region, addressing the issue of unrealistic spectral representation in the boundary region of hyperspectral data. This achieves a more detailed and realistic representation of spectral information, improving the accuracy and detail reproduction capability of the final hyperspectral image. Through the combination of these steps, this application can reconstruct hyperspectral images with higher accuracy and richer information from conventional RGB images, providing reliable technical support for remote sensing image analysis and applications.

[0057] Further, step 201 involves performing pixel-level semantic segmentation on the acquired RGB image to obtain a semantic category map, specifically including:

[0058] A fully convolutional network (FCN) model is used to perform pixel-level semantic segmentation on the acquired RGB images to obtain semantic category maps. The FCN model adopts an encoder-decoder architecture, consisting of a sequentially connected encoder module (Feature ResNet), a decoder module (Seg ResNet), and a pixel-level classifier. The encoder module is built based on a modified deep residual network, specifically ResNet-34, used to extract multi-scale hierarchical features from the image. ResNet-34 is currently an efficient image feature extractor, and its residual connection structure facilitates the training of deeper networks. The decoder module includes multiple sequentially connected upsampling fusion submodules. Each upsampling fusion submodule includes sequentially connected upsampling units and feature fusion units, used to progressively restore the resolution of deep feature maps through a series of transposed convolution operations. The pixel-level classifier includes convolutional layers and a sigmoid activation function.

[0059] The core of this application lies in introducing a skip connection mechanism during the upsampling process of the decoder module, which fuses deep feature maps from the corresponding layers of the encoder module. This skip connection mechanism effectively combines the high-resolution, detail-rich shallow features of the encoder module with the deep semantic features of the decoder module, significantly improving the precision of the segmentation results and the accuracy of object edge contour recognition. Through end-to-end training, this application can accurately predict the land cover category for each pixel of the input RGB image, achieving high-precision semantic segmentation.

[0060] Image semantic segmentation is a fundamental and crucial task in the field of computer vision. Its goal is to assign a predefined land cover category label (such as road, building, pedestrian, vehicle, etc.) to each pixel in an RGB image. Traditional deep learning methods based on sliding windows or region nomination (such as the R-CNN series) have achieved great success in object detection tasks, but they suffer from computational redundancy and cannot guarantee pixel-level accuracy.

[0061] The core idea of ​​the FCN model is to replace the fully connected layers at the end of a traditional convolutional neural network (CNN) with convolutional layers. This allows the network to accept input images of arbitrary size and generate a two-dimensional feature map (or heatmap) corresponding to the spatial dimensions of the input image, thus achieving end-to-end pixel-to-pixel prediction. However, due to the multiple downsampling operations (pooling or straddle convolution) during the encoding process, the basic FCN results in a low resolution feature map in the final output, losing a large amount of spatial detail information, leading to a coarse segmentation result and blurred object edges.

[0062] Based on this, this application adopts an improved FCN model, which is based on a deep residual network and a fully convolutional network model with skip connections. It integrates multi-level feature information to generate refined segmentation results, so as to solve the problems of blurred edge details and low accuracy in semantic segmentation results in the prior art.

[0063] Furthermore, the main functions of the FCN model structure include: the encoder module, which is used to extract multi-scale features from the acquired RGB image through the ResNet-34 network and modify its forward propagation process to obtain a set of deep feature maps; the set of deep feature maps includes multiple deep feature maps of different scales (i.e., different spatial resolutions and semantic levels) arranged in sequence.

[0064] The current upsampling fusion submodule in the decoder module is used to first upsample the previous feature fusion result using a transposed convolutional layer, gradually enlarging the size of the deep feature map to obtain the current upsampling result. Then, it matches a deep feature map with the same scale (i.e., the corresponding spatial resolution) as the current upsampling result from the deep feature map set, and uses a skip connection mechanism (such as element-level addition or channel-level concatenation) to perform feature fusion operation on the current upsampling result and the deep feature map with the same scale to obtain the current feature fusion result. The current upsampling fusion submodule can be any upsampling fusion submodule. When the current upsampling fusion submodule is the first upsampling fusion submodule, the previous feature fusion result is the last deep feature map. When the current upsampling fusion submodule is the last upsampling fusion submodule, the current feature fusion result is a high-resolution feature map.

[0065] The pixel-level classifier is used to perform a convolution operation on the high-resolution feature map through a final convolutional layer, mapping the number of channels of the high-resolution feature map to the number of land cover categories, obtaining a category score map for each pixel, and converting the category score map of each pixel into a category probability map for each pixel through a sigmoid activation function. Then, based on the category probability map of each pixel, a semantic category map is obtained.

[0066] The beneficial effects of the FCN model in this application are as follows:

[0067] (1) Effective feature fusion: Through the skip connection mechanism, the shallow features (such as edges and textures) with rich spatial details provided by the encoder module are combined with the deep features with high-level semantic information gradually recovered by the decoder module, which achieves information complementarity and greatly improves the clarity and accuracy of the segmentation boundary.

[0068] (2) Using pre-trained weights: The module directly uses ResNet-34 weights pre-trained on large datasets (such as ImageNet) for initialization, which accelerates the convergence speed of the model and improves the generalization ability of the model.

[0069] (3) End-to-end pixel-level prediction: The entire FCN model is a complete, differentiable neural network that can be trained end-to-end and directly optimize pixel-level segmentation tasks.

[0070] Furthermore, this application also provides a specific implementation example of the FCN model:

[0071] A. Encoder Module: The encoder module for multi-scale feature extraction in this application utilizes a ResNet-34 network. Unlike the classification task of the standard ResNet network, its purpose is no longer to output a final classification vector, but rather to generate a feature pyramid map, capturing and returning a series of deep feature maps from intermediate processes. Its output is represented as a set of deep feature maps:

[0072] ;

[0073] in, For encoder module number The deep feature maps output from each level, along with the index... The increase, While the spatial resolution is reduced, the semantic information it contains has a higher level of abstraction (i.e., semantic level), as shown in the above-mentioned set. This is the deepest, lowest-resolution feature map output by the encoder module.

[0074] B. Decoder Module: This application employs a cascaded feature fusion decoder, the core task of which is to progressively reconstruct the deep feature maps extracted by the encoder module into pixel-level semantic category maps. For example, from... Initially, through a series of upsampling and feature fusion operations, the resolution is gradually restored, and a final pixel-level semantic category map is generated. The decoding process iterates as follows: Let... For the decoder module The output feature map of each stage (i.e., the current feature fusion result), the input of the decoder module in the initial stage is the last deep feature map output by the encoder module. :

[0075] ;

[0076] in, This is the first transposed convolutional layer, its function is to... Spatial resolution magnification, Indicates batch normalization. This represents the nonlinear activation function (ReLU).

[0077] In subsequent stages, a cross-level feature fusion mechanism (i.e., a skip connection mechanism) is employed to integrate the upsampling results from the previous stage of the decoder module. Deep feature maps with corresponding spatial resolution in the encoder module To integrate.

[0078] ;

[0079] in, This is an element-wise feature fusion operation. For the first The transposed convolutional layer of the stage.

[0080] Cross-level feature fusion mechanism solves the problem of deep semantic information Detailed information in shallow space The "semantic gap" between the two can be bridged by adding them element-wise. This not only allows for upsampling based on existing high-level semantic inference, but also enables real-time calibration and refinement using precise spatial information provided by the encoder module, thereby improving the boundary clarity of the segmentation results and the ability to recognize small objects.

[0081] C. Pixel-level classifier: After the above cross-level feature fusion, the final high-resolution feature map is obtained. Finally, its channel dimensions are mapped to the number of land cover categories through an output convolutional layer. Generate the final category score map. .

[0082] ;

[0083] in, This is the output convolutional layer.

[0084] Next, the final category score chart will be generated. The Sigmoid activation function is used to convert the probability of each pixel in each land cover category into a predicted probability. :

[0085] ;

[0086] in, Indicates in batch In the middle, pixel position Belongs to the category of land features The original prediction score.

[0087] Finally, the FCN model outputs a semantic category map, in which each pixel is assigned a specific land feature category label (such as "road", "vegetation", "building" etc.).

[0088] Furthermore, the core of the FCN model in this application lies in two key operations: two-dimensional convolution operation and upsampling operation.

[0089] 1. Two-dimensional convolution operation:

[0090] A two-dimensional convolution operation can be defined by the following formula for an input feature map. and a convolution kernel Output feature map At pixel value for:

[0091] ;

[0092] in, It is the size of The convolution kernel; It is the step size; bias It is a bias term; It is an activation function; It is the row index of the convolution kernel in two-dimensional space; It is the column index of the convolution kernel.

[0093] 2. Upsampling / transposed convolution operation:

[0094] The FCN model no longer includes fully connected layers; instead, it uses upsampling layers to restore the low-resolution feature maps downsampled by the encoder module to their original image size. This application uses transposed convolutional layers to achieve this function. Transposed convolution is not the inverse operation of convolution, but a special type of convolution whose output size is larger than its input size. It achieves the effect of enlarging the feature map size by inserting zero values ​​between pixels in the input feature map and then performing a standard convolution on the padded feature map. It can be regarded as a learnable upsampling layer.

[0095] Further, in step 202, based on the semantic category map and the pre-built endmember spectral library, the pure endmember spectra of each pixel are matched and filled to obtain the initial hyperspectral data cube, specifically including:

[0096] Step 2021: First, initialize a zero-value hyperspectral data cube HSI with dimensions H×W×B; then, parse the semantic category map into a two-dimensional integer matrix L with dimensions H×W (H is the image height, W is the image width). Each element in the matrix... Represents pixel coordinates The label of the land feature category to which it belongs.

[0097] Step 2022: Based on the land cover category of each pixel in the semantic category map, match the corresponding clean endmember spectra from the pre-built endmember spectral library, and fill the corresponding clean endmember spectra into the corresponding spatial positions in the zero-value hyperspectral data cube to obtain the initial hyperspectral data cube. The clean endmember spectrum filling result (i.e., the initial hyperspectral data cube) is as follows: Figure 4 As shown; that is, traversing the matrix L, for any pixel with land cover category c ,Right now Retrieve the corresponding pure endmember spectra from the endmember spectral library. and perform the assignment operation. .

[0098] The endmember spectral library is constructed based on the USGS spectral database. The pure endmember spectra in the USGS spectral database are ground cover spectral reflectance data obtained through spectrometer measurements and standardized processing. When loading the endmember spectral library, the number of reference bands is first determined, and all spectral data in the library are verified to have the same dimension to ensure consistency in subsequent processing.

[0099] Further, in step 203, the pure endmember spectra within the boundary region of the target pixel are subjected to spectral mixing processing to obtain the mixed pixel spectrum of the target pixel, specifically including:

[0100] Step a1: Based on the land cover category corresponding to the target pixel in the semantic category map obtained from semantic segmentation, an initial binary abundance map of discrete target pixels is obtained. The initial binary abundance map represents the spatial distribution of plant categories in various regions, and it is represented by pixels. The value at that point is defined by the Kronecker function, specifically:

[0101] According to the formula Obtain the initial binary abundance map of the target pixel;

[0102] in, For pixels The initial binary abundance plot; The Kronecker function; For pixels Corresponding land feature categories; This refers to the category of land features.

[0103] This step initially assumes that each pixel is 100% covered by a single land cover class. However, this representation introduces abrupt changes at land cover class boundaries that are not physically accurate, requiring smoothing through spatial regularization.

[0104] Step a2 involves constructing a smooth transition region at the boundaries of land cover categories and simulating the spatial continuity of land cover category distribution in the real world, based on the initial binary abundance map. Spatial regularization is performed using a two-dimensional Gaussian kernel. Spatial convolution is performed on the initial binary abundance map of the target pixels to obtain a spatially regularized continuous abundance map of the target pixels. :

[0105] ;

[0106] Among them, the standard deviation of the two-dimensional Gaussian kernel This is a key parameter that directly controls the degree of spatial smoothing, i.e., the width of the mixing region. The physical meaning of spatial convolution is to "diffuse" the abundance value (which is 1) of the pure region to the neighboring boundary region, thereby forming a smooth abundance gradient at the boundary of different land cover categories; the continuous abundance map after spatial regularization of small targets is as follows: Figure 4 As shown.

[0107] Step a3: Normalize the continuous abundance map of each pixel to obtain the normalized abundance coefficient of each pixel; specifically, to ensure the validity of its physical meaning, the abundance coefficient must meet the following two constraints:

[0108] Abundance Non-negativity Constraint (ANC):

[0109] ;

[0110] Abundance Sum-to-One Constraint (ASC):

[0111] .

[0112] The continuous abundance map obtained after Gaussian smoothing It satisfies the nonnegativity constraint, but at any pixel point and It is not equal to 1, which violates the constraint that the sum is one, so it needs to be normalized further.

[0113] For any pixel First, identify all species in its neighborhood that have non-zero abundance contributions. Land cover categories Then, calculate each land cover category. Normalized abundance coefficients :

[0114] ;

[0115] in, Let (i,j) be the unnormalized abundance value corresponding to the k-th land cover category at pixel (i,j); Let be the unnormalized abundance value corresponding to the k-th land cover category at pixel (i,j); t is the category index. The normalized abundance coefficients are... It strictly satisfies both ANC and ASC physical constraints.

[0116] Based on the normalized abundance coefficients of the target pixels and the pure endmember spectra within the boundary regions of the target pixels, the mixed pixel spectra of the target pixels are synthesized using a linear spectral mixing model (ignoring noise terms). The linear spectral mixing model is expressed as follows:

[0117] ;

[0118] in, For pixels Mixed pixel spectra; The abundance coefficients are the normalized values ​​for each pixel. For the first The first land cover category corresponding to the first A pure endmember spectrum; This represents the total number of pure endmember spectra.

[0119] Furthermore, after obtaining the final hyperspectral image, the method for generating the hyperspectral image further includes:

[0120] To the final hyperspectral image (also the final hyperspectral data cube, such as...) Figure 5 Additive white Gaussian noise (AWGN) is added to the image to simulate the noise characteristics of a real sensor and obtain a real hyperspectral image; the intensity of the additive white Gaussian noise is controlled by the target signal-to-noise ratio (SNR) parameter.

[0121] The beneficial effects of the hyperspectral image generation method provided in this application embodiment are as follows:

[0122] 1. No need for large-scale paired training data: This application decouples the task into two stages: semantic segmentation and spectral filling. The main training pressure is concentrated on the semantic segmentation model. It can utilize a large number of existing RGB semantic segmentation datasets, avoiding strong dependence on expensive paired RGB-HSI datasets.

[0123] 2. Enhanced physical authenticity of generated spectra: By introducing an endmember spectral library based on physical measurements, this application assigns spectral information with clear physical meaning to each pixel, rather than a statistical mapping learned purely by the network, which significantly improves the accuracy and interpretability of the generated spectra.

[0124] 3. Enhanced flexibility and scalability: The semantic segmentation model and the endmember spectral library are two independent modules. Users can flexibly replace the segmentation model with a more advanced one, or build a more professional and refined endmember spectral library (such as for specific crops or minerals), according to specific application scenarios, thus enhancing the versatility and generalizability of the method.

[0125] Example 2

[0126] This application also provides an application scenario in which the above-described hyperspectral image generation method is applied. Specifically, the hyperspectral image generation method provided in this embodiment can be applied in a hyperspectral image generation scenario. The hyperspectral image generation scenario includes a semantic segmentation stage and a spectral filling stage; the semantic segmentation stage is used to perform semantic segmentation on the RGB image to obtain a semantic category map; the spectral filling stage is used to obtain the final hyperspectral image based on the semantic category map and the endmember spectral library. The hyperspectral image generation method provided in this embodiment belongs to the semantic segmentation stage and the spectral filling stage.

[0127] Example 3

[0128] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 6 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores processed data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a method for generating hyperspectral images.

[0129] Those skilled in the art will understand that Figure 6 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0130] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0131] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0132] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0133] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0134] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0135] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for generating a hyperspectral image, characterized in that, The method for generating the hyperspectral image includes: The acquired RGB image is subjected to pixel-level semantic segmentation to obtain a semantic category map; the semantic category map includes multiple pixels and their corresponding land cover categories; Based on the semantic category map and the pre-built endmember spectral library, the pure endmember spectra of each pixel are matched and filled to obtain the initial hyperspectral data cube; the endmember spectral library includes pure endmember spectra corresponding to various land cover categories. The boundary regions of target pixels in the semantic category map are determined, and the pure endmember spectra within the boundary regions of the target pixels are subjected to spectral mixing to obtain the mixed pixel spectra of the target pixels. Then, the mixed pixel spectra of the target pixels are used to update the initial hyperspectral data cube to obtain the final hyperspectral image. The target pixel is a pixel whose land cover category is different from any of its adjacent pixels.

2. The method for generating hyperspectral images according to claim 1, characterized in that, Perform pixel-level semantic segmentation on the acquired RGB image to obtain a semantic category map, specifically including: A fully convolutional network model is used to perform pixel-level semantic segmentation on the acquired RGB image to obtain a semantic category map; the fully convolutional network model includes an encoder module, a decoder module, and a pixel-level classifier connected in sequence; The encoder module is constructed based on a deep residual network, which is a ResNet-34 network. The decoder module includes multiple upsampling fusion sub-modules connected in sequence, and each upsampling fusion sub-module includes an upsampling unit and a feature fusion unit connected in sequence. Pixel-level classifiers consist of convolutional layers and a sigmoid activation function.

3. The method for generating hyperspectral images according to claim 2, characterized in that, The encoder module is used to perform multi-scale feature extraction on the acquired RGB image to obtain a deep feature map set; the deep feature map set includes multiple deep feature maps of different scales arranged in sequence. The current upsampling fusion submodule in the decoder module is used to first upsample the previous feature fusion result using a transposed convolutional layer to obtain the current upsampling result. Then, it matches a deep feature map with the same scale as the current upsampling result from the deep feature map set, and uses a skip connection mechanism to perform feature fusion on the current upsampling result and the deep feature map with the same scale to obtain the current feature fusion result. The current upsampling fusion submodule can be any upsampling fusion submodule. When the current upsampling fusion submodule is the first upsampling fusion submodule, the previous feature fusion result is the last deep feature map. When the current upsampling fusion submodule is the last upsampling fusion submodule, the current feature fusion result is a high-resolution feature map. The pixel-level classifier is used to perform convolution operations on the high-resolution feature map to obtain the category score map of each pixel, and to convert the category score map of each pixel into a category probability map of each pixel through the Sigmoid activation function. Then, based on the category probability map of each pixel, a semantic category map is obtained.

4. The method for generating hyperspectral images according to claim 1, characterized in that, Based on the semantic category map and a pre-built endmember spectral library, the pure endmember spectra of each pixel are matched and filled to obtain an initial hyperspectral data cube, which specifically includes: Initialize a zero-value hyperspectral data cube; Based on the land cover category of each pixel in the semantic category map, the corresponding clean endmember spectrum is matched from the pre-built endmember spectrum library, and the corresponding clean endmember spectrum is filled into the corresponding spatial position in the zero-value hyperspectral data cube to obtain the initial hyperspectral data cube; the endmember spectrum library is built based on the USGS spectral database.

5. The method for generating hyperspectral images according to claim 1, characterized in that, The pure endmember spectra within the boundary region of the target pixel are subjected to spectral mixing processing to obtain the mixed pixel spectra of the target pixel, specifically including: Based on the land cover category corresponding to the target pixel in the semantic category map, an initial binary abundance map of the target pixel is obtained; A spatial convolution operation is performed on the initial binary abundance map of the target pixel using a two-dimensional Gaussian kernel to obtain a spatially regularized continuous abundance map of the target pixel. The continuous abundance map of the target pixels is normalized to obtain the normalized abundance coefficients of the target pixels. Based on the normalized abundance coefficient of the target pixel and the pure endmember spectrum in the boundary region of the target pixel, the mixed pixel spectrum of the target pixel is synthesized by a linear spectral mixing model.

6. The method for generating a hyperspectral image according to claim 5, characterized in that, Based on the land cover category corresponding to the target pixel in the semantic category map, an initial binary abundance map of the target pixel is obtained, specifically including: According to the formula Obtain the initial binary abundance map of the target pixel; in, For pixels The initial binary abundance plot; The Kronecker function; For pixels Corresponding land feature categories; This refers to the category of land features.

7. The method for generating a hyperspectral image according to claim 5, characterized in that, The linear spectral mixing model is expressed as follows: ； in, For pixels Mixed pixel spectra; For the first Each land cover category at the pixel The normalized abundance coefficients for any pixel First, identify all species in its neighborhood that have non-zero abundance contributions. Land cover categories Then calculate each land cover category Normalized abundance coefficients ; Category of land features The corresponding pure endmember spectrum.

8. The method for generating a hyperspectral image according to claim 1, characterized in that, After obtaining the final hyperspectral image, the method for generating the hyperspectral image further includes: Additive Gaussian white noise is added to the final hyperspectral image to obtain the true hyperspectral image.

9. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that the processor executes the computer program to implement the method for generating a hyperspectral image according to any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method for generating a hyperspectral image according to any one of claims 1-8.

Images