Hyperspectral scene generation method and system based on dynamic target embedding

Abstract

The invention discloses a hyperspectral scene generation method based on dynamic target embedding, and the method comprises the steps: inputting an RGB base map of a scene object and correlation parameters of the scene object, generating a hyperspectral image of the scene object through a spatial-spectral combined generative adversarial network, and updating network parameters, the associated parameters of the scene object at least comprise latitude and longitude coordinates, a wave band range, spectral resolution, spatial resolution and environmental factors; obtaining dynamic target embedded data based on the spectral information and the motion track of a target object, and obtaining a hyperspectral image containing the target object based on the hyperspectral image of the scene object and the dynamic target embedded data; and obtaining evaluation data of the scene object through performance evaluation based on the hyperspectral image containing the target object and the truth value of the hyperspectral image containing the target object, wherein the evaluation data at least comprises data of an Euclidean distance, a relative quadratic error, a spectral angle and a peak signal-to-noise ratio.

Description

Technical Field

[0001] This invention belongs to the field of hyperspectral remote sensing scene generation technology, and particularly relates to a hyperspectral scene generation method and system based on dynamic target embedding. Background Technology

[0002] Space-based systems, through constellation deployment, can overcome geographical limitations and achieve multi-dimensional global coverage, meeting the wide-area detection needs of various fields and types of targets. Hyperspectral payloads possess spectral fusion capabilities, enabling the acquisition of rich spectral information and effective identification of complex, multi-layered targets. This significantly enhances the on-orbit precision target detection capabilities of space-based systems, demonstrating important application value in fields such as resource surveys, environmental monitoring, and military reconnaissance. However, the development of hyperspectral detection technology in space-based systems currently faces two major challenges. First, there is a severe shortage of high-quality hyperspectral data samples. Due to limitations imposed by revisit cycles and weather conditions, and the high cost of satellite data acquisition, the number of available data samples is limited, making it difficult to meet the large-scale training requirements of algorithms such as deep learning. Second, data quality control is challenging. Remote sensing images are highly susceptible to various distortions such as noise, blurring, and non-uniformity during imaging, compression, and transmission. Current hyperspectral scene quality assessments primarily rely on manual judgment, which is not only inefficient and highly subjective but also involves payload calibration cycles lasting several weeks, failing to meet the requirements of real-time processing. Summary of the Invention

[0003] To address the aforementioned issues, this invention proposes a hyperspectral scene generation method and system based on dynamic target embedding. It generates hyperspectral images of scene objects and updates network parameters through a spatial-spectral joint generative adversarial network, adaptively extracting scene data features and generating scene data. This solves the problem of difficulty in generating high-quality hyperspectral scene samples due to the complex distribution of ground features and high data dimensionality in high-dimensional and complex remote sensing image data. To address the challenges of complex data distribution and modeling difficulties, spatial reconstruction discriminant networks and spectral fidelity discriminant networks are introduced during scene data acquisition, enhancing the adaptive distribution of scene feature level and pixel level data, ensuring that the data generated by the generator more closely approximates the real-world distribution of ground features and scenes.

[0004] A first aspect of the present invention provides a hyperspectral scene generation method based on dynamic target embedding, comprising: The input scene object's RGB base map and the scene object's associated parameters are used to generate a hyperspectral image of the scene object through a spatial-spectral joint generative adversarial network and update the network parameters. The associated parameters of the scene object include at least latitude and longitude coordinates, band range, spectral resolution, spatial resolution, and environmental factors. Dynamic target embedding data is obtained based on the spectral information and motion trajectory of the target object; a hyperspectral image containing the target object is obtained based on the hyperspectral image of the scene object and the dynamic target embedding data. Evaluation data of the scene object is obtained through performance evaluation based on the hyperspectral image containing the target object and the ground truth of the hyperspectral image containing the target object. The evaluation data includes at least Euclidean distance, relative quadratic error, spectral angle and peak signal-to-noise ratio.

[0005] Preferably, the step of generating a hyperspectral image of the scene object and updating the network parameters by using a spatiotemporal joint generative adversarial network based on the input RGB base image of the scene object and the associated parameters of the scene object further includes: Based on the RGB base map of the scene object and the associated parameters of the scene object, a hyperspectral image of the scene object is generated through a spatiotemporal joint generative adversarial network. Based on the hyperspectral image of the scene object, the RGB base image of the scene object, and the correlation parameters of the scene object, a spatial reconstruction discriminant network is used to obtain the discrimination result of the authenticity of the hyperspectral image of the scene object; A spectral curve dataset is obtained from the hyperspectral image of the scene object through global random sampling. Based on the aforementioned spectral curve dataset, a spectral fidelity discrimination network is used to obtain the true or false discrimination results of the spectral features of the hyperspectral image of the scene object; Construct a loss function for network training and update network parameters.

[0006] Preferably, the step of generating a hyperspectral image of the scene object using a spatiotemporal joint generative adversarial network based on the RGB base image of the scene object and the associated parameters of the scene object further includes: Based on the RGB image of the scene object and the associated parameters of the scene object, the feature data of the scene object is obtained through a spatial information encoder. The spatial information encoder has 8 consecutive 4×4 convolution kernels with a stride of 2, and the activation function of each layer is LeakyReLU. Based on the feature data of the scene object, a hyperspectral image of the scene object is obtained through an image information decoder. The image information decoder has a 4×4 deconvolution kernel with a stride of 2. The dropout of the first three layers of the image information decoder is set to 0.5 and the activation function of the output layer is Tanh.

[0007] Preferably, the step of obtaining the discrimination result of the hyperspectral image of the scene object through a spatial reconstruction discriminant network based on the hyperspectral image of the scene object, the RGB base image of the scene object, and the correlation parameters of the scene object further includes: By stacking two 3×3 convolutional layers and using the LeakyReLU activation function; The first instruction is passed twice. The first instruction includes: a 4×4 convolutional kernel with a stride of 2, a 3×3 convolutional layer stack, and the activation function LeakyReLU. The convolution kernel is 4×4 with a stride of 2 and the activation function is LeakyReLU. Spectral curve data and authenticity evaluation results are obtained by stacking 4×4 convolutional layers and using the Sigmoid activation function. The authenticity evaluation results are in the range of [0,1] probability.

[0008] Preferably, the step of obtaining a spectral curve dataset based on the hyperspectral image of the scene object through global random sampling further includes: The hyperspectral image of the scene object is divided into several image block data by uniformly dividing the width dimension; Based on several image block data, a random algorithm is used to obtain the location information of an appropriate number of image block data. Based on the location information of the image patch data, a spectral curve dataset is obtained by extraction, and the spectral curve dataset includes spectral data of at least 150 bands.

[0009] Preferably, the step of obtaining the authenticity determination result of the spectral features of the hyperspectral image of the scene object based on the spectral curve data through a spectral fidelity discrimination network further includes: Based on the spectral curve data, the softmax layer data is obtained sequentially through a multilayer perceptron algorithm and four linear transformations. The structure of the multilayer perceptron algorithm is 150-128-256-128-2-1. Based on the softmax layer data, the first value is extracted to obtain the true or false discrimination result of the spectral features of the hyperspectral image of the scene object. The range of the true or false discrimination result is [0,1] and the value is positively correlated with the true or false nature.

[0010] Preferably, the step of constructing the loss function for network training and updating network parameters further includes: The network training loss function is constructed based on pixel-level fidelity loss, spectral curve fidelity loss, spatial structure fidelity loss, and adversarial discrimination loss, and its calculation expression is as follows: In the formula Let be the training loss function of the network. For the pixel-level fidelity loss, For the loss of fidelity of the spectral curve, For the loss of fidelity of the spatial structure, For the adversarial discriminant loss, , , These are the weights for loss of spectral curve fidelity, loss of spatial structure fidelity, and loss of adversarial discrimination, respectively.

[0011] Preferably, the step of obtaining dynamic target embedding data based on the spectral information and motion trajectory of the target object further includes: Intermediate data is obtained by adding Gaussian noise along the direction of travel of the target object, based on its travel direction, height, speed, and acceleration. Based on the intermediate data, the motion trajectory of the target object is obtained by incorporating the pixel distance of the target motion; Dynamic target embedding data is obtained based on the spectral information and motion trajectory of the target object.

[0012] A second aspect of the present invention provides a hyperspectral scene generation system based on dynamic target embedding, employing any one of the above-mentioned hyperspectral scene generation methods based on dynamic target embedding, comprising: The scene generation module is used to generate a hyperspectral image of the scene object based on the RGB base map of the scene object and the associated parameters of the scene object through a generative adversarial network that combines spatial and spectral data. The target embedding module is used to obtain a hyperspectral image containing the target object by incorporating the dynamic target embedding data into the hyperspectral image of the scene object; The performance evaluation module is used to calculate the evaluation data of the scene object based on the hyperspectral image containing the target object and the ground truth of the hyperspectral image containing the target object, respectively, by Euclidean distance, relative quadratic error, spectral angle and peak signal-to-noise ratio.

[0013] Preferably, the scene generation module includes a generation network module, a spatial reconstruction discrimination network module, and a spectral fidelity discrimination network module. The generation network module is used to obtain a hyperspectral image of the scene object based on the RGB base map of the scene object and the association parameters of the scene object. The spatial reconstruction discrimination network module is used to obtain a discrimination result of the authenticity of the hyperspectral image of the scene object based on the hyperspectral image of the scene object, the RGB base map of the scene object, and the association parameters of the scene object. The spectral fidelity discrimination network module is used to obtain a discrimination result of the authenticity of the spectral features of the hyperspectral image of the scene object based on the spectral curve dataset.

[0014] Because the present invention adopts the above technical solution, it has the following advantages and positive effects compared with the prior art: By using a spatial-spectral joint generative adversarial network to acquire hyperspectral images of scene objects and update network parameters, the network can adaptively extract scene data features and generate scene data, thus solving the problem that the complex distribution of ground objects and the high dimensionality of data in high-dimensional and complex remote sensing image data make it difficult to generate high-quality hyperspectral scene samples.

[0015] To address the challenges of complex data distribution and modeling difficulties, spatial reconstruction discriminant network and spectral fidelity discriminant network are introduced during the acquisition of scene data. This enhances the adaptive distribution of scene feature level and pixel level data, ensuring that the data generated by the generator is closer to the distribution of real ground objects. Attached Figure Description

[0016] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a main flowchart of a hyperspectral scene generation method based on dynamic target embedding according to the present invention; Figure 2 This is a schematic diagram of the process for acquiring hyperspectral scene data in this invention; Figure 3 This is a process result diagram of a hyperspectral scene generation method based on dynamic target embedding according to the present invention; Figure 4 This is a schematic diagram of the framework of a hyperspectral scene generation system based on dynamic target embedding according to the present invention; Figure 5 This is a diagram showing the unit composition of the display and control interaction module of a hyperspectral scene generation system based on dynamic target embedding according to the present invention. Detailed Implementation

[0017] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer from the following description and claims. It should be noted that the drawings are all in a very simplified form and use non-precise ratios, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0018] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0019] First Embodiment See Figure 1 , Figure 2 and Figure 3 , A first aspect of the present invention provides a hyperspectral scene generation method based on dynamic target embedding, comprising: S100: The input scene object's RGB base map and the scene object's associated parameters are used to generate a hyperspectral image of the scene object through a spatial-spectral joint generative adversarial network and update the network parameters. The associated parameters of the scene object include at least latitude and longitude coordinates, band range, spectral resolution, spatial resolution, and environmental factors. S200: Obtain dynamic target embedding data based on the spectral information and motion trajectory of the target object; obtain a hyperspectral image containing the target object based on the hyperspectral image of the scene object and the dynamic target embedding data. S300: Based on the hyperspectral image containing the target object and the ground truth of the hyperspectral image containing the target object, the evaluation data of the scene object is obtained through performance evaluation. The evaluation data includes at least Euclidean distance, relative quadratic error, spectral angle and peak signal-to-noise ratio.

[0020] The specific steps include: S100: The RGB base map of the input scene object and the associated data of the object are used to obtain hyperspectral scene data through a spatiotemporal joint generative adversarial network and update the network parameters. The steps further include: S110: Generate a hyperspectral image of the scene object using a spatial-spectral joint generative adversarial network based on the RGB base map and the associated parameters of the scene object. This step further includes: obtaining feature data of the scene object using a spatial information encoder based on the RGB base map and the associated parameters of the scene object. The spatial information encoder has 8 consecutive 4×4 convolutional kernels with a stride of 2, and the activation function of each layer is LeakyReLU. Based on the feature data of the scene object, obtain hyperspectral scene data using an image information decoder. The image information decoder has 4×4 deconvolutional kernels with a stride of 2, and the dropout of the first three layers of the image information decoder is set to 0.5 and the activation function of the output layer is Tanh.

[0021] S120: Based on the hyperspectral image of the scene object, the RGB base map of the scene object, and the correlation parameters of the scene object, a spatial reconstruction discriminant network is used to generate a discrimination result for the authenticity of the hyperspectral image of the scene object; this step further includes: stacking two 3×3 convolutional layers and using the LeakyReLU activation function; passing the first instruction twice, the first instruction including: a convolutional kernel of size 4×4 and stride 2, stacking 3×3 convolutional layers and using the LeakyReLU activation function; using a convolutional kernel of size 4×4 and stride 2 and using the LeakyReLU activation function; obtaining spectral curve data and authenticity judgment results through stacking 4×4 convolutional layers and using the Sigmoid activation function, the authenticity judgment result is in the range of [0,1] probability.

[0022] S130: Hyperspectral images based on scene objects are used to obtain spectral curve datasets through global random sampling; S140: Based on the spectral curve dataset, obtain the true / false discrimination results of the spectral features of the hyperspectral image of the scene object through a spectral fidelity discrimination network; this step includes: obtaining softmax layer data by sequentially passing a multilayer perceptron algorithm and four linear transformations based on the spectral curve dataset. The structure of the multilayer perceptron algorithm is 150-128-256-128-2-1; obtaining the true / false discrimination results of the spectral features of the hyperspectral image of the scene object by extracting the first value based on the softmax layer data. The range of the true / false discrimination results is [0,1] and the magnitude of the value is positively correlated with the authenticity.

[0023] S150: Construct the loss function for network training and update network parameters. This step includes: constructing the network training loss function based on pixel-level fidelity loss, spectral curve fidelity loss, spatial structure fidelity loss, and adversarial discrimination loss. The calculation expression is as follows: In the formula The loss function for network training, For pixel-level fidelity loss, To preserve the fidelity of the spectral curve, To preserve the fidelity of the spatial structure, Adversarial loss judgment , , These are the weights for loss of spectral curve fidelity, loss of spatial structure fidelity, and loss of adversarial discrimination, respectively.

[0024] See Figure 3 For example, by executing the above method: S100: Input an RGB base map with a resolution of 2048*2048, latitude and longitude data (longitude 120.28, latitude 22.55), band range (range 0.4~0.7μm), spectral resolution data of 2.5nm, and spatial resolution data (25m) to acquire hyperspectral scene data. Hyperspectral scene data includes marine hyperspectral data. See [link / reference]. Figure 3 (a) is the hyperspectral scene data for display; S200: Magnified result after dynamic target embedding. The input target object is an airplane. The airplane's basic data is: altitude 2000m, direction of travel 0°, speed Mach 1.5, and acceleration 0. Gaussian noise is added to the direction of travel, and the airplane's speed determines its inter-frame movement pixel distance, obtaining the airplane's inter-frame trajectory. The airplane's initial position is used as frame 0 to replace the corresponding position in the original ocean hyperspectral scene, and background interpolation is used for pixel filling. Based on the generated airplane trajectory, the target is dynamically embedded into the original ocean hyperspectral scene. See [link to documentation]. Figure 3 (b) Display the embedded data of the target object (in this case, an aircraft integrated into a sea scene); see [link / reference] Figure 3 (c) Embed the spectral information of the target, compare the mixed pixel spectrum with the target spectrum, and combine the shape of the spectral curve to determine that it is an aircraft target of the same type.

[0025] S300: Evaluation data for hyperspectral images of scene objects: Performance evaluation is performed using hyperspectral images of the embedded target object (in this case, an aircraft). Figure 3 (d) shows the evaluation data results, specifically: Euclidean distance of 51.57, relative quadratic error of 3.32e-09, spectral angle of 0.01, and peak signal-to-noise ratio of 67.56.

[0026] Preferably, the step of generating a hyperspectral image of the scene object and updating the network parameters by using a spatiotemporal joint generative adversarial network based on the input RGB base image of the scene object and the associated parameters of the scene object further includes: Hyperspectral images of scene objects are generated using a generative adversarial network that combines spatial and spectral data, based on the RGB base map of the scene objects and the associated parameters of the scene objects. Based on the hyperspectral image of the scene object, the RGB base map of the scene object, and the correlation parameters of the scene object, a spatial reconstruction discriminant network is used to obtain the discrimination result of the hyperspectral image of the scene object to determine the authenticity of the scene object; Hyperspectral images based on scene objects are used to obtain spectral curve datasets through global random sampling. Based on the spectral curve dataset, a spectral fidelity discrimination network is used to obtain the true and false discrimination results of the spectral features of the hyperspectral images of scene objects. Construct a loss function for network training and update network parameters.

[0027] By combining spatial and spectral generation with a dual discrimination mechanism, the spatial-spectral consistency, physical authenticity, and generation efficiency of hyperspectral data are improved, which has significant application potential in remote sensing, visual computing, and other fields.

[0028] Preferably, the step of generating a hyperspectral image of the scene object based on the RGB base map of the scene object and the associated parameters of the scene object through a spatiotemporal joint generative adversarial network further includes: The feature data of the scene object is obtained by using the RGB image of the scene object and the associated parameters of the scene object through a spatial information encoder. The spatial information encoder has 8 consecutive 4×4 convolution kernels with a stride of 2, and the activation function of each layer is LeakyReLU. The hyperspectral image of the scene object is obtained by using the feature data of the scene object through the image information decoder. The image information decoder has a 4×4 deconvolution kernel with a stride of 2. The dropout of the first three layers of the image information decoder is set to 0.5 and the activation function of the output layer is Tanh.

[0029] The encoder and decoder are symmetrically designed to ensure spatial alignment between the generated hyperspectral data and the input RGB base image, avoiding geometric distortions (such as blurring and ghosting). The hyperspectral data output by the decoder is obtained by randomly sampling spectral curves, covering diverse land cover types and lighting conditions, ensuring inter-band correlation and matching degree of the material spectral library. The Tanh activation function limits the range of pixel values ​​to avoid spectral energy overflow and improve resistance to false spectra. Dropout (0.5) in the first three layers of the decoder randomly discards neurons, enhancing the model's generalization ability to unknown scenes and reducing over-reliance on training data. The LeakyReLU activation function alleviates the gradient vanishing problem in the encoder, improving the training stability of deep networks. Through joint spatial-spectral generation and dual discrimination mechanism, the spatial-spectral consistency, physical realism, and generation efficiency of hyperspectral images of scene objects are improved.

[0030] Preferably, the step of obtaining the discrimination result of the hyperspectral image of the scene object through a spatial reconstruction discriminant network based on the hyperspectral image of the scene object, the RGB base image of the scene object, and the correlation parameters of the scene object further includes: By stacking two 3×3 convolutional layers and using the LeakyReLU activation function; The first instruction is passed twice. The first instruction includes: a 4×4 convolutional kernel with a stride of 2, a stack of 3×3 convolutional layers, and the activation function LeakyReLU. The convolution kernel is 4×4 with a stride of 2 and the activation function is LeakyReLU. Spectral curve data and authenticity assessment results are obtained by stacking 4×4 convolutional layers and using the Sigmoid activation function. The authenticity assessment results are in the range of [0,1] probability.

[0031] This discriminative network achieves efficient evaluation of the spatial-spectral consistency of hyperspectral images of scene objects by combining multi-level convolution and downsampling operations with LeakyReLU nonlinear modeling and Sigmoid probability output. Its effectiveness is reflected in: accurate capture of multi-scale spatial features, implicit fusion of spectral information, output of high-confidence authenticity probabilities, and good computational efficiency and generalization ability, making it suitable for tasks involving the identification of real and fake data in fields such as remote sensing and computer vision.

[0032] Preferably, the step of obtaining the spectral curve dataset from the hyperspectral image of the scene object through global random sampling further includes: Hyperspectral images based on scene objects are divided into several image patch data by uniformly dividing the width dimension; The location information of a suitable number of image blocks is obtained through a random algorithm based on several image block data. The location information of the image patch data is used to extract the spectral curve dataset, which includes spectral data of at least 150 bands.

[0033] This processing flow achieves spatial localization analysis through uniform block division, ensures data diversity through random sampling, and retains key information through multi-band spectral extraction, ultimately achieving a balance between computational efficiency, feature richness, and model performance.

[0034] Preferably, the step of obtaining the true / false discrimination result of the spectral features of the hyperspectral image of the scene object based on the spectral curve dataset through a spectral fidelity discrimination network further includes: Based on the spectral curve dataset, the softmax layer data is obtained sequentially through a multilayer perceptron algorithm and four linear transformations. The structure of the multilayer perceptron algorithm is 150-128-256-128-2-1. Based on the softmax layer data, the first value is extracted to obtain the true or false discrimination result of the spectral features of the hyperspectral image of the scene object. The range of the true or false discrimination result is [0,1] and the value is positively correlated with the true or false status.

[0035] This scheme achieves complex modeling of spectral features through deep MLP, and combines the output of Softmax to determine the probability of authenticity, which can effectively distinguish between real and generated hyperspectral data. The output results have both interpretability and fine-grained scoring characteristics.

[0036] Preferably, the step of constructing the loss function for network training and updating network parameters further includes: The network training loss function is constructed based on pixel-level fidelity loss, spectral curve fidelity loss, spatial structure fidelity loss, and adversarial discrimination loss, and its calculation expression is as follows: In the formula The loss function for network training, For pixel-level fidelity loss, To preserve the fidelity of the spectral curve, To preserve the fidelity of the spatial structure, To counteract the loss of judgment, , , These are the weights for loss of spectral curve fidelity, loss of spatial structure fidelity, and loss of adversarial discrimination, respectively.

[0037] Optionally, pixel-level fidelity loss is obtained based on real image data, image data at a first resolution, and image data at a second resolution. The calculation expression is as follows: In the formula, S represents the actual image data, S r To obtain second-resolution image data based on real image data and first-resolution image data, where Ds represents the data in the training dataset, I represents the pixel spectrum, and D... I Pixel spectral data in the training dataset, I l S represents the pixel spectral data of a real image. l Spectral data of real image data at point l, S rl The second resolution image data is the spectral data at point l, and ω represents the weight of different bands. Generally, a vector consisting entirely of 1s is chosen, meaning that the weight of each band is equal. This embodiment does not impose any restrictions.

[0038] Optionally, the expression for calculating the fidelity loss of the spectral curve is: In the formula S l为 Pixel spectral data at position l, S rl The image data at the second resolution is the spectral data at position l. The spectral curve fidelity loss is mainly used to constrain the similarity of shapes between spectral curves, and the cosine similarity between the synthesized spectral vector and the true spectrum is selected for calculation.

[0039] Optionally, the expression for calculating the spatial structure fidelity loss is: ; Structural similarity (SSIM) measures the degree of similarity between real image data and hyperspectral images of scene objects in terms of brightness, contrast, and structure. It constructs a sliding window using a Gaussian function and calculates the mean, variance, and covariance of each window using weighted averages.

[0040] Optionally, the expression for calculating the adversarial discriminative loss is as follows: In the formula , The spatial fidelity discrimination loss and the spectral fidelity discrimination loss can be obtained respectively through the following calculations: .

[0041] The adversarial discriminant loss is mainly responsible for balancing the generator and the discriminator, making them compete with each other. It mainly includes two components: spectral fidelity discriminant loss and spatial fidelity discriminant loss.

[0042] Preferably, the step of obtaining dynamic target embedding data based on the spectral information and motion trajectory of the target object further includes: Intermediate data is obtained by adding Gaussian noise to the direction of travel, height, speed, and acceleration of the target object. The motion trajectory of the target object is obtained by incorporating the pixel distance of the target motion based on intermediate data; Dynamic target embedding data is obtained based on the spectral information and motion trajectory of the target object.

[0043] By employing multi-dimensional noise modeling, physical coordinate transformation, filtering and smoothing, and adaptive fusion, the authenticity and robustness of trajectory data can be significantly improved.

[0044] Second Embodiment See Figure 4 and Figure 5 A second aspect of the present invention provides a hyperspectral scene generation system based on dynamic target embedding, employing any one of the above-mentioned hyperspectral scene generation methods based on dynamic target embedding, comprising: The scene generation module is used to generate a hyperspectral image of the scene object based on the RGB base map of the scene object and the associated parameters of the scene object through a generative adversarial network that combines spatial and spectral data. The target embedding module is used to obtain a hyperspectral image containing the target object by incorporating the dynamic target embedding data into the hyperspectral image of the scene object; The performance evaluation module is used to calculate the evaluation data of the scene object based on the hyperspectral image containing the target object and the ground truth of the hyperspectral image containing the target object, respectively, by Euclidean distance, relative quadratic error, spectral angle and peak signal-to-noise ratio.

[0045] The network architecture of the scene generation module is a generative adversarial network module composed of a U-Net network, further including a generation network module, a spatial reconstruction discriminant network module, and a spectral fidelity discriminant network module. The generation network module is used to obtain the hyperspectral image of the scene object based on the RGB base map data of the scene object. The spatial reconstruction discriminant network module is used to obtain the authenticity discrimination result of the hyperspectral image of the scene object based on the hyperspectral image of the scene object, the RGB base map of the scene object, and the correlation parameters of the scene object. The spectral fidelity discriminant network module is used to obtain the authenticity discrimination result of the spectral features of the hyperspectral image of the scene object based on the hyperspectral image of the scene object.

[0046] Optionally, the hyperspectral scene generation system also includes a display and control interaction module and a database module. The display and control interaction module is used to receive user input data, control scene generation, target embedding, and output information such as performance index data of the hyperspectral images of the target object and the scene object, as well as display visualization content. Specifically, the process is as follows: based on the input data, the scene generation module obtains the hyperspectral image of the scene object through parsing and selection; the hyperspectral image of the target object after embedding is obtained through input commands; and the performance index data of the hyperspectral image of the scene object is obtained through input commands. The database module provides data for the scene generation module, target embedding module, and performance evaluation module; the data includes structured data and files.

[0047] See Figure 5 Optionally, the display and control interaction module includes: a main application unit, a scene unit, a scene application unit, and a display unit. The main application unit is used to implement the display of the interface interaction area, as well as initialization, data display, status display, scene generation, prototype imaging, image display, data storage settings, and real-time status display functions. The scene unit is used for the parameter configuration of the scene generation module, displaying the interface, scene images, and status. The scene application unit is used for displaying the imaging parameter configuration of the hyperspectral image of the scene object and the status display of the hyperspectral image of the scene object. The display unit is used for converting and displaying data from the scene unit and the scene application unit.

[0048] See Figure 2 Preferably, the scene generation module includes a generation network module, a spatial reconstruction discrimination network module, and a spectral fidelity discrimination network module. The generation network module is used to obtain the hyperspectral image of the scene object based on the RGB base map of the scene object and the association parameters of the scene object. The spatial reconstruction discrimination network module is used to obtain the authenticity discrimination result of the hyperspectral image of the scene object based on the hyperspectral image of the scene object, the RGB base map of the scene object, and the association parameters of the scene object. The spectral fidelity discrimination network module is used to obtain the authenticity discrimination result of the spectral features of the hyperspectral image of the scene object based on the spectral curve dataset.

[0049] This module achieves mode conversion from RGB to hyperspectral from RGB base maps through collaborative optimization of the generative and discriminative models, striking a balance between spatial realism and spectral physicality, and providing an efficient solution for the scarcity of hyperspectral data.

[0050] In the description of this application, it should be noted that the terms "inner" and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product is in use. They are used only for the convenience of describing this application and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0051] It should also be noted that, unless otherwise explicitly specified and limited, the terms "setup" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0052] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific identification content executed by the system and device described above can be referred to the corresponding process in the foregoing method embodiments.

[0053] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they shall still fall within the protection scope of the present invention.

Claims

1. A hyperspectral scene generation method based on dynamic target embedding, characterized in that, include: The input scene object's RGB base image and the scene object's associated parameters are used to generate a hyperspectral image of the scene object through a spatial-spectral joint generative adversarial network and update the network parameters. The associated parameters of the scene object include at least latitude and longitude coordinates, band range, spectral resolution, spatial resolution, and environmental factors. Dynamic target embedding data is obtained based on the spectral information and motion trajectory of the target object; a hyperspectral image containing the target object is obtained based on the hyperspectral image of the scene object and the dynamic target embedding data. Evaluation data of the scene object is obtained through performance evaluation based on the hyperspectral image containing the target object and the ground truth of the hyperspectral image containing the target object. The evaluation data includes at least Euclidean distance, relative quadratic error, spectral angle and peak signal-to-noise ratio.

2. The hyperspectral scene generation method based on dynamic target embedding according to claim 1, characterized in that, The steps of generating a hyperspectral image of the scene object from an input RGB base image and its associated parameters via a spatial-spectral joint generative adversarial network, and then updating the network parameters, further include: Based on the RGB base map of the scene object and the associated parameters of the scene object, a hyperspectral image of the scene object is generated through a spatiotemporal joint generative adversarial network. Based on the hyperspectral image of the scene object, the RGB base image of the scene object, and the correlation parameters of the scene object, a spatial reconstruction discriminant network is used to obtain the discrimination result of the authenticity of the hyperspectral image of the scene object; A spectral curve dataset is obtained from the hyperspectral image of the scene object through global random sampling. Based on the aforementioned spectral curve dataset, a spectral fidelity discrimination network is used to obtain the true or false discrimination results of the spectral features of the hyperspectral image of the scene object; Construct a loss function for network training and update network parameters.

3. The hyperspectral scene generation method based on dynamic target embedding according to claim 2, characterized in that, The step of generating a hyperspectral image of the scene object based on the RGB base image and the associated parameters of the scene object through a spatiotemporal joint generative adversarial network further includes: Based on the RGB image of the scene object and the associated parameters of the scene object, the feature data of the scene object is obtained through a spatial information encoder. The spatial information encoder has 8 consecutive 4×4 convolution kernels with a stride of 2, and the activation function of each layer is LeakyReLU. Based on the feature data of the scene object, a hyperspectral image of the scene object is obtained through an image information decoder. The image information decoder has a 4×4 deconvolution kernel with a stride of 2. The dropout of the first three layers of the image information decoder is set to 0.5 and the activation function of the output layer is Tanh.

4. The hyperspectral scene generation method based on dynamic target embedding according to claim 2, characterized in that, The step of obtaining the authenticity judgment result of the hyperspectral image of the scene object through a spatial reconstruction discriminant network based on the hyperspectral image of the scene object, the RGB base image of the scene object, and the correlation parameters of the scene object further includes: By stacking two 3×3 convolutional layers and using the LeakyReLU activation function; The first instruction is passed twice. The first instruction includes: a 4×4 convolutional kernel with a stride of 2, a 3×3 convolutional layer stack, and the activation function LeakyReLU. The convolution kernel is 4×4 with a stride of 2 and the activation function is LeakyReLU. Spectral curve data and authenticity evaluation results are obtained by stacking 4×4 convolutional layers and using the Sigmoid activation function. The authenticity evaluation results are in the range of [0,1] probability.

5. The hyperspectral scene generation method based on dynamic target embedding according to claim 2, characterized in that, The step of obtaining a spectral curve dataset based on the hyperspectral image of the scene object through global random sampling further includes: The hyperspectral image of the scene object is divided into several image block data by uniformly dividing the width dimension; Based on several image block data, a random algorithm is used to obtain the location information of an appropriate number of image block data. Based on the location information of the image patch data, a spectral curve dataset is obtained by extraction, and the spectral curve dataset includes spectral data of at least 150 bands.

6. The hyperspectral scene generation method based on dynamic target embedding according to claim 2, characterized in that, The step of obtaining the true / false discrimination result of the spectral features of the hyperspectral image of the scene object through the spectral curve dataset using a spectral fidelity discrimination network further includes: Based on the spectral curve dataset, softmax layer data is obtained sequentially through a multilayer perceptron algorithm and four linear transformations. The structure of the multilayer perceptron algorithm is 150-128-256-128-2-1. Based on the softmax layer data, the first value is extracted to obtain the true or false discrimination result of the spectral features of the hyperspectral image of the scene object. The range of the true or false discrimination result is [0,1] and the value is positively correlated with the true or false nature.

7. The hyperspectral scene generation method based on dynamic target embedding according to claim 2, characterized in that, The steps of constructing the loss function for network training and updating network parameters further include: The network training loss function is constructed based on pixel-level fidelity loss, spectral curve fidelity loss, spatial structure fidelity loss, and adversarial discrimination loss, and its calculation expression is as follows: In the formula Let be the training loss function of the network. For the pixel-level fidelity loss, For the loss of fidelity of the spectral curve, For the loss of fidelity of the spatial structure, For the adversarial discriminant loss, , , These are the weights for loss of spectral curve fidelity, loss of spatial structure fidelity, and loss of adversarial discrimination, respectively.

8. The hyperspectral scene generation method based on dynamic target embedding according to claim 1, characterized in that, The step of obtaining dynamic target embedding data based on the spectral information and motion trajectory of the target object further includes: Intermediate data is obtained by adding Gaussian noise along the dimension of the travel direction based on the target object's travel direction, height, speed, and acceleration. Based on the intermediate data, the motion trajectory of the target object is obtained by incorporating the pixel distance of the target motion; Dynamic target embedding data is obtained based on the spectral information and motion trajectory of the target object.

9. A hyperspectral scene generation system based on dynamic target embedding, employing the hyperspectral scene generation method based on dynamic target embedding according to any one of claims 1-8, characterized in that, include: The scene generation module is used to generate a hyperspectral image of the scene object based on the RGB base map of the scene object and the associated parameters of the scene object through a spatiotemporal joint generative adversarial network. The target embedding module is used to obtain a hyperspectral image containing the target object by incorporating the dynamic target embedding data into the hyperspectral image of the scene object; The performance evaluation module is used to calculate the evaluation data of the scene object based on the hyperspectral image containing the target object and the ground truth of the hyperspectral image containing the target object, respectively, by Euclidean distance, relative quadratic error, spectral angle and peak signal-to-noise ratio.

10. The hyperspectral scene generation system based on dynamic target embedding according to claim 9, characterized in that, The scene generation module includes a generation network module, a spatial reconstruction discrimination network module, and a spectral fidelity discrimination network module. The generation network module is used to obtain the hyperspectral image of the scene object based on the RGB base map of the scene object and the association parameters of the scene object. The spatial reconstruction discrimination network module is used to obtain the authenticity discrimination result of the hyperspectral image of the scene object based on the hyperspectral image of the scene object, the RGB base map of the scene object, and the association parameters of the scene object. The spectral fidelity discrimination network module is used to obtain the authenticity discrimination result of the spectral features of the hyperspectral image of the scene object based on the spectral curve dataset.

Images