Ship hyperspectral image generation method, terminal device and storage medium

Through spectral generation adversarial network and mask-guided spectral converter based on content consistency constraints, the problem of hyperspectral image generation in complex sea conditions is solved, and high-precision and low-cost image conversion is achieved, suitable for ships under complex sea conditions Ship monitoring.

CN118822876BActive Publication Date: 2024-11-26CENT SOUTH UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411297303.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-18
Publication Date
2024-11-26
Estimated Expiration
2044-09-18

AI Technical Summary

Technical Problem

The prior art is difficult to quickly generate hyperspectral images under complex sea conditions, and traditional image enhancement technology cannot process spectral information. Generating adversarial networks requires a large amount of labeled data and have limited generalization capabilities, making it difficult to adapt to different marine environments and ship types.

Method used

Using spectral generation adversarial network and mask-guided spectral transformer based on content consistency constraints, generate hyperspectral images by preprocessing infrared images, extract and convert spectral features using multi-layer perceptrons and spectrum attention mechanisms to reduce hyperspectral imaging Hardware dependence improves the realism and resolution of images.

Benefits of technology

It realizes high-precision conversion from infrared images to hyperspectral images, reduces the cost and time of generating images, improves the quality and efficiency of images, enhances the adaptability to complex sea conditions and different marine environments, and is suitable for real-time monitoring tasks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118822876B_ABST
    Figure CN118822876B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for generating a hyperspectral image of a ship, a terminal device and a storage medium. The spectral generative adversarial network based on content consistency constraints ensures a high visual consistency between the visible light image and the original infrared image, while enhancing the realism of the image. In the feature extraction and transformation stage, the embedding layer and the spectral attention mechanism are used to accurately extract key features, and the convolution layer is used to perform effective feature transformation, which provides rich information for generating hyperspectral images. Finally, the mask-guided spectral transformer accurately converts the extracted spectral features into a hyperspectral image, ensuring the high spectral resolution and quality of the generated image. The present invention improves the quality and efficiency of image processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for generating a ship hyperspectral image, a terminal device and a storage medium. Background Art

[0002] In the field of ocean monitoring, there is an increasing demand for real-time and accurate monitoring of ships under complex sea conditions. The performance of visible light images in complex sea conditions is limited, and although infrared images can reflect the temperature information of the target, they are limited in detail resolution. Hyperspectral images have irreplaceable advantages for ship detection and identification because they can reveal detailed material information of the target. However, due to the cumbersome and costly data acquisition process of hyperspectral imaging technology, there are obvious limitations in its direct application to real-time monitoring. In addition, complex sea conditions such as sea surface reflections, waves, fog, etc., pose higher challenges to image acquisition and processing. Therefore, developing a technology that can quickly generate hyperspectral images under complex sea conditions is crucial to improving the accuracy and efficiency of intelligence acquisition.

[0003] Although there are many image generation and conversion technologies, there are still obvious deficiencies in the existing solutions for the generation of hyperspectral images of ships in complex sea conditions. Traditional image enhancement techniques mainly focus on improving the contrast and brightness of images, which helps to improve the visual effect of images. However, these technologies usually do not involve the processing and analysis of spectral information and cannot provide detailed information about the target material and chemical composition. In complex sea conditions, the spectral characteristics of ships may change due to various factors (such as reflection on the water surface, fog, etc.), and traditional technologies are difficult to adapt to these changes. Multispectral and hyperspectral imaging systems can provide rich spectral data, which is conducive to detailed target analysis and identification. However, these systems are often expensive and require professional hardware support, which is not conducive to large-scale deployment. The data acquisition process may be slow and is not suitable for real-time monitoring tasks that require fast response. Generative adversarial networks have shown great potential in image conversion tasks and can generate realistic images. However, in terms of generating hyperspectral images, generative adversarial networks still face the problem of large training data requirements, and a large amount of labeled data is required to train the model. The generalization ability of the model is limited, and it may be difficult to adapt to different marine environments and ship types. In addition, since the spectral characteristics of ships are closely related to their physical and environmental conditions, this places higher requirements on the adaptability and accuracy of the image generation algorithm. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method for generating a hyperspectral image of a ship, a terminal device and a storage medium to achieve high-precision conversion from infrared images to hyperspectral images in view of the deficiencies in the prior art.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for generating a hyperspectral image of a ship, comprising the following steps:

[0006] S1. Preprocessing infrared images of ocean scenes;

[0007] S2, inputting the preprocessed infrared image into a spectrum generative adversarial network based on content consistency constraints to generate an intermediate state image, that is, obtaining a visible light image;

[0008] S3, inputting the visible light image into an embedding layer of a mask-guided spectral transformer based on a spectral attention mechanism, that is, embedding the visible light image features into an initial feature space of the mask-guided spectral transformer based on a spectral attention mechanism, to obtain preliminary spectral features;

[0009] S4, using the preliminary spectral features as input of a mask-guided spectral transformer based on a spectral attention mechanism, and processing the output of the spectral transformer to obtain a generated image;

[0010] The loss function of the spectral generative adversarial network based on content consistency constraint is expressed as:

[0011] ;

[0012] in, , and They are the generator loss function, discriminator loss function and perceptual loss function of the GAN network. is the content consistency constraint loss, , and They are the external correction loss trade-off parameter and the internal correction loss trade-off parameter, , , is the temperature scale parameter, Indicates the synthetic frame in the multilayer perceptron l The output features of the layer spatial position s, represents the corresponding positive feature, represents other negative features, L represents the number of layers of the multilayer perceptron, and S represents the l The position of the multilayer perceptron, N represents the number of negative samples, X is the preprocessed infrared image as input, represents the infrared image after continuous input preprocessing, represents the continuously generated intermediate state images, express In the multi-layer perceptron l The difference in layers, express In the multi-layer perceptron l The difference in layers, express In the multi-layer perceptron l The difference in layers, express In the multi-layer perceptron l Difference in layers; each layer of the GAN network is followed by a multi-layer perceptron.

[0013] In the present invention, in the implementation details of the SpectralGAN network, a multilayer perceptron is used to embed the features extracted from each layer of the encoder. After the features extracted at each level of the GAN encoder, a two-layer multilayer perceptron is applied, with 256 units in each layer. Such a multilayer perceptron is used to embed the features extracted from each layer of the encoder, and fuse and transform the high-level features extracted from the encoder to better capture the semantic information of the input frame.

[0014] The present invention ensures a high degree of visual consistency between visible light images and original infrared images through a spectral generative adversarial network based on content consistency constraints, while enhancing the realism of the image. In the feature extraction and transformation stage, the embedding layer and the spectral attention mechanism are used to accurately extract key features, and the convolution layer is used to perform effective feature transformation, providing rich information for generating hyperspectral images. Finally, the mask-guided spectral transformer accurately converts the extracted spectral features into hyperspectral images, ensuring the high spectral resolution and quality of the generated images. The present invention improves the quality and efficiency of image processing.

[0015] In step S4, the spectrum attention mechanism SA is expressed as: ;in, is the intermediate state image used as input to the self-attention mechanism, and are learnable parameters, H represents the image height, W represents the image width, C represents the number of color channels, Q represents the query, K represents the key, and V represents the value. The spectral attention mechanism can intelligently identify the key spectral features in the image, significantly enhancing the expressiveness of the features. It not only improves the details of the image, making the generated hyperspectral image more visually realistic, but also improves the model's ability to recognize and classify targets by focusing on specific spectral regions of the image.

[0016] In step S4, the mask-guided spectral transformer performs multiple downsampling and upsampling on the spectral features to obtain multi-scale fusion features, and the multi-scale fusion features are input into the mapping layer to obtain a generated image. The mask-guided spectral transformer includes multiple serially connected attention blocks, the input side of the first attention block is connected to the embedding layer; for the first to i-th attention blocks, two adjacent attention blocks are connected through the downsampling layer; for the i-th to M-th attention blocks, two adjacent attention blocks are connected through the upsampling layer and the convolution layer in sequence; the output of the first attention block is channel-connected to the output of the last upsampling layer, the second attention block is channel-connected to the output of the K-1-th upsampling layer, and so on; the last attention block is connected to the mapping layer, and the output of the mapping layer is spliced ​​with the input of the embedding layer; wherein M is the number of attention blocks, 1<i≤M, and K is the number of upsampling layers. The downsampling stage effectively extracts high-level features of the image and captures the global information of the scene, while the upsampling stage focuses on restoring the details of the image so that high-frequency information is retained. This multi-scale processing strategy not only enhances the model's ability to express spectral features, but also improves the clarity and realism of the generated images by fusing image information of different resolutions. In addition, multi-scale feature fusion also enhances the generalization ability of the model, enabling it to better adapt to various image contents and styles, while optimizing computational efficiency and avoiding waste of resources.

[0017] The spectral generative adversarial network based on content consistency constraint includes a generator, a multi-layer perceptron and a discriminator connected in sequence. The generator uses the input noise vector to learn the data distribution and create an image with rich details and spectral information. The multi-layer perceptron further accurately characterizes these image features and extracts deep information, while the content consistency constraint ensures that the generated image is highly consistent with the original infrared image visually, enhancing the realism of the image.

[0018] The generator includes a plurality of serially connected convolutional layers, residual blocks, upsampling layers and output convolutional layers; the discriminator includes a plurality of serially connected convolutional layers and fully connected layers.

[0019] The residual block includes a first convolutional layer, a first normalization layer, an activation function, a second convolutional layer, and a second normalization layer connected in sequence; the output of the second normalization layer is spliced ​​with the input of the first convolutional layer and used as the input of the next residual block, and the output of the last residual block is spliced ​​with the input of the first convolutional layer of the residual block and used as the input of the upsampling layer. Through the jump connection between the convolutional layers, the network is allowed to learn the residual between the input and output, which effectively solves the gradient vanishing problem in the training of deep networks and promotes the learning and utilization of deep features. The normalization layer follows each convolutional layer and accelerates the convergence of the network and improves the generalization ability of the model by normalizing the features. The activation function introduces nonlinearity to the network and enhances the model's ability to capture complex data patterns. The upsampling layer is used to restore the resolution of the image while retaining important high-frequency details to ensure that the generated image is both clear and realistic.

[0020] The mask-guided spectral transformer is composed of a series of attention blocks and mapping layers. The attention blocks are the core of this structure. They adjust the feature channels through convolutional layers and use the attention mechanism to identify and enhance the key information in the image. Each attention block contains two normalization layers, a spectral attention and a feedforward network, which work together on the feature map to extract richer and more discriminative feature representations. With this design, the network can capture the details and global information of the image at different scales. The upsampling layer and the downsampling layer play a crucial role in the transformer. The upsampling layer increases the resolution of the feature map through interpolation technology and gradually restores the detailed information of the image, while the downsampling layer is responsible for reducing the spatial dimension of the feature map while increasing the level of abstraction of the features, which helps the network learn more advanced feature representations. Finally, the features processed by multiple layers are finally adjusted through the output mapping layer.

[0021] In the present invention, the specific implementation process of processing the output of the spectrum converter includes: performing a convolution operation on the output of the spectrum converter, adding the output result of the convolution operation to the preliminary spectrum feature to obtain a generated image.

[0022] As an inventive concept, the present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the steps of the above method.

[0023] As an inventive concept, the present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon; the computer program / instruction implements the steps of the above method when executed by a processor.

[0024] As an inventive concept, the present invention also provides a computer program product, including a computer program / instruction; when the computer program / instruction is executed by a processor, the steps of the above method are implemented.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention uses infrared images as input, which reduces the technical dependence on hyperspectral imaging hardware, thereby reducing the overall cost. The use of spectral generative adversarial networks and spectral converter technology can not only meet the real-time requirements, but also reduce the time required for hyperspectral imaging. At the same time, content consistency constraints and spectrum attention mechanisms make the generated images more realistic and further improve the image quality. Special consideration is given to the impact of complex marine environments on the image generation process. Through multi-feature and multi-level analysis, the adaptability of the algorithm to different sea conditions is enhanced, while ensuring the high spectral resolution and quality of the generated images. The present invention can not only improve the monitoring and identification capabilities of ships, but also be extended to the monitoring of other marine targets, such as civilian ships, marine life, oil pollution on the sea surface, etc., and has a wide range of civilian value. In addition, the present invention has a high degree of automation, can achieve fast and accurate image generation, is suitable for real-time monitoring and automated analysis, and is expected to significantly improve the technical level of marine monitoring. The method of the present invention is applied to images with ocean as background scenes in the public datasets VAIS and NUDT-SIRST and target detection datasets of ships in the distant sea (10-12 km). Experiments show that the method of the present invention is suitable for generating hyperspectral images of ships under complex sea conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of a method according to an embodiment of the present invention;

[0027] Figure 2 A hyperspectral image generation framework for an embodiment of the present invention;

[0028] Figure 3 A framework for generating a spectral generative adversarial network based on content consistency constraints according to an embodiment of the present invention;

[0029] Figure 4 A generator of a spectrum generative adversarial network based on content consistency constraints according to an embodiment of the present invention;

[0030] Figure 5 A discriminator of a spectral generative adversarial network based on content consistency constraints according to an embodiment of the present invention;

[0031] Figure 6 This is a structural diagram of a mask-guided spectrum converter according to an embodiment of the present invention;

[0032] Figure 71 is an example of an original infrared image, a visible light image, and a generated hyperspectral image (with wavelengths of 480 nm, 520 nm, 580 nm, and 660 nm, respectively) according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] Example 1

[0035] like Figure 1 and Figure 2 As shown, the embodiment of the present invention proposes a method for generating a ship hyperspectral image, aiming to achieve efficient and accurate conversion from infrared images to hyperspectral images under complex sea conditions. Figure 2 As shown in Figure 1, the hyperspectral image generation framework is divided into two parts: the first part is the infrared image intermediate state correction mechanism (using 30 infrared images of ocean scenes from the public datasets VAIS and NUDT-SIRST and 300 short-wave infrared images and long-wave infrared images of ships in the open sea (10-12km) as the dataset. In the training stage, 1088 paired infrared images and 1088 visible light images from the public dataset VAIS were used as the training set, and the training set was input into the spectral generation adversarial network (SpectralGAN, see Figure 3 , Figure 4 and Figure 5 ); in the testing phase, 154 infrared images without paired visible light images in VAIS, 30 infrared images of the ocean in NUDT-SIRST, and 300 infrared images of the target detection dataset of distant sea (10-12km) ships are used as the test set for testing). The second part is the generation of hyperspectral images.

[0036] In the infrared image conversion stage, a spectral generative adversarial network (SpectralGAN) for infrared image alignment is proposed, which can capture the complex relationship between image modalities. In order to generate higher resolution and clearer visible light image output, by introducing content consistency loss as a correction mechanism for image generation, the content consistency of the generated image can be enhanced to make it more realistic. The content consistency loss is divided into two parts: external correction loss and internal correction loss.

[0037] The external correction loss is defined as:

[0038] ;

[0039] in, is the temperature scale parameter, Output features representing the layer spatial position of the synthesized frame in the multilayer perceptron. represents the corresponding positive feature, Represents other negative features. L represents the number of layers of the multilayer perceptron, S represents the position of the multilayer perceptron in this layer, N represents the number of negative samples, and X is the preprocessed infrared image as input.

[0040] The internal correction loss is defined as:

[0041]

[0042] in, represents continuous input images, represents the continuously generated images, express In the multi-layer perceptron l Layer differences.

[0043] So the content consistency loss is defined as:

[0044] ;

[0045] in, and are the external and internal correction loss trade-off parameters, respectively.

[0046] The final objective function is as follows:

[0047]

[0048] in, and They are the generator loss function, discriminator loss function and perceptual loss function of GAN respectively.

[0049] In the hyperspectral image generation stage, this embodiment proposes a spectrum converter technology. Figure 6 As shown in Figure 1, the mask-guided spectral transformer is a symmetric U-Net whose basic building block is the mask-guided attention block. The attention block contains two normalization layers, a spectral attention and a feed-forward network. The spectral attention can be decomposed into two parts, namely the mask-guided mechanism and the spectral attention mechanism.

[0050] The Spectral Attention mechanism is as follows:

[0051] ;

[0052] in, As the input of the self-attention mechanism, represents the transposed matrix of K, are learnable parameters.

[0053] This hyperspectral transformation mechanism not only ensures real-time processing capabilities, but also enhances the robustness against noise and interference in complex sea conditions, ensuring stable operation and high reliability in dynamic marine environments.

[0054] The image conversion technology proposed in the embodiment of the present invention realizes efficient and accurate conversion from infrared images to hyperspectral images by combining content consistency loss and spectrum attention mechanism, and significantly improves the generation quality of ship images under complex sea conditions. Figure 1 The method of the embodiment of the present invention comprises the following steps:

[0055] Step 1: Preprocess the infrared image;

[0056] Step 2: Input the preprocessed infrared image into the spectral generative adversarial network based on content consistency constraints (see Figure 3 ), generate an intermediate state image, that is, obtain a visible light image;

[0057] Step 3: Input the visible light image into the embedding layer, embed the visible light image features into the initial feature space of the network, and obtain preliminary spectral features;

[0058] Step 4: Use the preliminary spectral features as a mask-guided spectral transformer based on the spectral attention mechanism (see Figure 6 ) as input and processes the output of the spectral transformer to obtain the generated image.

[0059] The embodiment of the present invention only requires infrared images, reducing the reliance on expensive hyperspectral imaging hardware, thereby reducing the overall cost. The generated hyperspectral images surpass traditional methods in detail resolution and spectral information, provide richer target information, and help improve the accuracy of target detection and recognition. The spectral transformer technology enhances the generalization ability of the model for different sea conditions and target types, enabling the model to cope with complex and changing ocean conditions. The embodiment of the present invention reduces manual intervention, improves the level of automated processing, and makes the system more suitable for large-scale and long-term continuous monitoring tasks. The embodiment of the present invention introduces content consistency constraints for intermediate states (visible light images) in the generative adversarial network to make the generated images more realistic, and proposes a spectral transformer technology based on the spectral attention mechanism to further improve image quality and cope with complex changes in sea conditions.

[0060] Figure 7The conversion process from the original infrared image to the visible light image and finally to the generated hyperspectral image is clearly presented. The original infrared image mainly shows the temperature information of the scene, while the visible light image provides the natural color of the scene. The generated hyperspectral image shows richer and more detailed spectral features at wavelengths of 480nm, 520nm, 580nm, and 660nm.

[0061] Example 2

[0062] Embodiment 2 of the present invention provides a terminal device corresponding to the above-mentioned embodiment 1. The terminal device may be a processing device for a client, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to execute the method of the above-mentioned embodiment.

[0063] The terminal device of this embodiment includes a memory, a processor, and a computer program stored in the memory; the processor executes the computer program in the memory to implement the steps of the method in the above-mentioned embodiment 1.

[0064] In some implementations, the memory may be a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0065] In some other implementations, the processor may be a central processing unit (CPU), a digital signal processor (DSP), or other general-purpose processors of various types, which are not limited herein.

[0066] Example 3

[0067] Embodiment 3 of the present invention provides a computer-readable storage medium corresponding to the above embodiment 1, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the method of the above embodiment 1 are implemented.

[0068] Computer readable storage media can be tangible devices that hold and store instructions used by instruction execution devices. Computer readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any combination thereof.

[0069] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present application may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0070] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0071] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0072] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0073] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for generating hyperspectral images of ships, It is characterized in that The following steps are involved: S1. Preprocessing infrared images of ocean scenes; S2, inputting the preprocessed infrared image into a spectrum generative adversarial network based on content consistency constraints to generate an intermediate state image, that is, obtaining a visible light image; S3, embedding the visible light image features into the initial feature space of the mask-guided spectrum transformer based on the spectrum attention mechanism to obtain preliminary spectrum features; S4, using the preliminary spectral features as input of a mask-guided spectral transformer based on a spectral attention mechanism, and processing the output of the spectral transformer to obtain a generated image; The loss function of the spectral generative adversarial network based on content consistency constraint is expressed as: ; in, , and They are the generator loss function, discriminator loss function and perceptual loss function of the GAN network. is the content consistency constraint loss, , and They are the external correction loss trade-off parameter and the internal correction loss trade-off parameter, , , is the temperature scale parameter, Indicates the synthetic frame in the multilayer perceptron l The output features of the layer spatial position s, represents the corresponding positive feature, represents other negative features, L represents the number of layers of the multilayer perceptron, and S represents the l The position of the multilayer perceptron, N represents the number of negative samples, X is the preprocessed infrared image as input, represents the infrared image after continuous input preprocessing, represents the continuously generated intermediate state images, express In the multi-layer perceptron l The difference in layers, express In the multi-layer perceptron l The difference in layers, express In the multi-layer perceptron l The difference in layers, express In the multi-layer perceptron l Difference in layers; each layer of the GAN network is followed by a multi-layer perceptron.

2. The method for generating a ship hyperspectral image according to claim 1, It is characterized in that In step S4, the spectrum attention mechanism SA is expressed as: ;in, is the intermediate state image used as input to the self-attention mechanism, and are learnable parameters, H represents the image height, W represents the image width, C represents the number of color channels, Q represents the query, K represents the key, and V represents the value.

3. The method for generating a ship hyperspectral image according to claim 1, It is characterized in that In step S4, the mask-guided spectral transformer includes a plurality of attention blocks connected in series, and the input side of the first attention block is connected to the embedding layer; for the first to i-th attention blocks, two adjacent attention blocks are connected through a downsampling layer; For the i-th to M-th attention blocks, two adjacent attention blocks are connected through the upsampling layer and the convolution layer in sequence; the output of the first attention block is channel-connected with the output of the last upsampling layer, the second attention block is channel-connected with the output of the K-1-th upsampling layer, and so on; the last attention block is connected to the mapping layer, and the output of the mapping layer is concatenated with the input of the embedding layer; where M is the number of attention blocks, 1<i≤M, and K is the number of upsampling layers.

4. The method for generating a ship hyperspectral image according to claim 1, It is characterized in that The spectral generative adversarial network based on content consistency constraints includes a generator, a multi-layer perceptron and a discriminator connected in sequence.

5. The method for generating a ship hyperspectral image according to claim 4, It is characterized in that The generator includes a plurality of serially connected convolutional layers, residual blocks, upsampling layers and output convolutional layers; the discriminator includes a plurality of serially connected convolutional layers and fully connected layers.

6. The method for generating a ship hyperspectral image according to claim 5, It is characterized in that The residual block includes a first convolutional layer, a first normalization layer, an activation function, a second convolutional layer, and a second normalization layer connected in sequence; the output of the second normalization layer is spliced ​​with the input of the first convolutional layer and used as the input of the next residual block, and the output of the last residual block is spliced ​​with the input of the first convolutional layer of the residual block and used as the input of the upsampling layer.

7. The method for generating a ship hyperspectral image according to any one of claims 1 to 6, It is characterized in that In step S4, the specific implementation process of processing the output of the spectrum converter includes: performing a convolution operation on the output of the spectrum converter, adding the output result of the convolution operation to the preliminary spectrum feature to obtain a generated image.

8. A terminal device comprising a memory, a processor and a computer program stored in the memory; It is characterized in that The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instructions stored thereon; It is characterized in that When the computer program / instructions are executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions; It is characterized in that When the computer program / instructions are executed by a processor, the steps of the method of any one of claims 1 to 7 are implemented.