A hyperspectral image classification method in scenes with inconsistent spectral bands

Abstract

The present invention discloses a method for hyperspectral image classification in scenarios with inconsistent spectral bands. The method comprises: obtaining a source domain image of a first initial hyperspectral image and a target domain image of a second initial hyperspectral image; preprocessing the source domain image and the target domain image to obtain a preprocessed source domain sub-image and a target domain sub-image, wherein the source domain sub-image has n bands and the target domain sub-image has k bands; and based on the target domain band number k, randomly selecting m bands from the n bands of the source domain sub-image for masking, retaining k unmasked bands. The present invention addresses the technical limitations of cross-domain classification in scenarios with inconsistent source and target domain spectra.

Description

Technical Field

[0001] The present invention relates to the field of graphics processing, and in particular to a hyperspectral image classification method in a scene with inconsistent spectral bands. Background Art

[0002] Cross-domain hyperspectral image classification plays an important role in important issues such as land feature classification and recognition, agricultural and ecological environment monitoring, urban planning and development, disaster monitoring and emergency response. However, in cross-domain hyperspectral image tasks, there is often a large spectral offset between the source domain and the target domain samples, such as the different number of spectra. The model trained on the source domain cannot be directly used in the target domain. Therefore, there is currently no research on the case where the number of spectra in the source domain and the target domain is inconsistent. Most of the studies are to extract the spectra of the source domain and the target domain, obtain the same number of bands, and then conduct research. These studies can be roughly divided into two types: one is domain adaptation, which aligns the data of the source domain and the target domain into a common feature space to minimize the difference; the other is domain generalization, which generates an extended domain from the source domain image to learn domain-invariant representations on the source domain and the extended domain:

[0003] Cross-domain classification based on domain adaptation includes a method based on graph optimal transfer and maximum mean difference proposed by Y. Zhang et al. in the document "Y. Zhang, W. Li, M. Zhang, Y. Qu, R. Tao and H. Qi,"Topological Structure and Semantic Information Transfer Network for Cross-Scene Hyperspectral Image Classification," IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 6, pp. 2817-2830, 2023." This method assists the maximum mean difference in aligning the distribution of the source domain and the target domain by aligning the graph topological relationship through graph optimal transfer. Graph alignment and distribution alignment work together to classify the target domain.

[0004] For cross-domain classification based on domain generalization, H. Zhao et al. proposed a method based on unbiased randomization and contrastive adversarial learning in the document "H. Zhao, J. Zhang, L. Lin, J. Wang, S. Gao and Z. Zhang," Locally Linear Unbiased Randomization Network for Cross-Scene Hyperspectral Image Classification," IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1-12, 2023." This method uses unbiased randomization to generate an extended domain using source domain images, and uses contrastive adversarial learning to constrain the extended domain to have the domain-invariant features of the source domain and certain offset features. Then, learning is performed on the source domain and the extended domain to train a generalized network that is directly used for classification in the target domain.

[0005] These methods simply cut the bands of the source and target domain data and train them on the premise of the same number of bands. They cannot cope with the situation where the source and target domain bands are significantly different in actual applications. Summary of the Invention

[0006] The embodiment of the present invention provides a hyperspectral image classification method in a scenario with inconsistent spectral bands, so as to at least solve the technical problem of the limitation of cross-domain classification when the source domain and the target domain spectra are inconsistent.

[0007] According to one aspect of an embodiment of the present invention, a hyperspectral image classification method in a scene with inconsistent spectral bands is provided. The method may include: obtaining a source domain image of a first initial hyperspectral image and a target domain image of a second initial hyperspectral image, preprocessing the source domain image and the target domain image to obtain a preprocessed source domain sub-image and a target domain sub-image, wherein the number of bands of the source domain sub-image is n and the number of bands of the target domain sub-image is k; based on the number of bands of the target domain k, randomly selecting m bands from the n bands of the source domain sub-image for masking, and retaining k unmasked bands; using the k unmasked bands to restore the m masked bands through a linear interpolation function to obtain the number of bands of the restored source domain sub-image, wherein the number of bands of the restored source domain sub-image includes k unmasked bands and m new bands after restoration; position encoding the k unmasked bands and the m new bands after restoration to obtain the number of bands of the k unmasked bands. Position coding and position coding of m new bands; based on the number of bands k, m and the position coding of k unmasked bands and the position coding of m new bands, the reconstructed source domain reconstructed sub-image corresponding to the source domain sub-image is obtained; the source domain reconstructed sub-image is displayed and implicitly spectrally shifted to obtain the shifted source domain shifted sub-image, the source domain shifted sub-image is smoothed to obtain the simulation domain sub-image corresponding to the source domain sub-image, the source domain sub-image and the simulation domain sub-image are merged to obtain a merged sub-image; the merged sub-image is processed by a feature extractor and a classifier to obtain the classification result and feature projection of the merged sub-image, the target domain sub-image is processed based on the optimized feature extractor and classifier to obtain the classification result of the target domain sub-image, wherein the feature projection is used to calculate the loss function, and the loss function is used to optimize the generation and classification process of the simulation domain sub-image.

[0008] Alternatively, the expression for restoring the m masked bands using the k unmasked bands through a linear interpolation function is: Among them, x0, y0 and x1, y1 are the center wavelength and radiation intensity values ​​of k adjacent two bands respectively, and x and y are the center wavelength and radiation intensity values ​​of the center positions of k adjacent two bands.

[0009] Optionally, position encoding is performed on the k unmasked bands and the restored m new bands to obtain position encodings of the k unmasked bands and position encodings of the m new bands, including: the process expression for position encoding the k unmasked bands is: Among them, PE 0，int(pos) Encode the positions of the k unmasked bands, where pos represents the absolute index of the band position, dim_idx is the dimension index, and d_model is the embedding dimension; position encoding is performed on the restored m new bands, and the position encoding of the m new bands is zero.

[0010] Optionally, based on the number of bands k, m and the position codes of k unmasked bands and the position codes of m new bands, a reconstructed source domain reconstructed sub-image corresponding to the initial sub-image of the source domain is obtained, including: adding the position codes of k unmasked bands and the position codes of m new bands to the original features to obtain the reconstructed sub-image of the source domain, wherein, when the number of bands is k, the original feature is the sum of k and the position codes of k unmasked bands, and when the number of bands is m, the original feature is m plus zero.

[0011] Optionally, the source domain reconstructed sub-image is displayed and implicitly spectrally shifted to obtain a shifted source domain shifted sub-image, including: adding Gaussian noise to the original features of the source domain reconstructed sub-image to obtain the shifted features; performing dimensionality reduction processing on the shifted features to obtain the shifted reduced-dimensional original features; performing feature extraction on the shifted reduced-dimensional original features through a decoder to obtain a feature map; obtaining content features by calculating the mean and variance of the feature map; randomly generating a style feature from the content features through an adaptive instance normalization algorithm; calculating the mean and variance of a style feature, and changing the mean and variance of a style feature through an affine transformation; aligning the mean and variance of the changed style feature with the mean and variance of the feature map to determine that the display shift and implicit shift of the source domain reconstructed sub-image are completed; when the display shift and implicit shift of the source domain reconstructed sub-image are completed, decoding the feature map through a decoder to obtain the shifted source domain shifted sub-image.

[0012] Optionally, the source domain offset sub-image is smoothed to obtain a simulation domain sub-image corresponding to the source domain sub-image, including: smoothing the source domain offset sub-image through a 1×1×1 convolution kernel to obtain the simulation domain sub-image.

[0013] Optionally, the expression used to optimize the generation process of the simulation domain sub-image through the loss function is: Among them, L adv To optimize the loss function of the feature extractor in the generation process of the simulated domain sub-image, n c Refers to the number of samples in the cth class, zi represents the feature vector of the i-th sample, and Represent positive samples and negative samples respectively, P c (i) and N c (i) represents the i-th sample in the positive sample set of the c-th class and the i-th sample in the negative sample set of the c-th class, respectively. The c-th class in the merged sub-image is taken as the positive sample, and the other classes are taken as negative samples. S is the similarity, c represents the category, and τ is the temperature parameter.

[0014] Optionally, the loss function used to optimize the classification process of the simulated domain sub-image is expressed as: Among them, L con In order to optimize the loss function of the classifier in the process of generating the simulated domain sub-image, N is the total number of samples, the samples in the source domain are selected as positive samples as the positive sample set P(i), and the samples in the simulated domain are selected as the negative sample set N(i), z i represents the feature vector of the i-th sample, and represent positive samples and negative samples respectively, S is the similarity, and τ is the temperature parameter.

[0015] Beneficial effects of the present invention:

[0016] The present invention proposes a new hyperspectral image classification method in a scenario with inconsistent spectral bands. When the number of bands of a source domain hyperspectral image and a target domain hyperspectral image is inconsistent, the inconsistent bands of the source domain and the target are masked, and linear interpolation is performed using the masked bands. The new bands after linear interpolation are encoded, and display offset and privacy offset are performed after encoding to obtain a simulation domain. The source domain and the simulation domain are added and processed by a feature extractor and a classifier to obtain a classification result and a loss function is calculated using feature projection to optimize the generation and classification process of the simulation domain. The optimized feature extractor and classifier are used to process the target domain hyperspectral image to obtain a classification result of the target domain hyperspectral image. The technical problem of the limitation of cross-domain classification in the case of inconsistent spectra between the source domain and the target domain is solved, and the cross-domain classification task is regarded as a classification problem based on domain generalization, which can effectively solve the classification problem of inconsistent spectra between the source domain and the target domain, and has the technical effect of fast training speed and high classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] Figure 1 This is a flowchart of a hyperspectral image classification method in a scenario with inconsistent spectral bands according to an embodiment of the present invention; DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0020] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] Example 1

[0022] According to an embodiment of the present invention, a method for classifying hyperspectral images in a scenario with non-uniform spectral bands is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system comprising at least one set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown.

[0023] Figure 1 FIG. 1 is a flow chart of a hyperspectral image classification method in a scenario with inconsistent spectral bands according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps:

[0024] Step S101: obtaining a source domain image of a first initial hyperspectral image and a target domain image of a second initial hyperspectral image, preprocessing the source domain image and the target domain image to obtain a preprocessed source domain sub-image and a target domain sub-image, wherein the number of bands of the source domain sub-image is n and the number of bands of the target domain sub-image is k.

[0025] In the technical solution provided in step S101 of the present invention, a source domain image of the first initial hyperspectral image and a target domain image of the second initial hyperspectral image are obtained, a normalization operation is performed on the source domain image and the target domain image, and then the images are divided into blocks to obtain preprocessed source domain sub-images and target domain sub-images, wherein the source domain sub-images are labeled data and the target domain sub-images are unlabeled data.

[0026] Step S102 : Based on the number k of bands in the target domain, m bands are randomly selected from the n bands in the source domain sub-image for masking, and k unmasked bands are retained.

[0027] In the technical solution provided in the above step S102 of the present invention, to address the problem of inconsistent numbers of source domain sub-image bands and target domain sub-image bands, m bands are randomly selected from n bands of the source domain sub-image for masking, and k unmasked bands are retained.

[0028] Step S103 : Using k unmasked bands, the m masked bands are restored through a linear interpolation function to obtain the number of bands of the restored source domain sub-image, where the number of bands of the restored source domain sub-image includes the k unmasked bands and the m new bands after restoration.

[0029] In the technical solution provided in the above step S103 of the present invention, in order to ensure spectral continuity, the masked bands are restored by a linear interpolation function to restore the number of bands of the first initial source domain hyperspectral image.

[0030] Step S104 , position coding is performed on the k unmasked bands and the restored m new bands to obtain position codes of the k unmasked bands and position codes of the m new bands.

[0031] In the technical solution provided in step S104 of the present invention, position coding is performed on the k unmasked bands and the restored m new bands to obtain position codes of the k unmasked bands and position codes of the m new bands.

[0032] Step S105 : performing calculation based on the number of bands k, m, the position codes of the k unmasked bands, and the position codes of the m new bands to obtain a reconstructed source domain sub-image corresponding to the source domain sub-image.

[0033] In the technical solution provided in step S105 of the present invention, the number of bands k, m, the positions of the k unmasked bands, and the positions of the m new bands are encoded to obtain a reconstructed source domain sub-image corresponding to the source domain sub-image.

[0034] Step S106: Display and implicitly perform spectral shift on the source domain reconstructed sub-image to obtain a shifted source domain shifted sub-image; smooth the source domain shifted sub-image to obtain a simulation domain sub-image corresponding to the source domain sub-image; merge the source domain sub-image and the simulation domain sub-image to obtain a merged sub-image.

[0035] In the technical solution provided in the above step S106 of the present invention, the source domain reconstructed sub-image is displayed and implicitly spectrally shifted to obtain the shifted source domain shifted sub-image, the source domain shifted sub-image is smoothed to obtain the simulation domain sub-image corresponding to the source domain sub-image, and the source domain sub-image and the simulation domain sub-image are merged to obtain a merged sub-image.

[0036] Step S107: Process the merged sub-image through a feature extractor and a classifier to obtain the classification result and feature projection of the merged sub-image; process the target domain sub-image based on the optimized feature extractor and classifier to obtain the classification result of the target domain sub-image; wherein the feature projection is used to calculate the loss function, and the loss function is used to optimize the generation and classification process of the simulation domain sub-image.

[0037] In the technical solution provided in the above step S107 of the present invention, the merged sub-image is input into the feature extractor and the classifier to obtain the classification result and feature projection corresponding to the merged sub-image. The feature projection is used to calculate the loss function, and the loss function is used to optimize the feature extractor and the optimizer.

[0038] The above method of this embodiment is further introduced below.

[0039] As an optional embodiment, in step S103, the expression for restoring the m masked bands by using the k unmasked bands through the linear interpolation function is: Among them, x0, y0 and x1, y1 are the center wavelength and radiation intensity values ​​of k adjacent two bands respectively, and x and y are the center wavelength and radiation intensity values ​​of the center positions of k adjacent two bands.

[0040] In this embodiment, the k unmasked bands are used to restore the m masked bands through a linear interpolation function:

[0041] As an optional embodiment, step S104, position coding is performed on the k unmasked bands and the restored m new bands to obtain the position coding of the k unmasked bands and the position coding of the m new bands, including: the process expression of position coding of the k unmasked bands is: Among them, PE 0，int(pos)Encode the positions of the k unmasked bands, where pos represents the absolute index of the band position, dim_idx is the dimension index, and d_model is the embedding dimension; position encoding is performed on the restored m new bands, and the position encoding of the m new bands is zero.

[0042] In this embodiment, the process of position coding the k unmasked bands is calculated by formula 2, and the position codes of the m new bands are set to 0.

[0043] As an optional implementation method, step S105, based on the number of bands k, m and the position codes of k unmasked bands and the position codes of m new bands, obtains a reconstructed source domain reconstructed sub-image corresponding to the source domain initial sub-image, including: adding the position codes of k unmasked bands and the position codes of m new bands to the original features to obtain the source domain reconstructed sub-image, wherein, when the number of bands is k, the original feature is the sum of k and the position codes of k unmasked bands, and when the number of bands is m, the original feature is m plus zero.

[0044] In this embodiment, the position codes of k unmasked bands and the position codes of m new bands are added to the original features to obtain a source domain reconstructed sub-image. When the number of bands is k, the original feature is the sum of k and the position codes of k unmasked bands. When the number of bands is m, the original feature is m plus zero.

[0045] As an optional embodiment, step S106 displays and implicitly spectrally shifts the source domain reconstructed sub-image to obtain a shifted source domain shifted sub-image, including: adding Gaussian noise to the original features of the source domain reconstructed sub-image to obtain the shifted features; performing dimensionality reduction processing on the shifted features to obtain the shifted reduced-dimensional original features; performing feature extraction on the shifted reduced-dimensional original features through a decoder to obtain a feature map; obtaining content features by calculating the mean and variance of the feature map; randomly generating a style feature from the content features through an adaptive instance normalization algorithm; calculating the mean and variance of a style feature, and changing the mean and variance of a style feature through an affine transformation; aligning the mean and variance of the changed style feature with the mean and variance of the feature map to determine that the display shift and implicit shift of the source domain reconstructed sub-image are completed; when the display shift and implicit shift of the source domain reconstructed sub-image are completed, decoding the feature map through a decoder to obtain the shifted source domain shifted sub-image.

[0046] In this embodiment, the source domain reconstructed sub-image adds Gaussian noise within a certain range on the basis of the original features to obtain the offset features, which is an explicit offset. The offset features are then subjected to dimensionality reduction processing, such as principal component analysis, and then input into the encoder to extract features: first input into a 1×1 convolution kernel to generate a 13×13×3 image block, and a 3×3×3 convolution kernel is used to extract features as a feature map representing spatial-spectral information. After obtaining the feature map of the image block, the mean and variance are calculated to obtain the content feature x, and a style randomization algorithm, such as the adaptive instance normalization (AdaIN) algorithm, is used to randomly generate a style feature y. By adaptively learning the affine transformation parameters, the style of y is transferred to the previous image x, that is, the mean and variance of the content feature x are aligned with the mean and variance of the style feature y. This process is called an implicit offset, and the expression of the privacy offset is:

[0047]

[0048] μ and σ represent the mean and variance of the image patch, respectively.

[0049] As an optional embodiment, step S106 smoothes the source domain offset sub-image to obtain a simulation domain sub-image corresponding to the source domain sub-image, including: smoothing the source domain offset sub-image through a 1×1×1 convolution kernel to obtain a simulation domain sub-image.

[0050] In this embodiment, spectral shifting may result in spectral discontinuities, so a 1×1×1 convolution kernel is used to smooth the spectral curve. The random convolution spectrum requires a threshold of, for example, 0.41, and a random number between [0, 1] is generated. If the generated random number is less than 0.41, the source domain shifted sub-image is smoothed, generating the simulated domain sub-image.

[0051] As an optional embodiment, in step S107, the expression for optimizing the generation process of the simulation domain sub-image through the loss function is:

[0052] Among them, L adv To optimize the loss function of the feature extractor in the generation process of the simulated domain sub-image, n c Refers to the number of samples in the cth class, zi represents the feature vector of the i-th sample, and Represent positive samples and negative samples respectively, P c (i) and N c(i) represents the i-th sample in the positive sample set of the c-th class and the i-th sample in the negative sample set of the c-th class, respectively. The c-th class in the merged sub-image is taken as the positive sample, and the other classes are taken as negative samples. S is the similarity, c represents the category, and τ is the temperature parameter.

[0053] In this embodiment, the feature extractor and the classifier process the merged sub-image to obtain feature projections, which are used to optimize the loss function of the feature extractor, and formula (3) is used as a constraint of the loss function of the feature extractor.

[0054] As an optional embodiment, in step S107, the loss function used to optimize the classification process of the simulation domain sub-image is expressed as follows:

[0055] Among them, L con In order to optimize the loss function of the classifier in the process of generating the simulated domain sub-image, N is the total number of samples, the samples in the source domain are selected as positive samples as the positive sample set P(i), and the samples in the simulated domain are selected as the negative sample set N(i), z i represents the feature vector of the i-th sample, and represent positive samples and negative samples respectively, S is the similarity, and τ is the temperature parameter.

[0056] In this embodiment, the feature extractor and the classifier process the merged sub-image to obtain feature projections, which are used to optimize the loss function of the classifier, and formula (4) is used as a constraint of the classifier loss function.

[0057] The effects of the embodiments of the present invention can be further illustrated by the following simulation experiments.

[0058] 1. Simulation conditions

[0059] The simulation was performed using Visual Studio Code on an NVIDIA GeForce RTX 4090 GPU. The data used in the simulation consisted of two sets of hyperspectral images with consistent source and target domain spectra.

[0060] 2. Simulation content

[0061] First, a part of the bands of the target domain hyperspectral image is extracted according to a certain ratio as a new test image; then, the same number of bands are extracted from the source domain hyperspectral image according to the above steps, and the above operations are performed on the two. The trained model directly classifies the target domain and calculates the three indicators of AA, OA, and Kappa coefficient.

[0062] In order to prove the effectiveness of the algorithm, the single source domain extension network (SDENet), the language-aware domain generalization network (LDGNet) and the locally linear unbiased randomization network (LLURNet) were selected as comparison algorithms. The comparison results are shown in Table 1.

[0063] Table 1 Comparison of results of different neural networks

[0064] Method SDENet LDGNet LLURNet OA 81.94％ 82.53％ 83.53％ Method LLURNet(1 / 2) SCNet(1 / 2) SCnet OA 74.94％ 81.48％ 84.93％

[0065] As shown in Table 1, the classification accuracy of the proposed method is significantly higher than that of previous cross-domain hyperspectral classification algorithms for both 1 / 2-scale and full-scale bands. Within the network, a generator is implemented to generate a simulated domain with spectral shifts that exhibit discriminative representations of the source domain. During the generator optimization process, an asymmetric encoding / decoding structure is utilized to process the bands, reconstruct the images, and generate different simulated domain images for training. Furthermore, a spectral shift simulation is designed to alter the spectral information of the reconstructed images, enabling the generalization of networks trained using the adversarial contrastive learning strategy.

[0066] In the embodiment of the present invention, a source domain image of a first initial hyperspectral image and a target domain image of a second initial hyperspectral image are obtained, and the source domain image and the target domain image are preprocessed to obtain a preprocessed source domain sub-image and a target domain sub-image, wherein the number of bands of the source domain sub-image is n, and the number of bands of the target domain sub-image is k; based on the number of bands k of the target domain, m bands are randomly selected from the n bands of the source domain sub-image for masking, and k unmasked bands are retained; and the k unmasked bands are used to mask the source domain sub-image. The bands are restored by a linear interpolation function on the m masked bands to obtain the number of bands of the restored source domain sub-image, wherein the number of bands of the restored source domain sub-image includes k unmasked bands and m new bands after restoration; the k unmasked bands and the m new bands after restoration are position-coded to obtain the position coding of the k unmasked bands and the position coding of the m new bands; based on the number of bands k, m and the position coding of the k unmasked bands and the position coding of the m new bands, the position coding of the k unmasked bands and the position coding of the m new bands are obtained. The code is used to obtain the reconstructed source domain reconstructed sub-image corresponding to the source domain sub-image; the source domain reconstructed sub-image is explicitly and implicitly spectrally shifted to obtain the shifted source domain shifted sub-image; the source domain shifted sub-image is smoothed to obtain the simulation domain sub-image corresponding to the source domain sub-image; the source domain sub-image and the simulation domain sub-image are merged to obtain a merged sub-image; the merged sub-image is processed by a feature extractor and a classifier to obtain the classification result and feature projection of the merged sub-image; the target domain sub-image is processed based on the optimized feature extractor and classifier to obtain the classification result of the target domain sub-image, wherein the feature projection is used to calculate the loss function, and the loss function is used to optimize the generation and classification process of the simulation domain sub-image, thereby solving the technical problem of the limitations of cross-domain classification when the source domain and target domain spectra are inconsistent, and achieving the goal of treating the cross-domain classification task as a classification problem based on domain generalization, which can effectively solve the classification problem of inconsistent source domain and target domain spectra, and has the technical effect of fast training speed and high classification accuracy.

[0067] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0068] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0069] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0070] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0071] In addition, the functional units in various embodiments of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0072] The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A hyperspectral image classification method in a scene with inconsistent spectral bands, characterized by: include: Acquire a source domain image of the first initial hyperspectral image and a target domain image of the second initial hyperspectral image, and preprocess the source domain image and the target domain image to obtain a preprocessed source domain sub-image and a target domain sub-image, wherein the number of bands of the source domain sub-image is n and the number of bands of the target domain sub-image is k; Based on the number k of bands in the target domain, m bands are randomly selected from the n bands of the source domain sub-image for masking, and k unmasked bands are retained; Restoring the m masked bands using k unmasked bands through a linear interpolation function to obtain the number of bands of the restored source domain sub-image, wherein the number of bands of the restored source domain sub-image includes the k unmasked bands and the m new bands after restoration; Performing position coding on the k unmasked bands and the restored m new bands to obtain position codes of the k unmasked bands and position codes of the m new bands; Obtaining a reconstructed source domain sub-image corresponding to the source domain sub-image based on the number of bands k, m, the position codes of the k unmasked bands, and the position codes of the m new bands; Performing display and implicit spectral shifting on the source domain reconstructed sub-image to obtain a shifted source domain shifted sub-image, performing smoothing on the source domain shifted sub-image to obtain a simulation domain sub-image corresponding to the source domain sub-image, and merging the source domain sub-image and the simulation domain sub-image to obtain a merged sub-image; The merged sub-image is processed by a feature extractor and a classifier to obtain a classification result and a feature projection of the merged sub-image. The target domain sub-image is processed based on the optimized feature extractor and classifier to obtain a classification result of the target domain sub-image, wherein the feature projection is used to calculate a loss function, and the loss function is used to optimize the generation and classification process of the simulation domain sub-image.

2. The method according to claim 1, characterized in that The expression for restoring the m masked bands by using the k unmasked bands through the linear interpolation function is: Among them, x0, y0 and x1, y1 are the center wavelength and radiation intensity values ​​of k adjacent two bands respectively, and x and y are the center wavelength and radiation intensity values ​​of the center positions of k adjacent two bands.

3. The method according to claim 1, characterized in that The step of performing position coding on the k unmasked bands and the restored m new bands to obtain position coding of the k unmasked bands and position coding of the m new bands includes: The process expression for position encoding of the k unmasked bands is: Among them, PE 0，int(pos) Encode the positions of k unmasked bands, where pos represents the absolute index of the band position, dim_idx is the dimension index, and d_model is the embedding dimension; Position encoding is performed on the restored m new bands, and the position encodings of the m new bands are zero.

4. The method according to claim 1, wherein The obtaining of a reconstructed source domain reconstructed sub-image corresponding to the source domain initial sub-image based on the number of bands k, m, the position codes of the k unmasked bands, and the position codes of the m new bands includes: The position codes of k unmasked bands and the position codes of m new bands are added to the original features to obtain a source domain reconstructed sub-image, where when the number of bands is k, the original feature is the sum of k and the position codes of k unmasked bands; when the number of bands is m, the original feature is m plus zero.

5. The method according to claim 1, wherein The display and implicit spectrum shifting of the source domain reconstructed sub-image to obtain the shifted source domain shifted sub-image includes: Adding Gaussian noise to the original features of the source domain reconstructed sub-image to obtain the offset features; Performing dimensionality reduction processing on the offset features to obtain offset dimensionality-reduced original features; Performing feature extraction on the offset dimension reduction original features through a decoder to obtain a feature map; By calculating the mean and variance of the feature map, the content features are obtained; The content features are subjected to an adaptive instance normalization algorithm to randomly generate a style feature; Calculating a mean and a variance of a pattern feature, and changing the mean and the variance of the pattern feature by an affine transformation; Aligning the mean and variance of the changed style feature with the mean and variance of the feature map, and determining whether the explicit offset and implicit offset of the source domain reconstructed sub-image are completed; After the explicit offset and implicit offset of the source domain reconstructed sub-image are completed, the feature map is decoded by a decoder to obtain the offset source domain offset sub-image.

6. The method according to claim 1, characterized in that The smoothing process is performed on the source domain offset sub-image to obtain a simulation domain sub-image corresponding to the source domain sub-image, including: The source domain offset sub-image is smoothed by a 1×1×1 convolution kernel to obtain the simulation domain sub-image.

7. The method according to claim 1, characterized in that The expression for optimizing the generation process of the simulation domain sub-image by the loss function is: Among them, L adv To optimize the loss function of the feature extractor in the generation process of the simulated domain sub-image, n c refers to the number of samples in the cth class, z i represents the feature vector of the i-th sample, and Represent positive samples and negative samples respectively, P c (i) and N c (i) represents the i-th sample in the positive sample set of the c-th class and the i-th sample in the negative sample set of the c-th class, respectively. The c-th class in the merged sub-image is taken as the positive sample, and the other classes are taken as negative samples. S is the similarity, c represents the category, and τ is the temperature parameter.

8. The method according to claim 1, characterized in that The expression for optimizing the classification process of the simulation domain sub-image by the loss function is: Among them, L con In order to optimize the loss function of the classifier in the process of generating the simulated domain sub-image, N is the total number of samples, the samples in the source domain are selected as positive samples as the positive sample set P(i), and the samples in the simulated domain are selected as the negative sample set N(i), z i represents the feature vector of the i-th sample, and represent positive samples and negative samples respectively, S is the similarity, and τ is the temperature parameter.

9. A processor, characterized in that: The processor is configured to run a program, wherein the program executes the method according to any one of claims 1 to 8 when running.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 8.

Images