A cross-scene hyperspectral image segmentation method based on style transfer

Abstract

The present invention relates to a cross-scene hyperspectral image segmentation method based on style transfer. The present invention comprises the following steps: constructing a cross-scene hyperspectral image segmentation dataset; constructing a multi-view information fusion segmentation network; constructing a cross-scene hyperspectral image segmentation network based on style transfer; training and testing the cross-scene hyperspectral image segmentation network based on style transfer. Compared with the prior art, the present invention solves the problem that in the existing hyperspectral classification task, it is difficult to migrate the source domain training model to the target domain dataset due to the style differences between different satellite hyperspectral images through a cross-scene hyperspectral image segmentation network. In addition, the present invention adopts a multi-view information fusion segmentation network to aggregate the spatial-spectral information of the hyperspectral image, thereby realizing an innovative method for effectively migrating the model trained on the source domain satellite data to the target domain satellite data.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral remote sensing image processing, and in particular to a cross-scene hyperspectral image segmentation method based on style conversion. Background Art

[0002] Hyperspectral remote sensing technology is widely used in environmental monitoring, agricultural management, urban planning and other fields because it can provide rich spectral information. Hyperspectral images can effectively identify and classify objects by capturing data from visible light to near-infrared bands. However, in practical applications, the processing of hyperspectral images faces many challenges, especially the model migration problem in cross-scene segmentation tasks.

[0003] Traditional hyperspectral image segmentation methods usually rely on deep learning techniques for feature extraction, but most methods assume that the training and test data come from the same scene or data source. However, satellite images have different shooting conditions, viewing angles, lighting and other factors, resulting in significant style differences between hyperspectral images from different satellites. This difference makes it difficult to directly migrate the source domain model to the target domain, affecting the model's cross-scene adaptability.

[0004] To solve this problem, style transfer technology has been proposed as an effective cross-domain migration method. Style transfer can adjust the style of the source domain image to make it close to the target domain image, thereby reducing the impact of style differences. However, the application of existing style transfer methods in hyperspectral image segmentation still has certain limitations, especially in cross-scene segmentation tasks. How to retain the detailed information of hyperspectral images and effectively handle style differences is still a difficulty.

[0005] Based on this, the present invention proposes a cross-scene hyperspectral image segmentation method based on style transfer. This method effectively aggregates the spatial and spectral information of hyperspectral images by constructing a cross-scene dataset and introducing a multi-view information fusion segmentation network. At the same time, the style transfer technology is used to solve the problem of style differences between different satellite images, and the effective migration of the source domain model to the target domain is realized. By combining style transfer and multi-view information fusion, the present invention significantly improves the accuracy and cross-scene adaptability of hyperspectral image segmentation, providing a more stable and accurate solution for remote sensing data analysis. Summary of the invention

[0006] The purpose of the present invention is to overcome the limitations of existing methods, improve the detail retention of cross-scene hyperspectral image segmentation results and the adaptability to the situation where there are style differences in hyperspectral images of different scenes, and provide a cross-scene hyperspectral image segmentation method based on style transfer.

[0007] In order to achieve the above object, the technical solution of the present invention is as follows:

[0008] A cross-scene hyperspectral image segmentation method based on style transfer includes the following steps:

[0009] 11) Constructing a cross-scene hyperspectral image segmentation dataset: Obtaining source domain hyperspectral images and source domain hyperspectral image segmentation labels ,in Represents the source domain; obtains the target domain hyperspectral image ,in Represent the target domain: Perform gamma correction to obtain the target domain hyperspectral image The brightness of the source domain is used as a reference to adjust the source domain hyperspectral image Brightness; Get the corrected source domain hyperspectral image ;

[0010] 12) Construct a multi-view information fusion segmentation network: Construct a multi-view information fusion segmentation network with hyperspectral image as input; the multi-view information fusion segmentation network consists of an encoder based on multi-view cross-attention and a decoder based on a diffusion model;

[0011] 13) Construct a cross-scene hyperspectral image segmentation network based on style transfer: Through the style transfer module based on Fourier transform, combined with the multi-view information fusion segmentation network, a cross-scene hyperspectral image segmentation network based on style transfer is constructed;

[0012] 14) Train and test a cross-scene hyperspectral image segmentation network based on style transfer: by minimizing the cross entropy loss Optimize the network parameters of cross-scene hyperspectral image segmentation and perform segmentation on the target domain hyperspectral image The final classification results are obtained by testing.

[0013] The construction of a cross-scene hyperspectral image segmentation dataset comprises the following steps:

[0014] 21) Obtain source domain hyperspectral images , source domain hyperspectral image segmentation label and target domain hyperspectral image , where the source domain hyperspectral image and target domain hyperspectral image All have bands with the same range and the same height and width ;

[0015] 22) Calculate source domain hyperspectral image and target domain hyperspectral image exist The mean brightness value of each band in the source domain hyperspectral image In the The average brightness of the band Calculated by the following formula:

[0016] ,

[0017] in and Represent the height and width of the hyperspectral image, Represents the indexing operation of the source domain hyperspectral image;

[0018] Target domain hyperspectral image In the The average brightness of the band Calculated by the following formula:

[0019] ,

[0020] in and Represent the height and width of the hyperspectral image, Represents the indexing operation of the target domain hyperspectral image;

[0021] 23) Using target domain hyperspectral images The brightness of the source domain is used as a reference to calculate the source domain hyperspectral image When doing gamma correction, Gamma value in each band ; Gamma value Calculated by the following formula:

[0022] ,

[0023] in Representative source domain hyperspectral image In the The average brightness of the bands, Representing the target domain hyperspectral image In the The average brightness of each band;

[0024] 24) Source domain hyperspectral image Apply a gamma correction transform, The transformation process on a band can be expressed by the following formula:

[0025] ,

[0026] in Represents the indexing operation of the corrected source domain hyperspectral image;

[0027] 25) Obtain corrected source domain hyperspectral image .

[0028] The construction of a multi-view information fusion segmentation network comprises the following steps:

[0029] 31) Multi-view information fusion segmentation network accepts hyperspectral images As input; the multi-view information fusion segmentation network consists of an encoder based on multi-view cross-attention and a decoder based on a diffusion model;

[0030] 311) Construct an encoder based on multi-view cross-attention; the encoder accepts hyperspectral images As input, a dual-branch structure is used to process hyperspectral images in different dimensions, and the cross-attention mechanism is combined to fuse the spatial-spectral information; the detailed structure is as follows:

[0031] 312) In the spatial branch, the hyperspectral image First, two-dimensional deep convolution generates spatial conditional position encoding , and then encode the spatial conditional position and hyperspectral images Addition; the two-dimensional deep convolution uses a convolution kernel of size 3×3 and is padded with 1; then the spatial branch embedding feature is generated by linear projection , the shape is , Represents the embedding dimension size;

[0032] 313) In the spectral branch, the hyperspectral image is first It is flattened according to the spatial dimension and then a one-dimensional deep convolution operation is used to generate the spectral conditional position encoding. , and finally encode the spectral condition position And the flattened hyperspectral image Addition; Among them, the one-dimensional deep convolution uses a convolution kernel of size 3 and is padded with 1; then the spectral branch embedding feature is generated by linear projection , the shape is ;

[0033] 314) Spatial branch embedding features and spectral branch embedding features Belong to the same hyperspectral image Features at different perspectives; the encoder consists of four consecutive feature extraction modules, each of which contains a Transformer Encoder on the spatial branch, a Transformer Encoder on the spectral branch, and a cross-attention module connecting the spatial branch and the spectral branch;

[0034] 3141) In the cross-attention module, the spatial branch is embedded into the feature Generate query matrix through linear mapping , key matrix , value matrix ; Spectral branch embedding features Generate the corresponding , , ;

[0035] 3142) The fusion process from spatial branch to spectral branch can be calculated by the following formula:

[0036] ,

[0037] in represents the transpose operation, represents the softmax function, represents the fusion process;

[0038] 3143) The fusion process from the spatial branch to the spectral branch can be calculated by the following formula:

[0039] ,

[0040] 3144) The output of each feature extraction module in the encoder is and Element-wise addition of ;

[0041] 315) After four consecutive feature extraction modules of the encoder, the hyperspectral image The final result is the encoded feature ;

[0042] 32) Construct a decoder based on a diffusion model; the diffusion model accepts the encoded features As input, its corresponding label is the hyperspectral image segmentation label ;

[0043] 321) Construct a standard Unet network to predict the mean of the multidimensional Gaussian distribution in the denoising process;

[0044] 322) In the forward process of diffusion from hyperspectral image segmentation label to noise, the maximum number of steps for adding noise is ,but Represents the hyperspectral image segmentation label The feature map obtained by forward diffusion of steps, where is less than A positive integer from Sampling from a related normal distribution The forward diffusion process representing one step can be expressed by the following formula;

[0045] ,

[0046] ,

[0047] in Uniformly distributed between 0.0001 and 0.02, representing the intensity of adding noise at each step; represents a state with a mean of , the variance is Normal distribution of represents the identity matrix; represents a standard normal distribution;

[0048] 323) Randomly select steps ,from Generated by The vector obtained by adding noise , which can be expressed by the following formula:

[0049] ,

[0050] in ,and It follows a uniform distribution between 0.0001 and 0.02. ; Obey standard normal distribution;

[0051] 324) From Recover from The ideal mean It can be expressed by the following formula:

[0052] ,

[0053] in It obeys the standard normal distribution;

[0054] from Recover from The ideal variance It can be expressed by the following formula:

[0055] ,

[0056] In the reverse diffusion process of generating the hyperspectral image segmentation result, the denoising process can be expressed by the following formula:

[0057] ,

[0058] The reverse diffusion mean ,variance The value of is determined by the vector , Diffusion step index and encoding features Predict through Unet network;

[0059] 325) Execution The denoising operation generates the result , the denoising process of each step can be expressed by the following formula:

[0060] ,

[0061] in The value is less than greater than 0, Represents the Unet network.

[0062] The construction of a cross-scene hyperspectral image segmentation network based on style transfer includes the following steps:

[0063] 41) Corrected source domain hyperspectral image The corresponding source domain encoding features are generated by the encoder based on multi-view cross attention ; Target domain hyperspectral image The encoder based on multi-view cross attention generates the corresponding target domain encoding features ;

[0064] 42) Encoding features of the source domain Perform Fourier transform and save its phase, then the Fourier transform process can be expressed by the following formula:

[0065] ,

[0066] in represents Fourier transform, represents the source domain encoding feature phase, represents the source domain encoding feature amplitude, and Represents the horizontal and vertical frequency coordinates in the frequency domain;

[0067] 43) Encoding features of the target domain Perform Fourier transform and save its amplitude. The Fourier transform process can be expressed by the following formula:

[0068] ,

[0069] in represents Fourier transform, represents the target domain encoding feature phase, represents the target domain encoding feature amplitude, and Represents the horizontal and vertical frequency coordinates in the frequency domain;

[0070] 45) Save the source domain encoding feature phase and the target domain encoding feature amplitude As the input of the inverse Fourier transform, the source domain encodes the characteristic phase Represents the source domain encoding features The domain invariant information of the target domain encodes the feature amplitude represents the style information of the target domain; then the process of inverse Fourier transform can be expressed by the following formula:

[0071] ,

[0072] The style transfer feature Represents source domain encoding features Features after style transfer to the target domain;

[0073] 46) Style transfer features Replace source domain encoding features The final segmentation result is generated as input to the decoder based on the diffusion model.

[0074] The training and testing of the cross-scene hyperspectral image segmentation network based on style transfer includes the following steps:

[0075] 51) Corrected source domain hyperspectral image The corresponding source domain encoding features are generated by the encoder based on multi-view cross attention , and then generate style conversion features through style conversion Finally, the decoder based on the diffusion model generates the final source domain hyperspectral image segmentation result ;

[0076] 52) Calculate the source domain hyperspectral image segmentation results and source domain hyperspectral image segmentation labels The cross entropy loss between , through the cross entropy loss Perform gradient descent to train network parameters; cross entropy loss It can be expressed by the following formula;

[0077] ,

[0078] in Represents the cross entropy loss calculation process;

[0079] 53) Determine whether the set number of training rounds has been reached. If so, complete the training of the cross-scene hyperspectral image segmentation network based on style transfer, otherwise continue training;

[0080] 54) The target domain hyperspectral image The image is input into the trained cross-scene hyperspectral image segmentation network based on style transfer to obtain the final target domain hyperspectral image segmentation result. Beneficial Effects

[0081] The cross-scene hyperspectral image segmentation method based on style transfer proposed in this invention has significant innovation and superiority compared with the existing technology. First, the style transfer technology is used to solve the problem of style differences between different satellite hyperspectral images, thereby effectively improving the migration ability of the source domain model on the target domain dataset. Through this method, the model can be seamlessly migrated between different shooting conditions and satellite platforms, significantly improving the accuracy and robustness of cross-scene segmentation.

[0082] Secondly, the present invention introduces a multi-view information fusion segmentation network, which makes full use of the spatial-spectral information in the hyperspectral image and enhances the model's ability to extract image spatial and spectral features. This method can accurately identify different types of objects in complex scenes, especially when the target domain data is missing or of low quality, and still maintains a high segmentation effect. The fusion of multi-view information enables the model to analyze the image from multiple angles, thereby improving the stability and reliability of the segmentation results.

[0083] In addition, the cross-scene segmentation method proposed in this invention not only maintains the key details of the image when processing style differences, but also significantly improves the adaptability of the segmentation network in different scenes through the combination of style conversion and information fusion. This innovative design can effectively deal with the style differences, resolution differences, and shooting angle differences between different satellite hyperspectral images, and enhance the cross-domain generalization ability of the model.

[0084] The technical solution of the present invention has strong practicality and can be widely applied to hyperspectral image segmentation tasks in different scenes and data sources, especially in the fields of remote sensing image analysis, environmental monitoring, agricultural resource management, etc. It solves the migration problem caused by style differences in the prior art, while improving segmentation accuracy and efficiency, and provides more reliable technical support for the practical application of hyperspectral images. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 It is a cross-scene hyperspectral image segmentation method based on style transfer;

[0086] Figure 2 This is a multi-view information fusion segmentation network framework diagram involved in the present invention;

[0087] Figure 3 This is a cross-scene hyperspectral image segmentation network framework diagram based on style transfer involved in the present invention; DETAILED DESCRIPTION

[0088] In order to have a further understanding and recognition of the structural features and the effects achieved by the present invention, a preferred embodiment and accompanying drawings are used for detailed description as follows:

[0089] like Figure 1 As shown, the cross-scene hyperspectral image segmentation method based on style conversion described in the present invention is characterized by comprising the following steps:

[0090] The first step is to build a cross-scene hyperspectral image segmentation dataset:

[0091] Acquire source domain hyperspectral images and source domain hyperspectral image segmentation labels ,in Represents the source domain; obtains the target domain hyperspectral image ,in Represent the target domain: Perform gamma correction to obtain the target domain hyperspectral image The brightness of the source domain is used as a reference to adjust the source domain hyperspectral image Brightness; Get the corrected source domain hyperspectral image , the specific steps are as follows:

[0092] (1) Obtaining source domain hyperspectral images , source domain hyperspectral image segmentation label and target domain hyperspectral image , where the source domain hyperspectral image and target domain hyperspectral image All have bands with the same range and the same height and width ;

[0093] (2) Calculation of source domain hyperspectral image and target domain hyperspectral image exist The mean brightness value of each band in the source domain hyperspectral image In the The average brightness of the band Calculated by the following formula:

[0094] ,

[0095] in and Represent the height and width of the hyperspectral image, Represents the indexing operation of the source domain hyperspectral image;

[0096] Target domain hyperspectral image In the The average brightness of the band Calculated by the following formula:

[0097] ,

[0098] in and Represent the height and width of the hyperspectral image, Represents the indexing operation of the target domain hyperspectral image;

[0099] (3) Target domain hyperspectral image The brightness of the source domain is used as a reference to calculate the source domain hyperspectral image When doing gamma correction, Gamma value in each band ; Gamma value Calculated by the following formula:

[0100] ,

[0101] in Representative source domain hyperspectral image In the The average brightness of the bands, Representing the target domain hyperspectral image In the The average brightness of each band;

[0102] (4) Source domain hyperspectral image Apply a gamma correction transform, The transformation process on a band can be expressed by the following formula:

[0103] ,

[0104] in Represents the indexing operation of the corrected source domain hyperspectral image;

[0105] (5) Obtain corrected source domain hyperspectral image .

[0106] The second step is to build a multi-view information fusion segmentation network:

[0107] The specific steps are as follows:

[0108] (1) Multi-view information fusion segmentation network accepts hyperspectral images As input; the multi-view information fusion segmentation network consists of an encoder based on multi-view cross-attention and a decoder based on a diffusion model;

[0109] (1-1) Construct an encoder based on multi-view cross-attention; the encoder accepts hyperspectral images As input, a dual-branch structure is used to process hyperspectral images in different dimensions, and the cross-attention mechanism is combined to fuse the spatial-spectral information; the detailed structure is as follows:

[0110] (1-2) In the spatial branch, the hyperspectral image First, two-dimensional deep convolution generates spatial conditional position encoding , and then encode the spatial conditional position and hyperspectral images Addition; the two-dimensional deep convolution uses a convolution kernel of size 3×3 and is padded with 1; then the spatial branch embedding feature is generated by linear projection , the shape is , Represents the embedding dimension size;

[0111] (1-3) In the spectral branch, the hyperspectral image is first It is flattened according to the spatial dimension and then a one-dimensional deep convolution operation is used to generate the spectral conditional position encoding. , and finally encode the spectral condition position And the flattened hyperspectral image Addition; Among them, the one-dimensional deep convolution uses a convolution kernel of size 3 and is padded with 1; then the spectral branch embedding feature is generated by linear projection , the shape is ;

[0112] (1-4) Spatial branch embedding features and spectral branch embedding features Belong to the same hyperspectral image Features at different perspectives; the encoder consists of four consecutive feature extraction modules, each of which contains a Transformer Encoder on the spatial branch, a Transformer Encoder on the spectral branch, and a cross-attention module connecting the spatial branch and the spectral branch;

[0113] (1-4-1) In the cross-attention module, embed the spatial branch into the feature Generate query matrix through linear mapping , key matrix , value matrix ; Spectral branch embedding features Generate the corresponding , , ;

[0114] (1-4-2) The fusion process from spatial branch to spectral branch can be calculated by the following formula:

[0115] ,

[0116] in represents the transpose operation, represents the softmax function, represents the fusion process;

[0117] (1-4-3) The fusion process from spatial branch to spectral branch can be calculated by the following formula:

[0118] ,

[0119] (1-4-4) The output of each feature extraction module in the encoder is and Element-wise addition of ;

[0120] (1-5) After four consecutive feature extraction modules of the encoder, the hyperspectral image The final result is the encoded feature ;

[0121] (2) Construct a decoder based on the diffusion model; the diffusion model accepts the encoded features As input, its corresponding label is the hyperspectral image segmentation label ;

[0122] (2-1) Construct a standard Unet network to predict the mean of the multidimensional Gaussian distribution in the denoising process;

[0123] (2-2) In the forward process of diffusion from hyperspectral image segmentation label to noise, the maximum number of steps for adding noise is ,but Represents the hyperspectral image segmentation label The feature map obtained by forward diffusion of steps, where is less than A positive integer from Sampling from a related normal distribution The forward diffusion process representing one step can be expressed by the following formula;

[0124] ,

[0125] ,

[0126] in Uniformly distributed between 0.0001 and 0.02, representing the intensity of adding noise at each step; represents a state with a mean of , the variance is Normal distribution of represents the identity matrix; represents a standard normal distribution;

[0127] (2-3) Randomly select the number of steps ,from Generated by The vector obtained by adding noise , which can be expressed by the following formula:

[0128] ,

[0129] in ,and It follows a uniform distribution between 0.0001 and 0.02. ; Obey standard normal distribution;

[0130] (2-4) From Recover from The ideal mean It can be expressed by the following formula:

[0131] ,

[0132] in It obeys the standard normal distribution;

[0133] from Recover from The ideal variance It can be expressed by the following formula:

[0134] ,

[0135] In the reverse diffusion process of generating the hyperspectral image segmentation result, the denoising process can be expressed by the following formula:

[0136] ,

[0137] The reverse diffusion mean ,variance The value of is determined by the vector , Diffusion step index and encoding features Predict through Unet network;

[0138] (2-5) Execution The denoising operation generates the result , the denoising process of each step can be expressed by the following formula:

[0139] ,

[0140] in The value is less than greater than 0, Represents the Unet network.

[0141] The third step is to build a cross-scene hyperspectral image segmentation network based on style transfer:

[0142] The specific steps are as follows:

[0143] (1) Corrected source domain hyperspectral image The corresponding source domain encoding features are generated by the encoder based on multi-view cross attention ; Target domain hyperspectral image The encoder based on multi-view cross attention generates the corresponding target domain encoding features ;

[0144] (2) Encoding features of the source domain Perform Fourier transform and save its phase, then the Fourier transform process can be expressed by the following formula:

[0145] ,

[0146] in represents Fourier transform, represents the source domain encoding feature phase, represents the source domain encoding feature amplitude, and Represents the horizontal and vertical frequency coordinates in the frequency domain;

[0147] (3) Encoding features of the target domain Perform Fourier transform and save its amplitude. The Fourier transform process can be expressed by the following formula:

[0148] ,

[0149] in represents Fourier transform, represents the target domain encoding feature phase, represents the target domain encoding feature amplitude, and Represents the horizontal and vertical frequency coordinates in the frequency domain;

[0150] (4) The saved source domain encoding feature phase and the target domain encoding feature amplitude As the input of the inverse Fourier transform, the source domain encodes the characteristic phase Represents the source domain encoding features The domain invariant information of the target domain encodes the feature amplitude represents the style information of the target domain; then the process of inverse Fourier transform can be expressed by the following formula:

[0151] ,

[0152] The style transfer feature Represents source domain encoding features Features after style transfer to the target domain;

[0153] (5) Style transfer features Replace source domain encoding features The final segmentation result is generated as input to the decoder based on the diffusion model.

[0154] The fourth step is to train and test the cross-scene hyperspectral image segmentation network based on style transfer:

[0155] The specific steps are as follows:

[0156] (1) Corrected source domain hyperspectral image The corresponding source domain encoding features are generated by the encoder based on multi-view cross attention , and then generate style conversion features through style conversion Finally, the decoder based on the diffusion model generates the final source domain hyperspectral image segmentation result ;

[0157] (2) Calculate the source domain hyperspectral image segmentation results and source domain hyperspectral image segmentation labels The cross entropy loss between , through the cross entropy loss Perform gradient descent to train network parameters; cross entropy loss It can be expressed by the following formula;

[0158] ,

[0159] in Represents the cross entropy loss calculation process;

[0160] (3) Determine whether the set number of training rounds has been reached. If so, complete the training of the cross-scene hyperspectral image segmentation network based on style transfer; otherwise, continue training;

[0161] (4) Target domain hyperspectral image The image is input into the trained cross-scene hyperspectral image segmentation network based on style transfer to obtain the final target domain hyperspectral image segmentation result.

[0162] The cross-scene hyperspectral image segmentation method based on style transfer proposed in the present invention significantly improves the migration ability of the source domain model to the target domain data, and successfully solves the problem of decreased segmentation accuracy caused by style differences in satellite images. Compared with the prior art, the present invention enhances the adaptability of the model to different scenes and data sources, while improving the accuracy and robustness of the segmentation results through the combination of style transfer and multi-view information fusion network. This method makes full use of the spatial and spectral information of hyperspectral images, especially when the target domain data is incomplete, it can still maintain a high segmentation accuracy. Through innovative design, the present invention improves the stability and generalization ability of segmentation and reduces the impact of style differences. The present invention provides a reliable solution for hyperspectral image segmentation and has wide application potential, especially in the fields of environmental monitoring, agricultural management and geological exploration. It has important practical value.

[0163] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A cross-scene hyperspectral image segmentation method based on style transfer, characterized in that: The following steps are involved: 11) Build a cross-scene hyperspectral image segmentation dataset: Get the source domain hyperspectral image X s and the source domain hyperspectral image segmentation label Y s , where s represents the source domain; Get the target domain hyperspectral image X t , where t represents the target domain: gamma correction is performed to obtain the target domain hyperspectral image X t The brightness of the source domain hyperspectral image X is used as a reference to adjust s Brightness; Obtain the corrected source domain hyperspectral image X′ s ; 12) Construct a multi-view information fusion segmentation network: Construct a multi-view information fusion segmentation network with hyperspectral image as input; the multi-view information fusion segmentation network consists of an encoder based on multi-view cross-attention and a decoder based on a diffusion model; The specific steps are as follows: 121) The multi-view information fusion segmentation network accepts the hyperspectral image X as input; the multi-view information fusion segmentation network consists of an encoder based on multi-view cross-attention and a decoder based on a diffusion model; 1211) construct an encoder based on multi-view cross-attention; the encoder accepts a hyperspectral image X as input, uses a dual-branch structure to process the hyperspectral image in different dimensions, and combines the cross-attention mechanism to fuse the spatial-spectral information; The detailed structure is as follows: 1212) In the spatial branch, the hyperspectral image X is first subjected to a two-dimensional deep convolution to generate a spatial conditional position code P spatial , and then the spatial conditional position code P spatial And the hyperspectral image X is added; the two-dimensional deep convolution uses a convolution kernel of size 3×3 and is filled with 1; then the spatial branch embedding feature X is generated by linear projection spatial , the shape is hw×d, d represents the embedding dimension size; 1213) In the spectral branch, the hyperspectral image X is first flattened according to the spatial dimension, and then a one-dimensional deep convolution operation is used to generate the spectral conditional position code P spectral , and finally the spectral condition position code P spectral And the flattened hyperspectral image X is added; the one-dimensional deep convolution uses a convolution kernel of size 3 and is filled with 1; then the spectral branch embedding feature X is generated by linear projection spectral , the shape is c×d; 1214) Spatial branch embedding feature X spatial and spectral branch embedding feature X spectral The features belonging to the same hyperspectral image X at different viewing angles; the encoder includes four consecutive feature extraction modules, each of which contains a Transformer Encoder on the spatial branch, a Transformer Encoder on the spectral branch, and a cross-attention module connecting the spatial branch and the spectral branch; 12141) In the cross-attention module, the spatial branch is embedded into the feature X spatial Generate query matrix Q1, key matrix K1, value matrix V1 through linear mapping; spectral branch embedding feature X spectral Generate corresponding Q2, K2, V2; 12142) The fusion process from the spatial branch to the spectral branch can be calculated by the following formula: Where T represents the transposition operation, softmax() represents the softmax function, and Fusion() represents the fusion process; 12143) The fusion process from the spatial branch to the spectral branch can be calculated by the following formula: 12144) The output of each feature extraction module in the encoder is the element-by-element addition of Fusion(Q1, K2, V2) and Fusion(Q2, K1, V1); 1215) After four consecutive feature extraction modules of the encoder, the hyperspectral image X finally obtains the encoding feature F; 122) constructing a decoder based on a diffusion model; the diffusion model accepts the encoded feature F as input, and its corresponding label is the hyperspectral image segmentation label Y; 1221) Construct a standard Unet network to predict the mean of the multidimensional Gaussian distribution in the denoising process; 1222) In the forward process of diffusion from hyperspectral image segmentation label to noise, let the maximum number of steps of adding noise be K, then Y (k) Represents the feature map obtained by forward diffusion of hyperspectral image segmentation labels after k steps, where k is a positive integer less than K; (k-1) Sample Y from the associated normal distribution (k) The forward diffusion process representing one step can be expressed by the following formula; where β (k) ∈{β (1) , β (2) , ..., β (K) } is uniformly distributed between 0.0001 and 0.02, representing the intensity of noise added at each step; represents a normal distribution with mean μ and variance σ; I represents the identity matrix; represents a standard normal distribution; 1223) Randomly select step number k from Y (0) The vector Y obtained by k-step noise addition operation is generated in (k) , which can be expressed by the following formula: where α (k) =1-β (k) , and β (k) ∈{β (1) , β (2) , ..., β (K) } follows a uniform distribution between 0.0001 and 0.02, ε (k) Obey standard normal distribution; 1224) from Y (k) Recover Y (k-1) The ideal mean It can be expressed by the following formula: where ε (k) It obeys the standard normal distribution; From Y (k) Recover Y (k-1) The ideal variance It can be expressed by the following formula: In the reverse diffusion process of generating the hyperspectral image segmentation result, the denoising process can be expressed by the following formula: The reverse diffusion mean variance The value of is determined by the vector Y (k) , diffusion step index k and encoding feature F are predicted through the Unet network; 1225) Perform K-step denoising operation to generate result Y (0) , the denoising process of each step can be expressed by the following formula: Where the value of k is less than K and greater than 0, and Unet() represents the Unet network; 13) Construct a cross-scene hyperspectral image segmentation network based on style transfer: Through the style transfer module based on Fourier transform, combined with the multi-view information fusion segmentation network, a cross-scene hyperspectral image segmentation network based on style transfer is constructed; the specific steps are as follows: 131) Corrected source domain hyperspectral image X′ s The corresponding source domain encoding feature F is generated by the encoder based on multi-view cross attention s ; Target domain hyperspectral image X t The encoder based on multi-view cross attention generates the corresponding target domain encoding feature F t ; 132) Encode the source domain feature F s Perform Fourier transform and save its phase, then the Fourier transform process can be expressed by the following formula: Where Fourier() stands for Fourier transform, represents the source domain encoding feature phase, |A s (a, b)| represents the source domain coding feature amplitude, a and b represent the horizontal and vertical frequency coordinates in the frequency domain; 133) Encode the target domain feature F t Perform Fourier transform and save its amplitude. The Fourier transform process can be expressed by the following formula: Where Fourier() stands for Fourier transform, represents the target domain encoding feature phase, |A t (a, b)| represents the target domain encoding feature amplitude, a and b represent the horizontal and vertical frequency coordinates in the frequency domain; 134) The saved source domain encoding feature phase and the target domain encoding feature amplitude |A t (a, b) | As the input of the inverse Fourier transform, the source domain encodes the characteristic phase Represents the source domain encoding feature F s The domain invariant information of the target domain is encoded by the feature amplitude |A t (a, b)| represents the style information of the target domain; then the process of inverse Fourier transform can be expressed by the following formula: The style transfer feature F s→t Represents the source domain encoding feature F s Features after style transfer to the target domain; 135) Style transfer feature F s→t Instead of the source domain encoding feature F s The final segmentation result is generated as the input of the decoder based on the diffusion model; 14) Train and test a cross-scene hyperspectral image segmentation network based on style transfer: By minimizing the cross entropy loss L ce Optimize the network parameters of cross-scene hyperspectral image segmentation and perform segmentation on the target domain hyperspectral image X t The final classification results are obtained by testing.

2. According to the method of claim 1, the method is characterized in that: The construction of a cross-scene hyperspectral image segmentation dataset comprises the following steps: 21) Obtain source domain hyperspectral image X s , source domain hyperspectral image segmentation label Y s and the target domain hyperspectral image X t , where the source domain hyperspectral image X s and the target domain hyperspectral image X t All have c bands with the same band range, and all have the same height h and width w; 22) Calculate the source domain hyperspectral image X s and the target domain hyperspectral image X t The mean brightness value in each band of c bands, where the source domain hyperspectral image X s The mean brightness in the rth band Calculated by the following formula: Where h and w represent the height and width of the hyperspectral image, respectively, s () represents the indexing operation of the source domain hyperspectral image; Target domain hyperspectral image X t The mean brightness in the rth band Calculated by the following formula: Where h and w represent the height and width of the hyperspectral image, respectively, t () represents the indexing operation of the target domain hyperspectral image; 23) Take the target domain hyperspectral image X t The brightness of the source domain is used as a reference to calculate the source domain hyperspectral image X s The gamma value γ in the rth band when doing gamma correction r ; Gamma value γ r Calculated by the following formula: in Represents the source domain hyperspectral image X s The mean brightness in the rth band, Represents the target domain hyperspectral image X t The mean brightness in the rth band; 24) For source domain hyperspectral image X s Applying the gamma correction transform, the transform process on the rth band can be expressed by the following formula: where X′ s () represents the indexing operation of the corrected source domain hyperspectral image; 25) Obtain the corrected source domain hyperspectral image X′ s .

3. The cross-scene hyperspectral image segmentation method based on style transfer according to claim 1, characterized in that: The training and testing of the cross-scene hyperspectral image segmentation network based on style transfer includes the following steps: 31) Corrected source domain hyperspectral image X′ s The corresponding source domain encoding feature F is generated by the encoder based on multi-view cross attention s , and then generate style conversion features F through style conversion s→t Finally, the decoder based on the diffusion model generates the final source domain hyperspectral image segmentation result 32) Calculate the source domain hyperspectral image segmentation results and the source domain hyperspectral image segmentation label Y s The cross entropy loss L between ce , through the cross entropy loss L ce Perform gradient descent to train network parameters; cross entropy loss L ce It can be expressed by the following formula; Where cross entropy() represents the cross entropy loss calculation process; 33) Determine whether the set number of training rounds has been reached. If so, complete the training of the cross-scene hyperspectral image segmentation network based on style transfer, otherwise continue training; 34) The target domain hyperspectral image X t The image is input into the trained cross-scene hyperspectral image segmentation network based on style transfer to obtain the final target domain hyperspectral image segmentation result.

Images